Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-19212

4GB Limit on large_pages (innodb_buffer_pool_chunk_size) - integer overflow

Details

    Description

      From upstream bug, https://bugs.mysql.com/bug.php?id=94747, publicity thanks to Valerii

      Allocations of 4G+ innodb_buffer_pool_chunk_size results in int overflow:

      10.4.4

      sql/mysqld --no-defaults --skip-networking --datadir=/tmp/datadir --log-bin=/tmp/datadir/mysqlbin --socket /tmp/s.sock --lc-messages-dir=/home/dan/repos/build-mariadb-server-10.4-upstream/sql/share --innodb_buffer_pool_instances=1 --large-pages --innodb_buffer_pool_chunk_size=4G --innodb_buffer_pool_size=8G --verbose
       
      2019-04-08 10:20:50 0 [Note] /home/dan/repos/build-mariadb-server-10.4-upstream/sql/mysqld (mysqld 10.4.4-MariaDB-log) starting as process 32260 ...
       
      Thread 1 "mysqld" hit Breakpoint 1, os_mem_alloc_large (n=n@entry=0x7fffffff59b0) at /home/dan/repos/mariadb-server/storage/innobase/os/os0proc.cc:66
      66	{
      (gdb) p *n
      $1 = 4391436288
      (gdb) n
      73		if (!os_use_large_pages || !os_large_page_size) {
      (gdb) p os_large_page_size 
      $2 = 2097152
      (gdb) n
      79		size = ut_2pow_round(*n + (os_large_page_size - 1),
      (gdb) n
      82		shmid = shmget(IPC_PRIVATE, (size_t) size, SHM_HUGETLB | SHM_R | SHM_W);
      (gdb) p size
      $3 = 96468992
      (gdb) n
      83		if (shmid < 0) {
      (gdb) 
      84			ib::warn() << "Failed to allocate " << size
      (gdb) 
      85				<< " bytes. errno " << errno;
      (gdb) 
      2019-04-08 10:25:48 0 [Warning] InnoDB: Failed to allocate 96468992 bytes. errno 22
      108		ib::warn() << "Using conventional memory pool";
      

      5.5 instance doesn't support chunk size - but seemed unaffected

      190408 10:29:29 [Note] /home/dan/repos/build-mariadb-5.5/sql/mysqld (mysqld 5.5.64-MariaDB) starting as process 26248 ...
      Warning: Failed to allocate 130023424 bytes from HugeTLB memory. errno 22
      Warning: Using conventional memory pool
      [New Thread 0x7ffff6dff700 (LWP 26255)]
      190408 10:29:29 InnoDB: The InnoDB memory heap is disabled
      190408 10:29:29 InnoDB: Mutexes and rw_locks use GCC atomic builtins
      190408 10:29:29 InnoDB: Compressed tables use zlib 1.2.11
      190408 10:29:29 InnoDB: Using Linux native AIO
      190408 10:29:29 InnoDB: Initializing buffer pool, size = 8.0G
       
      Thread 1 "mysqld" hit Breakpoint 1, os_mem_alloc_large (n=n@entry=0x7ffff72eb258, populate=populate@entry=0)
          at /home/dan/repos/mariadb-server-5.5/storage/xtradb/os/os0proc.c:112
      112		if (!os_use_large_pages || !os_large_page_size) {
      Missing separate debuginfos, use: dnf debuginfo-install jemalloc-5.1.0-2.fc29.x86_64 libxcrypt-4.4.4-2.fc29.x86_64
      (gdb) p os_large_page_size 
      $1 = 2097152
      (gdb) p *n
      $2 = 8816427008
      (gdb) n
      118		size = ut_2pow_round(*n + (os_large_page_size - 1),
      (gdb) n
      121		shmid = shmget(IPC_PRIVATE, (size_t)size, SHM_HUGETLB | SHM_R | SHM_W);
      (gdb) p size
      $3 = 8816427008
      (gdb) n
      122		if (shmid < 0) {
      (gdb) 
      123			fprintf(stderr, "InnoDB: HugeTLB: Warning: Failed to allocate"
      (gdb) 
      InnoDB: HugeTLB: Warning: Failed to allocate 8816427008 bytes. errno 22
      141		if (ptr) {
      (gdb) quit
      

      After changing os_large_page_size back to ulint (from uint where it was changed in 2e814d4702d71a04388386a9f591d14a35980bfe Merge InnoDB 5.7 from mysql-5.7.9) this is corrected:

      2019-04-08 10:57:07 0 [Note] /home/dan/repos/build-mariadb-server-10.4/sql/mysqld (mysqld 10.4.5-MariaDB-debug-log) starting as process 21743 ...
      2019-04-08 10:57:07 0 [Warning] Could not increase number of max_open_files to more than 4096 (request: 4190)
      2019-04-08 10:57:07 0 [Warning] Changed limits: max_open_files: 4096  max_connections: 151 (was 151)  table_cache: 1957 (was 2000)
      [New Thread 0x7ffff70e3700 (LWP 21772)]
      Warning: Failed to allocate 130023424 bytes from HugeTLB memory. errno 22
      Warning: Using conventional memory pool
      [New Thread 0x7fffe25ff700 (LWP 21782)]
      2019-04-08 10:57:07 0 [Note] InnoDB: Using Linux native AIO
      2019-04-08 10:57:07 0 [Note] InnoDB: !!!!!!!! UNIV_DEBUG switched on !!!!!!!!!
      2019-04-08 10:57:07 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
      2019-04-08 10:57:07 0 [Note] InnoDB: Uses event mutexes
      2019-04-08 10:57:07 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
      2019-04-08 10:57:07 0 [Note] InnoDB: Number of pools: 1
      2019-04-08 10:57:07 0 [Note] InnoDB: Using SSE2 crc32 instructions
      2019-04-08 10:57:07 0 [Note] mysqld: O_TMPFILE is not supported on /tmp (disabling future attempts)
      2019-04-08 10:57:07 0 [Note] InnoDB: Initializing buffer pool, total size = 8G, instances = 1, chunk size = 4G
       
      Thread 1 "mysqld" hit Breakpoint 1, os_mem_alloc_large (n=0x7fffffff8ba0) at /home/dan/repos/mariadb-server-10.4/storage/innobase/os/os0proc.cc:66
      66	{
      Missing separate debuginfos, use: dnf debuginfo-install libxcrypt-4.4.4-2.fc29.x86_64 openssl-libs-1.1.1b-3.fc29.x86_64
      (gdb) p *n
      $1 = 4294967296
      (gdb) n
      73		if (!os_use_large_pages || !os_large_page_size) {
      (gdb) p os_large_page_size 
      $2 = 2097152
      (gdb) n
      78		ut_ad(ut_is_2pow(os_large_page_size));
      (gdb) 
      79		size = ut_2pow_round(*n + (os_large_page_size - 1),
      (gdb) 
      82		shmid = shmget(IPC_PRIVATE, (size_t) size, SHM_HUGETLB | SHM_R | SHM_W);
      (gdb) p size
      $3 = 4294967296
      (gdb) n
      

      tested in 10.3 and also corrected.

      Note MDEV-18851 will correct the 10.4 version by removing os_large_page_size

      Attachments

        Activity

          Already MariaDB Server 10.2.2 is affected, due to merging the MySQL 5.7.4 change that introduced the regression.

          It is worth noting that the author of that change uses Microsoft Windows, and his intention wathat introduced the regressions to remove warnings that are analogous to gcc -Wconversion. I think that we should keep the type of the global variable as uint, but add conversions to a wider data type where appropriate.

          marko Marko Mäkelä added a comment - Already MariaDB Server 10.2.2 is affected, due to merging the MySQL 5.7.4 change that introduced the regression . It is worth noting that the author of that change uses Microsoft Windows, and his intention wathat introduced the regressions to remove warnings that are analogous to gcc -Wconversion . I think that we should keep the type of the global variable as uint , but add conversions to a wider data type where appropriate.

          To prevent similar errors from being introduced in the future, I will replace the type-unsafe macros ut_2pow_round() and friends with inline function templates.

          marko Marko Mäkelä added a comment - To prevent similar errors from being introduced in the future, I will replace the type-unsafe macros ut_2pow_round() and friends with inline function templates.
          danblack Daniel Black added a comment - Nice fixes: https://github.com/MariaDB/server/commit/e7f426d2c9be3ebc7c373e80873996ce8857ab48 https://github.com/MariaDB/server/commit/f120a15b93dcb88adc31ad4fc7caf63b813ca63e Had something similar to the second in https://github.com/MariaDB/server/pull/1221/commits/0ee8b8907e1905b420308a5cb49848f263be2410 , sigh rebase.

          People

            marko Marko Mäkelä
            danblack Daniel Black
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.