Details
-
Bug
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Fixed
-
10.2.2, 10.3.0, 10.4.0
-
None
-
x86_64, fc29, g++ (GCC) 8.3.1 20190223 (Red Hat 8.3.1-2)
Description
From upstream bug, https://bugs.mysql.com/bug.php?id=94747, publicity thanks to Valerii
Allocations of 4G+ innodb_buffer_pool_chunk_size results in int overflow:
10.4.4
sql/mysqld --no-defaults --skip-networking --datadir=/tmp/datadir --log-bin=/tmp/datadir/mysqlbin --socket /tmp/s.sock --lc-messages-dir=/home/dan/repos/build-mariadb-server-10.4-upstream/sql/share --innodb_buffer_pool_instances=1 --large-pages --innodb_buffer_pool_chunk_size=4G --innodb_buffer_pool_size=8G --verbose
|
|
2019-04-08 10:20:50 0 [Note] /home/dan/repos/build-mariadb-server-10.4-upstream/sql/mysqld (mysqld 10.4.4-MariaDB-log) starting as process 32260 ...
|
|
Thread 1 "mysqld" hit Breakpoint 1, os_mem_alloc_large (n=n@entry=0x7fffffff59b0) at /home/dan/repos/mariadb-server/storage/innobase/os/os0proc.cc:66
|
66 {
|
(gdb) p *n
|
$1 = 4391436288
|
(gdb) n
|
73 if (!os_use_large_pages || !os_large_page_size) {
|
(gdb) p os_large_page_size
|
$2 = 2097152
|
(gdb) n
|
79 size = ut_2pow_round(*n + (os_large_page_size - 1),
|
(gdb) n
|
82 shmid = shmget(IPC_PRIVATE, (size_t) size, SHM_HUGETLB | SHM_R | SHM_W);
|
(gdb) p size
|
$3 = 96468992
|
(gdb) n
|
83 if (shmid < 0) {
|
(gdb)
|
84 ib::warn() << "Failed to allocate " << size
|
(gdb)
|
85 << " bytes. errno " << errno;
|
(gdb)
|
2019-04-08 10:25:48 0 [Warning] InnoDB: Failed to allocate 96468992 bytes. errno 22
|
108 ib::warn() << "Using conventional memory pool";
|
5.5 instance doesn't support chunk size - but seemed unaffected
190408 10:29:29 [Note] /home/dan/repos/build-mariadb-5.5/sql/mysqld (mysqld 5.5.64-MariaDB) starting as process 26248 ...
|
Warning: Failed to allocate 130023424 bytes from HugeTLB memory. errno 22
|
Warning: Using conventional memory pool
|
[New Thread 0x7ffff6dff700 (LWP 26255)]
|
190408 10:29:29 InnoDB: The InnoDB memory heap is disabled
|
190408 10:29:29 InnoDB: Mutexes and rw_locks use GCC atomic builtins
|
190408 10:29:29 InnoDB: Compressed tables use zlib 1.2.11
|
190408 10:29:29 InnoDB: Using Linux native AIO
|
190408 10:29:29 InnoDB: Initializing buffer pool, size = 8.0G
|
|
Thread 1 "mysqld" hit Breakpoint 1, os_mem_alloc_large (n=n@entry=0x7ffff72eb258, populate=populate@entry=0)
|
at /home/dan/repos/mariadb-server-5.5/storage/xtradb/os/os0proc.c:112
|
112 if (!os_use_large_pages || !os_large_page_size) {
|
Missing separate debuginfos, use: dnf debuginfo-install jemalloc-5.1.0-2.fc29.x86_64 libxcrypt-4.4.4-2.fc29.x86_64
|
(gdb) p os_large_page_size
|
$1 = 2097152
|
(gdb) p *n
|
$2 = 8816427008
|
(gdb) n
|
118 size = ut_2pow_round(*n + (os_large_page_size - 1),
|
(gdb) n
|
121 shmid = shmget(IPC_PRIVATE, (size_t)size, SHM_HUGETLB | SHM_R | SHM_W);
|
(gdb) p size
|
$3 = 8816427008
|
(gdb) n
|
122 if (shmid < 0) {
|
(gdb)
|
123 fprintf(stderr, "InnoDB: HugeTLB: Warning: Failed to allocate"
|
(gdb)
|
InnoDB: HugeTLB: Warning: Failed to allocate 8816427008 bytes. errno 22
|
141 if (ptr) {
|
(gdb) quit
|
After changing os_large_page_size back to ulint (from uint where it was changed in 2e814d4702d71a04388386a9f591d14a35980bfe Merge InnoDB 5.7 from mysql-5.7.9) this is corrected:
2019-04-08 10:57:07 0 [Note] /home/dan/repos/build-mariadb-server-10.4/sql/mysqld (mysqld 10.4.5-MariaDB-debug-log) starting as process 21743 ...
|
2019-04-08 10:57:07 0 [Warning] Could not increase number of max_open_files to more than 4096 (request: 4190)
|
2019-04-08 10:57:07 0 [Warning] Changed limits: max_open_files: 4096 max_connections: 151 (was 151) table_cache: 1957 (was 2000)
|
[New Thread 0x7ffff70e3700 (LWP 21772)]
|
Warning: Failed to allocate 130023424 bytes from HugeTLB memory. errno 22
|
Warning: Using conventional memory pool
|
[New Thread 0x7fffe25ff700 (LWP 21782)]
|
2019-04-08 10:57:07 0 [Note] InnoDB: Using Linux native AIO
|
2019-04-08 10:57:07 0 [Note] InnoDB: !!!!!!!! UNIV_DEBUG switched on !!!!!!!!!
|
2019-04-08 10:57:07 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
|
2019-04-08 10:57:07 0 [Note] InnoDB: Uses event mutexes
|
2019-04-08 10:57:07 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
|
2019-04-08 10:57:07 0 [Note] InnoDB: Number of pools: 1
|
2019-04-08 10:57:07 0 [Note] InnoDB: Using SSE2 crc32 instructions
|
2019-04-08 10:57:07 0 [Note] mysqld: O_TMPFILE is not supported on /tmp (disabling future attempts)
|
2019-04-08 10:57:07 0 [Note] InnoDB: Initializing buffer pool, total size = 8G, instances = 1, chunk size = 4G
|
|
Thread 1 "mysqld" hit Breakpoint 1, os_mem_alloc_large (n=0x7fffffff8ba0) at /home/dan/repos/mariadb-server-10.4/storage/innobase/os/os0proc.cc:66
|
66 {
|
Missing separate debuginfos, use: dnf debuginfo-install libxcrypt-4.4.4-2.fc29.x86_64 openssl-libs-1.1.1b-3.fc29.x86_64
|
(gdb) p *n
|
$1 = 4294967296
|
(gdb) n
|
73 if (!os_use_large_pages || !os_large_page_size) {
|
(gdb) p os_large_page_size
|
$2 = 2097152
|
(gdb) n
|
78 ut_ad(ut_is_2pow(os_large_page_size));
|
(gdb)
|
79 size = ut_2pow_round(*n + (os_large_page_size - 1),
|
(gdb)
|
82 shmid = shmget(IPC_PRIVATE, (size_t) size, SHM_HUGETLB | SHM_R | SHM_W);
|
(gdb) p size
|
$3 = 4294967296
|
(gdb) n
|
tested in 10.3 and also corrected.
Note MDEV-18851 will correct the 10.4 version by removing os_large_page_size
Already MariaDB Server 10.2.2 is affected, due to merging the MySQL 5.7.4 change that introduced the regression.
It is worth noting that the author of that change uses Microsoft Windows, and his intention wathat introduced the regressions to remove warnings that are analogous to gcc -Wconversion. I think that we should keep the type of the global variable as uint, but add conversions to a wider data type where appropriate.