Details
-
Bug
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Fixed
-
10.2.2, 10.3.0, 10.4.0
-
None
-
x86_64, fc29, g++ (GCC) 8.3.1 20190223 (Red Hat 8.3.1-2)
Description
From upstream bug, https://bugs.mysql.com/bug.php?id=94747, publicity thanks to Valerii
Allocations of 4G+ innodb_buffer_pool_chunk_size results in int overflow:
10.4.4
sql/mysqld --no-defaults --skip-networking --datadir=/tmp/datadir --log-bin=/tmp/datadir/mysqlbin --socket /tmp/s.sock --lc-messages-dir=/home/dan/repos/build-mariadb-server-10.4-upstream/sql/share --innodb_buffer_pool_instances=1 --large-pages --innodb_buffer_pool_chunk_size=4G --innodb_buffer_pool_size=8G --verbose
|
|
|
2019-04-08 10:20:50 0 [Note] /home/dan/repos/build-mariadb-server-10.4-upstream/sql/mysqld (mysqld 10.4.4-MariaDB-log) starting as process 32260 ...
|
|
|
Thread 1 "mysqld" hit Breakpoint 1, os_mem_alloc_large (n=n@entry=0x7fffffff59b0) at /home/dan/repos/mariadb-server/storage/innobase/os/os0proc.cc:66
|
66 {
|
(gdb) p *n
|
$1 = 4391436288
|
(gdb) n
|
73 if (!os_use_large_pages || !os_large_page_size) {
|
(gdb) p os_large_page_size
|
$2 = 2097152
|
(gdb) n
|
79 size = ut_2pow_round(*n + (os_large_page_size - 1),
|
(gdb) n
|
82 shmid = shmget(IPC_PRIVATE, (size_t) size, SHM_HUGETLB | SHM_R | SHM_W);
|
(gdb) p size
|
$3 = 96468992
|
(gdb) n
|
83 if (shmid < 0) {
|
(gdb)
|
84 ib::warn() << "Failed to allocate " << size
|
(gdb)
|
85 << " bytes. errno " << errno;
|
(gdb)
|
2019-04-08 10:25:48 0 [Warning] InnoDB: Failed to allocate 96468992 bytes. errno 22
|
108 ib::warn() << "Using conventional memory pool";
|
5.5 instance doesn't support chunk size - but seemed unaffected
190408 10:29:29 [Note] /home/dan/repos/build-mariadb-5.5/sql/mysqld (mysqld 5.5.64-MariaDB) starting as process 26248 ...
|
Warning: Failed to allocate 130023424 bytes from HugeTLB memory. errno 22
|
Warning: Using conventional memory pool
|
[New Thread 0x7ffff6dff700 (LWP 26255)]
|
190408 10:29:29 InnoDB: The InnoDB memory heap is disabled
|
190408 10:29:29 InnoDB: Mutexes and rw_locks use GCC atomic builtins
|
190408 10:29:29 InnoDB: Compressed tables use zlib 1.2.11
|
190408 10:29:29 InnoDB: Using Linux native AIO
|
190408 10:29:29 InnoDB: Initializing buffer pool, size = 8.0G
|
|
|
Thread 1 "mysqld" hit Breakpoint 1, os_mem_alloc_large (n=n@entry=0x7ffff72eb258, populate=populate@entry=0)
|
at /home/dan/repos/mariadb-server-5.5/storage/xtradb/os/os0proc.c:112
|
112 if (!os_use_large_pages || !os_large_page_size) {
|
Missing separate debuginfos, use: dnf debuginfo-install jemalloc-5.1.0-2.fc29.x86_64 libxcrypt-4.4.4-2.fc29.x86_64
|
(gdb) p os_large_page_size
|
$1 = 2097152
|
(gdb) p *n
|
$2 = 8816427008
|
(gdb) n
|
118 size = ut_2pow_round(*n + (os_large_page_size - 1),
|
(gdb) n
|
121 shmid = shmget(IPC_PRIVATE, (size_t)size, SHM_HUGETLB | SHM_R | SHM_W);
|
(gdb) p size
|
$3 = 8816427008
|
(gdb) n
|
122 if (shmid < 0) {
|
(gdb)
|
123 fprintf(stderr, "InnoDB: HugeTLB: Warning: Failed to allocate"
|
(gdb)
|
InnoDB: HugeTLB: Warning: Failed to allocate 8816427008 bytes. errno 22
|
141 if (ptr) {
|
(gdb) quit
|
After changing os_large_page_size back to ulint (from uint where it was changed in 2e814d4702d71a04388386a9f591d14a35980bfe Merge InnoDB 5.7 from mysql-5.7.9) this is corrected:
2019-04-08 10:57:07 0 [Note] /home/dan/repos/build-mariadb-server-10.4/sql/mysqld (mysqld 10.4.5-MariaDB-debug-log) starting as process 21743 ...
|
2019-04-08 10:57:07 0 [Warning] Could not increase number of max_open_files to more than 4096 (request: 4190)
|
2019-04-08 10:57:07 0 [Warning] Changed limits: max_open_files: 4096 max_connections: 151 (was 151) table_cache: 1957 (was 2000)
|
[New Thread 0x7ffff70e3700 (LWP 21772)]
|
Warning: Failed to allocate 130023424 bytes from HugeTLB memory. errno 22
|
Warning: Using conventional memory pool
|
[New Thread 0x7fffe25ff700 (LWP 21782)]
|
2019-04-08 10:57:07 0 [Note] InnoDB: Using Linux native AIO
|
2019-04-08 10:57:07 0 [Note] InnoDB: !!!!!!!! UNIV_DEBUG switched on !!!!!!!!!
|
2019-04-08 10:57:07 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
|
2019-04-08 10:57:07 0 [Note] InnoDB: Uses event mutexes
|
2019-04-08 10:57:07 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
|
2019-04-08 10:57:07 0 [Note] InnoDB: Number of pools: 1
|
2019-04-08 10:57:07 0 [Note] InnoDB: Using SSE2 crc32 instructions
|
2019-04-08 10:57:07 0 [Note] mysqld: O_TMPFILE is not supported on /tmp (disabling future attempts)
|
2019-04-08 10:57:07 0 [Note] InnoDB: Initializing buffer pool, total size = 8G, instances = 1, chunk size = 4G
|
|
|
Thread 1 "mysqld" hit Breakpoint 1, os_mem_alloc_large (n=0x7fffffff8ba0) at /home/dan/repos/mariadb-server-10.4/storage/innobase/os/os0proc.cc:66
|
66 {
|
Missing separate debuginfos, use: dnf debuginfo-install libxcrypt-4.4.4-2.fc29.x86_64 openssl-libs-1.1.1b-3.fc29.x86_64
|
(gdb) p *n
|
$1 = 4294967296
|
(gdb) n
|
73 if (!os_use_large_pages || !os_large_page_size) {
|
(gdb) p os_large_page_size
|
$2 = 2097152
|
(gdb) n
|
78 ut_ad(ut_is_2pow(os_large_page_size));
|
(gdb)
|
79 size = ut_2pow_round(*n + (os_large_page_size - 1),
|
(gdb)
|
82 shmid = shmget(IPC_PRIVATE, (size_t) size, SHM_HUGETLB | SHM_R | SHM_W);
|
(gdb) p size
|
$3 = 4294967296
|
(gdb) n
|
tested in 10.3 and also corrected.
Note MDEV-18851 will correct the 10.4 version by removing os_large_page_size