[MDEV-13785] MariaDB 10.2. [7,8] does not use hugepages Created: 2017-09-12  Updated: 2018-05-29  Resolved: 2018-03-29

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - InnoDB
Affects Version/s: 10.2.7, 10.2.8, 10.2
Fix Version/s: 10.2.15

Type: Bug Priority: Major
Reporter: Devin Yu Assignee: Sergey Vojtovich
Resolution: Fixed Votes: 3
Labels: Hugepages
Environment:

CentOS Linux release 7.3.1611 (Core)
3.10.0-514.el7.x86_64
Mem:2.8G


Attachments: Text File alert_MGC.log     File my.cnf    

 Description   

It seems MariaDB 10.2.[7,8] does not use hugepages, but MariaDB 10.1.x,the same configuration, can use hugepages.

Thank you.

 
$ cat /etc/sysctl.conf
kernel.shmmax = 2705817600
kernel.shmall = 660600
vm.nr_hugepages = 500
vm.hugetlb_shm_group = 2001
 
$ id mysql
uid=2001(mysql) gid=2001(mysql) groups=2001(mysql)
 
$ cat /etc/security/limits.conf
mysql   soft   nproc    131072
mysql   hard   nproc    131072
mysql   soft   nofile   131072
mysql   hard   nofile   131072
mysql   soft   core     unlimited
mysql   hard   core     unlimited
mysql   soft   memlock  unlimited
mysql   hard   memlock  unlimited
 
$ free -h
              total        used        free      shared  buff/cache   available
Mem:           2.8G        1.1G        250M        2.2M        1.5G        1.5G
Swap:          1.5G         50M        1.5G
 
$ cat /proc/meminfo | grep -i huge
AnonHugePages:      2048 kB
HugePages_Total:     500
HugePages_Free:      500
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
 
$ cat /etc/my.cnf | grep large-pages
large-pages
 
$ mysqld_safe &
[1] 67750
170909 01:34:39 mysqld_safe Logging to '/MGC/datadg/log/alert_MGC.log'.
170909 01:34:40 mysqld_safe Starting mysqld daemon with databases from /MGC/datadg/db
 
$ cat /MGC/datadg/log/alert_MGC.log | grep -i "Warning: Using conventional memory pool"
 
$ cat /proc/meminfo | grep -i huge
AnonHugePages:     88064 kB
HugePages_Total:     500
HugePages_Free:      499
HugePages_Rsvd:      115
HugePages_Surp:        0
Hugepagesize:       2048 kB
 
 
01:36:47 (root@localhost) [(none)]> select version();
+--------------------+
| version()          |
+--------------------+
| 10.2.7-MariaDB-log |
+--------------------+
1 row in set (0.00 sec)
 
01:37:29 (root@localhost) [(none)]> select count(*) from test.a;
+----------+
| count(*) |
+----------+
|  3263442 |
+----------+
1 row in set (2.44 sec)
 
 
$ cat /proc/meminfo | grep -i huge
AnonHugePages:    167936 kB
HugePages_Total:     500
HugePages_Free:      499
HugePages_Rsvd:      115
HugePages_Surp:        0
Hugepagesize:       2048 kB
 
$ free -h
              total        used        free      shared  buff/cache   available
Mem:           2.8G        1.2G         76M        2.2M        1.4G        1.3G
Swap:          1.5G         50M        1.5G
 
$ ll -h
total 109M
-rw-rw---- 1 mysql mysql  432 Sep  8 23:34 a.frm
-rw-rw---- 1 mysql mysql 108M Sep  8 23:36 a.ibd
 
> show global variables like '%buffer%';
+-------------------------------------+----------------+
| Variable_name                       | Value          |
+-------------------------------------+----------------+
| aria_pagecache_buffer_size          | 134217728      |
| aria_sort_buffer_size               | 268434432      |
| bulk_insert_buffer_size             | 8388608        |
| innodb_buffer_pool_chunk_size       | 134217728      |
| innodb_buffer_pool_dump_at_shutdown | ON             |
| innodb_buffer_pool_dump_now         | OFF            |
| innodb_buffer_pool_dump_pct         | 25             |
| innodb_buffer_pool_filename         | ib_buffer_pool |
| innodb_buffer_pool_instances        | 1              |
| innodb_buffer_pool_load_abort       | OFF            |
| innodb_buffer_pool_load_at_startup  | ON             |
| innodb_buffer_pool_load_now         | OFF            |
| innodb_buffer_pool_populate         | OFF            |
| innodb_buffer_pool_size             | 134217728      |
| innodb_change_buffer_max_size       | 25             |
| innodb_change_buffering             | all            |
| innodb_log_buffer_size              | 16777216       |
| innodb_sort_buffer_size             | 1048576        |
| join_buffer_size                    | 262144         |
| join_buffer_space_limit             | 2097152        |
| key_buffer_size                     | 134217728      |
| mrr_buffer_size                     | 262144         |
| myisam_sort_buffer_size             | 134216704      |
| net_buffer_length                   | 16384          |
| preload_buffer_size                 | 32768          |
| read_buffer_size                    | 131072         |
| read_rnd_buffer_size                | 262144         |
| sort_buffer_size                    | 2097152        |
| sql_buffer_result                   | OFF            |
+-------------------------------------+----------------+



 Comments   
Comment by Elena Stepanova [ 2017-10-01 ]

Thanks for the report, I can reproduce it with InnoDB. Seems to work with MyISAM.

I think the problem might be caused by the change of the variable used in 10.2 vs 10.1. In 10.1, the logic is under HAVE_LARGE_PAGES, while in 10.2 it's under HAVE_LINUX_LARGE_PAGES. The change came with InnoDB 5.7 merge; but in MySQL 5.7, cmake files were changed accordingly, and the other code also uses the new name, while in 10.2 it's only checked in InnoDB, but isn't set anywhere, and the server still uses HAVE_LARGE_PAGES.

Comment by Daniel Black [ 2018-03-22 ]

so pushing `HAVE_LINUX_LARGE_PAGES` to the wider server code is the recommended fix for now?

Comment by Marc [ 2018-04-03 ]

Hi, this bug is closed and marked as fjxed in 10.2.14 but is not present in the change log of 10.2.14. Also 10.2.14 was released beginnig of march while the fixed status was set on march 29 and last the pull request is marked as declined ...

I installed 10.2.14 and I confirm that hugepages is trill not used by innodb.

Could someone clarify the real status of this issue?

Comment by Sergey Vojtovich [ 2018-04-03 ]

Sorry, wrong fix version was set. To be released along with 10.2.15.

Comment by Marc [ 2018-04-03 ]

thanks

Generated at Thu Feb 08 08:08:20 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.