Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-13785

MariaDB 10.2. [7,8] does not use hugepages

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Fixed
    • 10.2.7, 10.2.8, 10.2(EOL)
    • 10.2.15
    • CentOS Linux release 7.3.1611 (Core)
      3.10.0-514.el7.x86_64
      Mem:2.8G

    Description

      It seems MariaDB 10.2.[7,8] does not use hugepages, but MariaDB 10.1.x,the same configuration, can use hugepages.

      Thank you.

       
      $ cat /etc/sysctl.conf
      kernel.shmmax = 2705817600
      kernel.shmall = 660600
      vm.nr_hugepages = 500
      vm.hugetlb_shm_group = 2001
       
      $ id mysql
      uid=2001(mysql) gid=2001(mysql) groups=2001(mysql)
       
      $ cat /etc/security/limits.conf
      mysql   soft   nproc    131072
      mysql   hard   nproc    131072
      mysql   soft   nofile   131072
      mysql   hard   nofile   131072
      mysql   soft   core     unlimited
      mysql   hard   core     unlimited
      mysql   soft   memlock  unlimited
      mysql   hard   memlock  unlimited
       
      $ free -h
                    total        used        free      shared  buff/cache   available
      Mem:           2.8G        1.1G        250M        2.2M        1.5G        1.5G
      Swap:          1.5G         50M        1.5G
       
      $ cat /proc/meminfo | grep -i huge
      AnonHugePages:      2048 kB
      HugePages_Total:     500
      HugePages_Free:      500
      HugePages_Rsvd:        0
      HugePages_Surp:        0
      Hugepagesize:       2048 kB
       
      $ cat /etc/my.cnf | grep large-pages
      large-pages
       
      $ mysqld_safe &
      [1] 67750
      170909 01:34:39 mysqld_safe Logging to '/MGC/datadg/log/alert_MGC.log'.
      170909 01:34:40 mysqld_safe Starting mysqld daemon with databases from /MGC/datadg/db
       
      $ cat /MGC/datadg/log/alert_MGC.log | grep -i "Warning: Using conventional memory pool"
       
      $ cat /proc/meminfo | grep -i huge
      AnonHugePages:     88064 kB
      HugePages_Total:     500
      HugePages_Free:      499
      HugePages_Rsvd:      115
      HugePages_Surp:        0
      Hugepagesize:       2048 kB
       
       
      01:36:47 (root@localhost) [(none)]> select version();
      +--------------------+
      | version()          |
      +--------------------+
      | 10.2.7-MariaDB-log |
      +--------------------+
      1 row in set (0.00 sec)
       
      01:37:29 (root@localhost) [(none)]> select count(*) from test.a;
      +----------+
      | count(*) |
      +----------+
      |  3263442 |
      +----------+
      1 row in set (2.44 sec)
       
       
      $ cat /proc/meminfo | grep -i huge
      AnonHugePages:    167936 kB
      HugePages_Total:     500
      HugePages_Free:      499
      HugePages_Rsvd:      115
      HugePages_Surp:        0
      Hugepagesize:       2048 kB
       
      $ free -h
                    total        used        free      shared  buff/cache   available
      Mem:           2.8G        1.2G         76M        2.2M        1.4G        1.3G
      Swap:          1.5G         50M        1.5G
       
      $ ll -h
      total 109M
      -rw-rw---- 1 mysql mysql  432 Sep  8 23:34 a.frm
      -rw-rw---- 1 mysql mysql 108M Sep  8 23:36 a.ibd
       
      > show global variables like '%buffer%';
      +-------------------------------------+----------------+
      | Variable_name                       | Value          |
      +-------------------------------------+----------------+
      | aria_pagecache_buffer_size          | 134217728      |
      | aria_sort_buffer_size               | 268434432      |
      | bulk_insert_buffer_size             | 8388608        |
      | innodb_buffer_pool_chunk_size       | 134217728      |
      | innodb_buffer_pool_dump_at_shutdown | ON             |
      | innodb_buffer_pool_dump_now         | OFF            |
      | innodb_buffer_pool_dump_pct         | 25             |
      | innodb_buffer_pool_filename         | ib_buffer_pool |
      | innodb_buffer_pool_instances        | 1              |
      | innodb_buffer_pool_load_abort       | OFF            |
      | innodb_buffer_pool_load_at_startup  | ON             |
      | innodb_buffer_pool_load_now         | OFF            |
      | innodb_buffer_pool_populate         | OFF            |
      | innodb_buffer_pool_size             | 134217728      |
      | innodb_change_buffer_max_size       | 25             |
      | innodb_change_buffering             | all            |
      | innodb_log_buffer_size              | 16777216       |
      | innodb_sort_buffer_size             | 1048576        |
      | join_buffer_size                    | 262144         |
      | join_buffer_space_limit             | 2097152        |
      | key_buffer_size                     | 134217728      |
      | mrr_buffer_size                     | 262144         |
      | myisam_sort_buffer_size             | 134216704      |
      | net_buffer_length                   | 16384          |
      | preload_buffer_size                 | 32768          |
      | read_buffer_size                    | 131072         |
      | read_rnd_buffer_size                | 262144         |
      | sort_buffer_size                    | 2097152        |
      | sql_buffer_result                   | OFF            |
      +-------------------------------------+----------------+
      
      

      Attachments

        Activity

          Thanks for the report, I can reproduce it with InnoDB. Seems to work with MyISAM.

          I think the problem might be caused by the change of the variable used in 10.2 vs 10.1. In 10.1, the logic is under HAVE_LARGE_PAGES, while in 10.2 it's under HAVE_LINUX_LARGE_PAGES. The change came with InnoDB 5.7 merge; but in MySQL 5.7, cmake files were changed accordingly, and the other code also uses the new name, while in 10.2 it's only checked in InnoDB, but isn't set anywhere, and the server still uses HAVE_LARGE_PAGES.

          elenst Elena Stepanova added a comment - Thanks for the report, I can reproduce it with InnoDB. Seems to work with MyISAM. I think the problem might be caused by the change of the variable used in 10.2 vs 10.1. In 10.1, the logic is under HAVE_LARGE_PAGES , while in 10.2 it's under HAVE_LINUX_LARGE_PAGES . The change came with InnoDB 5.7 merge; but in MySQL 5.7, cmake files were changed accordingly, and the other code also uses the new name, while in 10.2 it's only checked in InnoDB, but isn't set anywhere, and the server still uses HAVE_LARGE_PAGES .
          danblack Daniel Black added a comment -

          so pushing `HAVE_LINUX_LARGE_PAGES` to the wider server code is the recommended fix for now?

          danblack Daniel Black added a comment - so pushing `HAVE_LINUX_LARGE_PAGES` to the wider server code is the recommended fix for now?
          marc.langevin@usherbrooke.ca Marc added a comment -

          Hi, this bug is closed and marked as fjxed in 10.2.14 but is not present in the change log of 10.2.14. Also 10.2.14 was released beginnig of march while the fixed status was set on march 29 and last the pull request is marked as declined ...

          I installed 10.2.14 and I confirm that hugepages is trill not used by innodb.

          Could someone clarify the real status of this issue?

          marc.langevin@usherbrooke.ca Marc added a comment - Hi, this bug is closed and marked as fjxed in 10.2.14 but is not present in the change log of 10.2.14. Also 10.2.14 was released beginnig of march while the fixed status was set on march 29 and last the pull request is marked as declined ... I installed 10.2.14 and I confirm that hugepages is trill not used by innodb. Could someone clarify the real status of this issue?

          Sorry, wrong fix version was set. To be released along with 10.2.15.

          svoj Sergey Vojtovich added a comment - Sorry, wrong fix version was set. To be released along with 10.2.15.
          marc.langevin@usherbrooke.ca Marc added a comment -

          thanks

          marc.langevin@usherbrooke.ca Marc added a comment - thanks

          People

            svoj Sergey Vojtovich
            920895156@qq.com Devin Yu
            Votes:
            3 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.