Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-14602

Reduce malloc()/free() usage in InnoDB

Details

    Description

      I load backup to mariadb and break it. In processlist still present one killed thread:
      {code}
      show processlist;
      ---------------------------------------------------------------------------------------------------------

      Id User Host db Command Time State Info Progress

      ---------------------------------------------------------------------------------------------------------

      85 loader 10.10.16.216:43971 phonebook_storage Killed 1845 Unlocking tables NULL 0.000
      184 root localhost NULL Query 0 init show processlist 0.000

      ---------------------------------------------------------------------------------------------------------
      {code}
      In same time I run strace for mysqld and see many calls to madvise:
      {code}

      1. strace -ff -p 33080
        ...
        [pid 60412] madvise(0x7f36cb638000, 331776, MADV_DONTNEED) = 0
        [pid 60412] madvise(0x7f36cb441000, 135168, MADV_DONTNEED) = 0
        [pid 60412] madvise(0x7f36cb638000, 335872, MADV_DONTNEED) = 0
        [pid 60412] madvise(0x7f36cb441000, 135168, MADV_DONTNEED) = 0
        [pid 60412] madvise(0x7f36cb638000, 335872, MADV_DONTNEED) = 0
        [pid 60412] madvise(0x7f36cb441000, 135168, MADV_DONTNEED) = 0
        [pid 60412] madvise(0x7f36cb638000, 335872, MADV_DONTNEED) = 0
        {code}

      Table which been loading:
      {code}
      /*!40101 SET @saved_cs_client = @@character_set_client */;
      /*!40101 SET character_set_client = utf8 */;
      CREATE TABLE `contact` (
      `id` int(10) unsigned NOT NULL AUTO_INCREMENT,
      `id_phonebook` int(10) unsigned NOT NULL DEFAULT '0',
      `id_user` int(10) unsigned NOT NULL DEFAULT '0',
      `firstname` varchar(255) NOT NULL DEFAULT '',
      `lastname` varchar(255) NOT NULL DEFAULT '',
      `local_key` varchar(255) NOT NULL DEFAULT '',
      PRIMARY KEY (`id`),
      KEY `id_phonebook` (`id_phonebook`) USING BTREE,
      KEY `id_user` (`id_user`) USING BTREE
      ) ENGINE=InnoDB AUTO_INCREMENT=133028725 DEFAULT CHARSET=utf8 ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=2;
      /*!40101 SET character_set_client = @saved_cs_client */;
      {code}

      Attachments

        1. vars.txt
          16 kB
        2. status.txt
          12 kB
        3. gdb.txt
          7 kB

        Issue Links

          Activity

            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            AirFocus AirFocus made changes -
            Description I load backup to mariadb and break it. In processlist still present one killed thread:
            {code}
            show processlist;
            +-----+--------+--------------------+-------------------+---------+------+------------------+------------------+----------+
            | Id | User | Host | db | Command | Time | State | Info | Progress |
            +-----+--------+--------------------+-------------------+---------+------+------------------+------------------+----------+
            | 85 | loader | 10.10.16.216:43971 | phonebook_storage | Killed | 1845 | Unlocking tables | NULL | 0.000 |
            | 184 | root | localhost | NULL | Query | 0 | init | show processlist | 0.000 |
            +-----+--------+--------------------+-------------------+---------+------+------------------+------------------+----------+
            {code}
            In same time I run strace for mysqld and see many calls to madvise:
            {code}
            # strace -ff -p 33080
            ...
            [pid 60412] madvise(0x7f36cb638000, 331776, MADV_DONTNEED) = 0
            [pid 60412] madvise(0x7f36cb441000, 135168, MADV_DONTNEED) = 0
            [pid 60412] madvise(0x7f36cb638000, 335872, MADV_DONTNEED) = 0
            [pid 60412] madvise(0x7f36cb441000, 135168, MADV_DONTNEED) = 0
            [pid 60412] madvise(0x7f36cb638000, 335872, MADV_DONTNEED) = 0
            [pid 60412] madvise(0x7f36cb441000, 135168, MADV_DONTNEED) = 0
            [pid 60412] madvise(0x7f36cb638000, 335872, MADV_DONTNEED) = 0
            {code}

            Table which been loading:
            {code}
            /*!40101 SET @saved_cs_client = @@character_set_client */;
            /*!40101 SET character_set_client = utf8 */;
            CREATE TABLE `contact` (
              `id` int(10) unsigned NOT NULL AUTO_INCREMENT,
              `id_phonebook` int(10) unsigned NOT NULL DEFAULT '0',
              `id_user` int(10) unsigned NOT NULL DEFAULT '0',
              `firstname` varchar(255) NOT NULL DEFAULT '',
              `lastname` varchar(255) NOT NULL DEFAULT '',
              `local_key` varchar(255) NOT NULL DEFAULT '',
              PRIMARY KEY (`id`),
              KEY `id_phonebook` (`id_phonebook`) USING BTREE,
              KEY `id_user` (`id_user`) USING BTREE
            ) ENGINE=InnoDB AUTO_INCREMENT=133028725 DEFAULT CHARSET=utf8 ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=2;
            /*!40101 SET character_set_client = @saved_cs_client */;
            {code}
            I load backup to mariadb and break it. In processlist still present one killed thread:
            \{code\}
            show processlist;
            +\-\-\-\--+\-\-\-\-\-\-\-\-+-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\--+\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-+-\-\-\-\-\-\-\--+\-\-\-\-\-\-+-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\--+\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-+-\-\-\-\-\-\-\-\-\-+

            | Id | User | Host | db | Command | Time | State | Info | Progress |

            +\-\-\-\--+\-\-\-\-\-\-\-\-+-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\--+\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-+-\-\-\-\-\-\-\--+\-\-\-\-\-\-+-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\--+\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-+-\-\-\-\-\-\-\-\-\-+

            | 85 | loader | 10.10.16.216:43971 | phonebook_storage | Killed | 1845 | Unlocking tables | NULL | 0.000 |
            | 184 | root | localhost | NULL | Query | 0 | init | show processlist | 0.000 |

            +\-\-\-\--+\-\-\-\-\-\-\-\-+-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\--+\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-+-\-\-\-\-\-\-\--+\-\-\-\-\-\-+-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\--+\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-+-\-\-\-\-\-\-\-\-\-+
            \{code\}
            In same time I run strace for mysqld and see many calls to madvise:
            \{code\}

            # strace -ff -p 33080
            ...
            \[pid 60412\] madvise(0x7f36cb638000, 331776, MADV\_DONTNEED) = 0
            \[pid 60412\] madvise(0x7f36cb441000, 135168, MADV\_DONTNEED) = 0
            \[pid 60412\] madvise(0x7f36cb638000, 335872, MADV\_DONTNEED) = 0
            \[pid 60412\] madvise(0x7f36cb441000, 135168, MADV\_DONTNEED) = 0
            \[pid 60412\] madvise(0x7f36cb638000, 335872, MADV\_DONTNEED) = 0
            \[pid 60412\] madvise(0x7f36cb441000, 135168, MADV\_DONTNEED) = 0
            \[pid 60412\] madvise(0x7f36cb638000, 335872, MADV\_DONTNEED) = 0
            \{code\}

            Table which been loading:
            \{code\}
            /*!40101 SET @saved_cs_client = @@character_set_client */;
            /*!40101 SET character_set_client = utf8 */;
            CREATE TABLE `contact` (
              `id` int(10) unsigned NOT NULL AUTO\_INCREMENT,
              `id\_phonebook` int(10) unsigned NOT NULL DEFAULT '0',
              `id\_user` int(10) unsigned NOT NULL DEFAULT '0',
              `firstname` varchar(255) NOT NULL DEFAULT '',
              `lastname` varchar(255) NOT NULL DEFAULT '',
              `local\_key` varchar(255) NOT NULL DEFAULT '',
              PRIMARY KEY (`id`),
              KEY `id_phonebook` (`id_phonebook`) USING BTREE,
              KEY `id_user` (`id_user`) USING BTREE
            ) ENGINE=InnoDB AUTO_INCREMENT=133028725 DEFAULT CHARSET=utf8 ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=2;
            /*!40101 SET character_set_client = @saved_cs_client */;
            \{code\}
            serg Sergei Golubchik made changes -
            Workflow MariaDB v3 [ 84341 ] MariaDB v4 [ 131800 ]
            marko Marko Mäkelä made changes -
            serg Sergei Golubchik made changes -
            Affects Version/s 10.2 [ 14601 ]
            Affects Version/s 5.5 [ 15800 ]
            Affects Version/s 10.0 [ 16000 ]
            Affects Version/s 10.1 [ 16100 ]
            Affects Version/s 10.3 [ 22126 ]
            Affects Version/s 10.4 [ 22408 ]
            Affects Version/s 10.1.29 [ 22636 ]
            Environment centos7 3.10.0-693.5.2.el7.x86_64
            Issue Type Bug [ 1 ] Task [ 3 ]
            marko Marko Mäkelä made changes -
            Fix Version/s 10.4 [ 22408 ]
            NRE Projects RM_long_term
            Affects Version/s 10.4 [ 22408 ]
            Labels memory performance
            Summary ROLLBACK of an INSERT into ROW_FORMAT=COMPRESSED table is very slow Reduce malloc()/free() usage in InnoDB
            marko Marko Mäkelä made changes -
            Fix Version/s 10.4 [ 22408 ]
            Fix Version/s 10.1 [ 16100 ]
            marko Marko Mäkelä made changes -
            Affects Version/s 5.5 [ 15800 ]
            Affects Version/s 10.0 [ 16000 ]
            Affects Version/s 10.1 [ 16100 ]
            Affects Version/s 10.2 [ 14601 ]
            Affects Version/s 10.3 [ 22126 ]
            marko Marko Mäkelä made changes -
            Status Open [ 1 ] Confirmed [ 10101 ]
            marko Marko Mäkelä made changes -
            Summary endless loop when rollback insert into compressed table ROLLBACK of an INSERT into ROW_FORMAT=COMPRESSED table is very slow
            marko Marko Mäkelä made changes -
            jplindst Jan Lindström (Inactive) made changes -
            Assignee Jan Lindström [ jplindst ] Marko Mäkelä [ marko ]
            elenst Elena Stepanova made changes -
            Fix Version/s 10.1 [ 16100 ]
            Assignee Jan Lindström [ jplindst ]
            chernomorets Sergey Chernomorets made changes -
            Attachment vars.txt [ 44751 ]
            Attachment status.txt [ 44752 ]
            chernomorets Sergey Chernomorets made changes -
            Field Original Value New Value
            Attachment gdb.txt [ 44750 ]
            chernomorets Sergey Chernomorets created issue -

            People

              marko Marko Mäkelä
              chernomorets Sergey Chernomorets
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.