Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-26740

Inplace alter rebuild increases file size

Details

    Description

      After inplace alter table, InnoDB increases the file size. InnoDB does the normal insert
      and file size is 32 mb after flushing all the pages. Inplace Alter table rebuilds the
      table and increases the file size(36MB) by 4MB.

      --source include/have_innodb.inc
      --source include/have_sequence.inc
      create table t1(f1 int not null primary key, b char(255) CHARACTER SET utf8)engine=innodb;
      INSERT INTO t1(f1) SELECT * FROM seq_1_to_1000000;
      --source include/restart_mysqld.inc # Here t1 size is 32 MB
      alter table t1 force, algorithm=inplace; 
      --source include/restart_mysqld.inc# Here t1 size is 36 MB
      drop table t1;
      

      Attachments

        Issue Links

          Activity

            thiru Thirunarayanan Balathandayuthapani created issue -
            thiru Thirunarayanan Balathandayuthapani made changes -
            Field Original Value New Value
            marko Marko Mäkelä made changes -
            Fix Version/s 10.2 [ 14601 ]
            Fix Version/s 10.3 [ 22126 ]
            Fix Version/s 10.4 [ 22408 ]
            Fix Version/s 10.5 [ 23123 ]
            Fix Version/s 10.6 [ 24028 ]
            marko Marko Mäkelä made changes -
            Status Open [ 1 ] Confirmed [ 10101 ]
            serg Sergei Golubchik made changes -
            Workflow MariaDB v3 [ 125812 ] MariaDB v4 [ 144415 ]

            I believe that the size difference must be related to the B-tree index page fill factor, or to the way how pages are allocated (MDEV-13013, MDEV-20094).

            I remember that there was some discussion of page fill factor during the implementation of the MySQL 5.7 bulk insert. I cannot remember if it was about intentional or accidental changes to the fill factor. Based on my findings in MDEV-19747, I would expect that the performance impact was not thoroughly tested.

            The page fill factor can affect the performance of subsequent operations, for the better or worse.

            • If there will be no changes to the index after the index was created or the table was rebuilt, a lower fill factor will unnecessarily increase the storage footprint and increase the I/O cost.
            • If there will be certain types of changes, a lower fill factor may significantly reduce, or maybe even completely avoid page splits.
            marko Marko Mäkelä added a comment - I believe that the size difference must be related to the B-tree index page fill factor, or to the way how pages are allocated ( MDEV-13013 , MDEV-20094 ). I remember that there was some discussion of page fill factor during the implementation of the MySQL 5.7 bulk insert. I cannot remember if it was about intentional or accidental changes to the fill factor. Based on my findings in MDEV-19747 , I would expect that the performance impact was not thoroughly tested. The page fill factor can affect the performance of subsequent operations, for the better or worse. If there will be no changes to the index after the index was created or the table was rebuilt, a lower fill factor will unnecessarily increase the storage footprint and increase the I/O cost. If there will be certain types of changes, a lower fill factor may significantly reduce, or maybe even completely avoid page splits.
            marko Marko Mäkelä made changes -
            ralf.gebhardt Ralf Gebhardt made changes -
            Fix Version/s 10.2 [ 14601 ]

            It turns out that MDEV-24621 did not enable the bulk load mechanism for ALTER TABLE…ALGORITHM=COPY:

            --source include/have_innodb.inc
            --source include/have_sequence.inc
             
            let $datadir=`SELECT @@datadir`;
             
            create table t1(f1 int not null primary key, b char(255) CHARACTER SET utf8)engine=innodb;
            INSERT INTO t1(f1) SELECT * FROM seq_1_to_100000;
            --exec wc -c $datadir/test/t1.ibd
            alter table t1 force, algorithm=copy;
            --exec wc -c $datadir/test/t1.ibd
            alter table t1 force, algorithm=inplace;
            --exec wc -c $datadir/test/t1.ibd
            drop table t1;
            

            This will report the file size 10,485,760 bytes (10 MiB) for the first two steps, and 9,437,184 bytes (9 MiB) for the last step:

            10.7 1e04cafcba88e1801e828a5bbab7fe9fdd7ca61c

            create table t1(f1 int not null primary key, b char(255) CHARACTER SET utf8)engine=innodb;
            INSERT INTO t1(f1) SELECT * FROM seq_1_to_100000;
            10485760 /dev/shm/10.7/mysql-test/var/mysqld.1/data//test/t1.ibd
            alter table t1 force, algorithm=copy;
            10485760 /dev/shm/10.7/mysql-test/var/mysqld.1/data//test/t1.ibd
            alter table t1 force, algorithm=inplace;
            9437184 /dev/shm/10.7/mysql-test/var/mysqld.1/data//test/t1.ibd
            drop table t1;
            

            If we insert 1,000,000 rows instead of 100,000, then the reported file sizes will be 32 MiB and 36 MiB (larger with the bulk load enabled).

            marko Marko Mäkelä added a comment - It turns out that MDEV-24621 did not enable the bulk load mechanism for ALTER TABLE…ALGORITHM=COPY : --source include/have_innodb.inc --source include/have_sequence.inc   let $datadir=` SELECT @@datadir`;   create table t1(f1 int not null primary key , b char (255) CHARACTER SET utf8)engine=innodb; INSERT INTO t1(f1) SELECT * FROM seq_1_to_100000; --exec wc -c $datadir/test/t1.ibd alter table t1 force , algorithm=copy; --exec wc -c $datadir/test/t1.ibd alter table t1 force , algorithm=inplace; --exec wc -c $datadir/test/t1.ibd drop table t1; This will report the file size 10,485,760 bytes (10 MiB) for the first two steps, and 9,437,184 bytes (9 MiB) for the last step: 10.7 1e04cafcba88e1801e828a5bbab7fe9fdd7ca61c create table t1(f1 int not null primary key, b char(255) CHARACTER SET utf8)engine=innodb; INSERT INTO t1(f1) SELECT * FROM seq_1_to_100000; 10485760 /dev/shm/10.7/mysql-test/var/mysqld.1/data//test/t1.ibd alter table t1 force, algorithm=copy; 10485760 /dev/shm/10.7/mysql-test/var/mysqld.1/data//test/t1.ibd alter table t1 force, algorithm=inplace; 9437184 /dev/shm/10.7/mysql-test/var/mysqld.1/data//test/t1.ibd drop table t1; If we insert 1,000,000 rows instead of 100,000, then the reported file sizes will be 32 MiB and 36 MiB (larger with the bulk load enabled).
            marko Marko Mäkelä made changes -
            julien.fritsch Julien Fritsch made changes -
            Fix Version/s 10.3 [ 22126 ]
            thiru Thirunarayanan Balathandayuthapani made changes -
            Status Confirmed [ 10101 ] In Progress [ 3 ]
            thiru Thirunarayanan Balathandayuthapani made changes -
            Assignee Thirunarayanan Balathandayuthapani [ thiru ] Marko Mäkelä [ marko ]
            Status In Progress [ 3 ] In Review [ 10002 ]

            Thank you, this is a very simple fix. I approved the 10.6 version, but I think that this should be applied to 10.4 already. As far as I understand, this regression was introduced in 10.2 by importing the MySQL 5.7 bulk index creation changes.

            marko Marko Mäkelä added a comment - Thank you, this is a very simple fix. I approved the 10.6 version, but I think that this should be applied to 10.4 already. As far as I understand, this regression was introduced in 10.2 by importing the MySQL 5.7 bulk index creation changes.
            marko Marko Mäkelä made changes -
            Assignee Marko Mäkelä [ marko ] Thirunarayanan Balathandayuthapani [ thiru ]
            Status In Review [ 10002 ] Stalled [ 10000 ]
            marko Marko Mäkelä made changes -
            JIraAutomate JiraAutomate made changes -
            Priority Major [ 3 ] Critical [ 2 ]
            thiru Thirunarayanan Balathandayuthapani made changes -
            Status Stalled [ 10000 ] In Testing [ 10301 ]
            thiru Thirunarayanan Balathandayuthapani made changes -
            Assignee Thirunarayanan Balathandayuthapani [ thiru ] Matthias Leich [ mleich ]

            origin/10.6-MDEV-26740 2005dedbcab3cf0550e5870ea80ea680875900b9 2024-01-05T13:11:38+05:30
            behaved well in RQG testing. No new problems.

            mleich Matthias Leich added a comment - origin/10.6- MDEV-26740 2005dedbcab3cf0550e5870ea80ea680875900b9 2024-01-05T13:11:38+05:30 behaved well in RQG testing. No new problems.
            mleich Matthias Leich made changes -
            Assignee Matthias Leich [ mleich ] Thirunarayanan Balathandayuthapani [ thiru ]
            Status In Testing [ 10301 ] Stalled [ 10000 ]
            thiru Thirunarayanan Balathandayuthapani made changes -
            Fix Version/s 10.4.33 [ 29516 ]
            Fix Version/s 10.4 [ 22408 ]
            Fix Version/s 10.5 [ 23123 ]
            Fix Version/s 10.6 [ 24028 ]
            Resolution Fixed [ 1 ]
            Status Stalled [ 10000 ] Closed [ 6 ]
            JIraAutomate JiraAutomate made changes -
            Fix Version/s 10.5.24 [ 29517 ]
            Fix Version/s 10.6.17 [ 29518 ]
            Fix Version/s 10.11.7 [ 29519 ]
            Fix Version/s 11.0.5 [ 29520 ]
            Fix Version/s 11.1.4 [ 29024 ]
            Fix Version/s 11.2.3 [ 29521 ]
            marko Marko Mäkelä made changes -

            People

              thiru Thirunarayanan Balathandayuthapani
              thiru Thirunarayanan Balathandayuthapani
              Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.