Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-10701

SET global TOKUDB_CHECKPOINT_LOCK=ON will not stop the tokudb data file change

Details

    Description

      How to reproduce the bug:

      Open a new shell "shell1" , and use sysbench to do the modification on the tokudb table, this sysbench
      will running through the whole test:

      sysbench --mysql-host=127.0.0.1 --mysql-port=3314 --mysql-user=test --mysql-password=test \
      --mysql-db=test1 --oltp-tables-count=1 --oltp-table-size=6000000 \
      --mysql-table-engine=tokudb --test=tests/db/oltp.lua prepare
      

      open a new shell "shell2" , to check the data file and log file modification of tokudb table in the disk:

      cd $datadir
      [root@mysql1 data3314]# ls -lh | grep test1
      drwx------. 2 mysql mysql 4.0K Aug 30 04:05 test1
      -rw-rw----. 1 mysql mysql 9.0M Aug 30 04:07 _test1_sbtest1_key_k_1_8c02_1_1c_B_1.tokudb
      -rw-rw----. 1 mysql mysql  85M Aug 30 04:07 _test1_sbtest1_main_8c02_1_1c_B_0.tokudb
      -rw-rw----. 1 mysql mysql  64K Aug 30 04:07 _test1_sbtest1_status_8bf7_1_1c.tokudb
      

      we can see the log file and data file is changing.

      Then open a new shell "shell3" login to the mariadb and acquire the checkpoint lock:

      root@localhost:mysql3314.sock  04:07:33 [(none)]>SET TOKUDB_CHECKPOINT_LOCK=ON;
      Query OK, 0 rows affected (0.32 sec)
      root@localhost:mysql3314.sock  04:07:36 [(none)]>SET global TOKUDB_CHECKPOINT_LOCK=ON;
      Query OK, 0 rows affected (0.04 sec)
      

      And during the test, this connection will not quit or broken.

      Back to the shell2, we still can see the data file changing!

      It seams that, the "TOKUDB_CHECKPOINT_LOCK=ON" fail to stop the data file modification on tokudb.

      Attachments

        Activity

          The upsteam bug was invalidated with the following comment:

          TokuDB checkpoint lock does not stop data files from changing, it only prevents the checkpoint from occurring. When a dirty node is evicted from memory due to cache pressure, it gets written to disk, The block file map header does not get updated on disk as it does during a checkpoint.

          There is one condition where a block file header will get updated even with the checkpointer disabled and that is when an index/dictionary file is closed and evicted from the cache table. This will be addressed in coming versions of Percona Server when the LOCK CHECKPOINT FOR BACKUP is introduced and tokudb_checkpoint_lock removed.

          Closing this replica accordingly.
          When the change described in the second paragraph is implemented in TokuDB, it will be merged into MariaDB.

          elenst Elena Stepanova added a comment - The upsteam bug was invalidated with the following comment: TokuDB checkpoint lock does not stop data files from changing, it only prevents the checkpoint from occurring. When a dirty node is evicted from memory due to cache pressure, it gets written to disk, The block file map header does not get updated on disk as it does during a checkpoint. There is one condition where a block file header will get updated even with the checkpointer disabled and that is when an index/dictionary file is closed and evicted from the cache table. This will be addressed in coming versions of Percona Server when the LOCK CHECKPOINT FOR BACKUP is introduced and tokudb_checkpoint_lock removed. Closing this replica accordingly. When the change described in the second paragraph is implemented in TokuDB, it will be merged into MariaDB.
          BohuTANG bohutang added a comment -

          We can stop the block header updated when setting TOKUDB_CHECKPOINT_LOCK=ON with this patch:
          https://github.com/BohuTANG/ft-index/commit/e48ac39c6644114c674f600dbdc4b34788e47091

          With it, now we can backup both InnoDB and TokuDB on PS5.6/PS5.7 and MariaDB.
          Patch is:
          https://github.com/BohuTANG/percona-xtrabackup/commit/b43ba1b25deff2ba21eb9030b49413e1851f263b

          BohuTANG bohutang added a comment - We can stop the block header updated when setting TOKUDB_CHECKPOINT_LOCK=ON with this patch: https://github.com/BohuTANG/ft-index/commit/e48ac39c6644114c674f600dbdc4b34788e47091 With it, now we can backup both InnoDB and TokuDB on PS5.6/PS5.7 and MariaDB. Patch is: https://github.com/BohuTANG/percona-xtrabackup/commit/b43ba1b25deff2ba21eb9030b49413e1851f263b

          People

            Unassigned Unassigned
            gao1738 dennis
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.