Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-8541

TokuDB file map table doesn't reflect the real situation of the files on the disk

Details

    Description

      Sometimes TokuDB tables get corrupted, the server crash and will not start anymore. The only way I found to have the server running, is to delete all the .tokudb, .frm and .par files related to the table that caused the crash. The problem is that the information_schema.TokuDB_file_map table still shows the records related to the table, and any subsequent attempt to recreate the table produces the error: 17 "File exists". As reported in MDEV-7841, I "created a table with the same structure (and partitions), renamed the files to reflect the files in the dictionary, then dropped the old table again". Files are deleted from disk, but the information_schema.TokuDB_file_map table still shows the records. So I can't recreate the table with the same name. Please, can you suggest me a workaround to the problem? Is there a way to synchronize the file map with the real situation of existing files?

      Attachments

        Activity

          Soltis Kai added a comment -

          Apparently, this is resolved in an upcoming version. Fixed in https://tokutek.atlassian.net/browse/FT-733

          Soltis Kai added a comment - Apparently, this is resolved in an upcoming version. Fixed in https://tokutek.atlassian.net/browse/FT-733
          Soltis Kai added a comment -

          For whatever reason, the solution for this bug is being removed from the upstream TokuDB code – see https://github.com/MariaDB/server/commit/dfeff40706
          If MariaDB maintainers are going to neuter the upstream solution, they need to provide their own fix.

          Soltis Kai added a comment - For whatever reason, the solution for this bug is being removed from the upstream TokuDB code – see https://github.com/MariaDB/server/commit/dfeff40706 If MariaDB maintainers are going to neuter the upstream solution, they need to provide their own fix.

          We might merge the upstream solution later, when it gets more mature. It's currently marked Experimental, and TokuDB storage engine in MariaDB is marked stable.

          serg Sergei Golubchik added a comment - We might merge the upstream solution later, when it gets more mature. It's currently marked Experimental , and TokuDB storage engine in MariaDB is marked stable .
          Soltis Kai added a comment -

          That commit message is pretty misleading, but your explanation makes more sense. It's definitely a mandatory tool when dealing with certain recovery cases - or, as in my case, when dealing with a table that was corrupted by an earlier bug in TokuDB that has since been fixed.

          Soltis Kai added a comment - That commit message is pretty misleading, but your explanation makes more sense. It's definitely a mandatory tool when dealing with certain recovery cases - or, as in my case, when dealing with a table that was corrupted by an earlier bug in TokuDB that has since been fixed.

          Having a similar situation, where I can't drop and recreate a table with a corrupted partition, with error "File exists", I report my workaround to resolve the problem (MariaDB 10.2.21):
          -create an identical table in another database (tmp), with the same partitions as the original table;
          -here the trick, copy the new files over the old files (if exists), looking at their names in the table information_schema.TokuDB_file_map, but using old file names. To do this, adapt this query to reflect your paths, db and table names, and run it:

          SELECT CONCAT('cp /media/data/data/', MID(new.internal_file_name,3), ' /media/data/data/four/', MID(old.internal_file_name,10)) command
          FROM information_schema.TokuDB_file_map old
          JOIN information_schema.TokuDB_file_map new ON old.table_name=new.table_name AND old.table_dictionary_name=new.table_dictionary_name
          WHERE old.table_schema='dbname' AND new.table_schema='tmp' AND old.table_name LIKE 'tablename#P%'

          -rows generated are similar to this one:
          cp /media/data/data/tmp/four_richiesta_P_p235_key_COD_FISCALE_CL_1f6c1337_6_1d.tokudb /media/data/data/four/four_richiesta_P_p235_key_COD_FISCALE_CL_18e4b633_6_1d.tokudb
          -export rows to a text editor and save as script
          -run the script (with sudo)
          -if necessary change permissions to the created files (sudo chown mysql:mysql /media/data/data/dbname/tablename_P*)
          -drop the table in sql (should't return errors this time)
          -recreate the table with partitions in original database

          steris Risato Stefano added a comment - Having a similar situation, where I can't drop and recreate a table with a corrupted partition, with error "File exists", I report my workaround to resolve the problem (MariaDB 10.2.21): -create an identical table in another database (tmp), with the same partitions as the original table; -here the trick, copy the new files over the old files (if exists), looking at their names in the table information_schema.TokuDB_file_map, but using old file names. To do this, adapt this query to reflect your paths, db and table names, and run it: SELECT CONCAT('cp /media/data/data/', MID(new.internal_file_name,3), ' /media/data/data/four/', MID(old.internal_file_name,10)) command FROM information_schema.TokuDB_file_map old JOIN information_schema.TokuDB_file_map new ON old.table_name=new.table_name AND old.table_dictionary_name=new.table_dictionary_name WHERE old.table_schema='dbname' AND new.table_schema='tmp' AND old.table_name LIKE 'tablename#P%' -rows generated are similar to this one: cp /media/data/data/tmp/four_richiesta_P_p235_key_COD_FISCALE_CL_1f6c1337_6_1d.tokudb /media/data/data/four/four_richiesta_P_p235_key_COD_FISCALE_CL_18e4b633_6_1d.tokudb -export rows to a text editor and save as script -run the script (with sudo) -if necessary change permissions to the created files (sudo chown mysql:mysql /media/data/data/dbname/tablename_P*) -drop the table in sql (should't return errors this time) -recreate the table with partitions in original database

          People

            Unassigned Unassigned
            steris Risato Stefano
            Votes:
            3 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.