Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-7259

Upgrade TokuDB from Tokutek.com binaries to MariaDB 10

Details

    Description

      After upgrading via a binary upgrade from Tokutek binaries to MariaDB 10.0.15, there are various headaches that make this transition problematic. I think at the very least, this is a minor documentation bug, but I think some of the issues might not be too crazy to fix.

      The first major issue is that Tokutek stores special values in the .frm row_type. When running SHOW CREATE TABLE against such tables MariaDB crashes.

      gdb shows this is caused by MariaDB accessing beyond the end of ha_row_type with an out of bounds create_info.row_type in:

      mariadb-10.0.15/sql/sql_show.cc:1970
      1970	      packet->append(ha_row_type[(uint) create_info.row_type]);

      This is also true if you ALTER a TokuDB table to MyISAM whose .frm inherits the old flags and continue to crash.

      A workaround for this issue is to run ALTER TABLE $table ROW_FORMAT=DEFAULT, as long as the tokudb_row_format variable is set appropriately. Selecting from the table otherwise seems to work correctly and gives expected results.

      The other issue is with TokuDB clustered indexes, where SHOW CREATE TABLE displays these as FULLTEXT. ALTER TABLE fails with:

      ERROR 1464 (HY000) The storage engine TokuDB doesn't support SPATIAL indexes

      In this case, I dropped and recreated the clustered indexes which resolved the issue. I imagine the actual data files aren't different, and if a correct .frm could be put into place even this major rebuild might not be necessary. I was hoping to hack a correct. frm into place, but table discovery makes that tricky.

      I think at the very least, perhaps some documentation could be added noting this is not a supported upgraded path and mysqldump + munging of the table DDL is necessary. I think perhaps there may be a bug here where MariaDB crashes with bad .frm values, where I would instead expect at least an ERR_NOT_FORM_FILE (1033) error. For comparison, Percona-Server 5.6+TokuDB gives this error with SHOW CREATE TABLE on a TokuDB table with clustered indexes from a Tokutek.com generated datadir.

      Of course, ideally if a direct binary upgrade could be supported that would be convenient, particularly in environment with a substantial existing dataset that wish to evaluate MariaDB 10 where rebuilding from logical dumps is a very time consuming process.

      Attachments

        Issue Links

          Activity

            Assigned to serg for consideration.

            elenst Elena Stepanova added a comment - Assigned to serg for consideration.

            The crash is fixed in MDEV-7333.

            serg Sergei Golubchik added a comment - The crash is fixed in MDEV-7333 .

            I think this is purely frm incompatibility, data are fully compatible. That is a workaround could be to create an empty table with the same structure, and then (shut the server down and) copy the new frm file over the one from the existing table. This should be much faster than reloading the table from a dump.

            I'll see whether this could be done automatically and internally in MariaDB.

            serg Sergei Golubchik added a comment - I think this is purely frm incompatibility, data are fully compatible. That is a workaround could be to create an empty table with the same structure, and then (shut the server down and) copy the new frm file over the one from the existing table. This should be much faster than reloading the table from a dump. I'll see whether this could be done automatically and internally in MariaDB.
            philsweeney Phil Sweeney added a comment -

            Just some feedback as I went through this upgrade path (Tokutek's MariaDB 5.5 -> Official MariaDB 10.0) with TokuDB tables. I did end up dumping (+modifying create table syntax) and reloading TokuDB tables after seeking some help on the tokudb google group (https://groups.google.com/d/msg/tokudb-user/GksdzAV8WZk/VWMwplGP5TgJ)

            Another thing I realised was that tables I had converted from TokuDB to MyISAM in the past still had the row_format variable in the frm and it was tripping up, so I had to dump and reload those too to get rid of the errors/warnings in the logs.

            I could have potentially just used your workaround for some of them by changing the row_format though! Some documentation around this would be really good - there'll be quite a few people in my position I think going forward.

            Thanks.

            philsweeney Phil Sweeney added a comment - Just some feedback as I went through this upgrade path (Tokutek's MariaDB 5.5 -> Official MariaDB 10.0) with TokuDB tables. I did end up dumping (+modifying create table syntax) and reloading TokuDB tables after seeking some help on the tokudb google group ( https://groups.google.com/d/msg/tokudb-user/GksdzAV8WZk/VWMwplGP5TgJ ) Another thing I realised was that tables I had converted from TokuDB to MyISAM in the past still had the row_format variable in the frm and it was tripping up, so I had to dump and reload those too to get rid of the errors/warnings in the logs. I could have potentially just used your workaround for some of them by changing the row_format though! Some documentation around this would be really good - there'll be quite a few people in my position I think going forward. Thanks.

            People

              serg Sergei Golubchik
              andrew.garner Andrew Garner
              Votes:
              2 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.