Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-9581

[PATCH] Innodb in Mariadb-10.1.x is unable to start up with data files generated by older version

Details

    • Bug
    • Status: Closed (View Workflow)
    • Critical
    • Resolution: Fixed
    • 10.1(EOL)
    • 10.1.21
    • 10.1.13

    Description

      The procedure to reproduce the issue as below, and analysis follows.

      1. Build and start up mariadb-10.0.10 to create a table with innodb storage engine, and insert some rows into it. Note that in my.cnf, set innodb-page-size to 4k.

      2. shutdown mysqld and keep the data directory intact.
      3. build and start up maraidb-10.1.9(the version I am using) and start up mysqld, using exactly the same my.cnf file and data directory as above.

      4. mysqld fails to startup with below error in mysqld.err:

      InnoDB: Error: Current page size 4096 !=  page size on page 16384
      2016-02-16 17:20:39 140521425086336 [ERROR] InnoDB: innodb-page-size mismatch in data file /my/data/dir/ibdata1
      2016-02-16 17:20:39 140521425086336 [ERROR] InnoDB: Could not open or create the system tablespace. If you tried to add new data files to the system tablespace, and it failed here, you should now edit innodb_data_file_path in my.cnf     back to what it was, and remove the new ibdata files InnoDB created in this failed attempt. InnoDB only wrote those files full of zeros, but did not yet use them in any way. But be careful: do not remove old data files which contai    n your precious data!
      2016-02-16 17:20:39 140521425086336 [ERROR] Plugin 'InnoDB' init function returned error.
      2016-02-16 17:20:39 140521425086336 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
      2016-02-16 17:20:39 140521425086336 [Note] Plugin 'FEEDBACK' is disabled.
      2016-02-16 17:20:39 140521425086336 [ERROR] Unknown/unsupported storage engine: innodb
      2016-02-16 17:20:39 140521425086336 [ERROR] Aborting
      

      The cause is that the newly added tablespace flags is wrongly placed: the PAGE_COMPRESSION, PAGE_COMPRESSION_LEVEL and ATOMIC_WRITES flags added in 10.1.x occupies the position of PAGE_SIZE, pushing PAGE_SIZE behind, so when new version mysqld reads the tablespace flags of old data file, the PAGE_SIZE is read as 0, so innodb thinks the default 16k is used, but UNIV_PAGE_SIZE is 4k as specified by my.cnf, hence the error.

      I have made a fix, please review. The patch passes all tests in mysql-test.

      I saw in latest release (mariadb-10.1.11) the relevant code keeps the same so the issue most probably exist in it too.

      Attachments

        Issue Links

          Activity

            DZW David Zhao added a comment -

            Thanks for the quick reply. My further questions:

            1. Is it possible for me to see your unreleased fix? Is it in some development branch somewhere? or could you please send me a patch of it?
            2. I want it because I want to know: whether the PAGE_COMPRESSION,PAGE_COMPRESSION_LEVEL, and ATOMIC_WRITES tablespace flags in future mariadb release will be placed the same way as in my patched version? I hope they will be the same otherwise I would face data format inconsistency when I have to upgrade to some future mariadb version. So if I can't see your code yet, could you describe the format of the tablespace flag integer here?

            and btw, in my above patch I did fix both innodb and xtradb, actually I am using innodb rather than xtradb.

            DZW David Zhao added a comment - Thanks for the quick reply. My further questions: 1. Is it possible for me to see your unreleased fix? Is it in some development branch somewhere? or could you please send me a patch of it? 2. I want it because I want to know: whether the PAGE_COMPRESSION,PAGE_COMPRESSION_LEVEL, and ATOMIC_WRITES tablespace flags in future mariadb release will be placed the same way as in my patched version? I hope they will be the same otherwise I would face data format inconsistency when I have to upgrade to some future mariadb version. So if I can't see your code yet, could you describe the format of the tablespace flag integer here? and btw, in my above patch I did fix both innodb and xtradb, actually I am using innodb rather than xtradb.

            Hi,

            (1) I have not yet committed the patch anywhere as it does not yet work fully.

            (2) There is following problem: We (MariaDB) and Oracle have done independent development, this means that we and Oracle have used same tablespace flags bits (fsp->flags) and dictionary flags bits (sys_tables.flags). This will become even worse with MySQL 5.7. Thus, I wanted solution that works for 10.1 as well as our 10.2 under development (I'm merging 5.7 InnoDB). My solution is simple, I remove our extended flags PAGE_COMPRESSION,PAGE_COMPRESSION_LEVEL, and ATOMIC_WRITES from both fsp flags and dictionary flags and use new system table SYS_TABLE_OPTIONS instead where this information is stored. This means your patch is not compatible with my solution. I'm in process adding a methods that will migrate existing 10.1 databases using those flags to our new solution.

            jplindst Jan Lindström (Inactive) added a comment - Hi, (1) I have not yet committed the patch anywhere as it does not yet work fully. (2) There is following problem: We (MariaDB) and Oracle have done independent development, this means that we and Oracle have used same tablespace flags bits (fsp->flags) and dictionary flags bits (sys_tables.flags). This will become even worse with MySQL 5.7. Thus, I wanted solution that works for 10.1 as well as our 10.2 under development (I'm merging 5.7 InnoDB). My solution is simple, I remove our extended flags PAGE_COMPRESSION,PAGE_COMPRESSION_LEVEL, and ATOMIC_WRITES from both fsp flags and dictionary flags and use new system table SYS_TABLE_OPTIONS instead where this information is stored. This means your patch is not compatible with my solution. I'm in process adding a methods that will migrate existing 10.1 databases using those flags to our new solution.
            mysolo Thierry Laurier added a comment - - edited

            Created with 10.0.25 debian version, upgraded with 10.1.14 of mariadb.org for debian jessie.

            InnoDB: Error: Current page size 8192 !=  page size on page 16384
            2016-06-28 11:31:57 140588507318208 [Note] InnoDB: Restoring page 0 of tablespace 0
            2016-06-28 11:31:57 140588507318208 [Warning] InnoDB: Doublewrite does not have page_no=0 of space: 0
            2016-06-28 11:31:57 140588507318208 [ERROR] InnoDB: innodb-page-size mismatch in data file ./ibdata1
            2016-06-28 11:31:57 140588507318208 [ERROR] InnoDB: Could not open or create the system tablespace. If you tried to add new data files to the system tablespace, and it failed here, you should now edit innodb_data_file_path in my.cnf back to what it was, and remove the new ibdata files InnoDB created in this failed attempt. InnoDB only wrote those files full of zeros, but did not yet use them in any way. But be careful: do not remove old data files which contain your precious data!
            2016-06-28 11:31:57 140588507318208 [ERROR] Plugin 'InnoDB' init function returned error.
            2016-06-28 11:31:57 140588507318208 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
            2016-06-28 11:31:57 140588507318208 [Note] Plugin 'FEEDBACK' is disabled.
            2016-06-28 11:31:57 140588507318208 [ERROR] Unknown/unsupported storage engine: InnoDB
            2016-06-28 11:31:57 140588507318208 [ERROR] Aborting
            
            

            mysolo Thierry Laurier added a comment - - edited Created with 10.0.25 debian version, upgraded with 10.1.14 of mariadb.org for debian jessie. InnoDB: Error: Current page size 8192 != page size on page 16384 2016 - 06 - 28 11 : 31 : 57 140588507318208 [Note] InnoDB: Restoring page 0 of tablespace 0 2016 - 06 - 28 11 : 31 : 57 140588507318208 [Warning] InnoDB: Doublewrite does not have page_no= 0 of space: 0 2016 - 06 - 28 11 : 31 : 57 140588507318208 [ERROR] InnoDB: innodb-page-size mismatch in data file ./ibdata1 2016 - 06 - 28 11 : 31 : 57 140588507318208 [ERROR] InnoDB: Could not open or create the system tablespace. If you tried to add new data files to the system tablespace, and it failed here, you should now edit innodb_data_file_path in my.cnf back to what it was, and remove the new ibdata files InnoDB created in this failed attempt. InnoDB only wrote those files full of zeros, but did not yet use them in any way. But be careful: do not remove old data files which contain your precious data! 2016 - 06 - 28 11 : 31 : 57 140588507318208 [ERROR] Plugin 'InnoDB' init function returned error. 2016 - 06 - 28 11 : 31 : 57 140588507318208 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed. 2016 - 06 - 28 11 : 31 : 57 140588507318208 [Note] Plugin 'FEEDBACK' is disabled. 2016 - 06 - 28 11 : 31 : 57 140588507318208 [ERROR] Unknown/unsupported storage engine: InnoDB 2016 - 06 - 28 11 : 31 : 57 140588507318208 [ERROR] Aborting

            Hi, can you provide your database, I will try to investigate ?

            jplindst Jan Lindström (Inactive) added a comment - Hi, can you provide your database, I will try to investigate ?

            This issue has been fixed on https://jira.mariadb.org/browse/MDEV-11623, thus closing this.

            jplindst Jan Lindström (Inactive) added a comment - This issue has been fixed on https://jira.mariadb.org/browse/MDEV-11623 , thus closing this.

            People

              ratzpo Rasmus Johansson (Inactive)
              DZW David Zhao
              Votes:
              2 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.