Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-26552

MyISAM index corruption on 400M rows

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Not a Bug
    • 10.5.11, 10.5.12
    • N/A
    • N/A
    • debian unstable

    Description

      Hi

      I have table corruption caught by check table.

      I narrowed my ~400M rows table to a single integer column. The error happens after index creation if you run check table.

      I managed to recreate it with a random set of 70K values distributed in 400M rows.

      See the attached script. It completes in 32' in my system (oldish xeon with NVM) and it fails
      with

      --------------
      		 
      CHECK TABLE bugtable
      		 
      --------------
      		 
       
      		 
      +---------------+-------+-------+------------------------------------------+
      		 
      | budb.bugtable | check | error | Key in wrong position at page 1023494144 |
      		 
      | budb.bugtable | check | error | Corrupt                                  |
      		 
      +---------------+-------+-------+------------------------------------------+
      		 
      2 rows in set (38.701 sec)

      The script is easily run with:

      time ./mysql-index-corruption-bug.sh bugdb bugtable;

      Attachments

        Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. MDEV-16461 MyISAM creates defect indexes on varchar, if rowcount is above some threshold between 480,000,000 and 490,000,000

          • Major - Major loss of function.
          • Confirmed

          Activity

            Ascending order - Click to sort in descending order
            vasvir Vassilis Virvilis created issue -
            alice Alice Sherepa made changes -
            Field Original Value New Value
            vasvir Vassilis Virvilis added a comment -

            I think I found another manifestation of this bug.

            GROUP BY fails to produce unique output (produces duplicate rows) in filesort mode.

            I theorize that this is because when in filesort mode it first creates a sort index and that part fails.

            Further experimentation showed that the bug manifests when the number of rows are more than 390136719 (lower than 390039063 works)

            Finally the bug does not exists in 10.6.4

            How to replicate this one: check the replicator script ./mariadb-groupby-fails-bug.sh

            You can run it with
            ./mariadb-groupby-fails-bug.sh database bugtable

            I would be great to know
            a) the commit that fixed the issue
            b) if a test that could guard against this could be devised?

            Vassilis

            vasvir Vassilis Virvilis added a comment - I think I found another manifestation of this bug. GROUP BY fails to produce unique output (produces duplicate rows) in filesort mode. I theorize that this is because when in filesort mode it first creates a sort index and that part fails. Further experimentation showed that the bug manifests when the number of rows are more than 390136719 (lower than 390039063 works) Finally the bug does not exists in 10.6.4 How to replicate this one: check the replicator script ./mariadb-groupby-fails-bug.sh You can run it with ./mariadb-groupby-fails-bug.sh database bugtable I would be great to know a) the commit that fixed the issue b) if a test that could guard against this could be devised? Vassilis
            vasvir Vassilis Virvilis made changes -
            Attachment mariadb-groupby-fails-bug.sh [ 60920 ]
            vasvir Vassilis Virvilis added a comment -

            First of all - my bad

            It was a configuration error after all.

            I upgraded to 10.6.4 and the bug persists. Since I had other machines with 10.6.4 where the bug doesn't manifest I thought it must be an error in configuration.

            Indeed I checked again and lo and behold I spot this

            sort-buffer-size = 8G

            It was supposed to be 8M.

            So I switched it to 8M and everything works now.

            Maybe this situation warrants a warning...

            Sorry for the noise.

            Vassilis

            vasvir Vassilis Virvilis added a comment - First of all - my bad It was a configuration error after all. I upgraded to 10.6.4 and the bug persists. Since I had other machines with 10.6.4 where the bug doesn't manifest I thought it must be an error in configuration. Indeed I checked again and lo and behold I spot this sort-buffer-size = 8G It was supposed to be 8M. So I switched it to 8M and everything works now. Maybe this situation warrants a warning... Sorry for the noise. Vassilis
            alice Alice Sherepa added a comment -

            Great - I'm glad to hear this!

            alice Alice Sherepa added a comment - Great - I'm glad to hear this!
            alice Alice Sherepa made changes -
            Component/s N/A [ 14411 ]
            Fix Version/s N/A [ 14700 ]
            Resolution Not a Bug [ 6 ]
            Status Open [ 1 ] Closed [ 6 ]
            serg Sergei Golubchik made changes -
            Workflow MariaDB v3 [ 124844 ] MariaDB v4 [ 159664 ]

            People

              Unassigned Unassigned
              vasvir Vassilis Virvilis
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.