Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-18370

InnoDB: Failing assertion: lsn % OS_FILE_LOG_BLOCK_SIZE == LOG_BLOCK_HDR_SIZE in log0log.cc with innodb_scrub_log=ON and high values of innodb_scrub_log_speed

Details

    Description

      10.2 dc91372d

      		 
      2019-08-17 20:43:39 0x7fed6c9f6700  InnoDB: Assertion failure in file /data/src/10.2/storage/innobase/log/log0log.cc line 2238
      		 
      InnoDB: Failing assertion: lsn % OS_FILE_LOG_BLOCK_SIZE == LOG_BLOCK_HDR_SIZE
      		 
       
      		 
      #5  0x00007fed6d93742a in __GI_abort () at abort.c:89
      		 
      #6  0x000055d1c6fc5752 in ut_dbg_assertion_failed (expr=0x55d1c7501290 "lsn % OS_FILE_LOG_BLOCK_SIZE == LOG_BLOCK_HDR_SIZE", file=0x55d1c7500470 "/data/src/10.2/storage/innobase/log/log0log.cc", line=2238) at /data/src/10.2/storage/innobase/ut/ut0dbg.cc:60
      		 
      #7  0x000055d1c6e4d818 in log_pad_current_log_block () at /data/src/10.2/storage/innobase/log/log0log.cc:2238
      		 
      #8  0x000055d1c6e4d883 in log_scrub () at /data/src/10.2/storage/innobase/log/log0log.cc:2254
      		 
      #9  0x000055d1c6e4d934 in log_scrub_thread () at /data/src/10.2/storage/innobase/log/log0log.cc:2281
      		 
      #10 0x00007fed6f4a34a4 in start_thread (arg=0x7fed6c9f6700) at pthread_create.c:456
      		 
      #11 0x00007fed6d9ebd0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97

      To reproduce, bootstrap a clean datadir with all defaults and start the server with

      --innodb_scrub_log=ON --innodb_scrub_log_speed=50000

      Do it in a loop if it doesn't fail right away.
      It happens pretty much every time for me on my local HDD and SSD disks, sporadically on an (apparently faster or otherwise different) SSD disk on a lab machine, and almost never in shm. So, don't run it in shm.

      Happens both on debug and non-debug builds.

      Couldn't reproduce on 10.1 and 10.5.

      Attachments

        Issue Links

          is blocked by

          Task - A task that needs to be done. MDEV-21870 Deprecate and ignore innodb_scrub_log and innodb_scrub_log_speed

          • Major - Major loss of function.
          • Closed
          relates to

          Task - A task that needs to be done. MDEV-14425 Change the InnoDB redo log format to reduce write amplification

          • Blocker - Blocks development and/or testing work, production could not run.
          • Closed

          Task - A task that needs to be done. MDEV-14425 Change the InnoDB redo log format to reduce write amplification

          • Blocker - Blocks development and/or testing work, production could not run.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-18038 Assertion failure in innodb.undo_truncate_recover: "pad_len >= len || i * 512U >= len - pad_len || log_block_get_hdr_no( buf + i * 512U) == log_block_get_hdr_no(buf) + i"

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-20474 Assertion `!recv_no_log_write' failed in log_pad_current_log_block upon server startup on a clean datadir

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-20475 Assertion `flushed_lsn == log_get_lsn()' failed in srv_prepare_to_delete_redo_log_files upon server startup

          • Major - Major loss of function.
          • Closed

          Task - A task that needs to be done. MDEV-21870 Deprecate and ignore innodb_scrub_log and innodb_scrub_log_speed

          • Major - Major loss of function.
          • Closed

          Activity

            Ascending order - Click to sort in descending order
            alice Alice Sherepa created issue -
            alice Alice Sherepa made changes -
            Field Original Value New Value
            Description {noformat}
            2019-01-24 17:54:39 0 [ERROR] InnoDB: The age of the last checkpoint is 12, which exceeds the log group capacity 0.
            2019-01-24 17:54:41 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
            2019-01-24 17:54:41 0x7f29881a8700 InnoDB: Assertion failure in file /src/storage/innobase/log/log0log.cc line 2087
            InnoDB: Failing assertion: lsn % OS_FILE_LOG_BLOCK_SIZE == LOG_BLOCK_HDR_SIZE
            InnoDB: We intentionally generate a memory trap.
            InnoDB: Submit a detailed bug report to https://jira.mariadb.org/
            InnoDB: If you get repeated assertion failures or crashes, even
            InnoDB: immediately after the mysqld startup, there may be
            InnoDB: corruption in the InnoDB tablespace. Please refer to
            InnoDB: https://mariadb.com/kb/en/library/innodb-recovery-modes/
            InnoDB: about forcing recovery.
            190124 17:54:41 [ERROR] mysqld got signal 6 ;

            linux/raise.c:54(__GI_raise)[0x7f2abd9fe428]
            stdlib/abort.c:91(__GI_abort)[0x7f2abda0002a]
            /bld/sql//mysqld(+0x24eeeed)[0x557b3595ceed]
            ut/ut0dbg.cc:61(_GLOBAL__sub_D_00099_0_ut0dbg.cc)[0x557b356aef2f]
            log/log0log.cc:2088(log_pad_current_log_block())[0x557b356af060]
            log/log0log.cc:2106(log_scrub())[0x557b356af1e8]
            /lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba)[0x7f2abe83f6ba]
            /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7f2abdad041d]
            {noformat}
            		{noformat}
            2019-01-24 17:54:39 0 [ERROR] InnoDB: The age of the last checkpoint is 12, which exceeds the log group capacity 0.
            2019-01-24 17:54:41 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
            2019-01-24 17:54:41 0x7f29881a8700 InnoDB: Assertion failure in file /src/storage/innobase/log/log0log.cc line 2087
            InnoDB: Failing assertion: lsn % OS_FILE_LOG_BLOCK_SIZE == LOG_BLOCK_HDR_SIZE
            InnoDB: We intentionally generate a memory trap.
            InnoDB: Submit a detailed bug report to https://jira.mariadb.org/
            InnoDB: If you get repeated assertion failures or crashes, even
            InnoDB: immediately after the mysqld startup, there may be
            InnoDB: corruption in the InnoDB tablespace. Please refer to
            InnoDB: https://mariadb.com/kb/en/library/innodb-recovery-modes/
            InnoDB: about forcing recovery.
            190124 17:54:41 [ERROR] mysqld got signal 6 ;

            linux/raise.c:54(__GI_raise)[0x7f2abd9fe428]
            stdlib/abort.c:91(__GI_abort)[0x7f2abda0002a]
            /bld/sql//mysqld(+0x24eeeed)[0x557b3595ceed]
            ut/ut0dbg.cc:61(_GLOBAL__sub_D_00099_0_ut0dbg.cc)[0x557b356aef2f]
            log/log0log.cc:2088(log_pad_current_log_block())[0x557b356af060]
            log/log0log.cc:2106(log_scrub())[0x557b356af1e8]
            /lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba)[0x7f2abe83f6ba]
            /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7f2abdad041d]
            {noformat}
            sporadic, start debug server on a clean datadir with my.cnf
            alice Alice Sherepa made changes -
            Attachment my.cnf [ 47160 ]
            marko Marko Mäkelä made changes -
            Component/s mariabackup [ 14500 ]
            Fix Version/s N/A [ 14700 ]
            Resolution Duplicate [ 3 ]
            Status Open [ 1 ] Closed [ 6 ]
            marko Marko Mäkelä made changes -
            Component/s Storage Engine - InnoDB [ 10129 ]
            marko Marko Mäkelä made changes -
            Resolution Duplicate [ 3 ]
            Status Closed [ 6 ] Stalled [ 10000 ]
            marko Marko Mäkelä made changes -
            elenst Elena Stepanova made changes -
            Assignee Alice Sherepa [ alice ] Elena Stepanova [ elenst ]
            elenst Elena Stepanova made changes -
            Component/s mariabackup [ 14500 ]
            Fix Version/s 10.2 [ 14601 ]
            Fix Version/s 10.3 [ 22126 ]
            Fix Version/s 10.4 [ 22408 ]
            Fix Version/s N/A [ 14700 ]
            Affects Version/s 10.2 [ 14601 ]
            Affects Version/s 10.3 [ 22126 ]
            Description {noformat}
            2019-01-24 17:54:39 0 [ERROR] InnoDB: The age of the last checkpoint is 12, which exceeds the log group capacity 0.
            2019-01-24 17:54:41 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
            2019-01-24 17:54:41 0x7f29881a8700 InnoDB: Assertion failure in file /src/storage/innobase/log/log0log.cc line 2087
            InnoDB: Failing assertion: lsn % OS_FILE_LOG_BLOCK_SIZE == LOG_BLOCK_HDR_SIZE
            InnoDB: We intentionally generate a memory trap.
            InnoDB: Submit a detailed bug report to https://jira.mariadb.org/
            InnoDB: If you get repeated assertion failures or crashes, even
            InnoDB: immediately after the mysqld startup, there may be
            InnoDB: corruption in the InnoDB tablespace. Please refer to
            InnoDB: https://mariadb.com/kb/en/library/innodb-recovery-modes/
            InnoDB: about forcing recovery.
            190124 17:54:41 [ERROR] mysqld got signal 6 ;

            linux/raise.c:54(__GI_raise)[0x7f2abd9fe428]
            stdlib/abort.c:91(__GI_abort)[0x7f2abda0002a]
            /bld/sql//mysqld(+0x24eeeed)[0x557b3595ceed]
            ut/ut0dbg.cc:61(_GLOBAL__sub_D_00099_0_ut0dbg.cc)[0x557b356aef2f]
            log/log0log.cc:2088(log_pad_current_log_block())[0x557b356af060]
            log/log0log.cc:2106(log_scrub())[0x557b356af1e8]
            /lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba)[0x7f2abe83f6ba]
            /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7f2abdad041d]
            {noformat}
            sporadic, start debug server on a clean datadir with my.cnf
            		{noformat:title=10.2 dc91372d}
            2019-08-17 20:43:39 0x7fed6c9f6700 InnoDB: Assertion failure in file /data/src/10.2/storage/innobase/log/log0log.cc line 2238
            InnoDB: Failing assertion: lsn % OS_FILE_LOG_BLOCK_SIZE == LOG_BLOCK_HDR_SIZE

            #5 0x00007fed6d93742a in __GI_abort () at abort.c:89
            #6 0x000055d1c6fc5752 in ut_dbg_assertion_failed (expr=0x55d1c7501290 "lsn % OS_FILE_LOG_BLOCK_SIZE == LOG_BLOCK_HDR_SIZE", file=0x55d1c7500470 "/data/src/10.2/storage/innobase/log/log0log.cc", line=2238) at /data/src/10.2/storage/innobase/ut/ut0dbg.cc:60
            #7 0x000055d1c6e4d818 in log_pad_current_log_block () at /data/src/10.2/storage/innobase/log/log0log.cc:2238
            #8 0x000055d1c6e4d883 in log_scrub () at /data/src/10.2/storage/innobase/log/log0log.cc:2254
            #9 0x000055d1c6e4d934 in log_scrub_thread () at /data/src/10.2/storage/innobase/log/log0log.cc:2281
            #10 0x00007fed6f4a34a4 in start_thread (arg=0x7fed6c9f6700) at pthread_create.c:456
            #11 0x00007fed6d9ebd0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
            {noformat}

            To reproduce, bootstrap a clean datadir with all defaults and start the server with
            {noformat}
            --innodb_scrub_log=ON --innodb_scrub_log_speed=50000
            {noformat}

            Do it in a loop if it doesn't fail right away.
            It happens pretty much every time for me on my local HDD and SSD disks, sporadically on an (apparently faster or otherwise different) SSD disk on a lab machine, and *never* in {{shm}}. So, don't run it in {{shm}}.

            Happens both on debug and non-debug builds.
            Summary assertion: lsn % OS_FILE_LOG_BLOCK_SIZE == LOG_BLOCK_HDR_SIZE in log0log.cc line 2087 InnoDB: Failing assertion: lsn % OS_FILE_LOG_BLOCK_SIZE == LOG_BLOCK_HDR_SIZE in log0log.cc with innodb_scrub_log=ON and high values of innodb_scrub_log_speed
            elenst Elena Stepanova made changes -
            Assignee Elena Stepanova [ elenst ] Marko Mäkelä [ marko ]
            elenst Elena Stepanova made changes -
            Description {noformat:title=10.2 dc91372d}
            2019-08-17 20:43:39 0x7fed6c9f6700 InnoDB: Assertion failure in file /data/src/10.2/storage/innobase/log/log0log.cc line 2238
            InnoDB: Failing assertion: lsn % OS_FILE_LOG_BLOCK_SIZE == LOG_BLOCK_HDR_SIZE

            #5 0x00007fed6d93742a in __GI_abort () at abort.c:89
            #6 0x000055d1c6fc5752 in ut_dbg_assertion_failed (expr=0x55d1c7501290 "lsn % OS_FILE_LOG_BLOCK_SIZE == LOG_BLOCK_HDR_SIZE", file=0x55d1c7500470 "/data/src/10.2/storage/innobase/log/log0log.cc", line=2238) at /data/src/10.2/storage/innobase/ut/ut0dbg.cc:60
            #7 0x000055d1c6e4d818 in log_pad_current_log_block () at /data/src/10.2/storage/innobase/log/log0log.cc:2238
            #8 0x000055d1c6e4d883 in log_scrub () at /data/src/10.2/storage/innobase/log/log0log.cc:2254
            #9 0x000055d1c6e4d934 in log_scrub_thread () at /data/src/10.2/storage/innobase/log/log0log.cc:2281
            #10 0x00007fed6f4a34a4 in start_thread (arg=0x7fed6c9f6700) at pthread_create.c:456
            #11 0x00007fed6d9ebd0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
            {noformat}

            To reproduce, bootstrap a clean datadir with all defaults and start the server with
            {noformat}
            --innodb_scrub_log=ON --innodb_scrub_log_speed=50000
            {noformat}

            Do it in a loop if it doesn't fail right away.
            It happens pretty much every time for me on my local HDD and SSD disks, sporadically on an (apparently faster or otherwise different) SSD disk on a lab machine, and *never* in {{shm}}. So, don't run it in {{shm}}.

            Happens both on debug and non-debug builds.
            		{noformat:title=10.2 dc91372d}
            2019-08-17 20:43:39 0x7fed6c9f6700 InnoDB: Assertion failure in file /data/src/10.2/storage/innobase/log/log0log.cc line 2238
            InnoDB: Failing assertion: lsn % OS_FILE_LOG_BLOCK_SIZE == LOG_BLOCK_HDR_SIZE

            #5 0x00007fed6d93742a in __GI_abort () at abort.c:89
            #6 0x000055d1c6fc5752 in ut_dbg_assertion_failed (expr=0x55d1c7501290 "lsn % OS_FILE_LOG_BLOCK_SIZE == LOG_BLOCK_HDR_SIZE", file=0x55d1c7500470 "/data/src/10.2/storage/innobase/log/log0log.cc", line=2238) at /data/src/10.2/storage/innobase/ut/ut0dbg.cc:60
            #7 0x000055d1c6e4d818 in log_pad_current_log_block () at /data/src/10.2/storage/innobase/log/log0log.cc:2238
            #8 0x000055d1c6e4d883 in log_scrub () at /data/src/10.2/storage/innobase/log/log0log.cc:2254
            #9 0x000055d1c6e4d934 in log_scrub_thread () at /data/src/10.2/storage/innobase/log/log0log.cc:2281
            #10 0x00007fed6f4a34a4 in start_thread (arg=0x7fed6c9f6700) at pthread_create.c:456
            #11 0x00007fed6d9ebd0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
            {noformat}

            To reproduce, bootstrap a clean datadir with all defaults and start the server with
            {noformat}
            --innodb_scrub_log=ON --innodb_scrub_log_speed=50000
            {noformat}

            Do it in a loop if it doesn't fail right away.
            It happens pretty much every time for me on my local HDD and SSD disks, sporadically on an (apparently faster or otherwise different) SSD disk on a lab machine, and *never* in {{shm}}. So, don't run it in {{shm}}.

            Happens both on debug and non-debug builds.

            Couldn't reproduce on 10.1 and 10.5.
            elenst Elena Stepanova made changes -
            Description {noformat:title=10.2 dc91372d}
            2019-08-17 20:43:39 0x7fed6c9f6700 InnoDB: Assertion failure in file /data/src/10.2/storage/innobase/log/log0log.cc line 2238
            InnoDB: Failing assertion: lsn % OS_FILE_LOG_BLOCK_SIZE == LOG_BLOCK_HDR_SIZE

            #5 0x00007fed6d93742a in __GI_abort () at abort.c:89
            #6 0x000055d1c6fc5752 in ut_dbg_assertion_failed (expr=0x55d1c7501290 "lsn % OS_FILE_LOG_BLOCK_SIZE == LOG_BLOCK_HDR_SIZE", file=0x55d1c7500470 "/data/src/10.2/storage/innobase/log/log0log.cc", line=2238) at /data/src/10.2/storage/innobase/ut/ut0dbg.cc:60
            #7 0x000055d1c6e4d818 in log_pad_current_log_block () at /data/src/10.2/storage/innobase/log/log0log.cc:2238
            #8 0x000055d1c6e4d883 in log_scrub () at /data/src/10.2/storage/innobase/log/log0log.cc:2254
            #9 0x000055d1c6e4d934 in log_scrub_thread () at /data/src/10.2/storage/innobase/log/log0log.cc:2281
            #10 0x00007fed6f4a34a4 in start_thread (arg=0x7fed6c9f6700) at pthread_create.c:456
            #11 0x00007fed6d9ebd0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
            {noformat}

            To reproduce, bootstrap a clean datadir with all defaults and start the server with
            {noformat}
            --innodb_scrub_log=ON --innodb_scrub_log_speed=50000
            {noformat}

            Do it in a loop if it doesn't fail right away.
            It happens pretty much every time for me on my local HDD and SSD disks, sporadically on an (apparently faster or otherwise different) SSD disk on a lab machine, and *never* in {{shm}}. So, don't run it in {{shm}}.

            Happens both on debug and non-debug builds.

            Couldn't reproduce on 10.1 and 10.5.             		{noformat:title=10.2 dc91372d}
            2019-08-17 20:43:39 0x7fed6c9f6700 InnoDB: Assertion failure in file /data/src/10.2/storage/innobase/log/log0log.cc line 2238
            InnoDB: Failing assertion: lsn % OS_FILE_LOG_BLOCK_SIZE == LOG_BLOCK_HDR_SIZE

            #5 0x00007fed6d93742a in __GI_abort () at abort.c:89
            #6 0x000055d1c6fc5752 in ut_dbg_assertion_failed (expr=0x55d1c7501290 "lsn % OS_FILE_LOG_BLOCK_SIZE == LOG_BLOCK_HDR_SIZE", file=0x55d1c7500470 "/data/src/10.2/storage/innobase/log/log0log.cc", line=2238) at /data/src/10.2/storage/innobase/ut/ut0dbg.cc:60
            #7 0x000055d1c6e4d818 in log_pad_current_log_block () at /data/src/10.2/storage/innobase/log/log0log.cc:2238
            #8 0x000055d1c6e4d883 in log_scrub () at /data/src/10.2/storage/innobase/log/log0log.cc:2254
            #9 0x000055d1c6e4d934 in log_scrub_thread () at /data/src/10.2/storage/innobase/log/log0log.cc:2281
            #10 0x00007fed6f4a34a4 in start_thread (arg=0x7fed6c9f6700) at pthread_create.c:456
            #11 0x00007fed6d9ebd0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
            {noformat}

            To reproduce, bootstrap a clean datadir with all defaults and start the server with
            {noformat}
            --innodb_scrub_log=ON --innodb_scrub_log_speed=50000
            {noformat}

            Do it in a loop if it doesn't fail right away.
            It happens pretty much every time for me on my local HDD and SSD disks, sporadically on an (apparently faster or otherwise different) SSD disk on a lab machine, and *almost never* in {{shm}}. So, don't run it in {{shm}}.

            Happens both on debug and non-debug builds.

            Couldn't reproduce on 10.1 and 10.5.
            elenst Elena Stepanova made changes -
            elenst Elena Stepanova made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            Fix Version/s 10.5.2 [ 24030 ]
            Fix Version/s 10.2 [ 14601 ]
            Fix Version/s 10.3 [ 22126 ]
            Fix Version/s 10.4 [ 22408 ]
            Resolution Won't Fix [ 2 ]
            Status Stalled [ 10000 ] Closed [ 6 ]
            serg Sergei Golubchik made changes -
            Workflow MariaDB v3 [ 92043 ] MariaDB v4 [ 155584 ]

            People

              marko Marko Mäkelä
              alice Alice Sherepa
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.