Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-22397

repeatable crashes: InnoDB: Assertion failure in file /home/buildbot/buildbot/padding_for_CPACK_RPM_BUILD_SOURCE_DIRS_PREFIX/mariadb-10.4.12/storage/innobase/btr/btr0cur.cc line 311

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Incomplete
    • 10.4(EOL)
    • N/A
    • Centos Linux 7.7
      MariaDB 10.4.12
      Linux 3.10.0-1062.9.1.el7.x86_64

    Description

      Got repeatable crashes with the following traces in logs.
      So far got no cores, will supply bts and cores asap.

      Apr 28 14:20:39 sso-db02 mysqld: 2020-04-28 14:20:39 0 [Note] WSREP:  cleaning up 75554af7 (tcp://188.186.154.182:4567)
      Apr 28 15:26:20 sso-db02 mysqld: 2020-04-28 15:26:20 0x7efa894c8700  InnoDB: Assertion failure in file /home/buildbot/buildbot/padding_for_CPACK_RPM_BUILD_SOURCE_DIRS_PREFIX/mariadb-10.4.12/storage/innobase/btr/btr0cur.cc line 311
      Apr 28 15:26:20 sso-db02 mysqld: InnoDB: Failing assertion: btr_page_get_next(latch_leaves.blocks[0]->frame) == page_get_page_no(page)
      Apr 28 15:26:20 sso-db02 mysqld: InnoDB: We intentionally generate a memory trap.
      Apr 28 15:26:20 sso-db02 mysqld: InnoDB: Submit a detailed bug report to https://jira.mariadb.org/
      Apr 28 15:26:20 sso-db02 mysqld: InnoDB: If you get repeated assertion failures or crashes, even
      Apr 28 15:26:20 sso-db02 mysqld: InnoDB: immediately after the mysqld startup, there may be
      Apr 28 15:26:20 sso-db02 mysqld: InnoDB: corruption in the InnoDB tablespace. Please refer to
      Apr 28 15:26:20 sso-db02 mysqld: InnoDB: https://mariadb.com/kb/en/library/innodb-recovery-modes/
      Apr 28 15:26:20 sso-db02 mysqld: InnoDB: about forcing recovery.
      Apr 28 15:26:20 sso-db02 mysqld: 200428 15:26:20 [ERROR] mysqld got signal 6 ;
      Apr 28 15:26:20 sso-db02 mysqld: This could be because you hit a bug. It is also possible that this binary
      Apr 28 15:26:20 sso-db02 mysqld: or one of the libraries it was linked against is corrupt, improperly built,
      Apr 28 15:26:20 sso-db02 mysqld: or misconfigured. This error can also be caused by malfunctioning hardware.
      Apr 28 15:26:20 sso-db02 mysqld: To report this bug, see https://mariadb.com/kb/en/reporting-bugs
      Apr 28 15:26:20 sso-db02 mysqld: We will try our best to scrape up some info that will hopefully help
      Apr 28 15:26:20 sso-db02 mysqld: diagnose the problem, but since we have already crashed,
      Apr 28 15:26:20 sso-db02 mysqld: something is definitely wrong and this may fail.
      Apr 28 15:26:20 sso-db02 mysqld: Server version: 10.4.12-MariaDB
      Apr 28 15:26:20 sso-db02 mysqld: key_buffer_size=16777216
      Apr 28 15:26:20 sso-db02 mysqld: read_buffer_size=8388608
      Apr 28 15:26:20 sso-db02 mysqld: max_used_connections=194
      Apr 28 15:26:20 sso-db02 mysqld: max_threads=1026
      Apr 28 15:26:20 sso-db02 mysqld: thread_count=190
      Apr 28 15:26:20 sso-db02 mysqld: It is possible that mysqld could use up to
      Apr 28 15:26:20 sso-db02 mysqld: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 8684712 K  bytes of memory
      Apr 28 15:26:20 sso-db02 mysqld: Hope that's ok; if not, decrease some variables in the equation.
      Apr 28 15:26:20 sso-db02 mysqld: Thread pointer: 0x7ef9b8124ff8
      Apr 28 15:26:20 sso-db02 mysqld: Attempting backtrace. You can use the following information to find out
      Apr 28 15:26:20 sso-db02 mysqld: where mysqld died. If you see no messages after this, something went
      Apr 28 15:26:20 sso-db02 mysqld: terribly wrong...
      Apr 28 15:26:20 sso-db02 mysqld: stack_bottom = 0x7efa894c7cf0 thread_stack 0x49000
      Apr 28 15:26:20 sso-db02 mysqld: /usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x563680ae2a1e]
      Apr 28 15:26:20 sso-db02 mysqld: /usr/sbin/mysqld(handle_fatal_signal+0x30f)[0x563680576d8f]
      Apr 28 15:26:20 sso-db02 mysqld: /lib64/libpthread.so.0(+0xf630)[0x7efcfdd32630]
      Apr 28 15:26:20 sso-db02 mysqld: /lib64/libc.so.6(gsignal+0x37)[0x7efcfc002377]
      Apr 28 15:26:20 sso-db02 mysqld: /lib64/libc.so.6(abort+0x148)[0x7efcfc003a68]
      Apr 28 15:26:20 sso-db02 mysqld: /usr/sbin/mysqld(+0x5b18be)[0x56368026a8be]
      Apr 28 15:26:20 sso-db02 mysqld: /usr/sbin/mysqld(+0xbcb9dc)[0x5636808849dc]
      Apr 28 15:26:20 sso-db02 mysqld: /usr/sbin/mysqld(+0xbd6d20)[0x56368088fd20]
      Apr 28 15:26:20 sso-db02 mysqld: /usr/sbin/mysqld(+0xb48a3b)[0x563680801a3b]
      Apr 28 15:26:20 sso-db02 mysqld: /usr/sbin/mysqld(+0xb4b2f3)[0x5636808042f3]
      Apr 28 15:26:20 sso-db02 mysqld: /usr/sbin/mysqld(+0xcbfe46)[0x563680978e46]
      Apr 28 15:26:20 sso-db02 mysqld: /usr/sbin/mysqld(+0xcc0f81)[0x563680979f81]
      Apr 28 15:26:20 sso-db02 mysqld: /usr/sbin/mysqld(+0xb55950)[0x56368080e950]
      Apr 28 15:26:20 sso-db02 mysqld: /usr/sbin/mysqld(+0xb00f5d)[0x5636807b9f5d]
      Apr 28 15:26:20 sso-db02 mysqld: /usr/sbin/mysqld(+0xb96f4b)[0x56368084ff4b]
      Apr 28 15:26:20 sso-db02 mysqld: /usr/sbin/mysqld(+0xb971ae)[0x5636808501ae]
      Apr 28 15:26:20 sso-db02 mysqld: /usr/sbin/mysqld(+0xa5d308)[0x563680716308]
      Apr 28 15:26:20 sso-db02 mysqld: /usr/sbin/mysqld(_Z17ha_rollback_transP3THDb+0x143)[0x56368057a373]
      Apr 28 15:26:21 sso-db02 mysqld: /usr/sbin/mysqld(_Z14trans_rollbackP3THD+0x57)[0x56368045adf7]
      Apr 28 15:26:21 sso-db02 mysqld: /usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x338f)[0x56368036320f]
      Apr 28 15:26:21 sso-db02 mysqld: /usr/sbin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_statebb+0x24b)[0x5636803694eb]
      Apr 28 15:26:21 sso-db02 mysqld: /usr/sbin/mysqld(+0x6b0d99)[0x563680369d99]
      Apr 28 15:26:21 sso-db02 mysqld: /usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcjbb+0x2b06)[0x56368036d296]
      Apr 28 15:26:21 sso-db02 mysqld: /usr/sbin/mysqld(_Z10do_commandP3THD+0x11c)[0x56368036da4c]
      Apr 28 15:26:21 sso-db02 mysqld: /usr/sbin/mysqld(_Z24do_handle_one_connectionP7CONNECT+0x1fa)[0x56368044c32a]
      Apr 28 15:26:21 sso-db02 mysqld: /usr/sbin/mysqld(handle_one_connection+0x3d)[0x56368044c40d]
      Apr 28 15:26:21 sso-db02 mysqld: /lib64/libpthread.so.0(+0x7ea5)[0x7efcfdd2aea5]
      Apr 28 15:26:21 sso-db02 mysqld: /lib64/libc.so.6(clone+0x6d)[0x7efcfc0ca8cd]
      Apr 28 15:26:21 sso-db02 mysqld: Trying to get some variables.
      Apr 28 15:26:21 sso-db02 mysqld: Some pointers may be invalid and cause the dump to abort.
      Apr 28 15:26:21 sso-db02 mysqld: Query (0x7ef9b822c0d0): ROLLBACK
      Apr 28 15:26:21 sso-db02 mysqld: Connection ID (thread ID): 295
      Apr 28 15:26:21 sso-db02 mysqld: Status: NOT_KILLED
      Apr 28 15:26:21 sso-db02 mysqld: Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=on,condition_pushdown_from_having=on
      Apr 28 15:26:21 sso-db02 mysqld: The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
      Apr 28 15:26:21 sso-db02 mysqld: information that should help you find out what is causing the crash.
      Apr 28 15:26:21 sso-db02 mysqld: Writing a core file...
      Apr 28 15:26:21 sso-db02 mysqld: Working directory at /var/lib/mysql
      Apr 28 15:26:21 sso-db02 mysqld: Resource Limits:
      Apr 28 15:26:21 sso-db02 mysqld: Limit                     Soft Limit           Hard Limit           Units
      Apr 28 15:26:21 sso-db02 mysqld: Max cpu time              unlimited            unlimited            seconds
      Apr 28 15:26:21 sso-db02 mysqld: Max file size             unlimited            unlimited            bytes
      Apr 28 15:26:21 sso-db02 mysqld: Max data size             unlimited            unlimited            bytes
      Apr 28 15:26:21 sso-db02 mysqld: Max stack size            8388608              unlimited            bytes
      Apr 28 15:26:21 sso-db02 mysqld: Max core file size        0                    unlimited            bytes
      Apr 28 15:26:21 sso-db02 mysqld: Max resident set          unlimited            unlimited            bytes
      Apr 28 15:26:21 sso-db02 mysqld: Max processes             63374                63374                processes
      Apr 28 15:26:21 sso-db02 mysqld: Max open files            16364                16364                files
      Apr 28 15:26:21 sso-db02 mysqld: Max locked memory         65536                65536                bytes
      Apr 28 15:26:21 sso-db02 mysqld: Max address space         unlimited            unlimited            bytes
      Apr 28 15:26:21 sso-db02 mysqld: Max file locks            unlimited            unlimited            locks
      Apr 28 15:26:21 sso-db02 mysqld: Max pending signals       63374                63374                signals
      Apr 28 15:26:21 sso-db02 mysqld: Max msgqueue size         819200               819200               bytes
      Apr 28 15:26:21 sso-db02 mysqld: Max nice priority         0                    0
      Apr 28 15:26:21 sso-db02 mysqld: Max realtime priority     0                    0
      Apr 28 15:26:21 sso-db02 mysqld: Max realtime timeout      unlimited            unlimited            us
      Apr 28 15:26:21 sso-db02 mysqld: Core pattern: core
      

      Attachments

        Issue Links

          Activity

            A similar bug MDEV-24014 was filed today. In order to be able to proceed, we would need more information, and something repeatable. Did the server ever crash before this? Were any DDL statements executed on the table in the past?

            marko Marko Mäkelä added a comment - A similar bug MDEV-24014 was filed today. In order to be able to proceed, we would need more information, and something repeatable. Did the server ever crash before this? Were any DDL statements executed on the table in the past?

            People

              marko Marko Mäkelä
              emz Eugene Z. Zheganin
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.