Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-38428

InnoDB: Assertion failure row0sel.cc line 604

    XMLWordPrintable

Details

    • Bug
    • Status: Open (View Workflow)
    • Major
    • Resolution: Unresolved
    • 10.6.24
    • None

    Description

      MariaDB crashes during 'normal' operation. It's a production environment of a Magento web-shop, we don't really know how to reproduce.

      The server is also consuming large amounts of memory - all calculations we could find point to the server using a maximum of 5 GB, but we see RSS of mariadbd rise well above 14 GB.

      2025-12-24 03:47:40 0x7f9c77361640  InnoDB: Assertion failure in file /home/buildbot/amd64-rhel-9-rpm-autobake/build/padding_for_CPACK_RPM_BUILD_SOURCE_DIRS_PREFIX/storage/innobase/row/row0sel.cc line 604
      InnoDB: Failing assertion: data
      InnoDB: We intentionally generate a memory trap.
      InnoDB: Submit a detailed bug report to https://jira.mariadb.org/
      InnoDB: If you get repeated assertion failures or crashes, even
      InnoDB: immediately after the mariadbd startup, there may be
      InnoDB: corruption in the InnoDB tablespace. Please refer to
      InnoDB: https://mariadb.com/kb/en/library/innodb-recovery-modes/
      InnoDB: about forcing recovery.
      251224  3:47:40 [ERROR] /usr/sbin/mariadbd got signal 6 ;
      Sorry, we probably made a mistake, and this is a bug.
       
      Your assistance in bug reporting will enable us to fix this for the next release.
      To report this bug, see https://mariadb.com/kb/en/reporting-bugs about how to report
      a bug on https://jira.mariadb.org/.
       
      Please include the information from the server start above, to the end of the
      information below.
       
      Server version: 10.6.24-MariaDB-log source revision: e5994025bec0d34ddef25df2e252b8f1d2e34e51
       
      The information page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mariadbd/
      contains instructions to obtain a better version of the backtrace below.
      Following these instructions will help MariaDB developers provide a fix quicker.
       
      Attempting backtrace. Include this in the bug report.
      (note: Retrieving this information may fail)
       
      Thread pointer: 0x7f9af54616d8
      stack_bottom = 0x7f9c77362000 thread_stack 0x49000
      2025-12-24  3:47:41 0 [Note] /usr/sbin/mariadbd (initiated by: unknown): Normal shutdown
      2025-12-24  3:47:41 980349 [Warning] Sort aborted, host: 192.168.125.46, user: sam_mag, thread: 980349, query: SELECT `e`.*, IF(at_is_active.value_id > 0, at_is_active.value, at_is_active_default.value) AS `is_active`, `parent`.`path` AS `parent_path` FROM `catalog_category_entity` AS `e`
       INNER JOIN `catalog_category_entity_int` AS `at_is_active_default` ON (`at_is_active_default`.`entity_id` = `e`.`entity_id`) AND (`at_is_active_default`.`attribute_id` = '46') AND `at_is_active_default`.`store_id` = 0
       LEFT JOIN `catalog_category_entity_int` AS `at_is_active` ON (`at_is_active`.`entity_id` = `e`.`entity_id`) AND (`at_is_active`.`attribute_id` = '46') AND (`at_is_active`.`store_id` = 1)
       LEFT JOIN `catalog_category_entity` AS `parent` ON e.parent_id = parent.entity_id WHERE (IF(at_is_active.value_id > 0, at_is_active.value, at_is_active_default.value) = '1') AND (`e`.`path` LIKE '1/5/10/%') AND (`e`.`level` > '2') ORDER BY `e`.`position` ASC
      /usr/sbin/mariadbd(my_print_stacktrace+0x2e)[0x56143134848e]
      /usr/sbin/mariadbd(handle_fatal_signal+0x305)[0x561430e08a15]
      /lib64/libc.so.6(+0x3fc30)[0x7f9d9d03fc30]
      /lib64/libc.so.6(+0x8d02c)[0x7f9d9d08d02c]
      /lib64/libc.so.6(raise+0x16)[0x7f9d9d03fb86]
      /lib64/libc.so.6(abort+0xd3)[0x7f9d9d029873]
      /usr/sbin/mariadbd(+0x69855c)[0x561430a6d55c]
      /usr/sbin/mariadbd(+0x68d270)[0x561430a62270]
      /usr/sbin/mariadbd(+0xe0a145)[0x5614311df145]
      /usr/sbin/mariadbd(+0xe0a35b)[0x5614311df35b]
      /usr/sbin/mariadbd(+0xdc53e5)[0x56143119a3e5]
      /usr/sbin/mariadbd(+0xeeccf9)[0x5614312c1cf9]
      /usr/sbin/mariadbd(+0xee2ac9)[0x5614312b7ac9]
      /usr/sbin/mariadbd(+0xeebb2a)[0x5614312c0b2a]
      /usr/sbin/mariadbd(+0xf036b3)[0x5614312d86b3]
      /usr/sbin/mariadbd(+0xf0374a)[0x5614312d874a]
      /usr/sbin/mariadbd(+0xf037a0)[0x5614312d87a0]
      /usr/sbin/mariadbd(+0xee9674)[0x5614312be674]
      /usr/sbin/mariadbd(+0xd41cb1)[0x561431116cb1]
      /usr/sbin/mariadbd(_ZN15Item_func_match11init_searchEP3THDb+0x55d)[0x561430e8e10d]
      /usr/sbin/mariadbd(_Z12init_ftfuncsP3THDP13st_select_lexb+0x63)[0x561430b57293]
      /usr/sbin/mariadbd(_ZN4JOIN15optimize_stage2Ev+0x1e1c)[0x561430c2b0ac]
      /usr/sbin/mariadbd(_ZN4JOIN14optimize_innerEv+0x148e)[0x561430c2d88e]
      /usr/sbin/mariadbd(_ZN4JOIN8optimizeEv+0xaa)[0x561430c2deba]
      /usr/sbin/mariadbd(_Z12mysql_selectP3THDP10TABLE_LISTR4ListI4ItemEPS4_jP8st_orderS9_S7_S9_yP13select_resultP18st_select_lex_unitP13st_select_lex+0xb3)[0x561430c2df83]
      /usr/sbin/mariadbd(_Z13handle_selectP3THDP3LEXP13select_resultm+0x154)[0x561430c2e774]
      /usr/sbin/mariadbd(+0x7dcd65)[0x561430bb1d65]
      /usr/sbin/mariadbd(_Z21mysql_execute_commandP3THDb+0x4214)[0x561430bc1234]
      /usr/sbin/mariadbd(_Z11mysql_parseP3THDPcjP12Parser_state+0x1e7)[0x561430bc2b47]
      /usr/sbin/mariadbd(_Z16dispatch_command19enum_server_commandP3THDPcjb+0x128d)[0x561430bc4e5d]
      /usr/sbin/mariadbd(_Z10do_commandP3THDb+0x13f)[0x561430bc6c8f]
      /usr/sbin/mariadbd(_Z24do_handle_one_connectionP7CONNECTb+0x395)[0x561430cdc365]
      /usr/sbin/mariadbd(handle_one_connection+0x5d)[0x561430cdc6bd]
      /usr/sbin/mariadbd(+0xc92439)[0x561431067439]
      /lib64/libc.so.6(+0x8b2ea)[0x7f9d9d08b2ea]
      /lib64/libc.so.6(+0x1103c0)[0x7f9d9d1103c0]
       
      Connection ID (thread ID): 980341
      Status: KILL_SERVER
      Query (0x7f9af58d96b0): SELECT `main_table`.* FROM `mst_quick_navigation_sequence` AS `main_table` WHERE (`store_id` = '1') AND (`category_id` = '9') AND (MATCH(sequence) AGAINST('+brand:10482 +brand:11465 +brand:6164 +brand:6214 +brand:6979 +brand:7063 +brand:7408 +brand:7503 +brand:7551 +brand:7943 +brand:7947 +brand:7966 +brand:7968 +brand:9339 +brand:9656' IN BOOLEAN MODE)) ORDER BY popularity DESC
       LIMIT 100
      Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=off,condition_pushdown_from_having=on,not_null_range_scan=off,hash_join_cardinality=off,cset_narrowing=off
       
      Writing a core file...
      Working directory at /var/lib/mysql
      Resource Limits (excludes unlimited resources):
      Limit                     Soft Limit           Hard Limit           Units     
      Max stack size            8388608              unlimited            bytes     
      Max processes             62740                62740                processes 
      Max open files            1073741816           1073741816           files     
      Max locked memory         524288               524288               bytes     
      Max pending signals       62740                62740                signals   
      Max msgqueue size         819200               819200               bytes     
      Max nice priority         0                    0                    
      Max realtime priority     0                    0                    
      Core pattern: |/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h
       
      Kernel version: Linux version 5.14.0-570.58.1.el9_6.x86_64 (mockbuild@iad1-prod-build001.bld.equ.rockylinux.org) (gcc (GCC) 11.5.0 20240719 (Red Hat 11.5.0-5), GNU ld version 2.35.2-63.el9) #1 SMP PREEMPT_DYNAMIC Fri Oct 31 13:55:05 UTC 2025
      

      There is a 'core dump', but coredumpctl lists it as 'truncated':

      # coredumpctl list
      TIME                          PID UID GID SIG     COREFILE  EXE                  SIZE
      Wed 2025-12-24 03:47:44 CET 13942 987 987 SIGABRT truncated /usr/sbin/mariadbd 287.1K
      
      

      Here is the server configuration:

      [mysqld]
      performance_schema=ON
      log-error = /var/log/mariadb/mariadb.log
      character-set-server = utf8mb4
      collation_server = utf8mb4_unicode_ci
      default-storage-engine = InnoDB
      skip-name-resolve
      lower_case_table_names=1
      key-buffer-size                = 32M
      myisam-recover-options         = FORCE,BACKUP
      max-allowed-packet             = 16M
      max-connect-errors             = 1000000
      sql-mode                       = STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_AUTO_VALUE_ON_ZERO,NO_ENGINE_SUBSTITUTION,NO_ZERO_DATE,NO_ZERO_IN_DATE
      innodb                         = FORCE
      log-bin                        = /var/lib/mysql/mysql-bin
      expire-logs-days               = 2
      sync-binlog                    = 1
      log_bin_trust_function_creators = 1
      skip-log-bin
      tmp-table-size                 = 16M
      max-heap-table-size            = 16M
      query-cache-type               = 0
      query-cache-size               = 0
      query_cache_limit	       = 4M
      max-connections                = 250
      thread-cache-size              = 50
      open-files-limit               = 16384
      table-definition-cache         = 1024
      table-open-cache               = 2048
      join-buffer-size               = 256k
      join-buffer-space-limit	       = 2M
      optimizer-switch               = optimize_join_buffer_size=on,rowid_filter=off
      optimizer_use_condition_selectivity = 1
      innodb-flush-method            = O_DIRECT
      #innodb-log-files-in-group      = 2 <- deprecated
      innodb-log-file-size           = 1G
      innodb-flush-log-at-trx-commit = 1
      innodb-file-per-table          = 1
      innodb-buffer-pool-size        = 4G
      innodb-stats-on-metadata       = False
      innodb-ft-cache-size           = 40000000
      general_log_file               = /var/log/mariadb/mariadb-query.log
      log-queries-not-using-indexes  = 0
      slow-query-log                 = 1
      slow-query-log-file            = /var/log/mariadb/mariadb-slow.log
       
      ignore_db_dirs                 = 'lost+found'
      aria_sort_buffer_size = 32M
      

      As this is a production server MariaDB has been restarted automatically via systemd.

      Please let me know if I should add more info.

      Attachments

        Activity

          People

            Unassigned Unassigned
            danci1973 Danilo Godec
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.