[MDEV-26050] InnoDB: Assertion failure in lock0lock.cc line 6942 Created: 2021-06-30  Updated: 2021-12-06  Resolved: 2021-12-06

Status: Closed
Project: MariaDB Server
Component/s: Galera, Storage Engine - InnoDB
Affects Version/s: 10.4.19
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: Valerii Kravchuk Assignee: Jan Lindström (Inactive)
Resolution: Incomplete Votes: 1
Labels: need_feedback

Issue Links:
Relates
relates to MDEV-25594 Crash in deadlock checker under high ... Closed

 Description   

Server crashes as follows while executing SELECT ... FOR UPDATE:

2021-06-28 16:53:32 0x7f924abbf700  InnoDB: Assertion failure in file /home/buildbot/buildbot/padding_for_CPACK_RPM_BUILD_SOURCE_DIRS_PREFIX/mariadb-10.4.19/storage/innobase/lock/lock0lock.cc line 6942
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to https://jira.mariadb.org/
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: https://mariadb.com/kb/en/library/innodb-recovery-modes/
InnoDB: about forcing recovery.
210628 16:53:32 [ERROR] mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
 
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
 
Server version: 10.4.19-MariaDB-log
key_buffer_size=134217728
read_buffer_size=131072
max_used_connections=328
max_threads=352
thread_count=367
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 905724 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
 
Thread pointer: 0x7f9b1ccc1448
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x7f924abbecf0 thread_stack 0x49000
mysys/stacktrace.c:175(my_print_stacktrace)[0x5655348af40e]
sql/signal_handler.cc:222(handle_fatal_signal)[0x56553433846f]
/lib64/libpthread.so.0(+0xf630)[0x7fa32edf1630]
:0(__GI_raise)[0x7fa32e23c387]
:0(__GI_abort)[0x7fa32e23da78]
/usr/sbin/mysqld(+0x5c96dc)[0x56553402d6dc]
/usr/sbin/mysqld(+0xad0bd6)[0x565534534bd6]
/usr/sbin/mysqld(+0xad0d1b)[0x565534534d1b]
/usr/sbin/mysqld(+0xad20aa)[0x5655345360aa]
/usr/sbin/mysqld(+0xad25e9)[0x5655345365e9]
/usr/sbin/mysqld(+0x5c8529)[0x56553402c529]
/usr/sbin/mysqld(+0xb709e9)[0x5655345d49e9]
/usr/sbin/mysqld(+0xa90de3)[0x5655344f4de3]
/usr/sbin/mysqld(_ZN7handler17ha_index_read_mapEPhPKhm16ha_rkey_function+0x1e8)[0x56553433d6f8]
/usr/sbin/mysqld(+0x70f117)[0x565534173117]
ut/ut0rbt.cc:218(??)[0x565534164479]
lock/lock0lock.cc:7003(Gis_read_stream::check_next_symbol(char))[0x565534187def]
lock/lock0lock.cc:1716(Gis_read_stream::check_next_symbol(char))[0x565534188103]
lock/lock0lock.cc:1951(Gis_read_stream::check_next_symbol(char))[0x565534186266]
lock/lock0lock.cc:5806(Gis_read_stream::check_next_symbol(char))[0x565534186e07]
row/row0sel.cc:1295(??)[0x565534018a6b]
row/row0sel.cc:5112(void std::__introsort_loop<unsigned char**, long>(unsigned char**, unsigned char**, long))[0x56553412a9cf]
handler/ha_innodb.cc:9363(Gis_read_stream::check_next_symbol(char))[0x56553412fa7d]
sql/handler.cc:2923(handler::ha_index_read_map(unsigned char*, unsigned char const*, unsigned long, ha_rkey_function))[0x565534130287]
sql/sql_select.cc:21301(cp_buffer_from_ref(THD*, TABLE*, st_table_ref*))[0x565534133b2f]
sql/sql_select.cc:20541(sub_select(JOIN*, st_join_table*, bool))[0x56553413427b]
sql/sql_select.cc:20082(JOIN::exec_inner())[0x5655342139aa]
sql/sql_select.cc:4308(JOIN::exec())[0x565534213a8d]
pthread_create.c:0(start_thread)[0x7fa32ede9ea5]
/lib64/libc.so.6(clone+0x6d)[0x7fa32e3049fd]
 
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (0x7f9b1cd3e550): SELECT ...
        FROM T
        WHERE T.`id1`= '7447620' AND (`id2` = '4065') AND (`id3` = '1261585') AND (`id4` IS NULL) AND (``status` IN ('active','incomplete','completed'))
         FOR UPDATE
 
Connection ID (thread ID): 134194709
Status: NOT_KILLED

Looks like the assertion failure happens in deadlocks checker, like in MDEV-25594.



 Comments   
Comment by Marko Mäkelä [ 2021-07-23 ]

The assertion failure was originally reported for 10.5 in MDEV-25594.

It looks like we’d better port the cleanup to 10.2.

Comment by Marko Mäkelä [ 2021-07-27 ]

To rule out any InnoDB problem, I ported the MDEV-25594 cleanup from 10.5 to earlier major versions.

Because Galera is involved here, and because Galera interacts with InnoDB locking, I think that this needs to be analyzed further by the Galera team.

Comment by Jan Lindström (Inactive) [ 2021-08-30 ]

valerii Can we have a show create table for problematic table, full query and if possible test case to reproduce the problem.

Comment by Jan Lindström (Inactive) [ 2021-08-30 ]

I do not follow why Gis_read_stream is on stack because table in query does not have any GIS columns.

Comment by Jan Lindström (Inactive) [ 2021-10-06 ]

valerii I would need full stack trace with resolved function names using either core file or debug symbol package. To me current stack trace does not look correct.

Comment by Elena Stepanova [ 2021-11-03 ]

I do not follow why Gis_read_stream is on stack because table in query does not have any GIS columns.

jplindst,
The occasion doesn't seem unique. You have MDEV-24473 which features similar frames, and I don't see any GIS in the test case which triggered the failure, either.

Generated at Thu Feb 08 09:42:22 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.