[MDEV-24141] Failing assertion Created: 2020-11-05  Updated: 2020-12-28  Resolved: 2020-12-28

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - InnoDB
Affects Version/s: 10.4.13
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: Ilya Bovbel Assignee: Unassigned
Resolution: Incomplete Votes: 0
Labels: crash, need_feedback

Issue Links:
Relates
relates to MDEV-24449 Corruption of system tablespace or la... Closed

 Description   

Hi!
Got this error in error.log

2020-11-05 16:42:09 0x7ef37c66c700 InnoDB: Assertion failure in file /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/row/row0ins.cc line 231
InnoDB: Failing assertion: !cursor->index->is_committed()
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to https://jira.mariadb.org/
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: https://mariadb.com/kb/en/library/innodb-recovery-modes/
InnoDB: about forcing recovery.
201105 16:42:09 [ERROR] mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.

To report this bug, see https://mariadb.com/kb/en/reporting-bugs

We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.

Server version: 10.4.13-MariaDB-1:10.4.13+maria~buster-log
key_buffer_size=4294967296
read_buffer_size=131072
max_used_connections=194
max_threads=802
thread_count=139
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 5959249 K bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x7ef21402d348
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x7ef37c66bcd8 thread_stack 0x40000

      • buffer overflow detected ***: /usr/sbin/mysqld terminated

So I'm here to report it)
Info:
The environment (Operating system = Debian GNU/Linux 10 (buster), MariaDB version = Server version: 10.4.13-MariaDB-1:10.4.13+maria~buster-log)



 Comments   
Comment by Marko Mäkelä [ 2020-11-05 ]

bovbel.ilya, this belongs to the MDEV-9663 family of bugs. We have not been able to reproduce this internally. The only thing that we can reproduce is MDEV-15532 and its siblings. That is, if you are using XA transactions and DDL statements, broken MDL may lead to corruption.

I am happy to debug this if you can provide the exact steps to reproduce this, using SQL statements. (The use of pre-corrupted data files is not useful.)

Comment by Ilya Bovbel [ 2020-11-06 ]

marko, thank you for answering! Unfortunately, I don't have exact steps. If I have one, I'll provide here or in MDEV-9663

Comment by Ilya Bovbel [ 2020-11-26 ]

marko, hello!
I have been collecting all sql queries to mariadb (option general_log=1), and now I have 33GB log... Do you need all the log file? Do you need anything else?

Comment by Elena Stepanova [ 2020-11-26 ]

bovbel.ilya,

How big is it when compressed?
And does it contain everything starting from an empty database – CREATE statements, etc.?

Comment by Ilya Bovbel [ 2020-11-27 ]

elenst,
2GB.
No, not from the beginning.

Comment by Marko Mäkelä [ 2020-11-30 ]

bovbel.ilya, I am sorry, but I think that without starting from an empty database there is a possibility that the data is corrupted to begin with. Some months ago, I a dataset that triggered another type of corruption-related assertion failure. I posted my findings in MDEV-23565. Unfortunately, I cannot move forward with that one. I would have to know how the initial corruption emerged in the first place.

How long does it take to replay your compressed log? Does it repeat the crash reliably every time? How big is the database on which it could be applied to reproduce the corruption? Can any of it be reduced such that the corruption still occurs?

Comment by Marko Mäkelä [ 2020-12-28 ]

MDEV-24449 could explain this.

Generated at Thu Feb 08 09:27:45 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.