[MDEV-18095] InnoDB assertion failure when access table Created: 2018-12-28  Updated: 2019-02-25  Resolved: 2019-02-25

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - InnoDB
Affects Version/s: 10.3.8
Fix Version/s: N/A

Type: Bug Priority: Critical
Reporter: Ze Yang Assignee: Unassigned
Resolution: Incomplete Votes: 0
Labels: need_feedback

Issue Links:
Relates
relates to MDEV-11369 Instant add column for InnoDB Closed
relates to MDEV-17483 Insert on delete-marked record can wr... Closed
relates to MDEV-17881 Assertion failure in cmp_dtuple_rec_w... Closed

 Description   

The mysqld crash when user access the table.
The assert line(storage/innobase/rem/rem0rec.cc line 1883) is:

1881   case REC_STATUS_COLUMNS_ADDED:
1882     /* We would have !index->is_instant() when rolling back
1883     an instant ADD COLUMN operation. */
1884     ut_ad(index->is_instant() || page_rec_is_default_row(rec));
1885     nulls = &rec[-REC_N_NEW_EXTRA_BYTES];
1886     const ulint n_rec = index->n_core_fields + 1
1887       + rec_get_n_add_field(nulls);
1888     const uint n_nullable = index->get_n_nullable(n_rec);
1889     lens = --nulls - UT_BITS_IN_BYTES(n_nullable);

I dont know how to get the line code by addr2line for MariaDB. The address has offset.
you could tell me the detail method to get the line code. Then I could give the detail line in backstrace.

2018-12-28 01:38:00 0x7fe92236e700 InnoDB: Assertion failure in file /home/admin/100_20180827173548855_57140248_code/rpm_workspace/storage/innobase/rem/rem0rec.cc line 1883
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to https://jira.mariadb.org/
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: https://mariadb.com/kb/en/library/xtradbinnodb-recovery-modes/
InnoDB: about forcing recovery.
181228 1:38:00 [ERROR] mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
 
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
 
Server version: 10.3.8-MariaDB-log
key_buffer_size=16777216
read_buffer_size=1048576
max_used_connections=26
max_threads=8534
thread_count=40
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 26424424 K bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
 
Thread pointer: 0x7fe8300009a8
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x7fe92236ddb0 thread_stack 0x40000
*** buffer overflow detected ***: /u01/mariadb/bin/mysqld terminated
 
 
 
/u01/mariadb/bin/mysqld(handle_fatal_signal+0x357)[0x7fbd22c334b7]
/lib64/libpthread.so.0(+0xf100)[0x7fbd22080100]
/lib64/libc.so.6(gsignal+0x37)[0x7fbd204065f7]
/lib64/libc.so.6(abort+0x148)[0x7fbd20407ce8]
/u01/mariadb/bin/mysqld(+0x4cc532)[0x7fbd2297d532]
/u01/mariadb/bin/mysqld(+0xad6093)[0x7fbd22f87093]
/u01/mariadb/bin/mysqld(+0xbb3068)[0x7fbd23064068]
/u01/mariadb/bin/mysqld(+0xb1a056)[0x7fbd22fcb056]
/u01/mariadb/bin/mysqld(+0xa46a29)[0x7fbd22ef7a29]
/u01/mariadb/bin/mysqld(_ZN7handler13ha_index_prevEPh+0x19f)[0x7fbd22c38f8f]
/u01/mariadb/bin/mysqld(+0x5ef93c)[0x7fbd22aa093c]
/u01/mariadb/bin/mysqld(_Z10sub_selectP4JOINP13st_join_tableb+0x20f)[0x7fbd22a95c7f]
/u01/mariadb/bin/mysqld(_ZN4JOIN10exec_innerEv+0xa42)[0x7fbd22ab7332]
/u01/mariadb/bin/mysqld(_ZN4JOIN4execEv+0x33)[0x7fbd22ab7583]
/u01/mariadb/bin/mysqld(_Z12mysql_selectP3THDP10TABLE_LISTjR4ListI4ItemEPS4_jP8st_orderS9_S7_S9_yP13select_resultP18st_select_lex_unitP13st_select_lex+0x11a)[0x7fbd22ab76da]
/u01/mariadb/bin/mysqld(_Z13handle_selectP3THDP3LEXP13select_resultm+0x1bc)[0x7fbd22ab81dc]
/u01/mariadb/bin/mysqld(+0x4bd66f)[0x7fbd2296e66f]
/u01/mariadb/bin/mysqld(_Z21mysql_execute_commandP3THD+0x71ac)[0x7fbd22a6493c]
/u01/mariadb/bin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_statebb+0x202)[0x7fbd22a66a12]
/u01/mariadb/bin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcjbb+0x1bbb)[0x7fbd22a694db]
/u01/mariadb/bin/mysqld(_Z10do_commandP3THD+0x13e)[0x7fbd22a6a44e]
/u01/mariadb/bin/mysqld(_Z24do_handle_one_connectionP7CONNECT+0x1aa)[0x7fbd22b3bbba]
/u01/mariadb/bin/mysqld(handle_one_connection+0x3d)[0x7fbd22b3bcdd]
/lib64/libpthread.so.0(+0x7dc5)[0x7fbd22078dc5]
/lib64/libc.so.6(clone+0x6d)[0x7fbd204c7ced]
 
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.



 Comments   
Comment by Marko Mäkelä [ 2018-12-28 ]

This looks related to MDEV-11369 (instant ADD COLUMN).

Since the MariaDB Server 10.3.8 release, a number of bugs that were caused by MDEV-11369 have been fixed. Two of the fixes affect DML code paths; others are for DDL. The DML fixes (MDEV-17483 and MDEV-17881) could explain the corruption that you are facing.

It might be possible to debug this if you know which table is causing this, and have a log of DDL and DML statements that have been executed on the table.

Comment by Elena Stepanova [ 2018-12-28 ]

yangze,

For starters, would it be possible for you to upgrade to the latest released 10.3 version and see if the problem still occurs there?

Comment by Ze Yang [ 2018-12-29 ]

There are 3200 tables with same schema created in different databases.
All of them were created and then add one 'text' column in 8 days ago.
One of them was inserted one row last day. And then when user access this table, the server crash as description in issue.

Comment by Elena Stepanova [ 2019-01-28 ]

yangze,

Did it happen on the latest 10.3, or are you still running 10.3.8?

Generated at Thu Feb 08 08:41:28 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.