[MDEV-22431] IMPORT TABLESPACE, service restart Created: 2020-05-01  Updated: 2020-06-17  Resolved: 2020-06-17

Status: Closed
Project: MariaDB Server
Component/s: Data Definition - Alter Table, Storage Engine - InnoDB
Affects Version/s: 10.3.21, 10.3.22
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: Vucomir Ianculov Assignee: Marko Mäkelä
Resolution: Incomplete Votes: 0
Labels: corruption, need_feedback
Environment:

Ubuntu 18.04


Issue Links:
Duplicate
duplicates MDEV-21549 IMPORT TABLESPACE fails to adjust all... Closed
Relates
relates to MDEV-21407 Crash when restarting server after IM... Closed

 Description   

Hi,

we are restoring DB between 2 DB instances via IMPORT TABLESPACE, this work with no problems until updating to MariaDB 10.3.22 from 10.3.18, after a DB is restored and a service restart is done all DB will be corrupted, it's the same on 10.3.22 and 10.3.21 but on 10.3.20 everything is working no DB corrupted

if you don't restart MariaDB service everything will work normally

MariaDB 10.3.22 logs

2020-05-01 13:39:29 0x7fe83c33c700  InnoDB: Assertion failure in file /home/buildbot/buildbot/build/mariadb-10.3.22/storage/innobase/btr/btr0btr.cc line 205
InnoDB: Failing assertion: mach_read_from_4(seg_header + FSEG_HDR_SPACE) == space
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to https://jira.mariadb.org/
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: https://mariadb.com/kb/en/library/innodb-recovery-modes/
InnoDB: about forcing recovery.
200501 13:39:29 [ERROR] mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
 
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
 
Server version: 10.3.22-MariaDB-1:10.3.22+maria~bionic-log
key_buffer_size=134217728
read_buffer_size=131072
max_used_connections=2
max_threads=502
thread_count=9
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 2262804 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
 
Thread pointer: 0x7fe6a0007b58
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x7fe83c33bdd8 thread_stack 0x49000
/usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x558276a392ee]
/usr/sbin/mysqld(handle_fatal_signal+0x515)[0x5582764d1005]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x12890)[0x7fe851e74890]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0xc7)[0x7fe851388e97]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x141)[0x7fe85138a801]
/usr/sbin/mysqld(+0x4b429e)[0x55827621b29e]
/usr/sbin/mysqld(+0xa40650)[0x5582767a7650]
/usr/sbin/mysqld(+0xa41fd5)[0x5582767a8fd5]
/usr/sbin/mysqld(+0xa420c1)[0x5582767a90c1]
/usr/sbin/mysqld(+0x4b4acb)[0x55827621bacb]
/usr/sbin/mysqld(+0xac2a7c)[0x558276829a7c]
/usr/sbin/mysqld(+0xac2dd1)[0x558276829dd1]
/usr/sbin/mysqld(+0xab275b)[0x55827681975b]
/usr/sbin/mysqld(+0x9b9596)[0x558276720596]
/usr/sbin/mysqld(+0x902905)[0x558276669905]
/usr/sbin/mysqld(_Z15ha_delete_tableP3THDP10handlertonPKcPK25st_mysql_const_lex_stringS7_b+0x11a)[0x5582764d942a]
/usr/sbin/mysqld(_Z23mysql_rm_table_no_locksP3THDP10TABLE_LISTbbbbbb+0x501)[0x558276371991]
/usr/sbin/mysqld(+0x54f630)[0x5582762b6630]
/usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x3ef3)[0x5582762f3153]
/usr/sbin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_statebb+0x20a)[0x5582762f754a]
/usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcjbb+0x1c57)[0x5582762f9ec7]
/usr/sbin/mysqld(_Z10do_commandP3THD+0x178)[0x5582762fac88]
/usr/sbin/mysqld(_Z24do_handle_one_connectionP7CONNECT+0x33e)[0x5582763cd99e]
/usr/sbin/mysqld(handle_one_connection+0x3d)[0x5582763cda6d]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x76db)[0x7fe851e696db]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7fe85146b88f]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (0x7fe6a0019b60): DROP DATABASE IF EXISTS mydb
Connection ID (thread ID): 7239
Status: NOT_KILLED
 
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on
 
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
Writing a core file...
Working directory at /var/lib/mysql
Resource Limits:
Limit                     Soft Limit           Hard Limit           Units
Max cpu time              unlimited            unlimited            seconds
Max file size             unlimited            unlimited            bytes
Max data size             unlimited            unlimited            bytes
Max stack size            8388608              unlimited            bytes
Max core file size        0                    unlimited            bytes
Max resident set          unlimited            unlimited            bytes
Max processes             31411                31411                processes
Max open files            16364                16364                files
Max locked memory         16777216             16777216             bytes
Max address space         unlimited            unlimited            bytes
Max file locks            unlimited            unlimited            locks
Max pending signals       31411                31411                signals
Max msgqueue size         819200               819200               bytes
Max nice priority         0                    0
Max realtime priority     0                    0
Max realtime timeout      unlimited            unlimited            us
Core pattern: |/usr/share/apport/apport %p %s %c %d %P %E

MariaDB 10.3.21 logs

Version: '10.3.21-MariaDB-1:10.3.21+maria~bionic-log'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  mariadb.org binary distribution
2020-05-01 13:54:37 0x7fa60c85d700  InnoDB: Assertion failure in file /home/buildbot/buildbot/build/mariadb-10.3.21/storage/innobase/btr/btr0btr.cc line 205
InnoDB: Failing assertion: mach_read_from_4(seg_header + FSEG_HDR_SPACE) == space
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to https://jira.mariadb.org/
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: https://mariadb.com/kb/en/library/innodb-recovery-modes/
InnoDB: about forcing recovery.
200501 13:54:37 [ERROR] mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
 
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
 
Server version: 10.3.21-MariaDB-1:10.3.21+maria~bionic-log
key_buffer_size=134217728
read_buffer_size=131072
max_used_connections=1
max_threads=502
thread_count=8
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 2262808 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
 
Thread pointer: 0x7fa4880094f8
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x7fa60c85cdd8 thread_stack 0x49000
/usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x560bbdb2509e]
/usr/sbin/mysqld(handle_fatal_signal+0x515)[0x560bbd5bca95]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x12890)[0x7fa63513c890]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0xc7)[0x7fa634650e97]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x141)[0x7fa634652801]
/usr/sbin/mysqld(+0x4a5e2c)[0x560bbd306e2c]
/usr/sbin/mysqld(+0xa32140)[0x560bbd893140]
/usr/sbin/mysqld(+0xa33985)[0x560bbd894985]
/usr/sbin/mysqld(+0xa33a71)[0x560bbd894a71]
/usr/sbin/mysqld(+0x4a65cc)[0x560bbd3075cc]
/usr/sbin/mysqld(+0xab43c0)[0x560bbd9153c0]
/usr/sbin/mysqld(+0xab4771)[0x560bbd915771]
/usr/sbin/mysqld(+0xaa41cb)[0x560bbd9051cb]
/usr/sbin/mysqld(+0x8ee482)[0x560bbd74f482]
/usr/sbin/mysqld(_ZN7handler7ha_openEP5TABLEPKcijP11st_mem_rootP4ListI6StringE+0x4a)[0x560bbd5c14ca]
/usr/sbin/mysqld(_Z21open_table_from_shareP3THDP11TABLE_SHAREPK25st_mysql_const_lex_stringjjjP5TABLEbP4ListI6StringE+0x912)[0x560bbd48ef42]
/usr/sbin/mysqld(_Z10open_tableP3THDP10TABLE_LISTP18Open_table_context+0xc1f)[0x560bbd3804bf]
/usr/sbin/mysqld(_Z11open_tablesP3THDRK14DDL_options_stPP10TABLE_LISTPjjP19Prelocking_strategy+0xc13)[0x560bbd383673]
/usr/sbin/mysqld(_Z30open_normal_and_derived_tablesP3THDP10TABLE_LISTjj+0x4f)[0x560bbd3842bf]
/usr/sbin/mysqld(_Z18mysqld_list_fieldsP3THDP10TABLE_LISTPKc+0x38)[0x560bbd43b2d8]
/usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcjbb+0x24cc)[0x560bbd3e631c]
/usr/sbin/mysqld(_Z10do_commandP3THD+0x178)[0x560bbd3e6868]
/usr/sbin/mysqld(_Z24do_handle_one_connectionP7CONNECT+0x33e)[0x560bbd4b923e]
/usr/sbin/mysqld(handle_one_connection+0x3d)[0x560bbd4b930d]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x76db)[0x7fa6351316db]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7fa63473388f]
 
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (0x7fa488016fd0):
Connection ID (thread ID): 25
Status: NOT_KILLED
 
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on
 
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
Writing a core file...
Working directory at /var/lib/mysql
Resource Limits:
Limit                     Soft Limit           Hard Limit           Units
Max cpu time              unlimited            unlimited            seconds
Max file size             unlimited            unlimited            bytes
Max data size             unlimited            unlimited            bytes
Max stack size            8388608              unlimited            bytes
Max core file size        0                    unlimited            bytes
Max resident set          unlimited            unlimited            bytes
Max processes             31411                31411                processes
Max open files            16364                16364                files
Max locked memory         16777216             16777216             bytes
Max address space         unlimited            unlimited            bytes
Max file locks            unlimited            unlimited            locks
Max pending signals       31411                31411                signals
Max msgqueue size         819200               819200               bytes
Max nice priority         0                    0
Max realtime priority     0                    0
Max realtime timeout      unlimited            unlimited            us
Core pattern: |/usr/share/apport/apport %p %s %c %d %P %E

logs during import

Version: '10.3.21-MariaDB-1:10.3.21+maria~bionic-log'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  mariadb.org binary distribution
2020-05-01 13:49:58 0 [Note] InnoDB: Buffer pool(s) load completed at 200501 13:49:58
2020-05-01 13:51:34 92 [Note] InnoDB: Importing tablespace for table 'mydb/mytable' that was exported from host 'db02'
2020-05-01 13:51:34 92 [Note] InnoDB: Phase I - Update all pages
2020-05-01 13:51:36 92 [Note] InnoDB: Sync to disk
2020-05-01 13:51:36 92 [Note] InnoDB: Sync to disk - done!
2020-05-01 13:51:36 92 [Note] InnoDB: Phase III - Flush changes to disk
2020-05-01 13:51:36 92 [Note] InnoDB: Phase IV - Flush complete
2020-05-01 13:51:36 92 [Note] InnoDB: `mydb`.`mytable` autoinc value set to 1102812



 Comments   
Comment by Marko Mäkelä [ 2020-05-05 ]

This looks like a duplicate of MDEV-21549. Please test again, once the fix has been released.

Generated at Thu Feb 08 09:14:41 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.