Details
-
Bug
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Fixed
-
10.1.13
-
None
-
10.1.19
Description
We're use mariadb 10.1.13 as a main database of our system.
We migrated data from develpment server to product server a week ago.
Below is our error case :
MariaDB shutdown with signal 6 when some query executes. (insert or update...)
The error log is below.
...
|
2016-07-18 9:44:14 139882584566528 [ERROR] InnoDB: Block in space_id 0 in file /data001/masvc01/ibdata1 encrypted.
|
2016-07-18 9:44:14 139882584566528 [ERROR] InnoDB: However key management plugin or used key_id 286 is not found or used encryption algorithm or method does not match.
|
2016-07-18 9:44:14 139882584566528 [ERROR] InnoDB: Marking tablespace as missing. You may drop this table or install correct key management plugin and key file.
|
2016-07-18 9:44:14 139882584566528 [ERROR] InnoDB: Block in space_id 0 in file /data001/masvc01/ibdata1 encrypted.
|
2016-07-18 9:44:14 139882584566528 [ERROR] InnoDB: However key management plugin or used key_id 286 is not found or used encryption algorithm or method does not match.
|
2016-07-18 9:44:14 139882584566528 [ERROR] InnoDB: Marking tablespace as missing. You may drop this table or install correct key management plugin and key file.
|
2016-07-18 09:44:14 7f38f3c36b00 InnoDB: Assertion failure in thread 139882584566528 in file buf0buf.cc line 4803
|
...
|
We check out system innodb table space with innochecksum.
The result is below.
/engn001/masvc01/mysql/bin/innochecksum -v ibdata1
|
|
|
InnoDB offline file checksum utility.
|
|
Variables (--variable-name=value)
|
and boolean options {FALSE|TRUE} Value (after reading options)
|
--------------------------------- ----------------------------------------
|
verbose TRUE
|
debug FALSE
|
skip-corrupt FALSE
|
count FALSE
|
start-page 0
|
end-page 0
|
page 0
|
per-page-details FALSE
|
leaf FALSE
|
merge 0
|
Table is uncompressed
|
Page size is 16384
|
file ibdata1_20160718 = 3221225472 bytes (196608 pages)...
|
InnoChecksum; checking pages in range 0 to 196607
|
Fail; page 0 invalid (fails innodb and crc32 checksum)
|
We hava ibdata1, ibdata2, ibdata3 and ibdata4 and results of innochecksum are same.
We compiled mariadb with debug option and test under same enviroment.
Below is the mysql.trace when db crash.
==========================================================================================================================
|
T@10 : | | | | | | | | | | | | | | | | | >print_buffer_to_file
|
T@10 : | | | | | | | | | | | | | | | | | | enter: buffer: InnoDB: Marking tablespace as missing. You may drop this table or install correct key management plugin and key file.
|
T@10 : | | | | | | | | | | | | | | | | | | mutex: LOCK_error_log (0x15b4b60) locking
|
T@10 : | | | | | | | | | | | | | | | | | | mutex: LOCK_error_log (0x15b4b60) locked
|
T@10 : | | | | | | | | | | | | | | | | | | mutex: LOCK_error_log (0x15b4b60) unlocking
|
T@10 : | | | | | | | | | | | | | | | | | <print_buffer_to_file
|
T@10 : | | | | | | | | | | | | | | | | <vprint_msg_to_log
|
T@10 : | | | | | | | | | | | | | | | <sql_print_error
|
T@10 : | | | | | | | | | | | | | | | >my_malloc
|
T@10 : | | | | | | | | | | | | | | | | my: size: 4096 my_flags: 16
|
T@10 : | | | | | | | | | | | | | | | | exit: ptr: 0x7f9e84159690
|
T@10 : | | | | | | | | | | | | | | | <my_malloc
|
T@10 : | | | | | | | | | | | | | | | >push_warning_printf
|
T@10 : | | | | | | | | | | | | | | | | enter: warning: 192
|
T@10 : | | | | | | | | | | | | | | | | >push_warning
|
T@10 : | | | | | | | | | | | | | | | | | enter: code: 192, msg: Table in tablespace 0 encrypted.However key management plugin or used key_id 290 is not found or used encryption algorithm or method does not match. Can't continue opening the table.
|
T@10 : | | | | | | | | | | | | | | | | | >THD::raise_condition
|
T@10 : | | | | | | | | | | | | | | | | | | >mysql_audit_acquire_plugins
|
T@10 : | | | | | | | | | | | | | | | | | | <mysql_audit_acquire_plugins
|
T@10 : | | | | | | | | | | | | | | | | | | mutex: l_perm->lock (0x5dcaeb00) locking
|
T@10 : | | | | | | | | | | | | | | | | | | >my_multi_malloc
|
T@10 : | | | | | | | | | | | | | | | | | | | >my_malloc
|
T@10 : | | | | | | | | | | | | | | | | | | | | my: size: 208 my_flags: 24
|
T@10 : | | | | | | | | | | | | | | | | | | | | exit: ptr: 0x7f9e8415a710
|
T@10 : | | | | | | | | | | | | | | | | | | | <my_malloc
|
T@10 : | | | | | | | | | | | | | | | | | | <my_multi_malloc
|
T@10 : | | | | | | | | | | | | | | | | | | >my_hash_init
|
T@10 : | | | | | | | | | | | | | | | | | | | enter: hash: 0x7f9e8415a710 size: 128
|
T@10 : | | | | | | | | | | | | | | | | | | | >init_dynamic_array2
|
T@10 : | | | | | | | | | | | | | | | | | | | | >my_malloc
|
So we export data from mariadb and import to new table space.
After migration, we shutdown mariadb and restart. but signal 11 occurred and db crash.
Below is our error log.
2016-07-19 8:55:09 140556344748800 [Note] /engn001/masvc01/mysql/bin/mysqld: Normal shutdown
|
|
|
2016-07-19 8:55:09 140556344748800 [Note] Event Scheduler: Killing the scheduler thread, thread id 1
|
2016-07-19 8:55:09 140556344748800 [Note] Event Scheduler: Waiting for the scheduler thread to reply
|
2016-07-19 8:55:09 140556344748800 [Note] Event Scheduler: Stopped
|
2016-07-19 8:55:09 140556344748800 [Note] Event Scheduler: Purging the queue. 0 events
|
160719 8:55:09 server_audit: STOPPED
|
2016-07-19 8:55:09 140556627830528 [Note] InnoDB: FTS optimize thread exiting.
|
2016-07-19 8:55:09 140556344748800 [Note] InnoDB: Starting shutdown...
|
2016-07-19 8:55:09 140556648810240 [Warning] InnoDB: Dumping buffer pool(s) to /data001/masvc01/ib_buffer_pool
|
2016-07-19 8:55:10 140556648810240 [Warning] InnoDB: Buffer pool(s) dump completed at 160719 8:55:10
|
2016-07-19 8:55:15 140556344748800 [Note] InnoDB: Shutdown completed; log sequence number 399145649446
|
2016-07-19 8:55:15 140556344748800 [Note] /engn001/masvc01/mysql/bin/mysqld: Shutdown complete
|
|
|
160719 08:55:16 mysqld_safe mysqld from pid file /engn001/masvc01/mysql/mysqld.pid ended
|
160719 08:57:05 mysqld_safe Starting mysqld daemon with databases from /data001/masvc01
|
2016-07-19 8:57:05 140304014350176 [Note] /engn001/masvc01/mysql/bin/mysqld (mysqld 10.1.14-MariaDB) starting as process 2608 ...
|
2016-07-19 8:57:06 140304014350176 [Note] InnoDB: Using mutexes to ref count buffer pool pages
|
2016-07-19 8:57:06 140304014350176 [Note] InnoDB: The InnoDB memory heap is disabled
|
2016-07-19 8:57:06 140304014350176 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
|
2016-07-19 8:57:06 140304014350176 [Note] InnoDB: Memory barrier is not used
|
2016-07-19 8:57:06 140304014350176 [Note] InnoDB: Compressed tables use zlib 1.2.3
|
2016-07-19 8:57:06 140304014350176 [Note] InnoDB: Using Linux native AIO
|
2016-07-19 8:57:06 140304014350176 [Note] InnoDB: Using SSE crc32 instructions
|
2016-07-19 8:57:06 140304014350176 [Note] InnoDB: Initializing buffer pool, size = 24.0G
|
2016-07-19 8:57:08 140304014350176 [Note] InnoDB: Completed initialization of buffer pool
|
2016-07-19 8:57:10 140304014350176 [Note] InnoDB: Highest supported file format is Barracuda.
|
2016-07-19 8:57:13 140304014350176 [Note] InnoDB: 128 rollback segment(s) are active.
|
2016-07-19 8:57:13 140304014350176 [Note] InnoDB: Waiting for purge to start
|
2016-07-19 8:57:13 140304014350176 [Note] InnoDB: Percona XtraDB (http://www.percona.com) 5.6.29-76.2 started; log sequence number 399145649446
|
2016-07-19 8:57:13 140273638147840 [Note] InnoDB: Dumping buffer pool(s) not yet started
|
2016-07-19 08:57:13 7f94005f8700 InnoDB: Loading buffer pool(s) from /data001/masvc01/ib_buffer_pool
|
2016-07-19 8:57:13 140304014350176 [Note] Plugin 'FEEDBACK' is disabled.
|
160719 8:57:13 server_audit: MariaDB Audit Plugin version 1.4.0 STARTED.
|
2016-07-19 8:57:13 140304014350176 [Note] Server socket created on IP: '::'.
|
2016-07-19 8:57:13 140304014350176 [Warning] 'user' entry 'root@lgedglap02v' ignored in --skip-name-resolve mode.
|
2016-07-19 8:57:13 140304014350176 [Warning] 'user' entry '@lgedglap02v' ignored in --skip-name-resolve mode.
|
2016-07-19 8:57:13 140304014350176 [Warning] 'proxies_priv' entry '@% root@lgedglap02v' ignored in --skip-name-resolve mode.
|
2016-07-19 8:57:13 140273595853568 [Note] Event Scheduler: scheduler thread started with id 1
|
2016-07-19 8:57:13 140304014350176 [Note] Reading of all Master_info entries succeded
|
2016-07-19 8:57:13 140304014350176 [Note] Added new Master_info '' to hash table
|
2016-07-19 8:57:13 140304014350176 [Note] /engn001/masvc01/mysql/bin/mysqld: ready for connections.
|
Version: '10.1.14-MariaDB' socket: '/engn001/masvc01/mysql/mysqld.sock' port: 3310 MariaDB Server
|
2016-07-19 8:57:47 140273638147840 [ERROR] InnoDB: Block in space_id 0 in file /data001/masvc01/ibdata1 encrypted.
|
2016-07-19 8:57:47 140273638147840 [ERROR] InnoDB: However key management plugin or used key_id 92 is not found or used encryption algorithm or method does not match.
|
2016-07-19 8:57:47 140273638147840 [ERROR] InnoDB: Marking tablespace as missing. You may drop this table or install correct key management plugin and key file.
|
2016-07-19 8:57:47 140273638147840 [ERROR] InnoDB: Block in space_id 0 in file /data001/masvc01/ibdata1 encrypted.
|
2016-07-19 8:57:47 140273638147840 [ERROR] InnoDB: However key management plugin or used key_id 92 is not found or used encryption algorithm or method does not match.
|
2016-07-19 8:57:47 140273638147840 [ERROR] InnoDB: Marking tablespace as missing. You may drop this table or install correct key management plugin and key file.
|
160719 8:57:47 [ERROR] mysqld got signal 11 ;
|
This could be because you hit a bug. It is also possible that this binary
|
or one of the libraries it was linked against is corrupt, improperly built,
|
or misconfigured. This error can also be caused by malfunctioning hardware.
|
|
|
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
|
|
|
We will try our best to scrape up some info that will hopefully help
|
diagnose the problem, but since we have already crashed,
|
something is definitely wrong and this may fail.
|
|
|
Server version: 10.1.14-MariaDB
|
key_buffer_size=67108864
|
read_buffer_size=50331648
|
max_used_connections=3
|
max_threads=102
|
thread_count=4
|
It is possible that mysqld could use up to
|
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 10094612 K bytes of memory
|
Hope that's ok; if not, decrease some variables in the equation.
|
|
|
Thread pointer: 0x0x0
|
Attempting backtrace. You can use the following information to find out
|
where mysqld died. If you see no messages after this, something went
|
terribly wrong...
|
stack_bottom = 0x0 thread_stack 0x48400
|
/engn001/masvc01/mysql/bin/mysqld(my_print_stacktrace+0x2e)[0xc028ae]
|
/engn001/masvc01/mysql/bin/mysqld(handle_fatal_signal+0x464)[0x7651f4]
|
/lib64/libpthread.so.0[0x3ffe40f7e0]
|
/engn001/masvc01/mysql/bin/mysqld[0x8fa6d3]
|
/engn001/masvc01/mysql/bin/mysqld[0xa3ba47]
|
/engn001/masvc01/mysql/bin/mysqld[0xa51a2e]
|
/engn001/masvc01/mysql/bin/mysqld[0xa52bc9]
|
/engn001/masvc01/mysql/bin/mysqld[0xa433ff]
|
/engn001/masvc01/mysql/bin/mysqld[0xa440b7]
|
/lib64/libpthread.so.0[0x3ffe407aa1]
|
/lib64/libc.so.6(clone+0x6d)[0x3ffe0e893d]
|
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
|
information that should help you find out what is causing the crash.
|
.....
|
|
I can't find solution for it. I attached my.cnf.
Is it a bug or hardware fault ?
Attachments
Issue Links
- includes
-
-
- Closed
-
- relates to
-
-
- Closed
-
Activity
>> First, could you please clarify the versions? In 'Affects version/s' you put 10.1.13, but your description starts with you using 10.1.3; and the error log is >> from 10.1.14.
>> Is 10.1.3 a typo, or did you actually migrate from 10.1.3 to 10.1.13/14?
I mistyped. correct version is 10.1.13 and We tested with 10.1.13 and 10.1.14.
So above error log is mariadb 10.1.14's log.
>> Further, when you say
>> So we export data from mariadb and import to new table space.
>> After migration, we shutdown mariadb and restart.
>> what do you mean by that? Do you export/import one table only? Can you specify what exactly you did to import data to a new table space (how you made sure it's really new)?
We reinstalled mariadb and export our database from test server and imported to new mariadb.
>>The error log insists you have something encrypted, but your has no sign of it. Are you using, or have you ever used, any encryption on this server?
No, I attaced my,cnf file. We don't use any encryption option.
Finally, We downgraded mariadb version from 10.1.13 to 10.0.22 and DB doesn't crash any more under same test.
I assume that it's bug of mariadb 10.1.xx .
Thanks.
Yes, it must be a 10.1-specific problem, since 10.0 does not have encryption, which is clearly involved here – somehow InnoDB (mis)interprets the data as being encrypted.
When you just re-install MariaDB, the datadir is preserved, so unless you removed it (or InnoDB files from it) manually, it's possible that even after re-installation the server was still using the old (corrupted?) system table space.
Do you have that data stored, by any chance? I mean the InnoDB files itself, not the data dump.
>>When you just re-install MariaDB, the datadir is preserved, so unless you removed it (or InnoDB files from it) manually, it's >>possible that even after re-installation the server was still using the old (corrupted?) system table space.
>>Do you have that data stored, by any chance? I mean the InnoDB files itself, not the data dump.
When I reinstalled MariaDB, I changed datadir to another position.
Also I installed MariaDB another server and got same result under test.
Our test is very simple.
1. Install MariaDB 10.1.13 or 10.1.14
2. Start MariaDB
3. Shutdown MariaDB
4. Check innodb tablespace by innochecksum. the result is ok.
5. Start MariaDB
6. Import database from our backup file which is created by mysqldump. ( about 15G)
7. Shutdown MariaDB. After shutdown, there is no error message. it's normal shutdown.
8. Check innodb tablespace by innochecksum. the result is below.
--> Fail; page 0 invalid (fails innodb and crc32 checksum)
So we changed MariaDB 10.1.13 to 10.0.22 and do same test.
There is no invalid page in innodb tablespace with 10.0.22
Thanks
windfree, thanks for clarification.
Would you be able to provide this data dump? You could upload it to our ftp.askmonty.org/private, this way only MariaDB developers will have access to it.
15 GB is a lot, but we would really want to find the reason of the problem, and if it's reliably reproducible with your dump, it might be our real chance.
Besides, if it's 15 GB uncompressed, it should be much less in an archive, dumps usually compress very well.
Thanks for response.
Ok, I'll check whether it is possible or not.
I can't real data because of our client's security policy.
But I made a test data and do same test and found a innodb system table space corrupted.
I uploaded our test files to ftp.askmonty.org/private. ( mdev-10394-bugtest-schema.sql, mdev-10394-data.zip ) .
Below is my test scenario.
1. create dababase .
2. create table using mdev-10394-bugtest-schema.sql.
3. bin/mysql --defaults-file=./my.cnf < data.sql ( bugtest table's row count is 7388877)
4. shutdown database.
5. check innodb table space. ( bin/innochecksum data/ibdata1 .
6 . At this time, innodb tablespace has no problem.
7. start database.
8. execute step3 again --> bin/mysql --defaults-file=./my.cnf < data.sql ( bugtest table's row count is twice of 14777754 )
9 .shutdown databse.
10. check innodb table space. ( bin/innochecksum data/ibdata1 .
11. Now, innodb tablespace corrupted. ( message - > Fail; page 0 invalid (fails innodb and crc32 checksum))
I wanna know the reason of this bug.
Mr. Kim,
With the test scenario described above, are you also getting the InnoDB complaints and assertion failure which you had in the description?
I've been trying to reproduce the issue, and with your test I'm getting exactly what you said in the previous comment, innochecksum complaining about the tablespace being corrputed; but further attempt to start the server again and to use the table does not cause any visible problem, no InnoDB errors in the log, no assertion, and InnoDB is able to continue working with the table. So, while innochecksum complaint definitely needs to be investigated, I think maybe there is something more to it when it comes to the original problem.
Yes, We also getting the InnoDB assertion failure error. In that case, server started sucessfuly while having innodb table space corruption. but the server crashed with followed error log as I mentioned.
...........................................................................
2016-07-18 9:44:14 139882584566528 [ERROR] InnoDB: Block in space_id 0 in file /data001/masvc01/ibdata1 encrypted.
2016-07-18 9:44:14 139882584566528 [ERROR] InnoDB: However key management plugin or used key_id 286 is not found or used encryption algorithm or method does not match.
2016-07-18 9:44:14 139882584566528 [ERROR] InnoDB: Marking tablespace as missing. You may drop this table or install correct key management plugin and key file.
2016-07-18 09:44:14 7f38f3c36b00 InnoDB: Assertion failure in thread 139882584566528 in file buf0buf.cc line 4803
.......................................................
Hi,
Can you try with 10.1.16, there was some bad bugs on 10.1.13.
I got the same error on current 10.1. The difference is that instead of an assertion failure I'm getting SIGSEGV.
2016-09-05 16:08:20 140463950751616 [ERROR] InnoDB: Space id in fsp header 131344,but in the page header 0
|
2016-09-05 16:08:20 140463950751616 [Note] InnoDB: Highest supported file format is Barracuda.
|
2016-09-05 16:08:21 140463950751616 [ERROR] InnoDB: Block in space_id 0 in file ./ibdata1 encrypted.
|
2016-09-05 16:08:21 140463950751616 [ERROR] InnoDB: However key management plugin or used key_id 5 is not found or used encryption algorithm or method does not match.
|
2016-09-05 16:08:21 140463950751616 [ERROR] InnoDB: Marking tablespace as missing. You may drop this table or install correct key management plugin and key file.
|
2016-09-05 16:08:21 140463950751616 [ERROR] InnoDB: Block in space_id 0 in file ./ibdata1 encrypted.
|
2016-09-05 16:08:21 140463950751616 [ERROR] InnoDB: However key management plugin or used key_id 5 is not found or used encryption algorithm or method does not match.
|
2016-09-05 16:08:21 140463950751616 [ERROR] InnoDB: Marking tablespace as missing. You may drop this table or install correct key management plugin and key file.
|
160905 16:08:21 [ERROR] mysqld got signal 11 ;
|
|
Stack trace from 10.1 747893a85451 |
Thread 1 (Thread 0x7fc04fe36780 (LWP 22646)):
|
#0 0x00007fc04e74af8c in pthread_kill () from /lib/x86_64-linux-gnu/libpthread.so.0
|
#1 0x00007fc0503e6005 in handle_fatal_signal (sig=11) at /home/elenst/mdev10499/10.1.16/sql/signal_handler.cc:294
|
#2 <signal handler called>
|
#3 ib_push_warning (trx=0x0, error=error@entry=67, format=format@entry=0x7fc05094b030 "Table in tablespace %lu encrypted.However key management plugin or used key_id %u is not found or used encryption algorithm or method does not match. Can't continue opening the table.") at /home/elenst/mdev10499/10.1.16/storage/xtradb/handler/ha_innodb.cc:22138
|
#4 0x00007fc05072bfc6 in buf_page_io_complete (bpage=bpage@entry=0x7fbf73af95c0) at /home/elenst/mdev10499/10.1.16/storage/xtradb/buf/buf0buf.cc:4817
|
#5 0x00007fc0507411d8 in buf_read_page_low (err=err@entry=0x7ffffd8e790c, sync=sync@entry=true, mode=mode@entry=132, space=space@entry=0, zip_size=zip_size@entry=0, unzip=unzip@entry=0, tablespace_version=1, offset=offset@entry=320, trx=trx@entry=0x0, rbpage=rbpage@entry=0x7ffffd8e79c8) at /home/elenst/mdev10499/10.1.16/storage/xtradb/buf/buf0rea.cc:262
|
#6 0x00007fc050741f20 in buf_read_page (space=space@entry=0, zip_size=zip_size@entry=0, offset=offset@entry=320, trx=trx@entry=0x0, bpage=bpage@entry=0x7ffffd8e79c8) at /home/elenst/mdev10499/10.1.16/storage/xtradb/buf/buf0rea.cc:474
|
#7 0x00007fc0507236bf in buf_page_get_gen (space=<optimised out>, zip_size=<optimised out>, offset=offset@entry=320, rw_latch=rw_latch@entry=2, guess=<optimised out>, guess@entry=0x0, mode=mode@entry=10, file=file@entry=0x7fc050944f28 "/home/elenst/mdev10499/10.1.16/storage/xtradb/include/trx0undo.ic", line=line@entry=171, mtr=mtr@entry=0x7ffffd8e7ba0, err=err@entry=0x0) at /home/elenst/mdev10499/10.1.16/storage/xtradb/buf/buf0buf.cc:2962
|
#8 0x00007fc0506e7750 in trx_undo_page_get (mtr=0x7ffffd8e7ba0, page_no=320, zip_size=<optimised out>, space=<optimised out>) at /home/elenst/mdev10499/10.1.16/storage/xtradb/include/trx0undo.ic:170
|
#9 trx_undo_mem_create_at_db_start (mtr=0x7ffffd8e7ba0, page_no=320, id=0, rseg=0x7fbf1de12268) at /home/elenst/mdev10499/10.1.16/storage/xtradb/trx/trx0undo.cc:1292
|
#10 trx_undo_lists_init (rseg=rseg@entry=0x7fbf1de12268) at /home/elenst/mdev10499/10.1.16/storage/xtradb/trx/trx0undo.cc:1424
|
#11 0x00007fc0506db057 in trx_rseg_mem_create (id=id@entry=82, space=<optimised out>, zip_size=<optimised out>, page_no=page_no@entry=255, ib_bh=ib_bh@entry=0x7fc04d05d400, mtr=mtr@entry=0x7ffffd8e81b0) at /home/elenst/mdev10499/10.1.16/storage/xtradb/trx/trx0rseg.cc:209
|
#12 0x00007fc0506dba3c in trx_rseg_create_instance (mtr=0x7ffffd8e81b0, ib_bh=0x7fc04d05d400, sys_header=0x7ffffd8e81b0 "") at /home/elenst/mdev10499/10.1.16/storage/xtradb/trx/trx0rseg.cc:287
|
#13 trx_rseg_array_init (sys_header=sys_header@entry=0x7fc03c3dc026 "", ib_bh=ib_bh@entry=0x7fc04d05d400, mtr=mtr@entry=0x7ffffd8e81b0) at /home/elenst/mdev10499/10.1.16/storage/xtradb/trx/trx0rseg.cc:358
|
#14 0x00007fc0506dd33e in trx_sys_init_at_db_start () at /home/elenst/mdev10499/10.1.16/storage/xtradb/trx/trx0sys.cc:660
|
#15 0x00007fc0506c4ca5 in innobase_start_or_create_for_mysql () at /home/elenst/mdev10499/10.1.16/storage/xtradb/srv/srv0start.cc:2540
|
#16 0x00007fc0505fe211 in innobase_init (p=<optimised out>) at /home/elenst/mdev10499/10.1.16/storage/xtradb/handler/ha_innodb.cc:4394
|
#17 0x00007fc0503e8471 in ha_initialize_handlerton (plugin=0x7fc04d108790) at /home/elenst/mdev10499/10.1.16/sql/handler.cc:513
|
#18 0x00007fc050290d48 in plugin_initialize (tmp_root=tmp_root@entry=0x7ffffd8ec650, plugin=plugin@entry=0x7fc04d108790, argc=argc@entry=0x7fc051091fb8 <remaining_argc>, argv=argv@entry=0x7fc04d02ba58, options_only=options_only@entry=false) at /home/elenst/mdev10499/10.1.16/sql/sql_plugin.cc:1408
|
#19 0x00007fc050291e8a in plugin_init (argc=argc@entry=0x7fc051091fb8 <remaining_argc>, argv=0x7fc04d02ba58, flags=2) at /home/elenst/mdev10499/10.1.16/sql/sql_plugin.cc:1686
|
#20 0x00007fc0501fbea2 in init_server_components () at /home/elenst/mdev10499/10.1.16/sql/mysqld.cc:5190
|
#21 0x00007fc050200d0f in mysqld_main (argc=50, argv=0x7fc04d02ba58) at /home/elenst/mdev10499/10.1.16/sql/mysqld.cc:5768
|
#22 0x00007fc04dda076d in __libc_start_main () from /lib/x86_64-linux-gnu/libc.so.6
|
#23 0x00007fc0501f4ef9 in _start ()
|
My scenario was simple, but time-consuming, so it will take some time to retry.
- i started the server with some non-default options (no encryption, no encryption plugin) on a clean datadir
- created one table
- populated it with some 120M rows
- shut down the server with normal shutdown
- tried to restart the server with the same configuration
=> crash (persistent)
I just got the same error after upgrading on ubuntu from 10.1.17+maria-1~trusty to 10.1.18+maria-1~trusty.
Restart gives the
2016-10-19 21:43:18 139834710702848 [ERROR] InnoDB: Block in space_id 0 in file ./ibdata1 encrypted.
|
2016-10-19 21:43:18 139834710702848 [ERROR] InnoDB: However key management plugin or used key_id 5808 is not found or used encryption algorithm or method does not match.
|
2016-10-19 21:43:18 139834710702848 [ERROR] InnoDB: Marking tablespace as missing. You may drop this table or install correct key management plugin and key file.
|
Anything I can provide to track down the issue and maybe solve this?
Is the tablespace now really corrupted and I need to rebuild the database server or any solution?
commit bc323727de312b32e80ae9590e2346414746a594
Author: Jan Lindström <jan.lindstrom@mariadb.com>
Date: Thu Oct 27 08:18:14 2016 +0300
MDEV-10977: [ERROR] InnoDB: Block in space_id 0 in file ibdata1 encrypted.
MDEV-10394: Innodb system table space corrupted
Analysis: After we have read the page in buf_page_io_complete try to
find if the page is encrypted or corrupted. Encryption was determined
by reading FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION field from FIL-header
as a key_version. However, this field is not always zero even when
encryption is not used. Thus, incorrect key_version could lead situation where
decryption is tried to page that is not encrypted.
Fix: We still read key_version information from FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION
field but also check if tablespace has encryption information before trying
encrypt the page.
windfree,
First, could you please clarify the versions? In 'Affects version/s' you put 10.1.13, but your description starts with you using 10.1.3; and the error log is from 10.1.14.
Is 10.1.3 a typo, or did you actually migrate from 10.1.3 to 10.1.13/14?
Further, when you say
what do you mean by that? Do you export/import one table only? Can you specify what exactly you did to import data to a new table space (how you made sure it's really new)?
The error log insists you have something encrypted, but your has no sign of it. Are you using, or have you ever used, any encryption on this server?
Thanks.