Details
-
Bug
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Fixed
-
10.1.17
-
CentOS Linux release 7.2.1511 (Core)
Linux 10-30-1-12-db 3.10.0-327.el7.x86_64 #1 SMP Thu Nov 19 22:10:57 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
-
10.1.19
Description
We are planning on using MariaDB 10.1.17 with galera options enabled as backup in our DR setup. Initial startup went fine and loaded data we got through mysqldump. After loading the data, We made changes in my.cnf to enable binary logging and tries to restart the server. After issuing a restart, server won't start and it started throwing an error. Below is the error that was recorded in /var/log/messages
This is the first instance we are trying to start in cluster by issuing galera_new_cluster.
Oct 7 10:44:54 systemd: Starting MariaDB database server...
|
Oct 7 10:44:55 sh: 2016-10-07 10:44:55 140116692777088 [Warning] 'table-open-cache-instances' is MySQL 5.6 compatible option. Not used or needed in MariaDB.
|
Oct 7 10:44:55 sh: 2016-10-07 10:44:55 140116692777088 [Warning] option 'wsrep_max_ws_size': unsigned value 4294901759 adjusted to 2147483647
|
Oct 7 10:44:55 sh: 2016-10-07 10:44:55 140116692777088 [Note] /usr/sbin/mysqld (mysqld 10.1.17-MariaDB-enterprise) starting as process 6322 ...
|
Oct 7 10:44:59 sh: WSREP: Failed to start mysqld for wsrep recovery: '2016-10-07 10:44:55 140116692777088 [Note] InnoDB: Using mutexes to ref count buffer pool pages
|
Oct 7 10:44:59 sh: 2016-10-07 10:44:55 140116692777088 [Note] InnoDB: The InnoDB memory heap is disabled
|
Oct 7 10:44:59 sh: 2016-10-07 10:44:55 140116692777088 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
|
Oct 7 10:44:59 sh: 2016-10-07 10:44:55 140116692777088 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
|
Oct 7 10:44:59 sh: 2016-10-07 10:44:55 140116692777088 [Note] InnoDB: Compressed tables use zlib 1.2.7
|
Oct 7 10:44:59 sh: 2016-10-07 10:44:55 140116692777088 [Note] InnoDB: Using Linux native AIO
|
Oct 7 10:44:59 sh: 2016-10-07 10:44:55 140116692777088 [Note] InnoDB: Using SSE crc32 instructions
|
Oct 7 10:44:59 sh: 2016-10-07 10:44:55 140116692777088 [Note] InnoDB: Initializing buffer pool, size = 30.0G
|
Oct 7 10:44:59 sh: 2016-10-07 10:44:56 140116692777088 [Note] InnoDB: Completed initialization of buffer pool
|
Oct 7 10:44:59 sh: 2016-10-07 10:44:57 140116692777088 [ERROR] InnoDB: Space id in fsp header 138032,but in the page header 0
|
Oct 7 10:44:59 sh: 2016-10-07 10:44:57 140116692777088 [Note] InnoDB: Highest supported file format is Barracuda.
|
Oct 7 10:44:59 sh: 2016-10-07 10:44:57 140116692777088 [ERROR] InnoDB: Block in space_id 0 in file /mysql_data/prod_ec_cms2/data/ibdata1 encrypted.
|
Oct 7 10:44:59 sh: 2016-10-07 10:44:57 140116692777088 [ERROR] InnoDB: However key management plugin or used key_id 32 is not found or used encryption algorithm or method does not match.
|
Oct 7 10:44:59 sh: 2016-10-07 10:44:57 140116692777088 [ERROR] InnoDB: Marking tablespace as missing. You may drop this table or install correct key management plugin and key file.
|
Oct 7 10:44:59 sh: 2016-10-07 10:44:57 140116692777088 [ERROR] InnoDB: Block in space_id 0 in file /mysql_data/prod_ec_cms2/data/ibdata1 encrypted.
|
Oct 7 10:44:59 sh: 2016-10-07 10:44:57 140116692777088 [ERROR] InnoDB: However key management plugin or used key_id 32 is not found or used encryption algorithm or method does not match.
|
Oct 7 10:44:59 sh: 2016-10-07 10:44:57 140116692777088 [ERROR] InnoDB: Marking tablespace as missing. You may drop this table or install correct key management plugin and key file.
|
Oct 7 10:44:59 sh: 161007 10:44:57 [ERROR] mysqld got signal 11 ;
|
Oct 7 10:44:59 systemd: mariadb.service: control process exited, code=exited status=1
|
Oct 7 10:44:59 systemd: Failed to start MariaDB database server.
|
Oct 7 10:44:59 systemd: Unit mariadb.service entered failed state.
|
Oct 7 10:44:59 systemd: mariadb.service failed.
|
Oct 7 10:44:59 sh: This could be because you hit a bug. It is also possible that this binary
|
Oct 7 10:44:59 sh: or one of the libraries it was linked against is corrupt, improperly built,
|
Oct 7 10:44:59 sh: or misconfigured. This error can also be caused by malfunctioning hardware.
|
Oct 7 10:44:59 sh: To report this bug, see https://mariadb.com/kb/en/reporting-bugs
|
Oct 7 10:44:59 sh: We will try our best to scrape up some info that will hopefully help
|
Oct 7 10:44:59 sh: diagnose the problem, but since we have already crashed,
|
Oct 7 10:44:59 sh: something is definitely wrong and this may fail.
|
Oct 7 10:44:59 sh: Server version: 10.1.17-MariaDB-enterprise
|
Oct 7 10:44:59 sh: key_buffer_size=536870912
|
Oct 7 10:44:59 sh: read_buffer_size=131072
|
Oct 7 10:44:59 sh: max_used_connections=0
|
Oct 7 10:44:59 sh: max_threads=1001
|
Oct 7 10:44:59 sh: thread_count=0
|
Oct 7 10:44:59 sh: It is possible that mysqld could use up to
|
Oct 7 10:44:59 sh: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 2743562 K bytes of memory
|
Oct 7 10:44:59 sh: Hope that's ok; if not, decrease some variables in the equation.
|
Oct 7 10:44:59 sh: Thread pointer: 0x0x0
|
Oct 7 10:44:59 sh: Attempting backtrace. You can use the following information to find out
|
Oct 7 10:44:59 sh: where mysqld died. If you see no messages after this, something went
|
Oct 7 10:44:59 sh: terribly wrong...
|
Oct 7 10:44:59 sh: stack_bottom = 0x0 thread_stack 0x48400
|
Oct 7 10:44:59 sh: /usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x7f6f765e00ce]
|
Oct 7 10:44:59 sh: /usr/sbin/mysqld(handle_fatal_signal+0x2d5)[0x7f6f76106a05]
|
Oct 7 10:44:59 sh: /lib64/libpthread.so.0(+0xf100)[0x7f6f75724100]
|
Oct 7 10:44:59 sh: /usr/sbin/mysqld(+0x73653e)[0x7f6f7628b53e]
|
Oct 7 10:44:59 sh: /usr/sbin/mysqld(+0x88a285)[0x7f6f763df285]
|
Oct 7 10:44:59 sh: /usr/sbin/mysqld(+0x8a3136)[0x7f6f763f8136]
|
Oct 7 10:44:59 sh: mysys/stacktrace.c:268(my_print_stacktrace)[0x7f6f763d65b8]
|
Oct 7 10:44:59 sh: handler/ha_innodb.cc:22138(ib_push_warning(trx_t*, unsigned long, char const*, ...))[0x7f6f7638f92c]
|
Oct 7 10:44:59 sh: trx/trx0rseg.cc:212(trx_rseg_mem_create)[0x7f6f763823c8]
|
Oct 7 10:44:59 sh: trx/trx0rseg.cc:289(trx_rseg_create_instance)[0x7f6f763830b5]
|
Oct 7 10:44:59 sh: trx/trx0sys.cc:660(trx_sys_init_at_db_start())[0x7f6f7638466e]
|
Oct 7 10:44:59 sh: srv/srv0start.cc:2541(innobase_start_or_create_for_mysql())[0x7f6f76369cf5]
|
Oct 7 10:44:59 sh: handler/ha_innodb.cc:4396(innobase_init)[0x7f6f7628af25]
|
Oct 7 10:44:59 sh: sql/handler.cc:513(ha_initialize_handlerton(st_plugin_int*))[0x7f6f76108c74]
|
Oct 7 10:44:59 sh: sql/sql_plugin.cc:1408(plugin_initialize)[0x7f6f75f976b5]
|
Oct 7 10:44:59 sh: sql/sql_plugin.cc:1685(plugin_init(int*, char**, int))[0x7f6f75f9901a]
|
Oct 7 10:44:59 sh: sql/mysqld.cc:5188(init_server_components)[0x7f6f75ef150a]
|
Oct 7 10:44:59 sh: sql/mysqld.cc:5768(mysqld_main(int, char**))[0x7f6f75ef5623]
|
Oct 7 10:44:59 sh: /lib64/libc.so.6(__libc_start_main+0xf5)[0x7f6f73a6ab15]
|
Oct 7 10:44:59 sh: /usr/sbin/mysqld(+0x39404d)[0x7f6f75ee904d]
|
Oct 7 10:44:59 sh: The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
|
Oct 7 10:44:59 sh: information that should help you find out what is causing the crash.'
|
Attachments
Issue Links
- relates to
-
-
- Closed
-
-
-
- Closed
-
Activity
Do I understand correctly that you've never, at any point, turned on data encryption on the instance?
commit bc323727de312b32e80ae9590e2346414746a594
Author: Jan Lindström <jan.lindstrom@mariadb.com>
Date: Thu Oct 27 08:18:14 2016 +0300
MDEV-10977: [ERROR] InnoDB: Block in space_id 0 in file ibdata1 encrypted.MDEV-10394: Innodb system table space corruptedAnalysis: After we have read the page in buf_page_io_complete try to
find if the page is encrypted or corrupted. Encryption was determined
by reading FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION field from FIL-header
as a key_version. However, this field is not always zero even when
encryption is not used. Thus, incorrect key_version could lead situation where
decryption is tried to page that is not encrypted.
Fix: We still read key_version information from FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION
field but also check if tablespace has encryption information before trying
encrypt the page.