Details
-
Bug
-
Status: Closed (View Workflow)
-
Critical
-
Resolution: Fixed
-
10.1.8
-
Linux 4.2.0-0.bpo.1-amd64 #1 SMP Debian 4.2.3-2~bpo8+1 (2015-10-20) x86_64 GNU/Linux
Debian Jessie 8.2
-
10.1.9-2
Description
We are migrating a bunch of databases from Percona 5.5.33 to MariaDB 10.1.8. Unfortunately, a lot of them crash during startup.
We do hot backups of the databases with LVM snapshots (flush tables with read lock, make snapshot, tar -cf the snapshot). We move the backup to a different server and extract it.
MariaDB 5.5.46 works fine with the backup
MariaDB 10.0.21 works fine with the backup
MariaDB 10.1.8 crashes
I've tried going through Percona 5.5 -> MariaDB 5.5.46 -> MariaDB 10.0.21 -> MariaDB 10.1.8 with full mysql_upgrade's and clean shutdowns, but it would still crash on startup with 10.1.8.
Looks similar to https://mariadb.atlassian.net/browse/MDEV-8672
Here are the logs from MariaDB:
2015-10-28 12:09:23 139878086616960 [Note] /usr/local/mariadb10.1/bin/mysqld (mysqld 10.1.8-MariaDB-log) starting as process 37700 ...
|
2015-10-28 12:09:23 139878086616960 [Note] InnoDB: Using mutexes to ref count buffer pool pages
|
2015-10-28 12:09:23 139878086616960 [Note] InnoDB: The InnoDB memory heap is disabled
|
2015-10-28 12:09:23 139878086616960 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
|
2015-10-28 12:09:23 139878086616960 [Note] InnoDB: Memory barrier is not used
|
2015-10-28 12:09:23 139878086616960 [Note] InnoDB: Compressed tables use zlib 1.2.3
|
2015-10-28 12:09:23 139878086616960 [Note] InnoDB: Using Linux native AIO
|
2015-10-28 12:09:23 139878086616960 [Note] InnoDB: Using CPU crc32 instructions
|
2015-10-28 12:09:23 139878086616960 [Note] InnoDB: Initializing buffer pool, size = 10.0G
|
2015-10-28 12:09:27 139878086616960 [Note] InnoDB: Completed initialization of buffer pool
|
2015-10-28 12:09:27 139878086616960 [Note] InnoDB: Highest supported file format is Barracuda.
|
2015-10-28 12:09:27 139878086616960 [Note] InnoDB: The log sequence numbers 17122163212 and 17122163212 in ibdata files do not match the log sequence number 17122164830 in the ib_logfiles!
|
2015-10-28 12:09:27 139878086616960 [Note] InnoDB: Database was not shutdown normally!
|
2015-10-28 12:09:27 139878086616960 [Note] InnoDB: Starting crash recovery.
|
2015-10-28 12:09:27 139878086616960 [Note] InnoDB: Reading tablespace information from the .ibd files...
|
2015-10-28 12:09:30 139878086616960 [Note] InnoDB: Restoring possible half-written data pages
|
2015-10-28 12:09:30 139878086616960 [Note] InnoDB: from the doublewrite buffer...
|
InnoDB: In a MySQL replication slave the last master binlog file
|
InnoDB: position 0 1716444, file name mysql-bin.020607
|
2015-10-28 12:09:30 139878086616960 [Note] InnoDB: 128 rollback segment(s) are active.
|
2015-10-28 12:09:30 139878086616960 [Note] InnoDB: Waiting for purge to start
|
2015-10-28 12:09:30 7f348a3fc700 InnoDB: Assertion failure in thread 139863634462464 in file btr0cur.cc line 5341
|
InnoDB: Failing assertion: rb_ctx != RB_NONE
|
InnoDB: We intentionally generate a memory trap.
|
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
|
InnoDB: If you get repeated assertion failures or crashes, even
|
InnoDB: immediately after the mysqld startup, there may be
|
InnoDB: corruption in the InnoDB tablespace. Please refer to
|
InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
|
InnoDB: about forcing recovery.
|
151028 12:09:30 [ERROR] mysqld got signal 6 ;
|
This could be because you hit a bug. It is also possible that this binary
|
or one of the libraries it was linked against is corrupt, improperly built,
|
or misconfigured. This error can also be caused by malfunctioning hardware.
|
|
To report this bug, see http://kb.askmonty.org/en/reporting-bugs
|
|
We will try our best to scrape up some info that will hopefully help
|
diagnose the problem, but since we have already crashed,
|
something is definitely wrong and this may fail.
|
|
Server version: 10.1.8-MariaDB-log
|
key_buffer_size=2147483648
|
read_buffer_size=4194304
|
max_used_connections=0
|
max_threads=502
|
thread_count=0
|
It is possible that mysqld could use up to
|
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 6219767 K bytes of memory
|
Hope that's ok; if not, decrease some variables in the equation.
|
|
Thread pointer: 0x0x0
|
Attempting backtrace. You can use the following information to find out
|
where mysqld died. If you see no messages after this, something went
|
terribly wrong...
|
stack_bottom = 0x0 thread_stack 0x40000
|
|
015-10-28 12:09:30 139878086616960 [Note] InnoDB: Percona XtraDB (http://www.percona.com) 5.6.26-74.0 started; log sequence number 17122164830
|
2015-10-28 12:09:30 139863609284352 [Note] InnoDB: Dumping buffer pool(s) not yet started
|
2015-10-28 12:09:30 139878086616960 [Note] Plugin 'FEEDBACK' is disabled.
|
2015-10-28 12:09:30 139878086616960 [Note] Recovering after a crash using /var/log/mysql44-3/mysql-bin
|
2015-10-28 12:09:30 139878086616960 [Note] Starting crash recovery...
|
2015-10-28 12:09:30 139878086616960 [Note] Crash recovery finished.
|
2015-10-28 12:09:30 139878086117120 [Warning] Failed to load slave replication state from table mysql.gtid_slave_pos: 1146: Table 'mysql.gtid_slave_pos' doesn't exist
|
2015-10-28 12:09:30 139878086616960 [Note] Server socket created on IP: '::'.
|
2015-10-28 12:09:30 139878086616960 [ERROR] Missing system table mysql.roles_mapping; please run mysql_upgrade to create it
|
2015-10-28 12:09:30 139878086616960 [ERROR] Incorrect definition of table mysql.event: expected column 'sql_mode' at position 14 to have type set('REAL_AS_FLOAT','PIPES_AS_CONCAT','ANSI_QUOTES','IGNORE_SPACE','IGNORE_BAD_TABLE_OPTIONS','ONLY_FULL_GROUP_BY','NO_UNSIGNED_SUBTRACTION','NO_DIR_IN_CREATE','POSTGRESQL','ORACLE','MSSQL','DB2','MAXDB','NO_KEY_OPTIONS','NO_TABLE_OPTIONS','NO_FIELD_OPTIONS','MYSQL323','MYSQL40','ANSI','NO_AUTO_VALUE_ON_ZERO','NO_BACKSLASH_ESCAPES','STRICT_TRANS_TABLES','STRICT_ALL_TABLES','NO_ZERO_IN_DATE','NO_ZERO_DATE','INVALID_DATES','ERROR_FOR_DIVISION_BY_ZERO','TRADITIONAL','NO_AUTO_CREATE_USER','HIGH_NOT_PRECEDENCE','NO_ENGINE_SUBSTITUTION','PAD_CHAR_TO_FULL_LENGTH'), found type set('REAL_AS_FLOAT','PIPES_AS_CONCAT','ANSI_QUOTES','IGNORE_SPACE','NOT_USED','ONLY_FULL_GROUP_BY','NO_UNSIGNED_SUBTRACTION','NO_DIR_IN_CREATE','POSTGRESQL','ORACLE','MSSQL','DB2','MAXDB','NO_KEY_OPTIONS','NO_TABLE_OPTIONS','NO_FIELD_OPTIONS','MYSQL323','MYSQL40','ANSI','NO_AUTO_VALUE_ON_ZERO','NO_BACKSLASH_ESCAPES','STRICT_TRANS_TABLES','STRICT_A
|
2015-10-28 12:09:30 139878086616960 [ERROR] Event Scheduler: An error occurred when initializing system tables. Disabling the Event Scheduler.
|
2015-10-28 12:09:30 139878085584640 [Warning] Failed to load slave replication state from table mysql.gtid_slave_pos: 1146: Table 'mysql.gtid_slave_pos' doesn't exist
|
2015-10-28 12:09:30 139878086616960 [Warning] Neither --relay-log nor --relay-log-index were used; so replication may break when this MySQL server acts as a slave and has his hostname changed!! Please use '--log-basename=#' or '--relay-log=mysqld44-3-relay-bin' to avoid this problem.
|
2015-10-28 12:09:30 139878086616960 [Note] /usr/local/mariadb10.1/bin/mysqld: ready for connections.
|
Version: '10.1.8-MariaDB-log' socket: '/var/run/mysqld/mysqld44-3.sock' port: 3740 MariaDB Server
|
/usr/local/mariadb10.1/bin/mysqld(my_print_stacktrace+0x2e)[0xbf55be]
|
/usr/local/mariadb10.1/bin/mysqld(handle_fatal_signal+0x464)[0x75e6c4]
|
/lib/x86_64-linux-gnu/libpthread.so.0(+0xf8d0)[0x7f37e76868d0]
|
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x37)[0x7f37e627a107]
|
/lib/x86_64-linux-gnu/libc.so.6(abort+0x148)[0x7f37e627b4e8]
|
/usr/local/mariadb10.1/bin/mysqld[0xa0e9d3]
|
/usr/local/mariadb10.1/bin/mysqld[0x9ae60c]
|
/usr/local/mariadb10.1/bin/mysqld[0x979bd4]
|
/usr/local/mariadb10.1/bin/mysqld[0x9dba74]
|
/usr/local/mariadb10.1/bin/mysqld[0x9c4507]
|
/usr/local/mariadb10.1/bin/mysqld[0x9ca5ca]
|
/lib/x86_64-linux-gnu/libpthread.so.0(+0x80a4)[0x7f37e767f0a4]
|
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7f37e632b04d]
|
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
|
information that should help you find out what is causing the crash.
|
151028 12:09:31 mysqld_safe Number of processes running now: 0
|
151028 12:09:31 mysqld_safe mysqld restarted
|
Attachments
Issue Links
- blocks
-
MDEV-8672 Mariadb 10.1.6 crashed on upgrade from 10.0.x
- Closed