MariaDB server crashes with assertion failure in file ha_innodb.c line 11652

2014-07-06 07:22:12 7f3788e48700  InnoDB: MySQL and InnoDB data dictionaries are out of sync.

InnoDB: Unable to find the AUTOINC column vid in the InnoDB table .

InnoDB: We set the next AUTOINC column value to 0,

InnoDB: in effect disabling the AUTOINC next value generation.

InnoDB: You can either set the next AUTOINC value explicitly using ALTER TABLE

InnoDB: or fix the data dictionary by recreating the table.

2014-07-06 07:22:12 7f3788e48700  InnoDB: Assertion failure in thread 139876496606976 in file ha_innodb.cc line 11652

InnoDB: Failing assertion: index->table->stat_initialized

InnoDB: We intentionally generate a memory trap.

InnoDB: Submit a detailed bug report to http://bugs.mysql.com.

InnoDB: If you get repeated assertion failures or crashes, even

InnoDB: immediately after the mysqld startup, there may be

InnoDB: corruption in the InnoDB tablespace. Please refer to

InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html

InnoDB: about forcing recovery.

140706  7:22:12 [ERROR] mysqld got signal 6 ;

This could be because you hit a bug. It is also possible that this binary

or one of the libraries it was linked against is corrupt, improperly built,

or misconfigured. This error can also be caused by malfunctioning hardware.

To report this bug, see http://kb.askmonty.org/en/reporting-bugs

We will try our best to scrape up some info that will hopefully help

diagnose the problem, but since we have already crashed, 

something is definitely wrong and this may fail.

Server version: 10.0.11-MariaDB-log

key_buffer_size=16777216

read_buffer_size=262144

max_used_connections=259

max_threads=502

thread_count=158

It is possible that mysqld could use up to 

key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 412284 K  bytes of memory

Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x0x16753968

Attempting backtrace. You can use the following information to find out

where mysqld died. If you see no messages after this, something went

terribly wrong...

stack_bottom = 0x7f3788e47e48 thread_stack 0x48000

(my_addr_resolve failure: fork)

/usr/sbin/mysqld(my_print_stacktrace+0x26) [0xb0d036]

/usr/sbin/mysqld(handle_fatal_signal+0x398) [0x6ef3d8]

/lib64/libpthread.so.0(+0x10c20) [0x7f3a5d7d1c20]

/lib64/libc.so.6(gsignal+0x35) [0x7f3a5c3ed265]

/lib64/libc.so.6(abort+0x17b) [0x7f3a5c3ee6db]

/usr/sbin/mysqld() [0x964ba7]

/usr/sbin/mysqld() [0x96b396]

/usr/sbin/mysqld(handler::ha_open(TABLE*, char const*, int, unsigned int)+0x3f) [0x6f2fff]

/usr/sbin/mysqld() [0xaebedd]

/usr/sbin/mysqld(handler::ha_open(TABLE*, char const*, int, unsigned int)+0x3f) [0x6f2fff]

/usr/sbin/mysqld(open_table_from_share(THD*, TABLE_SHARE*, char const*, unsigned int, unsigned int, unsigned int, TABLE*, bool)+0x86c) [0x64d20c]

/usr/sbin/mysqld(open_table(THD*, TABLE_LIST*, st_mem_root*, Open_table_context*)+0x786) [0x5891f6]

/usr/sbin/mysqld(open_tables(THD*, TABLE_LIST**, unsigned int*, unsigned int, Prelocking_strategy*)+0xd36) [0x58a326]

/usr/sbin/mysqld(open_and_lock_tables(THD*, TABLE_LIST*, bool, unsigned int, Prelocking_strategy*)+0x49) [0x58a509]

/usr/sbin/mysqld() [0x5b95cd]

/usr/sbin/mysqld(mysql_execute_command(THD*)+0x5088) [0x5c42b8]

/usr/sbin/mysqld(mysql_parse(THD*, char*, unsigned int, Parser_state*)+0x1fc) [0x5c6b5c]

/usr/sbin/mysqld(dispatch_command(enum_server_command, THD*, char*, unsigned int)+0x1530) [0x5c80e0]

/usr/sbin/mysqld(do_handle_one_connection(THD*)+0x19d) [0x66fa4d]

/usr/sbin/mysqld(handle_one_connection+0x40) [0x66fb10]

/usr/sbin/mysqld() [0x89bc04]

/lib64/libpthread.so.0(+0x8eb7) [0x7f3a5d7c9eb7]

/lib64/libc.so.6(clone+0x6d) [0x7f3a5c4a4a7d]

Trying to get some variables.

Some pointers may be invalid and cause the dump to abort.

Query (0x7f3776e82a60): is an invalid pointer

Connection ID (thread ID): 356991

Status: NOT_KILLED

Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=off

The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains

information that should help you find out what is causing the crash.

140706 11:51:40 [Note] CONNECT: Version 1.02.0002 March 16, 2014

2014-07-06 11:51:40 7f040bfdf740 InnoDB: Warning: Using innodb_additional_mem_pool_size is DEPRECATED. This option may be removed in future releases, together with the option innodb_use_sys_malloc and with the InnoDB's internal memory allocator.

140706 11:51:40 [Note] InnoDB: Using mutexes to ref count buffer pool pages

140706 11:51:40 [Note] InnoDB: The InnoDB memory heap is disabled

140706 11:51:40 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins

140706 11:51:40 [Note] InnoDB: Compressed tables use zlib 1.2.8

140706 11:51:40 [Note] InnoDB: Using Linux native AIO

140706 11:51:40 [Note] InnoDB: Using CPU crc32 instructions

140706 11:51:40 [Note] InnoDB: Initializing buffer pool, size = 10.0G

140706 11:51:42 [Note] InnoDB: Completed initialization of buffer pool

140706 11:51:42 [Note] InnoDB: Highest supported file format is Barracuda.

140706 11:51:42 [Note] InnoDB: Log scan progressed past the checkpoint lsn 740281760840

140706 11:51:42 [Note] InnoDB: Database was not shutdown normally!

140706 11:51:42 [Note] InnoDB: Starting crash recovery.

140706 11:51:42 [Note] InnoDB: Reading tablespace information from the .ibd files...

140706 11:51:50 [Note] InnoDB: Restoring possible half-written data pages 

140706 11:51:50 [Note] InnoDB: from the doublewrite buffer...

...

some stuff is omitted

...

140706 11:52:10 [Note] InnoDB: Starting an apply batch of log records to the database...

InnoDB: Progress in percent: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 

InnoDB: Apply batch completed

InnoDB: Last MySQL binlog file position 0 651862295, file name ./emsow-bin.000013

140706 11:52:19 [Note] InnoDB: 128 rollback segment(s) are active.

140706 11:56:21 [Note] CONNECT: Version 1.02.0002 March 16, 2014

140706 11:56:21 [ERROR] mysqld: Can't lock aria control file '/var/lib/mysql/aria_log_control' for exclusive use, error: 11. Will retry for 30 seconds

2014-07-06 11:56:49 7f9daf3ad740 InnoDB: Warning: Using innodb_additional_mem_pool_size is DEPRECATED. This option may be removed in future releases, together with the option innodb_use_sys_malloc and with the InnoDB's internal memory allocator.

140706 11:56:49 [Note] InnoDB: Using mutexes to ref count buffer pool pages

140706 11:56:49 [Note] InnoDB: The InnoDB memory heap is disabled

140706 11:56:49 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins

140706 11:56:49 [Note] InnoDB: Compressed tables use zlib 1.2.8

140706 11:56:49 [Note] InnoDB: Using Linux native AIO

140706 11:56:49 [Note] InnoDB: Using CPU crc32 instructions

140706 11:56:49 [Note] InnoDB: Initializing buffer pool, size = 10.0G

140706 11:56:50 [Note] InnoDB: Completed initialization of buffer pool

140706 11:56:50 [Note] InnoDB: Highest supported file format is Barracuda.

140706 11:56:50 [Note] InnoDB: Log scan progressed past the checkpoint lsn 740281760840

140706 11:56:50 [Note] InnoDB: Database was not shutdown normally!

140706 11:56:50 [Note] InnoDB: Starting crash recovery.

140706 11:56:50 [Note] InnoDB: Reading tablespace information from the .ibd files...

140706 11:56:51 [Note] InnoDB: Restoring possible half-written data pages 

140706 11:56:51 [Note] InnoDB: from the doublewrite buffer...

...

some stuff is omitted

...

140706 11:57:00 [Note] CONNECT: Version 1.02.0002 March 16, 2014

2014-07-06 11:57:00 7f81987da740 InnoDB: Warning: Using innodb_additional_mem_pool_size is DEPRECATED. This option may be removed in future releases, together with the option innodb_use_sys_malloc and with the InnoDB's internal memory allocator.

140706 11:57:00 [Note] InnoDB: Using mutexes to ref count buffer pool pages

140706 11:57:00 [Note] InnoDB: The InnoDB memory heap is disabled

140706 11:57:00 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins

140706 11:57:00 [Note] InnoDB: Compressed tables use zlib 1.2.8

140706 11:57:00 [Note] InnoDB: Using Linux native AIO

140706 11:57:00 [Note] InnoDB: Using CPU crc32 instructions

140706 11:57:00 [Note] InnoDB: Initializing buffer pool, size = 10.0G

140706 11:57:01 [Note] InnoDB: Completed initialization of buffer pool

140706 11:57:01 [Note] InnoDB: Highest supported file format is Barracuda.

140706 11:57:01 [Note] InnoDB: Log scan progressed past the checkpoint lsn 740281760840

140706 11:57:01 [Note] InnoDB: Database was not shutdown normally!

140706 11:57:01 [Note] InnoDB: Starting crash recovery.

140706 11:57:01 [Note] InnoDB: Reading tablespace information from the .ibd files...

140706 11:57:02 [Note] InnoDB: Restoring possible half-written data pages 

140706 11:57:02 [Note] InnoDB: from the doublewrite buffer...

...

some stuff is omitted

...

140706 11:57:22 [Note] InnoDB: Starting an apply batch of log records to the database...

InnoDB: Progress in percent: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 

InnoDB: Apply batch completed

InnoDB: Last MySQL binlog file position 0 651862295, file name ./emsow-bin.000013

140706 11:57:30 [Note] InnoDB: 128 rollback segment(s) are active.

140706 12:00:46 [Note] InnoDB: Waiting for purge to start

140706 12:00:46 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.17-65.0 started; log sequence number 740978429198

140706 12:00:46 [Note] Recovering after a crash using emsow-bin

140706 12:00:57 [Note] Starting crash recovery...

140706 12:00:57 [Note] Crash recovery finished.

140706 12:00:57 [Note] Server socket created on IP: '0.0.0.0'.

140706 12:00:57 [Note] Event Scheduler: Loaded 0 events

140706 12:00:57 [Note] /usr/sbin/mysqld: ready for connections.

Version: '10.0.11-MariaDB-log'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  Source distribution

# uname -a 

Linux db 3.10.25-gentoo #1 SMP PREEMPT Wed Jan 29 19:50:21 EST 2014 x86_64 Intel(R) Xeon(R) CPU E5504 @ 2.00GHz GenuineIntel GNU/Linux

# mysql -V 

mysql -V

mysql  Ver 15.1 Distrib 10.0.11-MariaDB, for Linux (x86_64) using readline 6.2