mysqld got signal 11

Jan 27 11:23:43 server01-prod mysqld: 140127 11:23:43 [ERROR] mysqld got signal 11 ;

Jan 27 11:23:43 server01-prod mysqld: This could be because you hit a bug. It is also possible that this binary

Jan 27 11:23:43 server01-prod mysqld: or one of the libraries it was linked against is corrupt, improperly built,

Jan 27 11:23:43 server01-prod mysqld: or misconfigured. This error can also be caused by malfunctioning hardware.

Jan 27 11:23:43 server01-prod mysqld: 

Jan 27 11:23:43 server01-prod mysqld: To report this bug, see http://kb.askmonty.org/en/reporting-bugs

Jan 27 11:23:43 server01-prod mysqld: 

Jan 27 11:23:43 server01-prod mysqld: We will try our best to scrape up some info that will hopefully help

Jan 27 11:23:43 server01-prod mysqld: diagnose the problem, but since we have already crashed, 

Jan 27 11:23:43 server01-prod mysqld: something is definitely wrong and this may fail.

Jan 27 11:23:43 server01-prod mysqld: 

Jan 27 11:23:43 server01-prod mysqld: Server version: 5.5.34-MariaDB-1~squeeze

Jan 27 11:23:43 server01-prod mysqld: key_buffer_size=12884901888

Jan 27 11:23:43 server01-prod mysqld: read_buffer_size=262144

Jan 27 11:23:43 server01-prod mysqld: max_used_connections=10

Jan 27 11:23:43 server01-prod mysqld: max_threads=102

Jan 27 11:23:43 server01-prod mysqld: thread_count=5

Jan 27 11:23:43 server01-prod mysqld: It is possible that mysqld could use up to 

Jan 27 11:23:43 server01-prod mysqld: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 14281995 K  bytes of memory

Jan 27 11:23:43 server01-prod mysqld: Hope that's ok; if not, decrease some variables in the equation.

Jan 27 11:23:43 server01-prod mysqld: 

Jan 27 11:23:43 server01-prod mysqld: Thread pointer: 0x0x7f90ed530090

Jan 27 11:23:43 server01-prod mysqld: Attempting backtrace. You can use the following information to find out

Jan 27 11:23:43 server01-prod mysqld: where mysqld died. If you see no messages after this, something went

Jan 27 11:23:43 server01-prod mysqld: terribly wrong...

Jan 27 11:23:43 server01-prod mysqld: stack_bottom = 0x7f8a57ffee58 thread_stack 0x48000

Jan 27 11:23:44 server01-prod mysqld: (my_addr_resolve failure: fork)

Jan 27 11:23:44 server01-prod mysqld: /usr/sbin/mysqld(my_print_stacktrace+0x2e) [0x7f90d3dfa21e]

Jan 27 11:23:44 server01-prod mysqld: /usr/sbin/mysqld(handle_fatal_signal+0x4ac) [0x7f90d39fa41c]

Jan 27 11:23:44 server01-prod mysqld: /lib/libpthread.so.0(+0xeff0) [0x7f90d30f2ff0]

Jan 27 11:23:44 server01-prod mysqld: /usr/sbin/mysqld(Item_func_group_concat::fix_fields(THD*, Item**)+0x173) [0x7f90d3aa7c83]

Jan 27 11:23:44 server01-prod mysqld: /usr/sbin/mysqld(setup_fields(THD*, Item**, List<Item>&, enum_mark_columns, List<Item>*, bool)+0x193) [0x7f90d384ec63]

Jan 27 11:23:44 server01-prod mysqld: /usr/sbin/mysqld(+0x3b868b) [0x7f90d38d868b]

Jan 27 11:23:44 server01-prod mysqld: /usr/sbin/mysqld(mysql_select(THD*, Item***, TABLE_LIST*, unsigned int, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*)+0x84f) [0x7f90d38f49bf]

Jan 27 11:23:44 server01-prod mysqld: /usr/sbin/mysqld(handle_select(THD*, LEX*, select_result*, unsigned long)+0x2b3) [0x7f90d38f4f13]

Jan 27 11:23:44 server01-prod mysqld: /usr/sbin/mysqld(+0x379f9f) [0x7f90d3899f9f]

Jan 27 11:23:44 server01-prod mysqld: /usr/sbin/mysqld(mysql_execute_command(THD*)+0x3427) [0x7f90d38a10c7]

Jan 27 11:23:44 server01-prod mysqld: /usr/sbin/mysqld(mysql_parse(THD*, char*, unsigned int, Parser_state*)+0x29b) [0x7f90d38a3f4b]

Jan 27 11:23:44 server01-prod mysqld: /usr/sbin/mysqld(dispatch_command(enum_server_command, THD*, char*, unsigned int)+0x17f2) [0x7f90d38a5802]

Jan 27 11:23:44 server01-prod mysqld: /usr/sbin/mysqld(do_handle_one_connection(THD*)+0x419) [0x7f90d396c219]

Jan 27 11:23:44 server01-prod mysqld: /usr/sbin/mysqld(handle_one_connection+0x51) [0x7f90d396c2d1]

Jan 27 11:23:44 server01-prod mysqld: /lib/libpthread.so.0(+0x68ca) [0x7f90d30ea8ca]

Jan 27 11:23:44 server01-prod mysqld: /lib/libc.so.6(clone+0x6d) [0x7f90d1a69b6d]

Jan 27 11:23:44 server01-prod mysqld: 

Jan 27 11:23:44 server01-prod mysqld: Trying to get some variables.

Jan 27 11:23:44 server01-prod mysqld: Some pointers may be invalid and cause the dump to abort.

Jan 27 11:23:44 server01-prod mysqld: Query (0x7f90eced5728): SELECT	pbw.date AS 'date',

Jan 27 11:23:44 server01-prod mysqld: 		s.name AS 'Website',

Jan 27 11:23:44 server01-prod mysqld: 		b.name AS 'Brand',

Jan 27 11:23:44 server01-prod mysqld: 		pat.name AS 'Pattern',

Jan 27 11:23:44 server01-prod mysqld: 		group_concat(DISTINCT oe.name ORDER BY oe.name SEPARATOR '/' ) AS 'OE Marking',

Jan 27 11:23:44 server01-prod mysqld: 		d.width AS 'Width',

Jan 27 11:23:44 server01-prod mysqld: 		d.ratio AS 'Ratio',

Jan 27 11:23:44 server01-prod mysqld: 		d.diameter AS 'Diameter',

Jan 27 11:23:44 server01-prod mysqld: 		d.load_index AS 'Load Index',

Jan 27 11:23:44 server01-prod mysqld: 		d.speed_index AS 'Speed Index',

Jan 27 11:23:44 server01-prod mysqld: 		IF (pat.run_flat =1 ,'Yes','no') AS 'Runflat',

Jan 27 11:23:44 server01-prod mysqld: 		pbw.wprice_min AS 'Min Price',

Jan 27 11:23:44 server01-prod mysqld: 		pbw.wprice_sum / pbw.wprice_count AS 'AVG Price',

Jan 27 11:23:44 server01-prod mysqld: 		pbw.wprice_max AS 'Max Price'

Jan 27 11:23:44 server01-prod mysqld: FROM t_price_by_week pbw

Jan 27 11:23:44 server01-prod mysqld: STRAIGHT_JOIN t_site s

Jan 27 11:23:44 server01-prod mysqld: 	ON pbw.website_id = s.id

Jan 27 11:23:44 server01-prod mysqld: STRAIGHT_JOIN t_tyre t

Jan 27 11:23:44 server01-prod mysqld: 	ON pbw.tyre_id = t.id

Jan 27 11:23:44 server01-prod mysqld: STRAIGHT_JOIN t_pattern pat

Jan 27 11:23:44 server01-prod mysqld: 	ON t.pattern_id = pat.id

Jan 27 11:23:44 server01-prod mysqld: STRAIGHT_JOIN t_product prod

Jan 27 11:23:44 server01-prod mysqld: 	ON pat.product_id = prod.id

Jan 27 11:23:44 server01-prod mysqld: STRAIGHT_JOIN t_range r

Jan 27 11:23:44 server01-prod mysqld: 	ON prod.range_id = r.id

Jan 27 11:23:44 server01-prod mysqld: STRAIGHT_JOIN t_brand b

Jan 27 11:23:44 server01-prod mysqld: 	ON r.brand_id = b.id

Jan 27 11:23:44 server01-prod mysqld: STRAIGHT_JOIN t_tyre_dimension d

Jan 27 11:23:44 server01-prod mysqld: 	ON t.tyre_dimension_id = d.id

Jan 27 11:23:44 server01-prod mysqld: STRAIGHT_JOIN t_pattern_oe_marking poe

Jan 27 11:23:44 server01-prod mysqld: 	ON poe.pattern_id = pat.id

Jan 27 11:23:44 server01-prod mysqld: STRAIGHT_JOIN t_oe_marking oe

Jan 27 11:23:44 server01-prod mysqld: 	ON poe.oe_marking_id = oe.id

Jan 27 11:23:44 server01-prod mysqld: WHERE pbw.date = '2014-01-20'

Jan 27 11:23:44 server01-prod mysqld: AND s.code IN ('360cn','amacn','tmabr','carcn','maicn','suncn','tmacn','tuhcn','tyrcn')

Jan 27 11:23:44 server01-prod mysqld: GROUP BY s.id,t.id

Jan 27 11:23:44 server01-prod mysqld: 

Jan 27 11:23:44 server01-prod mysqld: Connection ID (thread ID): 518526

Jan 27 11:23:44 server01-prod mysqld: Status: NOT_KILLED

Jan 27 11:23:44 server01-prod mysqld: 

Jan 27 11:23:44 server01-prod mysqld: Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=off

Jan 27 11:23:44 server01-prod mysqld: 

Jan 27 11:23:44 server01-prod mysqld: The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains

Jan 27 11:23:44 server01-prod mysqld: information that should help you find out what is causing the crash.

Jan 27 11:23:47 server01-prod mysqld_safe: Number of processes running now: 0

Jan 27 11:23:47 server01-prod mysqld_safe: mysqld restarted

Jan 27 11:23:48 server01-prod mysqld: 140127 11:23:48 InnoDB: The InnoDB memory heap is disabled

Jan 27 11:23:48 server01-prod mysqld: 140127 11:23:48 InnoDB: Mutexes and rw_locks use GCC atomic builtins

Jan 27 11:23:48 server01-prod mysqld: 140127 11:23:48 InnoDB: Compressed tables use zlib 1.2.3.4

Jan 27 11:23:48 server01-prod mysqld: 140127 11:23:48 InnoDB: Using Linux native AIO

Jan 27 11:23:48 server01-prod mysqld: 140127 11:23:48  InnoDB: Warning: io_setup() failed with EAGAIN. Will make 5 attempts before giving up.

Jan 27 11:23:48 server01-prod mysqld: InnoDB: Warning: io_setup() attempt 1 failed.

Jan 27 11:23:49 server01-prod mysqld: InnoDB: Warning: io_setup() attempt 2 failed.

Jan 27 11:23:49 server01-prod mysqld: InnoDB: Warning: io_setup() attempt 3 failed.

Jan 27 11:23:50 server01-prod mysqld: InnoDB: Warning: io_setup() attempt 4 failed.

Jan 27 11:23:50 server01-prod mysqld: InnoDB: Warning: io_setup() attempt 5 failed.

Jan 27 11:23:51 server01-prod mysqld: 140127 11:23:51  InnoDB: Error: io_setup() failed with EAGAIN after 5 attempts.

Jan 27 11:23:51 server01-prod mysqld: InnoDB: You can disable Linux Native AIO by setting innodb_use_native_aio = 0 in my.cnf

Jan 27 11:23:51 server01-prod mysqld:   InnoDB: Warning: Linux Native AIO disabled because os_aio_linux_create_io_ctx() failed. To get rid of this warning you can try increasing system fs.aio-max-nr to 1048576 or larger or setting innodb_use_native_aio = 0 in my.cnf

Jan 27 11:23:51 server01-prod mysqld: 140127 11:23:51 InnoDB: Initializing buffer pool, size = 12.0G

Jan 27 11:23:52 server01-prod mysqld: 140127 11:23:52 InnoDB: Completed initialization of buffer pool

Jan 27 11:23:52 server01-prod mysqld: 140127 11:23:52 InnoDB: highest supported file format is Barracuda.

Jan 27 11:23:52 server01-prod mysqld: InnoDB: The log sequence number in ibdata files does not match

Jan 27 11:23:52 server01-prod mysqld: InnoDB: the log sequence number in the ib_logfiles!

Jan 27 11:23:52 server01-prod mysqld: 140127 11:23:52  InnoDB: Database was not shut down normally!

Jan 27 11:23:52 server01-prod mysqld: InnoDB: Starting crash recovery.

Jan 27 11:23:52 server01-prod mysqld: InnoDB: Reading tablespace information from the .ibd files...

Jan 27 11:24:01 server01-prod mysqld: InnoDB: Restoring possible half-written data pages from the doublewrite

Jan 27 11:24:01 server01-prod mysqld: InnoDB: buffer...

Jan 27 11:24:05 server01-prod mysqld: InnoDB: Last MySQL binlog file position 0 41890574, file name /home/mysql/log/mariadb-bin.004155

Jan 27 11:24:05 server01-prod mysqld: 140127 11:24:05  InnoDB: Waiting for the background threads to start

Jan 27 11:24:06 server01-prod mysqld: 140127 11:24:06 Percona XtraDB (http://www.percona.com) 5.5.34-MariaDB-31.1 started; log sequence number 374319885837

Jan 27 11:24:06 server01-prod mysqld: 140127 11:24:06 [Note] Plugin 'FEEDBACK' is disabled.

Jan 27 11:24:06 server01-prod mysqld: 140127 11:24:06 [Note] Server socket created on IP: '0.0.0.0'.

Jan 27 11:24:06 server01-prod mysqld: 140127 11:24:06 [ERROR] mysqld: Table './mysql/user' is marked as crashed and should be repaired

Jan 27 11:24:06 server01-prod mysqld: 140127 11:24:06 [ERROR] mysqld: Table 'user' is marked as crashed and should be repaired

Jan 27 11:24:06 server01-prod mysqld: 140127 11:24:06 [Warning] Checking table:   './mysql/user'

Jan 27 11:24:06 server01-prod mysqld: 140127 11:24:06 [ERROR] mysql.user: 1 client is using or hasn't closed the table properly

Jan 27 11:24:06 server01-prod mysqld: 140127 11:24:06 [ERROR] mysqld: Table './mysql/db' is marked as crashed and should be repaired

Jan 27 11:24:06 server01-prod mysqld: 140127 11:24:06 [ERROR] mysqld: Table 'db' is marked as crashed and should be repaired

Jan 27 11:24:06 server01-prod mysqld: 140127 11:24:06 [Warning] Checking table:   './mysql/db'

Jan 27 11:24:06 server01-prod mysqld: 140127 11:24:06 [ERROR] mysql.db: 1 client is using or hasn't closed the table properly

Jan 27 11:24:06 server01-prod mysqld: 140127 11:24:06 [Note] Event Scheduler: Loaded 0 events

Jan 27 11:24:06 server01-prod mysqld: 140127 11:24:06 [Note] /usr/sbin/mysqld: ready for connections.

Jan 27 11:24:06 server01-prod mysqld: Version: '5.5.34-MariaDB-1~squeeze'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  mariadb.org binary distribution

Jan 27 11:29:52 server01-prod mysqld: 140127 11:29:52 [ERROR] mysqld: Table './mysql/proc' is marked as crashed and should be repaired

Jan 27 11:29:52 server01-prod mysqld: 140127 11:29:52 [ERROR] mysqld: Table 'proc' is marked as crashed and should be repaired

Jan 27 11:29:52 server01-prod mysqld: 140127 11:29:52 [Warning] Checking table:   './mysql/proc'

Jan 27 11:29:55 server01-prod mysqld: 140127 11:29:55 [ERROR] mysqld: Table './bidb/t_price_by_month' is marked as crashed and should be repaired

Jan 27 11:29:55 server01-prod mysqld: 140127 11:29:55 [ERROR] mysqld: Table 't_price_by_month' is marked as crashed and should be repaired

Jan 27 11:29:55 server01-prod mysqld: 140127 11:29:55 [Warning] Checking table:   './bidb/t_price_by_month'

Jan 27 11:30:06 server01-prod mysqld: 140127 11:30:06 [ERROR] mysqld: Table './bidb/t_price_by_week' is marked as crashed and should be repaired

Jan 27 11:30:06 server01-prod mysqld: 140127 11:30:06 [ERROR] mysqld: Table 't_price_by_week' is marked as crashed and should be repaired

Jan 27 11:30:06 server01-prod mysqld: 140127 11:30:06 [Warning] Checking table:   './bidb/t_price_by_week'

(...)