Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-5622

mysqld got signal 11

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Fixed
    • 5.5.34
    • 5.5.35
    • Parser
    • None
    • distro : GNU/Linux Debian 6.0.8
      kernel : Linux 2.6.32-5-amd64 #1 SMP Fri May 10 08:43:19 UTC 2013 x86_64 GNU/Linux
      hardware : vmware 5.0
      RAM : 48Gb
      CPU : 4 core Intel(R) Xeon(R) CPU X5690 @ 3.47GHz

    Description

      Hello,

      we had an issue with mariadb 5.5.34 and no idea why.
      We couldn't reproduce the problem after several retries but may be the stack trace is more readble and useful by developpers ?

      Jan 27 11:23:43 server01-prod mysqld: 140127 11:23:43 [ERROR] mysqld got signal 11 ;
      Jan 27 11:23:43 server01-prod mysqld: This could be because you hit a bug. It is also possible that this binary
      Jan 27 11:23:43 server01-prod mysqld: or one of the libraries it was linked against is corrupt, improperly built,
      Jan 27 11:23:43 server01-prod mysqld: or misconfigured. This error can also be caused by malfunctioning hardware.
      Jan 27 11:23:43 server01-prod mysqld: 
      Jan 27 11:23:43 server01-prod mysqld: To report this bug, see http://kb.askmonty.org/en/reporting-bugs
      Jan 27 11:23:43 server01-prod mysqld: 
      Jan 27 11:23:43 server01-prod mysqld: We will try our best to scrape up some info that will hopefully help
      Jan 27 11:23:43 server01-prod mysqld: diagnose the problem, but since we have already crashed, 
      Jan 27 11:23:43 server01-prod mysqld: something is definitely wrong and this may fail.
      Jan 27 11:23:43 server01-prod mysqld: 
      Jan 27 11:23:43 server01-prod mysqld: Server version: 5.5.34-MariaDB-1~squeeze
      Jan 27 11:23:43 server01-prod mysqld: key_buffer_size=12884901888
      Jan 27 11:23:43 server01-prod mysqld: read_buffer_size=262144
      Jan 27 11:23:43 server01-prod mysqld: max_used_connections=10
      Jan 27 11:23:43 server01-prod mysqld: max_threads=102
      Jan 27 11:23:43 server01-prod mysqld: thread_count=5
      Jan 27 11:23:43 server01-prod mysqld: It is possible that mysqld could use up to 
      Jan 27 11:23:43 server01-prod mysqld: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 14281995 K  bytes of memory
      Jan 27 11:23:43 server01-prod mysqld: Hope that's ok; if not, decrease some variables in the equation.
      Jan 27 11:23:43 server01-prod mysqld: 
      Jan 27 11:23:43 server01-prod mysqld: Thread pointer: 0x0x7f90ed530090
      Jan 27 11:23:43 server01-prod mysqld: Attempting backtrace. You can use the following information to find out
      Jan 27 11:23:43 server01-prod mysqld: where mysqld died. If you see no messages after this, something went
      Jan 27 11:23:43 server01-prod mysqld: terribly wrong...
      Jan 27 11:23:43 server01-prod mysqld: stack_bottom = 0x7f8a57ffee58 thread_stack 0x48000
      Jan 27 11:23:44 server01-prod mysqld: (my_addr_resolve failure: fork)
      Jan 27 11:23:44 server01-prod mysqld: /usr/sbin/mysqld(my_print_stacktrace+0x2e) [0x7f90d3dfa21e]
      Jan 27 11:23:44 server01-prod mysqld: /usr/sbin/mysqld(handle_fatal_signal+0x4ac) [0x7f90d39fa41c]
      Jan 27 11:23:44 server01-prod mysqld: /lib/libpthread.so.0(+0xeff0) [0x7f90d30f2ff0]
      Jan 27 11:23:44 server01-prod mysqld: /usr/sbin/mysqld(Item_func_group_concat::fix_fields(THD*, Item**)+0x173) [0x7f90d3aa7c83]
      Jan 27 11:23:44 server01-prod mysqld: /usr/sbin/mysqld(setup_fields(THD*, Item**, List<Item>&, enum_mark_columns, List<Item>*, bool)+0x193) [0x7f90d384ec63]
      Jan 27 11:23:44 server01-prod mysqld: /usr/sbin/mysqld(+0x3b868b) [0x7f90d38d868b]
      Jan 27 11:23:44 server01-prod mysqld: /usr/sbin/mysqld(mysql_select(THD*, Item***, TABLE_LIST*, unsigned int, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*)+0x84f) [0x7f90d38f49bf]
      Jan 27 11:23:44 server01-prod mysqld: /usr/sbin/mysqld(handle_select(THD*, LEX*, select_result*, unsigned long)+0x2b3) [0x7f90d38f4f13]
      Jan 27 11:23:44 server01-prod mysqld: /usr/sbin/mysqld(+0x379f9f) [0x7f90d3899f9f]
      Jan 27 11:23:44 server01-prod mysqld: /usr/sbin/mysqld(mysql_execute_command(THD*)+0x3427) [0x7f90d38a10c7]
      Jan 27 11:23:44 server01-prod mysqld: /usr/sbin/mysqld(mysql_parse(THD*, char*, unsigned int, Parser_state*)+0x29b) [0x7f90d38a3f4b]
      Jan 27 11:23:44 server01-prod mysqld: /usr/sbin/mysqld(dispatch_command(enum_server_command, THD*, char*, unsigned int)+0x17f2) [0x7f90d38a5802]
      Jan 27 11:23:44 server01-prod mysqld: /usr/sbin/mysqld(do_handle_one_connection(THD*)+0x419) [0x7f90d396c219]
      Jan 27 11:23:44 server01-prod mysqld: /usr/sbin/mysqld(handle_one_connection+0x51) [0x7f90d396c2d1]
      Jan 27 11:23:44 server01-prod mysqld: /lib/libpthread.so.0(+0x68ca) [0x7f90d30ea8ca]
      Jan 27 11:23:44 server01-prod mysqld: /lib/libc.so.6(clone+0x6d) [0x7f90d1a69b6d]
      Jan 27 11:23:44 server01-prod mysqld: 
      Jan 27 11:23:44 server01-prod mysqld: Trying to get some variables.
      Jan 27 11:23:44 server01-prod mysqld: Some pointers may be invalid and cause the dump to abort.
      Jan 27 11:23:44 server01-prod mysqld: Query (0x7f90eced5728): SELECT	pbw.date AS 'date',
      Jan 27 11:23:44 server01-prod mysqld: 		s.name AS 'Website',
      Jan 27 11:23:44 server01-prod mysqld: 		b.name AS 'Brand',
      Jan 27 11:23:44 server01-prod mysqld: 		pat.name AS 'Pattern',
      Jan 27 11:23:44 server01-prod mysqld: 		group_concat(DISTINCT oe.name ORDER BY oe.name SEPARATOR '/' ) AS 'OE Marking',
      Jan 27 11:23:44 server01-prod mysqld: 		d.width AS 'Width',
      Jan 27 11:23:44 server01-prod mysqld: 		d.ratio AS 'Ratio',
      Jan 27 11:23:44 server01-prod mysqld: 		d.diameter AS 'Diameter',
      Jan 27 11:23:44 server01-prod mysqld: 		d.load_index AS 'Load Index',
      Jan 27 11:23:44 server01-prod mysqld: 		d.speed_index AS 'Speed Index',
      Jan 27 11:23:44 server01-prod mysqld: 		IF (pat.run_flat =1 ,'Yes','no') AS 'Runflat',
      Jan 27 11:23:44 server01-prod mysqld: 		pbw.wprice_min AS 'Min Price',
      Jan 27 11:23:44 server01-prod mysqld: 		pbw.wprice_sum / pbw.wprice_count AS 'AVG Price',
      Jan 27 11:23:44 server01-prod mysqld: 		pbw.wprice_max AS 'Max Price'
      Jan 27 11:23:44 server01-prod mysqld: FROM t_price_by_week pbw
      Jan 27 11:23:44 server01-prod mysqld: STRAIGHT_JOIN t_site s
      Jan 27 11:23:44 server01-prod mysqld: 	ON pbw.website_id = s.id
      Jan 27 11:23:44 server01-prod mysqld: STRAIGHT_JOIN t_tyre t
      Jan 27 11:23:44 server01-prod mysqld: 	ON pbw.tyre_id = t.id
      Jan 27 11:23:44 server01-prod mysqld: STRAIGHT_JOIN t_pattern pat
      Jan 27 11:23:44 server01-prod mysqld: 	ON t.pattern_id = pat.id
      Jan 27 11:23:44 server01-prod mysqld: STRAIGHT_JOIN t_product prod
      Jan 27 11:23:44 server01-prod mysqld: 	ON pat.product_id = prod.id
      Jan 27 11:23:44 server01-prod mysqld: STRAIGHT_JOIN t_range r
      Jan 27 11:23:44 server01-prod mysqld: 	ON prod.range_id = r.id
      Jan 27 11:23:44 server01-prod mysqld: STRAIGHT_JOIN t_brand b
      Jan 27 11:23:44 server01-prod mysqld: 	ON r.brand_id = b.id
      Jan 27 11:23:44 server01-prod mysqld: STRAIGHT_JOIN t_tyre_dimension d
      Jan 27 11:23:44 server01-prod mysqld: 	ON t.tyre_dimension_id = d.id
      Jan 27 11:23:44 server01-prod mysqld: STRAIGHT_JOIN t_pattern_oe_marking poe
      Jan 27 11:23:44 server01-prod mysqld: 	ON poe.pattern_id = pat.id
      Jan 27 11:23:44 server01-prod mysqld: STRAIGHT_JOIN t_oe_marking oe
      Jan 27 11:23:44 server01-prod mysqld: 	ON poe.oe_marking_id = oe.id
      Jan 27 11:23:44 server01-prod mysqld: WHERE pbw.date = '2014-01-20'
      Jan 27 11:23:44 server01-prod mysqld: AND s.code IN ('360cn','amacn','tmabr','carcn','maicn','suncn','tmacn','tuhcn','tyrcn')
      Jan 27 11:23:44 server01-prod mysqld: GROUP BY s.id,t.id
      Jan 27 11:23:44 server01-prod mysqld: 
      Jan 27 11:23:44 server01-prod mysqld: Connection ID (thread ID): 518526
      Jan 27 11:23:44 server01-prod mysqld: Status: NOT_KILLED
      Jan 27 11:23:44 server01-prod mysqld: 
      Jan 27 11:23:44 server01-prod mysqld: Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=off
      Jan 27 11:23:44 server01-prod mysqld: 
      Jan 27 11:23:44 server01-prod mysqld: The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
      Jan 27 11:23:44 server01-prod mysqld: information that should help you find out what is causing the crash.
      Jan 27 11:23:47 server01-prod mysqld_safe: Number of processes running now: 0
      Jan 27 11:23:47 server01-prod mysqld_safe: mysqld restarted
      Jan 27 11:23:48 server01-prod mysqld: 140127 11:23:48 InnoDB: The InnoDB memory heap is disabled
      Jan 27 11:23:48 server01-prod mysqld: 140127 11:23:48 InnoDB: Mutexes and rw_locks use GCC atomic builtins
      Jan 27 11:23:48 server01-prod mysqld: 140127 11:23:48 InnoDB: Compressed tables use zlib 1.2.3.4
      Jan 27 11:23:48 server01-prod mysqld: 140127 11:23:48 InnoDB: Using Linux native AIO
      Jan 27 11:23:48 server01-prod mysqld: 140127 11:23:48  InnoDB: Warning: io_setup() failed with EAGAIN. Will make 5 attempts before giving up.
      Jan 27 11:23:48 server01-prod mysqld: InnoDB: Warning: io_setup() attempt 1 failed.
      Jan 27 11:23:49 server01-prod mysqld: InnoDB: Warning: io_setup() attempt 2 failed.
      Jan 27 11:23:49 server01-prod mysqld: InnoDB: Warning: io_setup() attempt 3 failed.
      Jan 27 11:23:50 server01-prod mysqld: InnoDB: Warning: io_setup() attempt 4 failed.
      Jan 27 11:23:50 server01-prod mysqld: InnoDB: Warning: io_setup() attempt 5 failed.
      Jan 27 11:23:51 server01-prod mysqld: 140127 11:23:51  InnoDB: Error: io_setup() failed with EAGAIN after 5 attempts.
      Jan 27 11:23:51 server01-prod mysqld: InnoDB: You can disable Linux Native AIO by setting innodb_use_native_aio = 0 in my.cnf
      Jan 27 11:23:51 server01-prod mysqld:   InnoDB: Warning: Linux Native AIO disabled because os_aio_linux_create_io_ctx() failed. To get rid of this warning you can try increasing system fs.aio-max-nr to 1048576 or larger or setting innodb_use_native_aio = 0 in my.cnf
      Jan 27 11:23:51 server01-prod mysqld: 140127 11:23:51 InnoDB: Initializing buffer pool, size = 12.0G
      Jan 27 11:23:52 server01-prod mysqld: 140127 11:23:52 InnoDB: Completed initialization of buffer pool
      Jan 27 11:23:52 server01-prod mysqld: 140127 11:23:52 InnoDB: highest supported file format is Barracuda.
      Jan 27 11:23:52 server01-prod mysqld: InnoDB: The log sequence number in ibdata files does not match
      Jan 27 11:23:52 server01-prod mysqld: InnoDB: the log sequence number in the ib_logfiles!
      Jan 27 11:23:52 server01-prod mysqld: 140127 11:23:52  InnoDB: Database was not shut down normally!
      Jan 27 11:23:52 server01-prod mysqld: InnoDB: Starting crash recovery.
      Jan 27 11:23:52 server01-prod mysqld: InnoDB: Reading tablespace information from the .ibd files...
      Jan 27 11:24:01 server01-prod mysqld: InnoDB: Restoring possible half-written data pages from the doublewrite
      Jan 27 11:24:01 server01-prod mysqld: InnoDB: buffer...
      Jan 27 11:24:05 server01-prod mysqld: InnoDB: Last MySQL binlog file position 0 41890574, file name /home/mysql/log/mariadb-bin.004155
      Jan 27 11:24:05 server01-prod mysqld: 140127 11:24:05  InnoDB: Waiting for the background threads to start
      Jan 27 11:24:06 server01-prod mysqld: 140127 11:24:06 Percona XtraDB (http://www.percona.com) 5.5.34-MariaDB-31.1 started; log sequence number 374319885837
      Jan 27 11:24:06 server01-prod mysqld: 140127 11:24:06 [Note] Plugin 'FEEDBACK' is disabled.
      Jan 27 11:24:06 server01-prod mysqld: 140127 11:24:06 [Note] Server socket created on IP: '0.0.0.0'.
      Jan 27 11:24:06 server01-prod mysqld: 140127 11:24:06 [ERROR] mysqld: Table './mysql/user' is marked as crashed and should be repaired
      Jan 27 11:24:06 server01-prod mysqld: 140127 11:24:06 [ERROR] mysqld: Table 'user' is marked as crashed and should be repaired
      Jan 27 11:24:06 server01-prod mysqld: 140127 11:24:06 [Warning] Checking table:   './mysql/user'
      Jan 27 11:24:06 server01-prod mysqld: 140127 11:24:06 [ERROR] mysql.user: 1 client is using or hasn't closed the table properly
      Jan 27 11:24:06 server01-prod mysqld: 140127 11:24:06 [ERROR] mysqld: Table './mysql/db' is marked as crashed and should be repaired
      Jan 27 11:24:06 server01-prod mysqld: 140127 11:24:06 [ERROR] mysqld: Table 'db' is marked as crashed and should be repaired
      Jan 27 11:24:06 server01-prod mysqld: 140127 11:24:06 [Warning] Checking table:   './mysql/db'
      Jan 27 11:24:06 server01-prod mysqld: 140127 11:24:06 [ERROR] mysql.db: 1 client is using or hasn't closed the table properly
      Jan 27 11:24:06 server01-prod mysqld: 140127 11:24:06 [Note] Event Scheduler: Loaded 0 events
      Jan 27 11:24:06 server01-prod mysqld: 140127 11:24:06 [Note] /usr/sbin/mysqld: ready for connections.
      Jan 27 11:24:06 server01-prod mysqld: Version: '5.5.34-MariaDB-1~squeeze'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  mariadb.org binary distribution
      Jan 27 11:29:52 server01-prod mysqld: 140127 11:29:52 [ERROR] mysqld: Table './mysql/proc' is marked as crashed and should be repaired
      Jan 27 11:29:52 server01-prod mysqld: 140127 11:29:52 [ERROR] mysqld: Table 'proc' is marked as crashed and should be repaired
      Jan 27 11:29:52 server01-prod mysqld: 140127 11:29:52 [Warning] Checking table:   './mysql/proc'
      Jan 27 11:29:55 server01-prod mysqld: 140127 11:29:55 [ERROR] mysqld: Table './bidb/t_price_by_month' is marked as crashed and should be repaired
      Jan 27 11:29:55 server01-prod mysqld: 140127 11:29:55 [ERROR] mysqld: Table 't_price_by_month' is marked as crashed and should be repaired
      Jan 27 11:29:55 server01-prod mysqld: 140127 11:29:55 [Warning] Checking table:   './bidb/t_price_by_month'
      Jan 27 11:30:06 server01-prod mysqld: 140127 11:30:06 [ERROR] mysqld: Table './bidb/t_price_by_week' is marked as crashed and should be repaired
      Jan 27 11:30:06 server01-prod mysqld: 140127 11:30:06 [ERROR] mysqld: Table 't_price_by_week' is marked as crashed and should be repaired
      Jan 27 11:30:06 server01-prod mysqld: 140127 11:30:06 [Warning] Checking table:   './bidb/t_price_by_week'
      (...)

      Attachments

        Activity

          People

            Unassigned Unassigned
            lomg-it Lizeo Online Media Group
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.