|
Stack trace from the attachment (to make it searchable):
Jun 21 16:29:21 mysqld: Server version: 10.0.25-MariaDB-1~wheezy-wsrep
|
Jun 21 16:29:21 mysqld: key_buffer_size=1073741824
|
Jun 21 16:29:21 mysqld: read_buffer_size=2097152
|
Jun 21 16:29:21 mysqld: max_used_connections=7
|
Jun 21 16:29:21 mysqld: max_threads=2002
|
Jun 21 16:29:21 mysqld: thread_count=4
|
Jun 21 16:29:21 mysqld: It is possible that mysqld could use up to
|
Jun 21 16:29:21 mysqld: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 13389306 K bytes of memory
|
Jun 21 16:29:21 mysqld: Hope that's ok; if not, decrease some variables in the equation.
|
Jun 21 16:29:21 mysqld:
|
Jun 21 16:29:21 mysqld: Thread pointer: 0x0x7f1c210b1008
|
Jun 21 16:29:21 mysqld: Attempting backtrace. You can use the following information to find out
|
Jun 21 16:29:21 mysqld: where mysqld died. If you see no messages after this, something went
|
Jun 21 16:29:21 mysqld: terribly wrong...
|
Jun 21 16:29:21 mysqld: stack_bottom = 0x7f1dc652ade0 thread_stack 0x40000
|
Jun 21 16:29:21 mysqld: /usr/sbin/mysqld(my_print_stacktrace+0x2b)[0x7f1dc5faf76b]
|
Jun 21 16:29:21 mysqld: /usr/sbin/mysqld(handle_fatal_signal+0x422)[0x7f1dc5b295a2]
|
Jun 21 16:29:21 mysqld: /lib/x86_64-linux-gnu/libpthread.so.0(+0xf0a0)[0x7f1dc51450a0]
|
Jun 21 16:29:21 mysqld: /usr/sbin/mysqld(_ZN8MDL_lock11Ticket_list13remove_ticketEP10MDL_ticket+0x11)[0x7f1dc5a88681]
|
Jun 21 16:29:21 mysqld: /usr/sbin/mysqld(_ZN8MDL_lock13remove_ticketEMS_NS_11Ticket_listEP10MDL_ticket+0x41)[0x7f1dc5a88c41]
|
Jun 21 16:29:21 mysqld: /usr/sbin/mysqld(_ZN11MDL_context27release_locks_stored_beforeE17enum_mdl_durationP10MDL_ticket+0x3a)[0x7f1dc5a89b2a]
|
Jun 21 16:29:21 mysqld: /usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x8fd)[0x7f1dc59af5ad]
|
Jun 21 16:29:21 mysqld: /usr/sbin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x26a)[0x7f1dc59b880a]
|
Jun 21 16:29:21 mysqld: /usr/sbin/mysqld(+0x4448dc)[0x7f1dc59b88dc]
|
Jun 21 16:29:21 mysqld: /usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x1c02)[0x7f1dc59bac22]
|
Jun 21 16:29:21 mysqld: /usr/sbin/mysqld(_Z10do_commandP3THD+0x2a7)[0x7f1dc59bb657]
|
Jun 21 16:29:21 mysqld: /usr/sbin/mysqld(_Z24do_handle_one_connectionP3THD+0x58b)[0x7f1dc5a7f38b]
|
Jun 21 16:29:21 mysqld: /usr/sbin/mysqld(handle_one_connection+0x47)[0x7f1dc5a7f487]
|
Jun 21 16:29:21 mysqld: /lib/x86_64-linux-gnu/libpthread.so.0(+0x6b50)[0x7f1dc513cb50]
|
Jun 21 16:29:21 mysqld: /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7f1dc3849fbd]
|
Jun 21 16:29:21 mysqld:
|
Jun 21 16:29:21 mysqld: Trying to get some variables.
|
Jun 21 16:29:21 mysqld: Some pointers may be invalid and cause the dump to abort.
|
Jun 21 16:29:21 mysqld: Query (0x7f1db4c77020): is an invalid pointer
|
Jun 21 16:29:21 mysqld: Connection ID (thread ID): 457704
|
Jun 21 16:29:21 mysqld: Status: NOT_KILLED
|
Jun 21 16:29:21 mysqld:
|
nirbhay_c, it seems to be the same problem as MDEV-8737 (and it's again Galera). Maybe looking at both issues together will help to figure out something.
|
|
with wsrep_debug:
15:44:36 mysqld: 160622 15:44:36 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT value FROM mdl_config WHERE name = 'siteidentifier'
|
15:44:36 mysqld: 160622 15:44:36 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT * FROM mdl_context WHERE contextlevel = '10'
|
15:44:36 mysqld: 160622 15:44:36 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT * FROM mdl_course WHERE category = '0'
|
15:44:36 mysqld: 160622 15:44:36 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT id FROM mdl_sessions WHERE sid = 'ef32uag70j5livshop2jcoc6k4'
|
15:44:36 mysqld: 160622 15:44:36 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT id, sessdata FROM mdl_sessions WHERE id = '13855436'
|
15:44:36 mysqld: 160622 15:44:36 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT id, sid, state, userid, lastip, timecreated, timemodified FROM mdl_sessions WHERE sid = 'ef32uag70j5livshop2jcoc6k4'
|
15:44:36 mysqld: 160622 15:44:36 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT * FROM mdl_context WHERE id = '165725'
|
15:44:36 mysqld: 160622 15:44:36 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT md.name
|
15:44:36 mysqld: FROM mdl_modules md
|
15:44:36 mysqld: JOIN mdl_course_modules cm ON cm.module = md.id
|
15:44:36 mysqld: WHERE cm.id = '97664'
|
15:44:36 mysqld: 160622 15:44:36 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT cm.*, m.name, md.name AS modname
|
15:44:36 mysqld: FROM mdl_course_modules cm
|
15:44:36 mysqld: JOIN mdl_modules md ON md.id = cm.module
|
15:44:36 mysqld: JOIN mdl_resource m ON m.id = cm.instance
|
15:44:36 mysqld:
|
15:44:36 mysqld: WHERE cm.id = '97664' AND md.name = 'resource'
|
15:44:36 mysqld: 160622 15:44:36 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT * FROM mdl_course WHERE id = '2042'
|
15:44:36 mysqld: 160622 15:44:36 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT * FROM mdl_context WHERE contextlevel = '50' AND instanceid = '2042'
|
15:44:36 mysqld: 160622 15:44:36 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT ctx.*
|
15:44:36 mysqld: FROM mdl_context ctx
|
15:44:36 mysqld: WHERE ctx.path LIKE '/1/137505/131/163550/163826/165590/%'
|
15:44:36 mysqld: 160622 15:44:36 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT name,value FROM mdl_user_preferences WHERE userid = '49922'
|
15:44:36 mysqld: 160622 15:44:36 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT name,value FROM mdl_cache_flags WHERE flagtype = 'accesslib/dirtycontexts' AND expiry >= '1466603076' AND timemodified > '1466601770'
|
15:44:36 mysqld: 160622 15:44:36 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT f.id AS id, f.contenthash, f.pathnamehash, f.contextid, f.component, f.filearea, f.itemid, f.filepath, f.filename, f.userid, f.filesize, f.mimetype, f.status, f.source, f.author, f.license, f.timecreated, f.timemodified, f.sortorder, f.referencefileid, r.repositoryid AS repositoryid, r.reference AS reference, r.lastsync AS referencelastsync
|
15:44:36 mysqld: FROM mdl_files f
|
15:44:36 mysqld: LEFT JOIN mdl_files_reference r
|
15:44:36 mysqld: ON f.referencefileid = r.id
|
15:44:36 mysqld: WHERE f.pathnamehash = '8f1bc760a3606fa58edcfce70a43d741331f4da9'
|
15:44:36 mysqld: 160622 15:44:36 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT value FROM mdl_config WHERE name = 'siteidentifier'
|
15:44:36 mysqld: 160622 15:44:36 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT * FROM mdl_context WHERE contextlevel = '10'
|
15:44:36 mysqld: 160622 15:44:36 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT * FROM mdl_course WHERE category = '0'
|
15:44:36 mysqld: 160622 15:44:36 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT id FROM mdl_sessions WHERE sid = 'ef32uag70j5livshop2jcoc6k4'
|
15:44:36 mysqld: 160622 15:44:36 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT id, sessdata FROM mdl_sessions WHERE id = '13855436'
|
15:44:36 mysqld: 160622 15:44:36 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT id, sid, state, userid, lastip, timecreated, timemodified FROM mdl_sessions WHERE sid = 'ef32uag70j5livshop2jcoc6k4'
|
15:44:36 mysqld: 160622 15:44:36 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT * FROM mdl_context WHERE id = '165725'
|
15:44:36 mysqld: 160622 15:44:36 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT md.name
|
15:44:36 mysqld: FROM mdl_modules md
|
15:44:36 mysqld: JOIN mdl_course_modules cm ON cm.module = md.id
|
15:44:36 mysqld: WHERE cm.id = '97664'
|
15:44:36 mysqld: 160622 15:44:36 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT cm.*, m.name, md.name AS modname
|
15:44:36 mysqld: FROM mdl_course_modules cm
|
15:44:36 mysqld: JOIN mdl_modules md ON md.id = cm.module
|
15:44:36 mysqld: JOIN mdl_resource m ON m.id = cm.instance
|
15:44:36 mysqld:
|
15:44:36 mysqld: WHERE cm.id = '97664' AND md.name = 'resource'
|
15:44:36 mysqld: 160622 15:44:36 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT * FROM mdl_course WHERE id = '2042'
|
15:44:36 mysqld: 160622 15:44:36 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT * FROM mdl_context WHERE contextlevel = '50' AND instanceid = '2042'
|
15:44:36 mysqld: 160622 15:44:36 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT ctx.*
|
15:44:36 mysqld: FROM mdl_context ctx
|
15:44:36 mysqld: WHERE ctx.path LIKE '/1/137505/131/163550/163826/165590/%'
|
15:44:37 mysqld: 160622 15:44:37 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT name,value FROM mdl_user_preferences WHERE userid = '49922'
|
15:44:37 mysqld: 160622 15:44:37 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT name,value FROM mdl_cache_flags WHERE flagtype = 'accesslib/dirtycontexts' AND expiry >= '1466603077' AND timemodified > '1466601770'
|
15:44:37 mysqld: 160622 15:44:37 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT f.id AS id, f.contenthash, f.pathnamehash, f.contextid, f.component, f.filearea, f.itemid, f.filepath, f.filename, f.userid, f.filesize, f.mimetype, f.status, f.source, f.author, f.license, f.timecreated, f.timemodified, f.sortorder, f.referencefileid, r.repositoryid AS repositoryid, r.reference AS reference, r.lastsync AS referencelastsync
|
15:44:37 mysqld: FROM mdl_files f
|
15:44:37 mysqld: LEFT JOIN mdl_files_reference r
|
15:44:37 mysqld: ON f.referencefileid = r.id
|
15:44:37 mysqld: WHERE f.pathnamehash = '8f1bc760a3606fa58edcfce70a43d741331f4da9'
|
15:44:37 mysqld: 160622 15:44:37 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT value FROM mdl_config WHERE name = 'siteidentifier'
|
15:44:37 mysqld: 160622 15:44:37 [ERROR] mysqld got signal 11 ;
|
15:44:37 mysqld: This could be because you hit a bug. It is also possible that this binary
|
15:44:37 mysqld: or one of the libraries it was linked against is corrupt, improperly built,
|
15:44:37 mysqld: or misconfigured. This error can also be caused by malfunctioning hardware.
|
15:44:37 mysqld:
|
15:44:37 mysqld: To report this bug, see https://mariadb.com/kb/en/reporting-bugs
|
15:44:37 mysqld:
|
15:44:37 mysqld: We will try our best to scrape up some info that will hopefully help
|
15:44:37 mysqld: diagnose the problem, but since we have already crashed,
|
15:44:37 mysqld: something is definitely wrong and this may fail.
|
15:44:37 mysqld:
|
15:44:37 mysqld: Server version: 10.0.25-MariaDB-1~wheezy-wsrep
|
15:44:37 mysqld: key_buffer_size=1073741824
|
15:44:37 mysqld: read_buffer_size=2097152
|
15:44:37 mysqld: max_used_connections=7
|
15:44:37 mysqld: max_threads=2002
|
15:44:37 mysqld: thread_count=4
|
15:44:37 mysqld: It is possible that mysqld could use up to
|
15:44:37 mysqld: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 13389306 K bytes of memory
|
15:44:37 mysqld: Hope that's ok; if not, decrease some variables in the equation.
|
15:44:37 mysqld:
|
15:44:37 mysqld: Thread pointer: 0x0x7fdb7c995008
|
15:44:37 mysqld: Attempting backtrace. You can use the following information to find out
|
15:44:37 mysqld: where mysqld died. If you see no messages after this, something went
|
15:44:37 mysqld: terribly wrong...
|
15:44:37 mysqld: stack_bottom = 0x7fdcedb0cde0 thread_stack 0x40000
|
15:44:37 mysqld: /usr/sbin/mysqld(my_print_stacktrace+0x2b)[0x7fdced4ce76b]
|
15:44:37 mysqld: /usr/sbin/mysqld(handle_fatal_signal+0x422)[0x7fdced0485a2]
|
15:44:37 mysqld: /lib/x86_64-linux-gnu/libpthread.so.0(+0xf0a0)[0x7fdcec6640a0]
|
15:44:37 mysqld: /usr/sbin/mysqld(_ZN8MDL_lock11Ticket_list13remove_ticketEP10MDL_ticket+0x11)[0x7fdcecfa7681]
|
15:44:37 mysqld: /usr/sbin/mysqld(_ZN8MDL_lock13remove_ticketEMS_NS_11Ticket_listEP10MDL_ticket+0x41)[0x7fdcecfa7c41]
|
15:44:37 mysqld: /usr/sbin/mysqld(_ZN11MDL_context27release_locks_stored_beforeE17enum_mdl_durationP10MDL_ticket+0x3a)[0x7fdcecfa8b2a]
|
15:44:37 mysqld: /usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x8fd)[0x7fdcecece5ad]
|
15:44:37 mysqld: /usr/sbin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x26a)[0x7fdceced780a]
|
15:44:37 mysqld: /usr/sbin/mysqld(+0x4448dc)[0x7fdceced78dc]
|
15:44:37 mysqld: /usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x1c02)[0x7fdceced9c22]
|
15:44:37 mysqld: /usr/sbin/mysqld(_Z10do_commandP3THD+0x2a7)[0x7fdceceda657]
|
15:44:37 mysqld: /usr/sbin/mysqld(_Z24do_handle_one_connectionP3THD+0x58b)[0x7fdcecf9e38b]
|
15:44:37 mysqld: /usr/sbin/mysqld(handle_one_connection+0x47)[0x7fdcecf9e487]
|
15:44:37 mysqld: /lib/x86_64-linux-gnu/libpthread.so.0(+0x6b50)[0x7fdcec65bb50]
|
15:44:37 mysqld: /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7fdcead68fbd]
|
15:44:37 mysqld:
|
15:44:37 mysqld: Trying to get some variables.
|
15:44:37 mysqld: Some pointers may be invalid and cause the dump to abort.
|
15:44:37 mysqld: Query (0x7fdcdc170020): is an invalid pointer
|
15:44:37 mysqld: Connection ID (thread ID): 99668
|
15:44:37 mysqld: Status: NOT_KILLED
|
15:44:37 mysqld:
|
15:44:37 mysqld: Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on
|
15:44:37 mysqld:
|
15:44:37 mysqld: The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
|
15:44:37 mysqld: information that should help you find out what is causing the crash.
|
15:44:37 mysqld_safe: Number of processes running now: 0
|
15:44:37 mysqld_safe: WSREP: not restarting wsrep node automatically
|
15:44:37 mysqld_safe: mysqld from pid file /var/run/mysqld/mysqld.pid ended
|
|
|
wow, 5 days w/o crash!
todays crash log:
Jun 28 12:59:11 B-DBCL-03 mysqld: 160628 12:59:11 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT id FROM scheduler_task WHERE lock_key = 0 ORDER BY id ASC
|
Jun 28 12:59:13 B-DBCL-03 mysqld: 160628 12:59:13 [Note] WSREP: cleanup transaction for LOCAL_STATE: select url,rotation,resolution from doorsigns.rooms where hostname = 'door1513'
|
Jun 28 12:59:13 B-DBCL-03 mysqld: 160628 12:59:13 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT * FROM `config` WHERE `cat` = 'config'
|
Jun 28 12:59:13 B-DBCL-03 mysqld: 160628 12:59:13 [ERROR] mysqld got signal 11 ;
|
Jun 28 12:59:13 B-DBCL-03 mysqld: This could be because you hit a bug. It is also possible that this binary
|
Jun 28 12:59:13 B-DBCL-03 mysqld: or one of the libraries it was linked against is corrupt, improperly built,
|
Jun 28 12:59:13 B-DBCL-03 mysqld: or misconfigured. This error can also be caused by malfunctioning hardware.
|
Jun 28 12:59:13 B-DBCL-03 mysqld:
|
Jun 28 12:59:13 B-DBCL-03 mysqld: To report this bug, see https://mariadb.com/kb/en/reporting-bugs
|
Jun 28 12:59:13 B-DBCL-03 mysqld:
|
Jun 28 12:59:13 B-DBCL-03 mysqld: We will try our best to scrape up some info that will hopefully help
|
Jun 28 12:59:13 B-DBCL-03 mysqld: diagnose the problem, but since we have already crashed,
|
Jun 28 12:59:13 B-DBCL-03 mysqld: something is definitely wrong and this may fail.
|
Jun 28 12:59:13 B-DBCL-03 mysqld:
|
Jun 28 12:59:13 B-DBCL-03 mysqld: Server version: 10.0.25-MariaDB-1~wheezy-wsrep
|
Jun 28 12:59:13 B-DBCL-03 mysqld: key_buffer_size=1073741824
|
Jun 28 12:59:13 B-DBCL-03 mysqld: read_buffer_size=2097152
|
Jun 28 12:59:13 B-DBCL-03 mysqld: max_used_connections=14
|
Jun 28 12:59:13 B-DBCL-03 mysqld: max_threads=2002
|
Jun 28 12:59:13 B-DBCL-03 mysqld: thread_count=4
|
Jun 28 12:59:13 B-DBCL-03 mysqld: It is possible that mysqld could use up to
|
Jun 28 12:59:13 B-DBCL-03 mysqld: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 13389306 K bytes of memory
|
Jun 28 12:59:13 B-DBCL-03 mysqld: Hope that's ok; if not, decrease some variables in the equation.
|
Jun 28 12:59:13 B-DBCL-03 mysqld:
|
Jun 28 12:59:13 B-DBCL-03 mysqld: Thread pointer: 0x0x7fb43bdca008
|
Jun 28 12:59:13 B-DBCL-03 mysqld: Attempting backtrace. You can use the following information to find out
|
Jun 28 12:59:13 B-DBCL-03 mysqld: where mysqld died. If you see no messages after this, something went
|
Jun 28 12:59:13 B-DBCL-03 mysqld: terribly wrong...
|
Jun 28 12:59:13 B-DBCL-03 mysqld: stack_bottom = 0x7fb5abd71de0 thread_stack 0x40000
|
Jun 28 12:59:14 B-DBCL-03 mysqld: /usr/sbin/mysqld(my_print_stacktrace+0x2b)[0x7fb5ac8b576b]
|
Jun 28 12:59:14 B-DBCL-03 mysqld: /usr/sbin/mysqld(handle_fatal_signal+0x422)[0x7fb5ac42f5a2]
|
Jun 28 12:59:14 B-DBCL-03 mysqld: /lib/x86_64-linux-gnu/libpthread.so.0(+0xf0a0)[0x7fb5aba4b0a0]
|
Jun 28 12:59:14 B-DBCL-03 mysqld: /usr/sbin/mysqld(free_root+0x33)[0x7fb5ac8aa1b3]
|
Jun 28 12:59:14 B-DBCL-03 mysqld: /usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x5ab)[0x7fb5ac2bf5cb]
|
Jun 28 12:59:14 B-DBCL-03 mysqld: /usr/sbin/mysqld(_Z10do_commandP3THD+0x2a7)[0x7fb5ac2c1657]
|
Jun 28 12:59:14 B-DBCL-03 mysqld: /usr/sbin/mysqld(_Z24do_handle_one_connectionP3THD+0x58b)[0x7fb5ac38538b]
|
Jun 28 12:59:14 B-DBCL-03 mysqld: /usr/sbin/mysqld(handle_one_connection+0x47)[0x7fb5ac385487]
|
Jun 28 12:59:14 B-DBCL-03 mysqld: /lib/x86_64-linux-gnu/libpthread.so.0(+0x6b50)[0x7fb5aba42b50]
|
Jun 28 12:59:14 B-DBCL-03 mysqld: /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7fb5aa14ffbd]
|
Jun 28 12:59:14 B-DBCL-03 mysqld:
|
Jun 28 12:59:14 B-DBCL-03 mysqld: Trying to get some variables.
|
Jun 28 12:59:14 B-DBCL-03 mysqld: Some pointers may be invalid and cause the dump to abort.
|
Jun 28 12:59:14 B-DBCL-03 mysqld: Query (0x0): is an invalid pointer
|
Jun 28 12:59:14 B-DBCL-03 mysqld: Connection ID (thread ID): 3283091
|
Jun 28 12:59:14 B-DBCL-03 mysqld: Status: NOT_KILLED
|
Jun 28 12:59:14 B-DBCL-03 mysqld:
|
Jun 28 12:59:14 B-DBCL-03 mysqld: Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on
|
Jun 28 12:59:14 B-DBCL-03 mysqld:
|
Jun 28 12:59:14 B-DBCL-03 mysqld: The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
|
Jun 28 12:59:14 B-DBCL-03 mysqld: information that should help you find out what is causing the crash.
|
Jun 28 12:59:14 B-DBCL-03 mysqld_safe: Number of processes running now: 0
|
Jun 28 12:59:14 B-DBCL-03 mysqld_safe: WSREP: not restarting wsrep node automatically
|
Jun 28 12:59:14 B-DBCL-03 mysqld_safe: mysqld from pid file /var/run/mysqld/mysqld.pid ended
|
last lines of mysql logging before crash:
3283088 Connect readonly@web-04 as anonymous on
|
3283088 Query select @@version_comment limit 1
|
3283088 Query SHOW STATUS LIKE 'wsrep_local_state'
|
3283088 Query SHOW STATUS LIKE 'wsrep_incoming_addresses'
|
3283088 Quit
|
3283089 Connect readonly@oc as anonymous on
|
3283089 Query select @@version_comment limit 1
|
3283089 Query SHOW STATUS LIKE 'wsrep_local_state'
|
3283089 Query SHOW STATUS LIKE 'wsrep_incoming_addresses'
|
3283089 Quit
|
3283090 Connect readonly@sso as anonymous on
|
3283090 Query select @@version_comment limit 1
|
3283090 Query SHOW STATUS LIKE 'wsrep_local_state'
|
3283090 Query SHOW STATUS LIKE 'wsrep_incoming_addresses'
|
3283090 Quit
|
3283091 Connect frienicadm@web1 as anonymous on frienicadb
|
3283091 Query SELECT * FROM `config` WHERE `cat` = 'config'
|
3283092 Connect readonly@web1 as anonymous on
|
3283092 Query select @@version_comment limit 1
|
3283092 Query SHOW STATUS LIKE 'wsrep_local_state'
|
3283092 Query SHOW STATUS LIKE 'wsrep_incoming_addresses'
|
3283092 Quit
|
160628 12:59:14 3283093 Connect readonly@web2 as anonymous on
|
3283093 Query select @@version_comment limit 1
|
3283093 Query SHOW STATUS LIKE 'wsrep_local_state'
|
3283093 Query SHOW STATUS LIKE 'wsrep_incoming_addresses'
|
3283093 Quit
|
|
|
2 days later again:
160630 17:08:10 [ERROR] mysqld got signal 11 ;
|
This could be because you hit a bug. It is also possible that this binary
|
or one of the libraries it was linked against is corrupt, improperly built,
|
or misconfigured. This error can also be caused by malfunctioning hardware.
|
|
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
|
|
We will try our best to scrape up some info that will hopefully help
|
diagnose the problem, but since we have already crashed,
|
something is definitely wrong and this may fail.
|
|
Server version: 10.0.25-MariaDB-1~wheezy-wsrep
|
key_buffer_size=1073741824
|
read_buffer_size=2097152
|
max_used_connections=5
|
max_threads=2002
|
thread_count=4
|
It is possible that mysqld could use up to
|
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 13389306 K bytes of memory
|
Hope that's ok; if not, decrease some variables in the equation.
|
|
Thread pointer: 0x0x7f4d8809c008
|
Attempting backtrace. You can use the following information to find out
|
where mysqld died. If you see no messages after this, something went
|
terribly wrong...
|
stack_bottom = 0x7f4d98f7bde0 thread_stack 0x40000
|
/usr/sbin/mysqld(my_print_stacktrace+0x2b)[0x7f4f19ae476b]
|
/usr/sbin/mysqld(handle_fatal_signal+0x422)[0x7f4f1965e5a2]
|
/lib/x86_64-linux-gnu/libpthread.so.0(+0xf0a0)[0x7f4f18c7a0a0]
|
/usr/sbin/mysqld(free_root+0x33)[0x7f4f19ad91b3]
|
/usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x5ab)[0x7f4f194ee5cb]
|
/usr/sbin/mysqld(_Z10do_commandP3THD+0x2a7)[0x7f4f194f0657]
|
/usr/sbin/mysqld(_Z24do_handle_one_connectionP3THD+0x58b)[0x7f4f195b438b]
|
/usr/sbin/mysqld(handle_one_connection+0x47)[0x7f4f195b4487]
|
/lib/x86_64-linux-gnu/libpthread.so.0(+0x6b50)[0x7f4f18c71b50]
|
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7f4f1737efbd]
|
|
Trying to get some variables.
|
Some pointers may be invalid and cause the dump to abort.
|
Query (0x0): is an invalid pointer
|
Connection ID (thread ID): 1117268
|
Status: NOT_KILLED
|
|
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on
|
|
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
|
information that should help you find out what is causing the crash.
|
mysqld_safe: Number of processes running now: 0
|
mysqld_safe: WSREP: not restarting wsrep node automatically
|
mysqld_safe: mysqld from pid file /var/run/mysqld/mysqld.pid ended
|
|
last lines of mysql logging before crash:
1117264 Connect readonly@web1 as anonymous on
|
1117264 Query select @@version_comment limit 1
|
1117264 Query SHOW STATUS LIKE 'wsrep_local_state'
|
1117264 Query SHOW STATUS LIKE 'wsrep_incoming_addresses'
|
1117264 Quit
|
1108245 Query UPDATE scheduler_future_task
|
SET lock_key = '100100028416', lock_time =
|
'2016-06-30 17:09:45'
|
|
WHERE lock_key = 0 AND execution_time <= '2016-06-30 17:09:45'
|
1108245 Query SELECT id, name, task_type, task_data, attempts
|
FROM scheduler_future_task
|
WHERE lock_key = '100100028416'
|
ORDER BY execution_time ASC
|
160630 17:08:10 1117266 Connect readonly@web2 as anonymous on
|
1117265 Connect readonly@sso as anonymous on
|
1117265 Query select @@version_comment limit 1
|
1117266 Query select @@version_comment limit 1
|
1117265 Query SHOW STATUS LIKE 'wsrep_local_state'
|
1117266 Query SHOW STATUS LIKE 'wsrep_local_state'
|
1117265 Query SHOW STATUS LIKE 'wsrep_incoming_addresses'
|
1117266 Query SHOW STATUS LIKE 'wsrep_incoming_addresses'
|
1117265 Quit
|
1117266 Quit
|
1117267 Connect debian-sys-maint@localhost as anonymous on
|
1117267 Query show /*!50002 GLOBAL */ status
|
1117267 Quit
|
1117268 Connect debian-sys-maint@localhost as anonymous on
|
1117268 Query show /*!50002 GLOBAL */ status
|
|
|
6 days later a new crash with some wsrep kill query before crash:
Jul 7 10:27:46 B-DBCL-03 mysqld: 160707 10:27:46 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT value FROM mdl_config WHERE name = 'siteidentifier'
|
Jul 7 10:27:46 B-DBCL-03 mysqld: 160707 10:27:46 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT * FROM mdl_context WHERE contextlevel = '10'
|
Jul 7 10:27:46 B-DBCL-03 mysqld: 160707 10:27:46 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT * FROM mdl_course WHERE category = '0'
|
Jul 7 10:27:46 B-DBCL-03 mysqld: 160707 10:27:46 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT id FROM mdl_sessions WHERE sid = 'tn14ekqcpp8ggcolprc75fse44'
|
Jul 7 10:27:46 B-DBCL-03 mysqld: 160707 10:27:46 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT id, sessdata FROM mdl_sessions WHERE id = '13925996'
|
Jul 7 10:27:46 B-DBCL-03 mysqld: 160707 10:27:46 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT id, sid, state, userid, lastip, timecreated, timemodified FROM mdl_sessions WHERE sid = 'tn14ekqcpp8ggcolprc75fse44'
|
Jul 7 10:27:46 B-DBCL-03 mysqld: 160707 10:27:46 [Note] WSREP: cluster conflict due to high priority abort for threads:
|
Jul 7 10:27:46 B-DBCL-03 mysqld: 160707 10:27:46 [Note] WSREP: Winning thread:
|
Jul 7 10:27:46 B-DBCL-03 mysqld: THD: 2, mode: applier, state: executing, conflict: no conflict, seqno: 46673369
|
Jul 7 10:27:46 B-DBCL-03 mysqld: SQL: (null)
|
Jul 7 10:27:46 B-DBCL-03 mysqld: 160707 10:27:46 [Note] WSREP: Victim thread:
|
Jul 7 10:27:46 B-DBCL-03 mysqld: THD: 3611946, mode: local, state: executing, conflict: no conflict, seqno: -1
|
Jul 7 10:27:46 B-DBCL-03 mysqld: SQL: UPDATE mdl_sessions SET timemodified = '1467880066' WHERE id='13925996'
|
Jul 7 10:27:46 B-DBCL-03 mysqld: 160707 10:27:46 [Note] WSREP: BF kill (1, seqno: 46673369), victim: (3611946) trx: 255275360
|
Jul 7 10:27:46 B-DBCL-03 mysqld: 160707 10:27:46 [Note] WSREP: Aborting query: UPDATE mdl_sessions SET timemodified = '1467880066' WHERE id='13925996'
|
Jul 7 10:27:46 B-DBCL-03 mysqld: 160707 10:27:46 [Note] WSREP: kill trx QUERY_EXEC for 255275360
|
Jul 7 10:27:46 B-DBCL-03 mysqld: 160707 10:27:46 [Note] WSREP: kill query for: 3611946
|
Jul 7 10:27:46 B-DBCL-03 mysqld: 160707 10:27:46 [Note] WSREP: client rollback due to BF abort for (3611946), query: UPDATE mdl_sessions SET timemodified = '1467880066' WHERE id='13925996'
|
Jul 7 10:27:46 B-DBCL-03 mysqld: 160707 10:27:46 [Note] WSREP: abort in exec query state, avoiding autocommit
|
Jul 7 10:27:46 B-DBCL-03 mysqld: 160707 10:27:46 [Note] WSREP: wsrep retrying AC query: UPDATE mdl_sessions SET timemodified = '1467880066' WHERE id='13925996'
|
Jul 7 10:27:46 B-DBCL-03 mysqld: 160707 10:27:46 [Note] WSREP: cleanup transaction for LOCAL_STATE: UPDATE mdl_sessions SET timemodified = '1467880066' WHERE id='13925996'
|
Jul 7 10:27:46 B-DBCL-03 mysqld: 160707 10:27:46 [Note] WSREP: releasing retry_query: conf 0 sent 0 kill 0 errno 0 SQL UPDATE mdl_sessions SET timemodified = '1467880066' WHERE id='13925996'
|
Jul 7 10:27:46 B-DBCL-03 mysqld: 160707 10:27:46 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT * FROM mdl_context WHERE id = '150167'
|
Jul 7 10:27:46 B-DBCL-03 mysqld: 160707 10:27:46 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT f.id AS id, f.contenthash, f.pathnamehash, f.contextid, f.component, f.filearea, f.itemid, f.filepath, f.filename, f.userid, f.filesize, f.mimetype, f.status, f.source, f.author, f.license, f.timecreated, f.timemodified, f.sortorder, f.referencefileid, r.repositoryid AS repositoryid, r.reference AS reference, r.lastsync AS referencelastsync
|
Jul 7 10:27:46 B-DBCL-03 mysqld: FROM mdl_files f
|
Jul 7 10:27:46 B-DBCL-03 mysqld: LEFT JOIN mdl_files_reference r
|
Jul 7 10:27:46 B-DBCL-03 mysqld: ON f.referencefileid = r.id
|
Jul 7 10:27:46 B-DBCL-03 mysqld: WHERE f.pathnamehash = '7a51781f23cca9d5712f54d97239ea93d6ff7a65'
|
Jul 7 10:27:46 B-DBCL-03 mysqld: 160707 10:27:46 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT f.id AS id, f.contenthash, f.pathnamehash, f.contextid, f.component, f.filearea, f.itemid, f.filepath, f.filename, f.userid, f.filesize, f.mimetype, f.status, f.source, f.author, f.license, f.timecreated, f.timemodified, f.sortorder, f.referencefileid, r.repositoryid AS repositoryid, r.reference AS reference, r.lastsync AS referencelastsync
|
Jul 7 10:27:46 B-DBCL-03 mysqld: FROM mdl_files f
|
Jul 7 10:27:46 B-DBCL-03 mysqld: LEFT JOIN mdl_files_reference r
|
Jul 7 10:27:46 B-DBCL-03 mysqld: ON f.referencefileid = r.id
|
Jul 7 10:27:46 B-DBCL-03 mysqld: WHERE f.pathnamehash = '94c54b0b30ac619a48be92214a6d84f4e778acf7'
|
Jul 7 10:27:48 B-DBCL-03 mysqld: 160707 10:27:48 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT id, name, lock_time
|
Jul 7 10:27:48 B-DBCL-03 mysqld: FROM scheduler_recurrent_task
|
Jul 7 10:27:48 B-DBCL-03 mysqld: WHERE task_type = 'GenericAgent'
|
Jul 7 10:27:48 B-DBCL-03 mysqld: AND lock_key <> 0
|
Jul 7 10:27:48 B-DBCL-03 mysqld: ORDER BY id ASC
|
Jul 7 10:27:50 B-DBCL-03 mysqld: 160707 10:27:50 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT option_name, option_value FROM mban_options WHERE autoload = 'yes'
|
Jul 7 10:27:50 B-DBCL-03 mysqld: 160707 10:27:50 [ERROR] mysqld got signal 11 ;
|
Jul 7 10:27:50 B-DBCL-03 mysqld: This could be because you hit a bug. It is also possible that this binary
|
Jul 7 10:27:50 B-DBCL-03 mysqld: or one of the libraries it was linked against is corrupt, improperly built,
|
Jul 7 10:27:50 B-DBCL-03 mysqld: or misconfigured. This error can also be caused by malfunctioning hardware.
|
Jul 7 10:27:50 B-DBCL-03 mysqld:
|
Jul 7 10:27:50 B-DBCL-03 mysqld: To report this bug, see https://mariadb.com/kb/en/reporting-bugs
|
Jul 7 10:27:50 B-DBCL-03 mysqld:
|
Jul 7 10:27:50 B-DBCL-03 mysqld: We will try our best to scrape up some info that will hopefully help
|
Jul 7 10:27:50 B-DBCL-03 mysqld: diagnose the problem, but since we have already crashed,
|
Jul 7 10:27:50 B-DBCL-03 mysqld: something is definitely wrong and this may fail.
|
Jul 7 10:27:50 B-DBCL-03 mysqld:
|
Jul 7 10:27:50 B-DBCL-03 mysqld: Server version: 10.0.25-MariaDB-1~wheezy-wsrep
|
Jul 7 10:27:50 B-DBCL-03 mysqld: key_buffer_size=1073741824
|
Jul 7 10:27:50 B-DBCL-03 mysqld: read_buffer_size=2097152
|
Jul 7 10:27:50 B-DBCL-03 mysqld: max_used_connections=7
|
Jul 7 10:27:50 B-DBCL-03 mysqld: max_threads=2002
|
Jul 7 10:27:50 B-DBCL-03 mysqld: thread_count=4
|
Jul 7 10:27:50 B-DBCL-03 mysqld: It is possible that mysqld could use up to
|
Jul 7 10:27:50 B-DBCL-03 mysqld: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 13389306 K bytes of memory
|
Jul 7 10:27:50 B-DBCL-03 mysqld: Hope that's ok; if not, decrease some variables in the equation.
|
Jul 7 10:27:50 B-DBCL-03 mysqld:
|
Jul 7 10:27:50 B-DBCL-03 mysqld: Thread pointer: 0x0x7fca998be008
|
Jul 7 10:27:50 B-DBCL-03 mysqld: Attempting backtrace. You can use the following information to find out
|
Jul 7 10:27:50 B-DBCL-03 mysqld: where mysqld died. If you see no messages after this, something went
|
Jul 7 10:27:50 B-DBCL-03 mysqld: terribly wrong...
|
Jul 7 10:27:50 B-DBCL-03 mysqld: stack_bottom = 0x7fcc29c3dde0 thread_stack 0x40000
|
Jul 7 10:27:50 B-DBCL-03 mysqld: /usr/sbin/mysqld(my_print_stacktrace+0x2b)[0x7fcc2964076b]
|
Jul 7 10:27:50 B-DBCL-03 mysqld: /usr/sbin/mysqld(handle_fatal_signal+0x422)[0x7fcc291ba5a2]
|
Jul 7 10:27:50 B-DBCL-03 mysqld: /lib/x86_64-linux-gnu/libpthread.so.0(+0xf0a0)[0x7fcc287d60a0]
|
Jul 7 10:27:50 B-DBCL-03 mysqld: /usr/sbin/mysqld(_ZN8MDL_lock11Ticket_list13remove_ticketEP10MDL_ticket+0x11)[0x7fcc29119681]
|
Jul 7 10:27:50 B-DBCL-03 mysqld: /usr/sbin/mysqld(_ZN8MDL_lock13remove_ticketEMS_NS_11Ticket_listEP10MDL_ticket+0x41)[0x7fcc29119c41]
|
Jul 7 10:27:50 B-DBCL-03 mysqld: /usr/sbin/mysqld(_ZN11MDL_context27release_locks_stored_beforeE17enum_mdl_durationP10MDL_ticket+0x3a)[0x7fcc2911ab2a]
|
Jul 7 10:27:50 B-DBCL-03 mysqld: /usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x8fd)[0x7fcc290405ad]
|
Jul 7 10:27:50 B-DBCL-03 mysqld: /usr/sbin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x26a)[0x7fcc2904980a]
|
Jul 7 10:27:50 B-DBCL-03 mysqld: /usr/sbin/mysqld(+0x4448dc)[0x7fcc290498dc]
|
Jul 7 10:27:50 B-DBCL-03 mysqld: /usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x1c02)[0x7fcc2904bc22]
|
Jul 7 10:27:50 B-DBCL-03 mysqld: /usr/sbin/mysqld(_Z10do_commandP3THD+0x2a7)[0x7fcc2904c657]
|
Jul 7 10:27:50 B-DBCL-03 mysqld: /usr/sbin/mysqld(_Z24do_handle_one_connectionP3THD+0x58b)[0x7fcc2911038b]
|
Jul 7 10:27:50 B-DBCL-03 mysqld: /usr/sbin/mysqld(handle_one_connection+0x47)[0x7fcc29110487]
|
Jul 7 10:27:50 B-DBCL-03 mysqld: /lib/x86_64-linux-gnu/libpthread.so.0(+0x6b50)[0x7fcc287cdb50]
|
Jul 7 10:27:50 B-DBCL-03 mysqld: /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7fcc26edafbd]
|
Jul 7 10:27:50 B-DBCL-03 mysqld:
|
Jul 7 10:27:50 B-DBCL-03 mysqld: Trying to get some variables.
|
Jul 7 10:27:50 B-DBCL-03 mysqld: Some pointers may be invalid and cause the dump to abort.
|
Jul 7 10:27:50 B-DBCL-03 mysqld: Query (0x7fcab983b020): is an invalid pointer
|
Jul 7 10:27:50 B-DBCL-03 mysqld: Connection ID (thread ID): 3611970
|
Jul 7 10:27:50 B-DBCL-03 mysqld: Status: NOT_KILLED
|
Jul 7 10:27:50 B-DBCL-03 mysqld:
|
Jul 7 10:27:50 B-DBCL-03 mysqld: Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on
|
Jul 7 10:27:50 B-DBCL-03 mysqld:
|
Jul 7 10:27:50 B-DBCL-03 mysqld: The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
|
Jul 7 10:27:50 B-DBCL-03 mysqld: information that should help you find out what is causing the crash.
|
Jul 7 10:27:51 B-DBCL-03 mysqld_safe: Number of processes running now: 0
|
Jul 7 10:27:51 B-DBCL-03 mysqld_safe: WSREP: not restarting wsrep node automatically
|
Jul 7 10:27:51 B-DBCL-03 mysqld_safe: mysqld from pid file /var/run/mysqld/mysqld.pid ended
|
query log:
160707 10:27:50 3611967 Connect readonly@www2 as anonymous on
|
3611967 Query select @@version_comment limit 1
|
3611967 Query SHOW STATUS LIKE 'wsrep_local_state'
|
3611967 Query SHOW STATUS LIKE 'wsrep_incoming_addresses'
|
3611967 Quit
|
3611968 Connect readonly@www4 as anonymous on
|
3611968 Query select @@version_comment limit 1
|
3611968 Query SHOW STATUS LIKE 'wsrep_local_state'
|
3611968 Query SHOW STATUS LIKE 'wsrep_incoming_addresses'
|
3611968 Quit
|
3611969 Connect readonly@fs1 as anonymous on
|
3611969 Query select @@version_comment limit 1
|
3611969 Query SHOW STATUS LIKE 'wsrep_local_state'
|
3611969 Query SHOW STATUS LIKE 'wsrep_incoming_addresses'
|
3611969 Quit
|
3611970 Connect wordpress@www1 as anonymous on
|
3611970 Query SELECT @@SESSION.sql_mode
|
3611970 Init DB wordpress
|
3611970 Query SELECT option_name, option_value FROM mban_options WHERE autoload = 'yes'
|
3611971 Connect readonly@www3 as anonymous on
|
3611971 Query select @@version_comment limit 1
|
3611971 Query SHOW STATUS LIKE 'wsrep_local_state'
|
3611971 Query SHOW STATUS LIKE 'wsrep_incoming_addresses'
|
3611971 Quit
|
plugins:
MariaDB [(none)]> SHOW PLUGINS;
|
+-----------------------------+----------+--------------------+---------+---------+
|
| Name | Status | Type | Library | License |
|
+-----------------------------+----------+--------------------+---------+---------+
|
| binlog | ACTIVE | STORAGE ENGINE | NULL | GPL |
|
| mysql_native_password | ACTIVE | AUTHENTICATION | NULL | GPL |
|
| mysql_old_password | ACTIVE | AUTHENTICATION | NULL | GPL |
|
| wsrep | ACTIVE | STORAGE ENGINE | NULL | GPL |
|
| MEMORY | ACTIVE | STORAGE ENGINE | NULL | GPL |
|
| MyISAM | ACTIVE | STORAGE ENGINE | NULL | GPL |
|
| CSV | ACTIVE | STORAGE ENGINE | NULL | GPL |
|
| MRG_MyISAM | ACTIVE | STORAGE ENGINE | NULL | GPL |
|
| PERFORMANCE_SCHEMA | ACTIVE | STORAGE ENGINE | NULL | GPL |
|
| InnoDB | ACTIVE | STORAGE ENGINE | NULL | GPL |
|
| XTRADB_READ_VIEW | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
|
| XTRADB_INTERNAL_HASH_TABLES | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
|
| XTRADB_RSEG | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
|
| INNODB_TRX | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
|
| INNODB_LOCKS | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
|
| INNODB_LOCK_WAITS | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
|
| INNODB_CMP | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
|
| INNODB_CMP_RESET | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
|
| INNODB_CMPMEM | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
|
| INNODB_CMPMEM_RESET | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
|
| INNODB_CMP_PER_INDEX | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
|
| INNODB_CMP_PER_INDEX_RESET | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
|
| INNODB_BUFFER_PAGE | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
|
| INNODB_BUFFER_PAGE_LRU | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
|
| INNODB_BUFFER_POOL_STATS | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
|
| INNODB_METRICS | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
|
| INNODB_FT_DEFAULT_STOPWORD | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
|
| INNODB_FT_DELETED | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
|
| INNODB_FT_BEING_DELETED | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
|
| INNODB_FT_CONFIG | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
|
| INNODB_FT_INDEX_CACHE | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
|
| INNODB_FT_INDEX_TABLE | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
|
| INNODB_SYS_TABLES | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
|
| INNODB_SYS_TABLESTATS | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
|
| INNODB_SYS_INDEXES | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
|
| INNODB_SYS_COLUMNS | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
|
| INNODB_SYS_FIELDS | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
|
| INNODB_SYS_FOREIGN | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
|
| INNODB_SYS_FOREIGN_COLS | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
|
| INNODB_SYS_TABLESPACES | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
|
| INNODB_SYS_DATAFILES | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
|
| INNODB_CHANGED_PAGES | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
|
| ARCHIVE | ACTIVE | STORAGE ENGINE | NULL | GPL |
|
| Aria | ACTIVE | STORAGE ENGINE | NULL | GPL |
|
| FEDERATED | ACTIVE | STORAGE ENGINE | NULL | GPL |
|
| BLACKHOLE | ACTIVE | STORAGE ENGINE | NULL | GPL |
|
| FEEDBACK | DISABLED | INFORMATION SCHEMA | NULL | GPL |
|
| partition | ACTIVE | STORAGE ENGINE | NULL | GPL |
|
+-----------------------------+----------+--------------------+---------+---------+
|
48 rows in set (0.00 sec)
|
|
|
10.0.26 don't fix the problem.
|
|
syslog (wsrep_debug=ON):
160712 16:24:12 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT * FROM mdl_course WHERE id = '2580'
|
160712 16:24:12 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT * FROM mdl_context WHERE contextlevel = '50' AND instanceid = '2580'
|
160712 16:24:12 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT ctx.*
|
FROM mdl_context ctx
|
WHERE ctx.path LIKE '/1/3400/86407/22229/207728/%'
|
160712 16:24:12 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT name,value FROM mdl_user_preferences WHERE userid = '6'
|
160712 16:24:12 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT f.id AS id, f.contenthash, f.pathnamehash, f.contextid, f.component, f.filearea, f.itemid, f.filepath, f.filename, f.userid, f.filesize, f.mimetype, f.status, f.source, f.author, f.license, f.timecreated, f.timemodified, f.sortorder, f.referencefileid, r.repositoryid AS repositoryid, r.reference AS reference, r.lastsync AS referencelastsync
|
FROM mdl_files f
|
LEFT JOIN mdl_files_reference r
|
ON f.referencefileid = r.id
|
WHERE f.pathnamehash = '525f8eb4688c460d58f8d0cea254c640feb79149'
|
160712 16:24:12 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT id, sessdata FROM mdl_sessions WHERE id = '13941027'
|
160712 16:24:12 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT id, sid, state, userid, lastip, timecreated, timemodified FROM mdl_sessions WHERE sid = 'if3fu5huo3u23im9mbnbkdq545'
|
160712 16:24:12 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT * FROM mdl_context WHERE id = '207972'
|
160712 16:24:12 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT md.name
|
FROM mdl_modules md
|
JOIN mdl_course_modules cm ON cm.module = md.id
|
WHERE cm.id = '121312'
|
160712 16:24:12 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT cm.*, m.name, md.name AS modname
|
FROM mdl_course_modules cm
|
JOIN mdl_modules md ON md.id = cm.module
|
JOIN mdl_scorm m ON m.id = cm.instance
|
|
WHERE cm.id = '121312' AND md.name = 'scorm'
|
160712 16:24:12 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT * FROM mdl_course WHERE id = '2580'
|
160712 16:24:12 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT * FROM mdl_context WHERE contextlevel = '50' AND instanceid = '2580'
|
160712 16:24:12 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT ctx.*
|
FROM mdl_context ctx
|
WHERE ctx.path LIKE '/1/3400/86407/22229/207728/%'
|
160712 16:24:12 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT value FROM mdl_config WHERE name = 'siteidentifier'
|
160712 16:24:12 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT * FROM mdl_context WHERE contextlevel = '10'
|
160712 16:24:12 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT * FROM mdl_course WHERE category = '0'
|
160712 16:24:12 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT id FROM mdl_sessions WHERE sid = 'if3fu5huo3u23im9mbnbkdq545'
|
160712 16:24:12 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT name,value FROM mdl_user_preferences WHERE userid = '6'
|
160712 16:24:12 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT f.id AS id, f.contenthash, f.pathnamehash, f.contextid, f.component, f.filearea, f.itemid, f.filepath, f.filename, f.userid, f.filesize, f.mimetype, f.status, f.source, f.author, f.license, f.timecreated, f.timemodified, f.sortorder, f.referencefileid, r.repositoryid AS repositoryid, r.reference AS reference, r.lastsync AS referencelastsync
|
FROM mdl_files f
|
LEFT JOIN mdl_files_reference r
|
ON f.referencefileid = r.id
|
WHERE f.pathnamehash = '08bc056250418e5ae16a1614c3e1b826ab8332d3'
|
160712 16:24:12 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT id, sessdata FROM mdl_sessions WHERE id = '13941027'
|
160712 16:24:12 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT id, sid, state, userid, lastip, timecreated, timemodified FROM mdl_sessions WHERE sid = 'if3fu5huo3u23im9mbnbkdq545'
|
160712 16:24:12 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT * FROM mdl_context WHERE id = '207972'
|
160712 16:24:12 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT md.name
|
FROM mdl_modules md
|
JOIN mdl_course_modules cm ON cm.module = md.id
|
WHERE cm.id = '121312'
|
160712 16:24:12 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT cm.*, m.name, md.name AS modname
|
FROM mdl_course_modules cm
|
JOIN mdl_modules md ON md.id = cm.module
|
JOIN mdl_scorm m ON m.id = cm.instance
|
|
WHERE cm.id = '121312' AND md.name = 'scorm'
|
160712 16:24:12 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT * FROM mdl_course WHERE id = '2580'
|
160712 16:24:12 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT * FROM mdl_context WHERE contextlevel = '50' AND instanceid = '2580'
|
160712 16:24:12 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT value FROM mdl_config WHERE name = 'siteidentifier'
|
160712 16:24:12 [ERROR] mysqld got signal 11 ;
|
This could be because you hit a bug. It is also possible that this binary
|
or one of the libraries it was linked against is corrupt, improperly built,
|
or misconfigured. This error can also be caused by malfunctioning hardware.
|
|
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
|
|
We will try our best to scrape up some info that will hopefully help
|
diagnose the problem, but since we have already crashed,
|
something is definitely wrong and this may fail.
|
|
Server version: 10.0.26-MariaDB-1~wheezy-wsrep
|
key_buffer_size=1073741824
|
read_buffer_size=2097152
|
max_used_connections=8
|
max_threads=2002
|
thread_count=6
|
It is possible that mysqld could use up to
|
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 13389306 K bytes of memory
|
Hope that's ok; if not, decrease some variables in the equation.
|
|
Thread pointer: 0x0x7f674980c008
|
Attempting backtrace. You can use the following information to find out
|
where mysqld died. If you see no messages after this, something went
|
terribly wrong...
|
stack_bottom = 0x7f68d3c5ede0 thread_stack 0x40000
|
160712 16:24:12 [Note] WSREP: cleanup transaction for LOCAL_STATE: SELECT ctx.*
|
FROM mdl_context ctx
|
WHERE ctx.path LIKE '/1/3400/86407/22229/207728/%'
|
/usr/sbin/mysqld(my_print_stacktrace+0x2b)[0x7f68d36a281b]
|
/usr/sbin/mysqld(handle_fatal_signal+0x422)[0x7f68d321a5c2]
|
/lib/x86_64-linux-gnu/libpthread.so.0(+0xf0a0)[0x7f68d28360a0]
|
/usr/sbin/mysqld(_ZN8MDL_lock11Ticket_list13remove_ticketEP10MDL_ticket+0x11)[0x7f68d31793e1]
|
/usr/sbin/mysqld(_ZN8MDL_lock13remove_ticketEMS_NS_11Ticket_listEP10MDL_ticket+0x41)[0x7f68d31799a1]
|
/usr/sbin/mysqld(_ZN11MDL_context27release_locks_stored_beforeE17enum_mdl_durationP10MDL_ticket+0x3a)[0x7f68d317a88a]
|
/usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x8fd)[0x7f68d309ff9d]
|
/usr/sbin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x26a)[0x7f68d30a91fa]
|
/usr/sbin/mysqld(+0x4442cc)[0x7f68d30a92cc]
|
/usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x1c02)[0x7f68d30ab612]
|
/usr/sbin/mysqld(_Z10do_commandP3THD+0x2a7)[0x7f68d30ac047]
|
/usr/sbin/mysqld(_Z24do_handle_one_connectionP3THD+0x58b)[0x7f68d31700eb]
|
/usr/sbin/mysqld(handle_one_connection+0x47)[0x7f68d31701e7]
|
/lib/x86_64-linux-gnu/libpthread.so.0(+0x6b50)[0x7f68d282db50]
|
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7f68d0f3afbd]
|
|
Trying to get some variables.
|
Some pointers may be invalid and cause the dump to abort.
|
Query (0x7f68c2844020): is an invalid pointer
|
Connection ID (thread ID): 7563
|
Status: NOT_KILLED
|
|
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on
|
|
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
|
information that should help you find out what is causing the crash.
|
Number of processes running now: 0
|
WSREP: not restarting wsrep node automatically
|
mysqld from pid file /var/run/mysqld/mysqld.pid ended
|
|
mysql query log:
7563 Connect moodle@192.168.80.54 as anonymous on moodle
|
7563 Query SET NAMES utf8
|
7563 Query SET SESSION sql_mode = 'STRICT_ALL_TABLES'
|
7563 Query SELECT value FROM mdl_config WHERE name = 'siteidentifier'
|
7562 Query SELECT ctx.*
|
FROM mdl_context ctx
|
WHERE ctx.path LIKE '/1/3400/86407/22229/207728/%'
|
|
|
root@esmt.org Will it possible for you to run debug mysqld (i.e. mysqld built with -D -DCMAKE_BUILD_TYPE=Debug) under same load?
|
|
no more crash!
my solution: don't use glbd! I switched to keepalived (no more load balancing, but still failover).
|
|
Support for 10.0-galera has ended.
|
my.cnf