I updated a cluster with wsrep_provider_options='socket.checksum=1' in the my.cnf of all nodes (from 5.5.29 to 5.5.37). I updated one node at a time and all when quite smoothly.
I then removed the wsrep_provider_options='socket.checksum=1' in node 1 and restarted it.
It crashed on startup with:
140502 8:43:56 [Note] WSREP: Received NON-PRIMARY.
|
140502 8:43:56 [Note] WSREP: Shifting SYNCED -> OPEN (TO: 9320857)
|
140502 8:43:56 [Note] WSREP: Received self-leave message.
|
140502 8:43:56 [Note] WSREP: Flow-control interval: [0, 0]
|
140502 8:43:56 [Note] WSREP: Received SELF-LEAVE. Closing connection.
|
140502 8:43:56 [Note] WSREP: Shifting OPEN -> CLOSED (TO: 9320857)
|
140502 8:43:56 [Note] WSREP: RECV thread exiting 0: Success
|
140502 8:43:56 [Note] WSREP: recv_thread() joined.
|
140502 8:43:56 [Note] WSREP: Closing replication queue.
|
140502 8:43:56 [Note] WSREP: Closing slave action queue.
|
140502 8:43:56 [Note] WSREP: New cluster view: global state: 727ac931-ae1d-11e2-0800-35fac11491b4:9320857, view# -1: non-Primary, number of nodes: 1, my index: 0, protocol version 2
|
140502 8:43:56 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
|
140502 8:43:56 [Note] WSREP: New cluster view: global state: 727ac931-ae1d-11e2-0800-35fac11491b4:9320857, view# -1: non-Primary, number of nodes: 0, my index: -1, protocol version 2
|
140502 8:43:56 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
|
140502 8:43:56 [Note] WSREP: applier thread exiting (code:0)
|
140502 8:43:56 [ERROR] mysqld got signal 11 ;
|
This could be because you hit a bug. It is also possible that this binary
|
or one of the libraries it was linked against is corrupt, improperly built,
|
or misconfigured. This error can also be caused by malfunctioning hardware.
|
|
To report this bug, see http://kb.askmonty.org/en/reporting-bugs
|
|
We will try our best to scrape up some info that will hopefully help
|
diagnose the problem, but since we have already crashed,
|
something is definitely wrong and this may fail.
|
|
Server version: 5.5.37-MariaDB-1~precise-wsrep-log
|
key_buffer_size=33554432
|
read_buffer_size=2097152
|
max_used_connections=4
|
max_threads=202
|
thread_count=1
|
It is possible that mysqld could use up to
|
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1277538 K bytes of memory
|
Hope that's ok; if not, decrease some variables in the equation.
|
|
Thread pointer: 0x0x7f6eff012000
|
Attempting backtrace. You can use the following information to find out
|
where mysqld died. If you see no messages after this, something went
|
terribly wrong...
|
stack_bottom = 0x7f6f003b2a20 thread_stack 0x48000
|
/usr/sbin/mysqld(my_print_stacktrace+0x2b)[0x7f6f14ee808b]
|
/usr/sbin/mysqld(handle_fatal_signal+0x471)[0x7f6f14b03ef1]
|
/lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0)[0x7f6f1334ecb0]
|
/usr/sbin/mysqld(+0x5a5e9c)[0x7f6f14bbbe9c]
|
/usr/sbin/mysqld(_ZN14Relay_log_infoD1Ev+0x1d2)[0x7f6f14a68e82]
|
/usr/sbin/mysqld(_ZN14Relay_log_infoD0Ev+0x11)[0x7f6f14a68f21]
|
/usr/sbin/mysqld(_ZN3THDD1Ev+0x18c)[0x7f6f14979a1c]
|
/usr/sbin/mysqld(_ZN3THDD0Ev+0x11)[0x7f6f1497a031]
|
/usr/sbin/mysqld(_Z29one_thread_per_connection_endP3THDb+0x18)[0x7f6f1491eab8]
|
/usr/sbin/mysqld(start_wsrep_THD+0x5d6)[0x7f6f1491f536]
|
/lib/x86_64-linux-gnu/libpthread.so.0(+0x7e9a)[0x7f6f13346e9a]
|
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7f6f12a773fd]
|
|
Trying to get some variables.
|
Some pointers may be invalid and cause the dump to abort.
|
Query (0x0): is an invalid pointer
|
Connection ID (thread ID): 2
|
Status: KILL_CONNECTION
|
|
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=off
|
|
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
|
information that should help you find out what is causing the crash.
|
140502 08:43:56 mysqld_safe Number of processes running now: 0
|
140502 08:43:56 mysqld_safe WSREP: not restarting wsrep node automatically
|
140502 08:43:56 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
|
There are no comments yet on this issue.
{"report":{"fcp":3169,"ttfb":290.69999980926514,"pageVisibility":"visible","entityId":36120,"key":"jira.project.issue.view-issue","isInitial":true,"threshold":1000,"elementTimings":{},"userDeviceMemory":8,"userDeviceProcessors":64,"apdex":0.5,"journeyId":"2c8d26bf-5886-42e8-8281-21c4f8e7fbec","navigationType":0,"readyForUser":3408.2999997138977,"redirectCount":0,"resourceLoadedEnd":3281.5,"resourceLoadedStart":406.7999997138977,"resourceTiming":[{"duration":354.2000002861023,"initiatorType":"link","name":"https://jira.mariadb.org/s/2c21342762a6a02add1c328bed317ffd-CDN/lu2cib/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/css/_super/batch.css","startTime":406.7999997138977,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":406.7999997138977,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":761,"responseStart":0,"secureConnectionStart":0},{"duration":354.2999997138977,"initiatorType":"link","name":"https://jira.mariadb.org/s/7ebd35e77e471bc30ff0eba799ebc151-CDN/lu2cib/820016/12ta74/494e4c556ecbb29f90a3d3b4f09cb99c/_/download/contextbatch/css/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true&whisper-enabled=true","startTime":407,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":407,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":761.2999997138977,"responseStart":0,"secureConnectionStart":0},{"duration":1901.8000001907349,"initiatorType":"script","name":"https://jira.mariadb.org/s/0917945aaa57108d00c5076fea35e069-CDN/lu2cib/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/js/_super/batch.js?locale=en","startTime":407.2999997138977,"connectEnd":407.2999997138977,"connectStart":407.2999997138977,"domainLookupEnd":407.2999997138977,"domainLookupStart":407.2999997138977,"fetchStart":407.2999997138977,"redirectEnd":0,"redirectStart":0,"requestStart":851.5999999046326,"responseEnd":2309.0999999046326,"responseStart":941.6999998092651,"secureConnectionStart":407.2999997138977},{"duration":2375.7999997138977,"initiatorType":"script","name":"https://jira.mariadb.org/s/2d8175ec2fa4c816e8023260bd8c1786-CDN/lu2cib/820016/12ta74/494e4c556ecbb29f90a3d3b4f09cb99c/_/download/contextbatch/js/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true&whisper-enabled=true","startTime":407.40000009536743,"connectEnd":407.40000009536743,"connectStart":407.40000009536743,"domainLookupEnd":407.40000009536743,"domainLookupStart":407.40000009536743,"fetchStart":407.40000009536743,"redirectEnd":0,"redirectStart":0,"requestStart":851.4000000953674,"responseEnd":2783.199999809265,"responseStart":913.1999998092651,"secureConnectionStart":407.40000009536743},{"duration":571.4000000953674,"initiatorType":"script","name":"https://jira.mariadb.org/s/a9324d6758d385eb45c462685ad88f1d-CDN/lu2cib/820016/12ta74/c92c0caa9a024ae85b0ebdbed7fb4bd7/_/download/contextbatch/js/atl.global,-_super/batch.js?locale=en","startTime":407.59999990463257,"connectEnd":407.59999990463257,"connectStart":407.59999990463257,"domainLookupEnd":407.59999990463257,"domainLookupStart":407.59999990463257,"fetchStart":407.59999990463257,"redirectEnd":0,"redirectStart":0,"requestStart":851.6999998092651,"responseEnd":979,"responseStart":976.2999997138977,"secureConnectionStart":407.59999990463257},{"duration":675.3000001907349,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-en/jira.webresources:calendar-en.js","startTime":407.7999997138977,"connectEnd":407.7999997138977,"connectStart":407.7999997138977,"domainLookupEnd":407.7999997138977,"domainLookupStart":407.7999997138977,"fetchStart":407.7999997138977,"redirectEnd":0,"redirectStart":0,"requestStart":851.7999997138977,"responseEnd":1083.0999999046326,"responseStart":977.2999997138977,"secureConnectionStart":407.7999997138977},{"duration":675.4000000953674,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-localisation-moment/jira.webresources:calendar-localisation-moment.js","startTime":408,"connectEnd":408,"connectStart":408,"domainLookupEnd":408,"domainLookupStart":408,"fetchStart":408,"redirectEnd":0,"redirectStart":0,"requestStart":852,"responseEnd":1083.4000000953674,"responseStart":979.5999999046326,"secureConnectionStart":408},{"duration":354.59999990463257,"initiatorType":"link","name":"https://jira.mariadb.org/s/b04b06a02d1959df322d9cded3aeecc1-CDN/lu2cib/820016/12ta74/a2ff6aa845ffc9a1d22fe23d9ee791fc/_/download/contextbatch/css/jira.global.look-and-feel,-_super/batch.css","startTime":408.19999980926514,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":408.19999980926514,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":762.7999997138977,"responseStart":0,"secureConnectionStart":0},{"duration":678.6000003814697,"initiatorType":"script","name":"https://jira.mariadb.org/rest/api/1.0/shortcuts/820016/47140b6e0a9bc2e4913da06536125810/shortcuts.js?context=issuenavigation&context=issueaction","startTime":408.2999997138977,"connectEnd":408.2999997138977,"connectStart":408.2999997138977,"domainLookupEnd":408.2999997138977,"domainLookupStart":408.2999997138977,"fetchStart":408.2999997138977,"redirectEnd":0,"redirectStart":0,"requestStart":852.0999999046326,"responseEnd":1086.9000000953674,"responseStart":1082,"secureConnectionStart":408.2999997138977},{"duration":354.5,"initiatorType":"link","name":"https://jira.mariadb.org/s/3ac36323ba5e4eb0af2aa7ac7211b4bb-CDN/lu2cib/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/css/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.css?jira.create.linked.issue=true","startTime":408.59999990463257,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":408.59999990463257,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":763.0999999046326,"responseStart":0,"secureConnectionStart":0},{"duration":997.7000002861023,"initiatorType":"script","name":"https://jira.mariadb.org/s/5d5e8fe91fbc506585e83ea3b62ccc4b-CDN/lu2cib/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/js/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.js?jira.create.linked.issue=true&locale=en","startTime":408.69999980926514,"connectEnd":408.69999980926514,"connectStart":408.69999980926514,"domainLookupEnd":408.69999980926514,"domainLookupStart":408.69999980926514,"fetchStart":408.69999980926514,"redirectEnd":0,"redirectStart":0,"requestStart":852.2999997138977,"responseEnd":1406.4000000953674,"responseStart":1003.9000000953674,"secureConnectionStart":408.69999980926514},{"duration":2307.5999999046326,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-js/jira.webresources:bigpipe-js.js","startTime":411.69999980926514,"connectEnd":411.69999980926514,"connectStart":411.69999980926514,"domainLookupEnd":411.69999980926514,"domainLookupStart":411.69999980926514,"fetchStart":411.69999980926514,"redirectEnd":0,"redirectStart":0,"requestStart":1404.2999997138977,"responseEnd":2719.2999997138977,"responseStart":2442.4000000953674,"secureConnectionStart":411.69999980926514},{"duration":2307.7000002861023,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-init/jira.webresources:bigpipe-init.js","startTime":411.69999980926514,"connectEnd":411.69999980926514,"connectStart":411.69999980926514,"domainLookupEnd":411.69999980926514,"domainLookupStart":411.69999980926514,"fetchStart":411.69999980926514,"redirectEnd":0,"redirectStart":0,"requestStart":1404.5,"responseEnd":2719.4000000953674,"responseStart":2516.0999999046326,"secureConnectionStart":411.69999980926514},{"duration":493,"initiatorType":"xmlhttprequest","name":"https://jira.mariadb.org/rest/webResources/1.0/resources","startTime":2449.5999999046326,"connectEnd":2449.5999999046326,"connectStart":2449.5999999046326,"domainLookupEnd":2449.5999999046326,"domainLookupStart":2449.5999999046326,"fetchStart":2449.5999999046326,"redirectEnd":0,"redirectStart":0,"requestStart":2449.5999999046326,"responseEnd":2942.5999999046326,"responseStart":2942.5999999046326,"secureConnectionStart":2449.5999999046326},{"duration":230.7000002861023,"initiatorType":"link","name":"https://jira.mariadb.org/s/d5715adaadd168a9002b108b2b039b50-CDN/lu2cib/820016/12ta74/be4b45e9cec53099498fa61c8b7acba4/_/download/contextbatch/css/jira.project.sidebar,-_super,-project.issue.navigator,-jira.general,-jira.browse.project,-jira.view.issue,-jira.global,-atl.general,-com.atlassian.jira.projects.sidebar.init/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true&whisper-enabled=true","startTime":3050.7999997138977,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":3050.7999997138977,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":3281.5,"responseStart":0,"secureConnectionStart":0},{"duration":197.19999980926514,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/e65b778d185daf5aee24936755b43da6/_/download/contextbatch/js/browser-metrics-plugin.contrib,-_super,-project.issue.navigator,-jira.view.issue,-atl.general/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true&whisper-enabled=true","startTime":3052,"connectEnd":3052,"connectStart":3052,"domainLookupEnd":3052,"domainLookupStart":3052,"fetchStart":3052,"redirectEnd":0,"redirectStart":0,"requestStart":3052,"responseEnd":3249.199999809265,"responseStart":3249.199999809265,"secureConnectionStart":3052},{"duration":371.7999997138977,"initiatorType":"script","name":"https://www.google-analytics.com/analytics.js","startTime":3145.4000000953674,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":3145.4000000953674,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":3517.199999809265,"responseStart":0,"secureConnectionStart":0}],"fetchStart":1,"domainLookupStart":1,"domainLookupEnd":1,"connectStart":1,"connectEnd":1,"requestStart":79,"responseStart":291,"responseEnd":406,"domLoading":307,"domInteractive":3466,"domContentLoadedEventStart":3466,"domContentLoadedEventEnd":3517,"domComplete":3933,"loadEventStart":3933,"loadEventEnd":3934,"userAgent":"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)","marks":[{"name":"bigPipe.sidebar-id.start","time":3449.199999809265},{"name":"bigPipe.sidebar-id.end","time":3450.2999997138977},{"name":"bigPipe.activity-panel-pipe-id.start","time":3450.5},{"name":"bigPipe.activity-panel-pipe-id.end","time":3451},{"name":"activityTabFullyLoaded","time":3521.199999809265}],"measures":[],"correlationId":"3d0875a970fbcf","effectiveType":"4g","downlink":9.9,"rtt":0,"serverDuration":95,"dbReadsTimeInMs":13,"dbConnsTimeInMs":24,"applicationHash":"9d11dbea5f4be3d4cc21f03a88dd11d8c8687422","experiments":[]}}