After a clean install and minimum configuration, MariaDB crashes on start up with semisync enabled.
bash# curl -sS https://downloads.mariadb.com/MariaDB/mariadb_repo_setup | sudo bash -s -- --mariadb-server-version=mariadb-10.3
|
bash# yum install MariaDB-server
|
[mysqld]
|
log_bin
|
binlog_format=ROW
|
server_id=1000
|
rpl_semi_sync_master_enabled=1
|
rpl_semi_sync_slave_enabled=1
|
bash# systemctl start MariaDB
Log output:
Apr 6 17:20:41 ip-10-0-1-250 mysqld: Status: NOT_KILLED
|
Apr 6 17:20:41 ip-10-0-1-250 mysqld: Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on
|
Apr 6 17:20:41 ip-10-0-1-250 mysqld: The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
|
Apr 6 17:20:41 ip-10-0-1-250 mysqld: information that should help you find out what is causing the crash.
|
Apr 6 17:20:46 ip-10-0-1-250 mysqld: 2018-04-06 17:20:46 0 [Note] /usr/sbin/mysqld (mysqld 10.3.5-MariaDB-log) starting as process 12508 ...
|
Apr 6 17:20:46 ip-10-0-1-250 mysqld: 2018-04-06 17:20:46 0 [Note] Semi-sync replication initialized for transactions.
|
Apr 6 17:20:46 ip-10-0-1-250 mysqld: 2018-04-06 17:20:46 0 [Note] Semi-sync replication enabled on the master.
|
Apr 6 17:20:46 ip-10-0-1-250 mysqld: 2018-04-06 17:20:46 0 [Warning] No argument was provided to --log-bin and neither --log-basename or --log-bin-index where used; This may cause repliction to break when this server acts as a master and has its hostname changed! Please use '--log-basename=ip-10-0-1-250.ec2.internal' or '--log-bin=ip-10-0-1-250-bin' to avoid this problem.
|
Apr 6 17:20:46 ip-10-0-1-250 mysqld: 180406 17:20:46 [ERROR] mysqld got signal 11 ;
|
Apr 6 17:20:46 ip-10-0-1-250 mysqld: This could be because you hit a bug. It is also possible that this binary
|
Apr 6 17:20:46 ip-10-0-1-250 mysqld: or one of the libraries it was linked against is corrupt, improperly built,
|
Apr 6 17:20:46 ip-10-0-1-250 mysqld: or misconfigured. This error can also be caused by malfunctioning hardware.
|
Apr 6 17:20:46 ip-10-0-1-250 mysqld: To report this bug, see https://mariadb.com/kb/en/reporting-bugs
|
Apr 6 17:20:46 ip-10-0-1-250 mysqld: We will try our best to scrape up some info that will hopefully help
|
Apr 6 17:20:46 ip-10-0-1-250 mysqld: diagnose the problem, but since we have already crashed,
|
Apr 6 17:20:46 ip-10-0-1-250 mysqld: something is definitely wrong and this may fail.
|
Apr 6 17:20:46 ip-10-0-1-250 mysqld: Server version: 10.3.5-MariaDB-log
|
Apr 6 17:20:46 ip-10-0-1-250 mysqld: key_buffer_size=0
|
Apr 6 17:20:46 ip-10-0-1-250 mysqld: read_buffer_size=131072
|
Apr 6 17:20:46 ip-10-0-1-250 mysqld: max_used_connections=0
|
Apr 6 17:20:46 ip-10-0-1-250 mysqld: max_threads=153
|
Apr 6 17:20:46 ip-10-0-1-250 mysqld: thread_count=0
|
Apr 6 17:20:46 ip-10-0-1-250 mysqld: It is possible that mysqld could use up to
|
Apr 6 17:20:46 ip-10-0-1-250 mysqld: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 336312 K bytes of memory
|
Apr 6 17:20:46 ip-10-0-1-250 mysqld: Hope that's ok; if not, decrease some variables in the equation.
|
Apr 6 17:20:46 ip-10-0-1-250 mysqld: Thread pointer: 0x7fd3000009a8
|
Apr 6 17:20:46 ip-10-0-1-250 mysqld: Attempting backtrace. You can use the following information to find out
|
Apr 6 17:20:46 ip-10-0-1-250 mysqld: where mysqld died. If you see no messages after this, something went
|
Apr 6 17:20:46 ip-10-0-1-250 mysqld: terribly wrong...
|
Apr 6 17:20:46 ip-10-0-1-250 mysqld: stack_bottom = 0x0 thread_stack 0x49000
|
Apr 6 17:20:46 ip-10-0-1-250 mysqld: mysys/stacktrace.c:270(my_print_stacktrace)[0x7fd31b4dfcce]
|
Apr 6 17:20:46 ip-10-0-1-250 mysqld: sql/signal_handler.cc:168(handle_fatal_signal)[0x7fd31af6dd17]
|
Apr 6 17:20:46 ip-10-0-1-250 mysqld: sigaction.c:0(__restore_rt)[0x7fd31a3da100]
|
Apr 6 17:20:46 ip-10-0-1-250 mysqld: sql/sql_plugin.cc:970(intern_plugin_lock)[0x7fd31adad374]
|
Apr 6 17:20:46 ip-10-0-1-250 mysqld: sql/sql_class.cc:1286(THD::init(bool))[0x7fd31ad618e8]
|
Apr 6 17:20:46 ip-10-0-1-250 mysqld: sql/sql_profile.h:318(PROFILING::set_thd(THD*))[0x7fd31ad62a5f]
|
Apr 6 17:20:46 ip-10-0-1-250 mysqld: sql/semisync_master_ack_receiver.cc:209(Ack_receiver::run())[0x7fd31aebdf47]
|
Apr 6 17:20:46 ip-10-0-1-250 mysqld: sql/semisync_master_ack_receiver.cc:36(ack_receive_handler)[0x7fd31aebe589]
|
Apr 6 17:20:46 ip-10-0-1-250 mysqld: pthread_create.c:0(start_thread)[0x7fd31a3d2dc5]
|
Apr 6 17:20:46 ip-10-0-1-250 mysqld: /lib64/libc.so.6(clone+0x6d)[0x7fd318a2a28d]
|
Apr 6 17:20:46 ip-10-0-1-250 mysqld: Trying to get some variables.
|
Apr 6 17:20:46 ip-10-0-1-250 mysqld: Some pointers may be invalid and cause the dump to abort.
|
Apr 6 17:20:46 ip-10-0-1-250 mysqld: Query (0x0):
|
Apr 6 17:20:46 ip-10-0-1-250 mysqld: Connection ID (thread ID): 1
|
Apr 6 17:20:46 ip-10-0-1-250 mysqld: Status: NOT_KILLED
|
|
{"report":{"fcp":1784.8000001907349,"ttfb":659.8000001907349,"pageVisibility":"visible","entityId":66710,"key":"jira.project.issue.view-issue","isInitial":true,"threshold":1000,"elementTimings":{},"userDeviceMemory":8,"userDeviceProcessors":32,"apdex":0.5,"journeyId":"6e01593b-beda-4ee5-90e0-9d537780620f","navigationType":0,"readyForUser":1883,"redirectCount":0,"resourceLoadedEnd":1630.6000003814697,"resourceLoadedStart":668.8000001907349,"resourceTiming":[{"duration":366.19999980926514,"initiatorType":"link","name":"https://jira.mariadb.org/s/2c21342762a6a02add1c328bed317ffd-CDN/lu2cib/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/css/_super/batch.css","startTime":668.8000001907349,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":668.8000001907349,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1035,"responseStart":0,"secureConnectionStart":0},{"duration":366.30000019073486,"initiatorType":"link","name":"https://jira.mariadb.org/s/7ebd35e77e471bc30ff0eba799ebc151-CDN/lu2cib/820016/12ta74/494e4c556ecbb29f90a3d3b4f09cb99c/_/download/contextbatch/css/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true&whisper-enabled=true","startTime":669,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":669,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1035.3000001907349,"responseStart":0,"secureConnectionStart":0},{"duration":466,"initiatorType":"script","name":"https://jira.mariadb.org/s/0917945aaa57108d00c5076fea35e069-CDN/lu2cib/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/js/_super/batch.js?locale=en","startTime":669.1999998092651,"connectEnd":669.1999998092651,"connectStart":669.1999998092651,"domainLookupEnd":669.1999998092651,"domainLookupStart":669.1999998092651,"fetchStart":669.1999998092651,"redirectEnd":0,"redirectStart":0,"requestStart":1040.5,"responseEnd":1135.1999998092651,"responseStart":1062.1999998092651,"secureConnectionStart":669.1999998092651},{"duration":737.5,"initiatorType":"script","name":"https://jira.mariadb.org/s/2d8175ec2fa4c816e8023260bd8c1786-CDN/lu2cib/820016/12ta74/494e4c556ecbb29f90a3d3b4f09cb99c/_/download/contextbatch/js/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true&whisper-enabled=true","startTime":669.6000003814697,"connectEnd":669.6000003814697,"connectStart":669.6000003814697,"domainLookupEnd":669.6000003814697,"domainLookupStart":669.6000003814697,"fetchStart":669.6000003814697,"redirectEnd":0,"redirectStart":0,"requestStart":1040.1000003814697,"responseEnd":1407.1000003814697,"responseStart":1067.3000001907349,"secureConnectionStart":669.6000003814697},{"duration":441.8999996185303,"initiatorType":"script","name":"https://jira.mariadb.org/s/a9324d6758d385eb45c462685ad88f1d-CDN/lu2cib/820016/12ta74/c92c0caa9a024ae85b0ebdbed7fb4bd7/_/download/contextbatch/js/atl.global,-_super/batch.js?locale=en","startTime":669.9000005722046,"connectEnd":669.9000005722046,"connectStart":669.9000005722046,"domainLookupEnd":669.9000005722046,"domainLookupStart":669.9000005722046,"fetchStart":669.9000005722046,"redirectEnd":0,"redirectStart":0,"requestStart":1043.1999998092651,"responseEnd":1111.8000001907349,"responseStart":1069.9000005722046,"secureConnectionStart":669.9000005722046},{"duration":431.19999980926514,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-en/jira.webresources:calendar-en.js","startTime":670.1000003814697,"connectEnd":670.1000003814697,"connectStart":670.1000003814697,"domainLookupEnd":670.1000003814697,"domainLookupStart":670.1000003814697,"fetchStart":670.1000003814697,"redirectEnd":0,"redirectStart":0,"requestStart":1045.6000003814697,"responseEnd":1101.3000001907349,"responseStart":1099.3000001907349,"secureConnectionStart":670.1000003814697},{"duration":406.5,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-localisation-moment/jira.webresources:calendar-localisation-moment.js","startTime":670.3000001907349,"connectEnd":670.3000001907349,"connectStart":670.3000001907349,"domainLookupEnd":670.3000001907349,"domainLookupStart":670.3000001907349,"fetchStart":670.3000001907349,"redirectEnd":0,"redirectStart":0,"requestStart":1047.3000001907349,"responseEnd":1076.8000001907349,"responseStart":1074.4000005722046,"secureConnectionStart":670.3000001907349},{"duration":376,"initiatorType":"link","name":"https://jira.mariadb.org/s/b04b06a02d1959df322d9cded3aeecc1-CDN/lu2cib/820016/12ta74/a2ff6aa845ffc9a1d22fe23d9ee791fc/_/download/contextbatch/css/jira.global.look-and-feel,-_super/batch.css","startTime":670.6000003814697,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":670.6000003814697,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1046.6000003814697,"responseStart":0,"secureConnectionStart":0},{"duration":409.30000019073486,"initiatorType":"script","name":"https://jira.mariadb.org/rest/api/1.0/shortcuts/820016/47140b6e0a9bc2e4913da06536125810/shortcuts.js?context=issuenavigation&context=issueaction","startTime":670.8000001907349,"connectEnd":670.8000001907349,"connectStart":670.8000001907349,"domainLookupEnd":670.8000001907349,"domainLookupStart":670.8000001907349,"fetchStart":670.8000001907349,"redirectEnd":0,"redirectStart":0,"requestStart":1061,"responseEnd":1080.1000003814697,"responseStart":1077.4000005722046,"secureConnectionStart":670.8000001907349},{"duration":386.3999996185303,"initiatorType":"link","name":"https://jira.mariadb.org/s/3ac36323ba5e4eb0af2aa7ac7211b4bb-CDN/lu2cib/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/css/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.css?jira.create.linked.issue=true","startTime":670.9000005722046,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":670.9000005722046,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1057.3000001907349,"responseStart":0,"secureConnectionStart":0},{"duration":430.5,"initiatorType":"script","name":"https://jira.mariadb.org/s/5d5e8fe91fbc506585e83ea3b62ccc4b-CDN/lu2cib/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/js/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.js?jira.create.linked.issue=true&locale=en","startTime":671,"connectEnd":671,"connectStart":671,"domainLookupEnd":671,"domainLookupStart":671,"fetchStart":671,"redirectEnd":0,"redirectStart":0,"requestStart":1061.8000001907349,"responseEnd":1101.5,"responseStart":1081.6999998092651,"secureConnectionStart":671},{"duration":958.5999994277954,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-js/jira.webresources:bigpipe-js.js","startTime":671.9000005722046,"connectEnd":671.9000005722046,"connectStart":671.9000005722046,"domainLookupEnd":671.9000005722046,"domainLookupStart":671.9000005722046,"fetchStart":671.9000005722046,"redirectEnd":0,"redirectStart":0,"requestStart":1428.4000005722046,"responseEnd":1630.5,"responseStart":1622.5,"secureConnectionStart":671.9000005722046},{"duration":953.4000005722046,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-init/jira.webresources:bigpipe-init.js","startTime":677.1999998092651,"connectEnd":677.1999998092651,"connectStart":677.1999998092651,"domainLookupEnd":677.1999998092651,"domainLookupStart":677.1999998092651,"fetchStart":677.1999998092651,"redirectEnd":0,"redirectStart":0,"requestStart":1436.6999998092651,"responseEnd":1630.6000003814697,"responseStart":1623.6999998092651,"secureConnectionStart":677.1999998092651},{"duration":175.19999980926514,"initiatorType":"xmlhttprequest","name":"https://jira.mariadb.org/rest/webResources/1.0/resources","startTime":1304.3000001907349,"connectEnd":1304.3000001907349,"connectStart":1304.3000001907349,"domainLookupEnd":1304.3000001907349,"domainLookupStart":1304.3000001907349,"fetchStart":1304.3000001907349,"redirectEnd":0,"redirectStart":0,"requestStart":1427.9000005722046,"responseEnd":1479.5,"responseStart":1477.3000001907349,"secureConnectionStart":1304.3000001907349}],"fetchStart":0,"domainLookupStart":0,"domainLookupEnd":0,"connectStart":0,"connectEnd":0,"requestStart":493,"responseStart":660,"responseEnd":675,"domLoading":664,"domInteractive":2019,"domContentLoadedEventStart":2019,"domContentLoadedEventEnd":2090,"domComplete":2752,"loadEventStart":2752,"loadEventEnd":2752,"userAgent":"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)","marks":[{"name":"bigPipe.sidebar-id.start","time":1985.1999998092651},{"name":"bigPipe.sidebar-id.end","time":1986},{"name":"bigPipe.activity-panel-pipe-id.start","time":1986.1000003814697},{"name":"bigPipe.activity-panel-pipe-id.end","time":1987.6000003814697},{"name":"activityTabFullyLoaded","time":2108}],"measures":[],"correlationId":"e77fd830a11ac9","effectiveType":"4g","downlink":10,"rtt":0,"serverDuration":100,"dbReadsTimeInMs":10,"dbConnsTimeInMs":20,"applicationHash":"9d11dbea5f4be3d4cc21f03a88dd11d8c8687422","experiments":[]}}
Bug
Blocker
--rpl_semi_sync_master_enabled=1 alone on startup is already enough to get the crash. Setting it at runtime, as all MTR tests seem to do, doesn't cause the trouble.
10.3 d8da97b09ab
#3 <signal handler called>#4 0x000055aa6a26b89e in intern_plugin_lock (lex=0x0, rc=0x0, state_mask=14) at /data/src/10.3/sql/sql_plugin.cc:970#5 0x000055aa6a27240c in plugin_thdvar_init (thd=0x7f0ab4001d50) at /data/src/10.3/sql/sql_plugin.cc:3177#6 0x000055aa6a1f0e51 in THD::init (this=0x7f0ab4001d50, skip_lock=true) at /data/src/10.3/sql/sql_class.cc:1282#7 0x000055aa6a1ef974 in THD::THD (this=0x7f0ab4001d50, id=1, is_wsrep_applier=false, skip_global_sys_var_lock=true) at /data/src/10.3/sql/sql_class.cc:904#8 0x000055aa6a4309cd in Ack_receiver::run (this=0x55aa6b9c41c0 <ack_receiver>) at /data/src/10.3/sql/semisync_master_ack_receiver.cc:209#9 0x000055aa6a4300c8 in ack_receive_handler (arg=0x55aa6b9c41c0 <ack_receiver>) at /data/src/10.3/sql/semisync_master_ack_receiver.cc:35#10 0x00007f0ac9236494 in start_thread (arg=0x7f0ac6c36700) at pthread_create.c:333#11 0x00007f0ac761c93f in clone () from /lib/x86_64-linux-gnu/libc.so.6