Details
-
Bug
-
Status: Closed (View Workflow)
-
Critical
-
Resolution: Fixed
-
10.6.20
-
debian bookwoorm
Description
We've experienced 3 crashes on three different hosts which are intermediate masters so:
Primary master -> secondary master -> slaves
The secondary master crashed on three different (and completely independent servers) while performing the following:
SET GLOBAL rpl_semi_sync_slave_enabled=OFF;
|
SET GLOBAL rpl_semi_sync_master_enabled=OFF;
|
stop slave; start slave;
|
After the above operation, we initiate a process of changing the replicas to hang from a secondary master
The process to do so is simple:
- Stop slave on the secondary master
- Move a replica under a different replica
- Start replication on the secondary master and the future new master
- Wait to catch up
- Start the process again (we have a number of replicas that is around 10).
In each case, the secondary master crashed when moving the last replica out from it.
These are the logs
db2214
2025-03-24T06:51:50.195163+00:00 db2214 mysqld[2434922]: 2025-03-24 6:51:50 843346037 [Note] Semi-sync replication switched OFF.
|
2025-03-24T06:51:50.197323+00:00 db2214 mysqld[2434922]: 2025-03-24 6:51:50 198149201 [Note] Stopping ack receiver thread
|
2025-03-24T06:51:55.074491+00:00 db2214 mysqld[2434922]: 2025-03-24 6:51:55 843094243 [Note] Slave SQL thread exiting, replication stopped in log 'db1173-bin.004212' at position 491243537; GTID position '0-180359184-3070029963,171966562-171966562-80913348,171970594-171970594-1063329989,171970705-171970705-239075862,171974662-171974662-2319916665,171974711-171974711-192010200,171974883-171974883-1921892293,171978766-171978766-1375989821,171978805-171978805-2751615153,171978904-171978904-198543478,172001036-172001036-184571046,180359184-180359184-35604081,180363274-180363274-7949150,180363370-180363370-7087924,180363389-180363389-877798080,180367474-180367474-111584823,180367475-180367475-1362966533,180367479-180367479-1635296654', master: db1173.eqiad.wmnet:3306
|
2025-03-24T06:51:55.075112+00:00 db2214 mysqld[2434922]: 2025-03-24 6:51:55 843094242 [Note] Slave I/O thread exiting, read up to log 'db1173-bin.004212', position 491243537; GTID position 171974883-171974883-1921892293,0-180359184-3070029963,180363389-180363389-877798080,180363274-180363274-7949150,171974711-171974711-192010200,171978904-171978904-198543478,171966562-171966562-80913348,171974662-171974662-2319916665,171970594-171970594-1063329989,180367475-180367475-1362966533,171978805-171978805-2751615153,180367474-180367474-111584823,180363370-180363370-7087924,180359184-180359184-35604081,171970705-171970705-239075862,171978766-171978766-1375989821,180367479-180367479-1635296654,172001036-172001036-184571046, master db1173.eqiad.wmnet:3306
|
2025-03-24T06:51:55.171537+00:00 db2214 mysqld[2434922]: 2025-03-24 6:51:55 843346050 [Note] Slave I/O thread: Start asynchronous replication to master 'repl2024@db1173.eqiad.wmnet:3306' in log 'db1173-bin.004212' at position 491243537
|
2025-03-24T06:51:55.355780+00:00 db2214 mysqld[2434922]: 2025-03-24 6:51:55 843346050 [Note] Slave I/O thread: connected to master 'repl2024@db1173.eqiad.wmnet:3306',replication starts at GTID position '171974883-171974883-1921892293,0-180359184-3070029963,180363389-180363389-877798080,180363274-180363274-7949150,171974711-171974711-192010200,171978904-171978904-198543478,171966562-171966562-80913348,171974662-171974662-2319916665,171970594-171970594-1063329989,180367475-180367475-1362966533,171978805-171978805-2751615153,180367474-180367474-111584823,180363370-180363370-7087924,180359184-180359184-35604081,171970705-171970705-239075862,171978766-171978766-1375989821,180367479-180367479-1635296654,172001036-172001036-184571046'
|
2025-03-24T06:52:05.900412+00:00 db2214 mysqld[2434922]: 2025-03-24 6:52:05 779841176 [Note] Stop semi-sync binlog_dump to slave (server_id: 180360462)
|
2025-03-24T06:53:57.876185+00:00 db2214 mysqld[2434922]: 2025-03-24 6:53:57 783776117 [Note] Stop semi-sync binlog_dump to slave (server_id: 180355296)
|
2025-03-24T06:53:58.405134+00:00 db2214 mysqld[2434922]: 2025-03-24 6:53:58 843346200 [Note] Start semi-sync binlog_dump to slave (server_id: 180355296), pos(./db2214-bin.001017, 163817059)
|
2025-03-24T06:54:15.150605+00:00 db2214 mysqld[2434922]: 2025-03-24 6:54:15 843346200 [Note] Stop semi-sync binlog_dump to slave (server_id: 180355296)
|
2025-03-24T06:54:19.050975+00:00 db2214 mysqld[2434922]: 2025-03-24 6:54:19 783505918 [Note] Stop semi-sync binlog_dump to slave (server_id: 180367385)
|
2025-03-24T06:54:19.578544+00:00 db2214 mysqld[2434922]: 2025-03-24 6:54:19 843346232 [Note] Start semi-sync binlog_dump to slave (server_id: 180367385), pos(./db2214-bin.001017, 164420908)
|
2025-03-24T06:54:36.339648+00:00 db2214 mysqld[2434922]: 2025-03-24 6:54:36 843346232 [Note] Stop semi-sync binlog_dump to slave (server_id: 180367385)
|
2025-03-24T06:54:40.267072+00:00 db2214 mysqld[2434922]: 2025-03-24 6:54:40 783299115 [Note] Stop semi-sync binlog_dump to slave (server_id: 180357384)
|
2025-03-24T06:54:40.800191+00:00 db2214 mysqld[2434922]: 2025-03-24 6:54:40 843346266 [Note] Start semi-sync binlog_dump to slave (server_id: 180357384), pos(./db2214-bin.001017, 165105894)
|
2025-03-24T06:54:57.579748+00:00 db2214 mysqld[2434922]: 2025-03-24 6:54:57 843346266 [Note] Stop semi-sync binlog_dump to slave (server_id: 180357384)
|
2025-03-24T06:55:01.513995+00:00 db2214 mysqld[2434922]: 2025-03-24 6:55:01 783051182 [Note] Stop semi-sync binlog_dump to slave (server_id: 180367482)
|
2025-03-24T06:55:02.054822+00:00 db2214 mysqld[2434922]: 2025-03-24 6:55:02 843346296 [Note] Start semi-sync binlog_dump to slave (server_id: 180367482), pos(./db2214-bin.001017, 165622170)
|
2025-03-24T06:55:18.844719+00:00 db2214 mysqld[2434922]: 2025-03-24 6:55:18 843346296 [Note] Stop semi-sync binlog_dump to slave (server_id: 180367482)
|
2025-03-24T06:55:22.761835+00:00 db2214 mysqld[2434922]: 2025-03-24 6:55:22 782902445 [Note] Stop semi-sync binlog_dump to slave (server_id: 180367576)
|
2025-03-24T06:55:23.290062+00:00 db2214 mysqld[2434922]: 2025-03-24 6:55:23 843346328 [Note] Start semi-sync binlog_dump to slave (server_id: 180367576), pos(./db2214-bin.001017, 166153524)
|
2025-03-24T06:55:40.115559+00:00 db2214 mysqld[2434922]: 2025-03-24 6:55:40 843346328 [Note] Stop semi-sync binlog_dump to slave (server_id: 180367576)
|
2025-03-24T06:55:44.040555+00:00 db2214 mysqld[2434922]: 2025-03-24 6:55:44 780189201 [Note] Stop semi-sync binlog_dump to slave (server_id: 180363366)
|
2025-03-24T06:55:44.565225+00:00 db2214 mysqld[2434922]: 2025-03-24 6:55:44 843346362 [Note] Start semi-sync binlog_dump to slave (server_id: 180363366), pos(./db2214-bin.001017, 166755743)
|
2025-03-24T06:56:26.310423+00:00 db2214 mysqld[2434922]: 2025-03-24 6:56:26 843346362 [Note] Stop semi-sync binlog_dump to slave (server_id: 180363366)
|
2025-03-24T06:56:30.209840+00:00 db2214 mysqld[2434922]: 2025-03-24 6:56:30 780048133 [Note] Stop semi-sync binlog_dump to slave (server_id: 180363363)
|
2025-03-24T06:56:30.209960+00:00 db2214 mysqld[2434922]: 250324 6:56:30 [ERROR] mysqld got signal 11 ;
|
2025-03-24T06:56:30.210013+00:00 db2214 mysqld[2434922]: Sorry, we probably made a mistake, and this is a bug.
|
2025-03-24T06:56:30.210071+00:00 db2214 mysqld[2434922]: Your assistance in bug reporting will enable us to fix this for the next release.
|
2025-03-24T06:56:30.210118+00:00 db2214 mysqld[2434922]: To report this bug, see https://mariadb.com/kb/en/reporting-bugs
|
2025-03-24T06:56:30.210170+00:00 db2214 mysqld[2434922]: We will try our best to scrape up some info that will hopefully help
|
2025-03-24T06:56:30.210465+00:00 db2214 mysqld[2434922]: diagnose the problem, but since we have already crashed,
|
2025-03-24T06:56:30.210512+00:00 db2214 mysqld[2434922]: something is definitely wrong and this may fail.
|
2025-03-24T06:56:30.210563+00:00 db2214 mysqld[2434922]: Server version: 10.6.18-MariaDB-log source revision: 887bb3f73555ff8a50138a580ca8308b9b5c069c
|
2025-03-24T06:56:30.210618+00:00 db2214 mysqld[2434922]: key_buffer_size=134217728
|
2025-03-24T06:56:30.210700+00:00 db2214 mysqld[2434922]: read_buffer_size=131072
|
2025-03-24T06:56:30.210747+00:00 db2214 mysqld[2434922]: max_used_connections=690
|
2025-03-24T06:56:30.210791+00:00 db2214 mysqld[2434922]: max_threads=2010
|
2025-03-24T06:56:30.210843+00:00 db2214 mysqld[2434922]: thread_count=12
|
2025-03-24T06:56:30.210893+00:00 db2214 mysqld[2434922]: It is possible that mysqld could use up to
|
2025-03-24T06:56:30.210962+00:00 db2214 mysqld[2434922]: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 4768845 K bytes of memory
|
2025-03-24T06:56:30.211018+00:00 db2214 mysqld[2434922]: Hope that's ok; if not, decrease some variables in the equation.
|
2025-03-24T06:56:30.211073+00:00 db2214 mysqld[2434922]: Thread pointer: 0x7ed9640956a8
|
2025-03-24T06:56:30.211119+00:00 db2214 mysqld[2434922]: Attempting backtrace. You can use the following information to find out
|
2025-03-24T06:56:30.211166+00:00 db2214 mysqld[2434922]: where mysqld died. If you see no messages after this, something went
|
2025-03-24T06:56:30.211211+00:00 db2214 mysqld[2434922]: terribly wrong...
|
2025-03-24T06:56:30.217189+00:00 db2214 mysqld[2434922]: stack_bottom = 0x7f3cb00d9c18 thread_stack 0x30000
|
2025-03-24T06:56:51.021751+00:00 db2214 mysqld[2434922]: /opt/wmf-mariadb106/bin/mysqld(my_print_stacktrace+0x2e)[0x559593b594be]
|
2025-03-24T06:56:52.271179+00:00 db2214 mysqld[2434922]: /opt/wmf-mariadb106/bin/mysqld(handle_fatal_signal+0x2b3)[0x5595935ee533]
|
2025-03-24T06:57:04.360757+00:00 db2214 mysqld[2434922]: /lib/x86_64-linux-gnu/libc.so.6(+0x3c050)[0x7f3cb985b050]
|
2025-03-24T06:57:20.633114+00:00 db2214 prometheus-mysqld-exporter[3407105]: ts=2025-03-24T06:57:20.632Z caller=exporter.go:174 level=error msg="Error from scraper" scraper=global_status err="context deadline exceeded"
|
2025-03-24T06:57:23.460765+00:00 db2214 prometheus-mysqld-exporter[3407105]: ts=2025-03-24T06:57:23.460Z caller=exporter.go:174 level=error msg="Error from scraper" scraper=global_variables err="context deadline exceeded"
|
2025-03-24T06:57:23.460932+00:00 db2214 prometheus-mysqld-exporter[3407105]: ts=2025-03-24T06:57:23.460Z caller=exporter.go:174 level=error msg="Error from scraper" scraper=slave_status err="context deadline exceeded"
|
2025-03-24T06:57:23.461050+00:00 db2214 prometheus-mysqld-exporter[3407105]: ts=2025-03-24T06:57:23.460Z caller=exporter.go:174 level=error msg="Error from scraper" scraper=heartbeat err="context deadline exceeded"
|
2025-03-24T06:57:23.461142+00:00 db2214 prometheus-mysqld-exporter[3407105]: ts=2025-03-24T06:57:23.460Z caller=exporter.go:174 level=error msg="Error from scraper" scraper=global_status err="context deadline exceeded"
|
2025-03-24T06:57:27.380627+00:00 db2214 mysqld[2434922]: /opt/wmf-mariadb106/bin/mysqld(_ZN12Active_tranx24clear_active_tranx_nodesEPKcyPFiP3THDS1_yE+0x14)[0x55959353a914]
|
2025-03-24T06:57:28.082120+00:00 db2214 mysqld[2434922]: /opt/wmf-mariadb106/bin/mysqld(_ZN21Repl_semi_sync_master12remove_slaveEv+0x54)[0x55959353afc4]
|
2025-03-24T06:57:28.189131+00:00 db2214 mysqld[2434922]: sql/semisync_master.cc:260(Active_tranx::clear_active_tranx_nodes(char const*, unsigned long long, int (*)(THD*, char const*, unsigned long long)))[0x55959353b01a]
|
2025-03-24T06:57:28.204104+00:00 db2214 mysqld[2434922]: sql/sql_repl.cc:3003(mysql_binlog_send(THD*, char*, unsigned long long, unsigned short))[0x5595933d01a7]
|
2025-03-24T06:57:28.224557+00:00 db2214 mysqld[2434922]: sql/sql_parse.cc:2165(dispatch_command(enum_server_command, THD*, char*, unsigned int, bool))[0x5595933b924b]
|
2025-03-24T06:57:28.225478+00:00 db2214 mysqld[2434922]: sql/sql_parse.cc:1409(do_command(THD*, bool))[0x5595933b9bd4]
|
2025-03-24T06:57:28.234842+00:00 db2214 mysqld[2434922]: sql/threadpool_common.cc:436(tp_callback(TP_connection*))[0x55959357ff6c]
|
2025-03-24T06:57:28.256045+00:00 db2214 mysqld[2434922]: sql/threadpool_generic.cc:1551(worker_main(void*))[0x55959379b15d]
|
2025-03-24T06:57:28.265218+00:00 db2214 mysqld[2434922]: perfschema/pfs.cc:2204(pfs_spawn_thread)[0x559593869c70]
|
2025-03-24T06:57:40.091384+00:00 db2214 mysqld[2434922]: /lib/x86_64-linux-gnu/libc.so.6(+0x89134)[0x7f3cb98a8134]
|
2025-03-24T06:57:45.097208+00:00 db2214 mysqld[2434922]: /lib/x86_64-linux-gnu/libc.so.6(+0x1097dc)[0x7f3cb99287dc]
|
2025-03-24T06:57:45.097368+00:00 db2214 mysqld[2434922]: Trying to get some variables.
|
2025-03-24T06:57:45.097452+00:00 db2214 mysqld[2434922]: Some pointers may be invalid and cause the dump to abort.
|
2025-03-24T06:57:45.097532+00:00 db2214 mysqld[2434922]: Query (0x0): (null)
|
2025-03-24T06:57:45.097601+00:00 db2214 mysqld[2434922]: Connection ID (thread ID): 780048133
|
2025-03-24T06:57:45.097673+00:00 db2214 mysqld[2434922]: Status: KILL_CONNECTION
|
2025-03-24T06:57:45.097831+00:00 db2214 mysqld[2434922]: Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=on,mrr_cost_based=on,mrr_sort_keys=on,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=off,condition_pushdown_from_having=on,not_null_range_scan=off,hash_join_cardinality=off,cset_narrowing=off
|
2025-03-24T06:57:45.098039+00:00 db2214 mysqld[2434922]: The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mariadbd/ contains
|
2025-03-24T06:57:45.098132+00:00 db2214 mysqld[2434922]: information that should help you find out what is causing the crash.
|
2025-03-24T06:57:45.098215+00:00 db2214 mysqld[2434922]: We think the query pointer is invalid, but we will try to print it anyway.
|
2025-03-24T06:57:45.098563+00:00 db2214 mysqld[2434922]: Query:
|
2025-03-24T06:57:45.098630+00:00 db2214 mysqld[2434922]: Writing a core file...
|
2025-03-24T06:57:45.098718+00:00 db2214 kernel: [30470235.375711] mysqld[2515377]: segfault at 30 ip 000055959353a914 sp 00007f3cb00d7ac0 error 4 in mariadbd[559593214000+a00000] likely on CPU 19 (core 10, socket 1)
|
2025-03-24T06:57:45.098763+00:00 db2214 mysqld[2434922]: Working directory at /srv/sqldata
|
2025-03-24T06:57:45.098853+00:00 db2214 mysqld[2434922]: Resource Limits:
|
2025-03-24T06:57:45.098914+00:00 db2214 mysqld[2434922]: Limit Soft Limit Hard Limit Units
|
2025-03-24T06:57:45.099005+00:00 db2214 mysqld[2434922]: Max cpu time unlimited unlimited seconds
|
2025-03-24T06:57:45.099085+00:00 db2214 mysqld[2434922]: Max file size unlimited unlimited bytes
|
2025-03-24T06:57:45.099153+00:00 db2214 mysqld[2434922]: Max data size unlimited unlimited bytes
|
2025-03-24T06:57:45.099230+00:00 db2214 mysqld[2434922]: Max stack size 8388608 unlimited bytes
|
2025-03-24T06:57:45.099296+00:00 db2214 mysqld[2434922]: Max core file size 0 0 bytes
|
2025-03-24T06:57:45.099373+00:00 db2214 mysqld[2434922]: Max resident set unlimited unlimited bytes
|
2025-03-24T06:57:45.099439+00:00 db2214 mysqld[2434922]: Max processes 2061519 2061519 processes
|
2025-03-24T06:57:45.099516+00:00 db2214 mysqld[2434922]: Max open files 200001 200001 files
|
2025-03-24T06:57:45.099583+00:00 db2214 mysqld[2434922]: Max locked memory 8388608 8388608 bytes
|
2025-03-24T06:57:45.099812+00:00 db2214 mysqld[2434922]: Max address space unlimited unlimited bytes
|
2025-03-24T06:57:45.099880+00:00 db2214 mysqld[2434922]: Max file locks unlimited unlimited locks
|
2025-03-24T06:57:45.099968+00:00 db2214 mysqld[2434922]: Max pending signals 2061519 2061519 signals
|
2025-03-24T06:57:45.100035+00:00 db2214 mysqld[2434922]: Max msgqueue size 819200 819200 bytes
|
2025-03-24T06:57:45.100113+00:00 db2214 mysqld[2434922]: Max nice priority 0 0
|
2025-03-24T06:57:45.100182+00:00 db2214 mysqld[2434922]: Max realtime priority 0 0
|
2025-03-24T06:57:45.100258+00:00 db2214 mysqld[2434922]: Max realtime timeout unlimited unlimited us
|
2025-03-24T06:57:45.100322+00:00 db2214 mysqld[2434922]: Core pattern: /var/tmp/core/core.%h.%e.%p.%t
|
2025-03-24T06:57:45.100405+00:00 db2214 mysqld[2434922]: Kernel version: Linux version 6.1.0-18-amd64 (debian-kernel@lists.debian.org) (gcc-12 (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40) #1 SMP PREEMPT_DYNAMIC Debian 6.1.76-1 (2024-02-01)
|
|
db2212
2025-03-24T07:27:28.078041+00:00 db2212 mysqld[1526]: 2025-03-24 7:27:28 722772472 [Note] Slave SQL thread exiting, replication stopped in log 'db1163-bin.009026' at position 998131926; GTID position '0-171970637-5484646134,171970572-171970572-3935877275,171970637-171970637-2116621969,171970661-171970661-3655324752,171970745-171970745-4198521467,171974720-171974720-2572451842,171978826-171978826-1257998662,180355171-180355171-176295269,180355190-180355190-1378262411,180357895-180357895-1629141008,180359172-180359172-49714369,180363268-180363268-3447080256,180363372-180363372-23365506,180367477-180367477-1469247192', master: db1163.eqiad.wmnet:3306
|
2025-03-24T07:27:28.078538+00:00 db2212 mysqld[1526]: 2025-03-24 7:27:28 722772471 [Note] Slave I/O thread exiting, read up to log 'db1163-bin.009026', position 998131926; GTID position 180355171-180355171-176295269,180359172-180359172-49714369,171970637-171970637-2116621969,171978826-171978826-1257998662,180363268-180363268-3447080256,171974720-171974720-2572451842,171970745-171970745-4198521467,180355190-180355190-1378262411,171970661-171970661-3655324752,171970572-171970572-3935877275,180367477-180367477-1469247192,0-171970637-5484646134,180363372-180363372-23365506,180357895-180357895-1629141008, master db1163.eqiad.wmnet:3306
|
2025-03-24T07:27:28.232973+00:00 db2212 mysqld[1526]: 2025-03-24 7:27:28 723714123 [Note] Slave I/O thread: Start asynchronous replication to master 'repl2024@db1163.eqiad.wmnet:3306' in log 'db1163-bin.009026' at position 998131926
|
2025-03-24T07:27:28.420070+00:00 db2212 mysqld[1526]: 2025-03-24 7:27:28 723714123 [Note] Slave I/O thread: connected to master 'repl2024@db1163.eqiad.wmnet:3306',replication starts at GTID position '180355171-180355171-176295269,180359172-180359172-49714369,171970637-171970637-2116621969,171978826-171978826-1257998662,180363268-180363268-3447080256,171974720-171974720-2572451842,171970745-171970745-4198521467,180355190-180355190-1378262411,171970661-171970661-3655324752,171970572-171970572-3935877275,180367477-180367477-1469247192,0-171970637-5484646134,180363372-180363372-23365506,180357895-180357895-1629141008'
|
2025-03-24T07:27:41.661334+00:00 db2212 mysqld[1526]: 2025-03-24 7:27:41 405298272 [Note] Stop semi-sync binlog_dump to slave (server_id: 180357895)
|
2025-03-24T10:32:29.280655+00:00 db2212 mysqld[1526]: 2025-03-24 10:32:29 583720964 [Note] Stop semi-sync binlog_dump to slave (server_id: 180367563)
|
2025-03-24T10:32:30.182507+00:00 db2212 mysqld[1526]: 2025-03-24 10:32:30 723771125 [Note] Start semi-sync binlog_dump to slave (server_id: 180367563), pos(./db2212-bin.001633, 651298193)
|
2025-03-24T10:32:47.059960+00:00 db2212 mysqld[1526]: 2025-03-24 10:32:47 723771125 [Note] Stop semi-sync binlog_dump to slave (server_id: 180367563)
|
2025-03-24T10:32:50.954690+00:00 db2212 mysqld[1526]: 2025-03-24 10:32:50 582470603 [Note] Stop semi-sync binlog_dump to slave (server_id: 180367564)
|
2025-03-24T10:32:51.857644+00:00 db2212 mysqld[1526]: 2025-03-24 10:32:51 723771247 [Note] Start semi-sync binlog_dump to slave (server_id: 180367564), pos(./db2212-bin.001633, 653416840)
|
2025-03-24T10:33:09.900816+00:00 db2212 mysqld[1526]: 2025-03-24 10:33:09 723771247 [Note] Stop semi-sync binlog_dump to slave (server_id: 180367564)
|
2025-03-24T10:33:13.809726+00:00 db2212 mysqld[1526]: 2025-03-24 10:33:13 578158592 [Note] Stop semi-sync binlog_dump to slave (server_id: 180367560)
|
2025-03-24T10:33:14.725581+00:00 db2212 mysqld[1526]: 2025-03-24 10:33:14 723771363 [Note] Start semi-sync binlog_dump to slave (server_id: 180367560), pos(./db2212-bin.001633, 655750103)
|
2025-03-24T10:33:31.560330+00:00 db2212 mysqld[1526]: 2025-03-24 10:33:31 723771363 [Note] Stop semi-sync binlog_dump to slave (server_id: 180367560)
|
2025-03-24T10:33:35.455373+00:00 db2212 mysqld[1526]: 2025-03-24 10:33:35 577015626 [Note] Stop semi-sync binlog_dump to slave (server_id: 180355076)
|
2025-03-24T10:33:36.359871+00:00 db2212 mysqld[1526]: 2025-03-24 10:33:36 723771486 [Note] Start semi-sync binlog_dump to slave (server_id: 180355076), pos(./db2212-bin.001633, 658924387)
|
2025-03-24T10:33:54.111871+00:00 db2212 mysqld[1526]: 2025-03-24 10:33:54 723771486 [Note] Stop semi-sync binlog_dump to slave (server_id: 180355076)
|
2025-03-24T10:33:59.949869+00:00 db2212 mysqld[1526]: 2025-03-24 10:33:59 498641245 [Note] Stop semi-sync binlog_dump to slave (server_id: 180355103)
|
2025-03-24T10:34:00.874589+00:00 db2212 mysqld[1526]: 2025-03-24 10:34:00 723771611 [Note] Start semi-sync binlog_dump to slave (server_id: 180355103), pos(./db2212-bin.001633, 661578939)
|
2025-03-24T10:34:19.768730+00:00 db2212 mysqld[1526]: 2025-03-24 10:34:19 723771611 [Note] Stop semi-sync binlog_dump to slave (server_id: 180355103)
|
2025-03-24T10:34:23.663254+00:00 db2212 mysqld[1526]: 2025-03-24 10:34:23 492015392 [Note] Stop semi-sync binlog_dump to slave (server_id: 180355297)
|
2025-03-24T10:34:24.602912+00:00 db2212 mysqld[1526]: 2025-03-24 10:34:24 723771743 [Note] Start semi-sync binlog_dump to slave (server_id: 180355297), pos(./db2212-bin.001633, 664248559)
|
2025-03-24T10:34:41.434622+00:00 db2212 mysqld[1526]: 2025-03-24 10:34:41 723771743 [Note] Stop semi-sync binlog_dump to slave (server_id: 180355297)
|
2025-03-24T10:34:45.337536+00:00 db2212 mysqld[1526]: 2025-03-24 10:34:45 447356004 [Note] Stop semi-sync binlog_dump to slave (server_id: 180355098)
|
2025-03-24T10:34:46.278523+00:00 db2212 mysqld[1526]: 2025-03-24 10:34:46 723771865 [Note] Start semi-sync binlog_dump to slave (server_id: 180355098), pos(./db2212-bin.001633, 666789833)
|
2025-03-24T10:35:03.077440+00:00 db2212 mysqld[1526]: 2025-03-24 10:35:03 723771865 [Note] Stop semi-sync binlog_dump to slave (server_id: 180355098)
|
2025-03-24T10:35:06.989983+00:00 db2212 mysqld[1526]: 2025-03-24 10:35:06 441157985 [Note] Stop semi-sync binlog_dump to slave (server_id: 180359172)
|
2025-03-24T10:35:07.917440+00:00 db2212 mysqld[1526]: 2025-03-24 10:35:07 723771990 [Note] Start semi-sync binlog_dump to slave (server_id: 180359172), pos(./db2212-bin.001633, 668919980)
|
2025-03-24T10:35:24.795043+00:00 db2212 mysqld[1526]: 2025-03-24 10:35:24 723771990 [Note] Stop semi-sync binlog_dump to slave (server_id: 180359172)
|
2025-03-24T10:35:28.731808+00:00 db2212 mysqld[1526]: 2025-03-24 10:35:28 405856454 [Note] Stop semi-sync binlog_dump to slave (server_id: 180367481)
|
2025-03-24T10:35:29.646969+00:00 db2212 mysqld[1526]: 2025-03-24 10:35:29 723772111 [Note] Start semi-sync binlog_dump to slave (server_id: 180367481), pos(./db2212-bin.001633, 671156918)
|
2025-03-24T10:35:47.484497+00:00 db2212 mysqld[1526]: 2025-03-24 10:35:47 723772111 [Note] Stop semi-sync binlog_dump to slave (server_id: 180367481)
|
2025-03-24T10:35:51.374735+00:00 db2212 mysqld[1526]: 2025-03-24 10:35:51 405022204 [Note] Stop semi-sync binlog_dump to slave (server_id: 180357894)
|
2025-03-24T10:35:51.374878+00:00 db2212 mysqld[1526]: 250324 10:35:51 [ERROR] mysqld got signal 11 ;
|
2025-03-24T10:35:51.374943+00:00 db2212 mysqld[1526]: Sorry, we probably made a mistake, and this is a bug.
|
2025-03-24T10:35:51.375188+00:00 db2212 mysqld[1526]: Your assistance in bug reporting will enable us to fix this for the next release.
|
2025-03-24T10:35:51.375284+00:00 db2212 mysqld[1526]: To report this bug, see https://mariadb.com/kb/en/reporting-bugs
|
2025-03-24T10:35:51.375338+00:00 db2212 mysqld[1526]: We will try our best to scrape up some info that will hopefully help
|
2025-03-24T10:35:51.375391+00:00 db2212 mysqld[1526]: diagnose the problem, but since we have already crashed,
|
2025-03-24T10:35:51.375488+00:00 db2212 mysqld[1526]: something is definitely wrong and this may fail.
|
2025-03-24T10:35:51.375577+00:00 db2212 mysqld[1526]: Server version: 10.6.20-MariaDB-log source revision: f00711bba2cd383825d0be1867f7d7d7f641c9e4
|
2025-03-24T10:35:51.375663+00:00 db2212 mysqld[1526]: key_buffer_size=134217728
|
2025-03-24T10:35:51.375721+00:00 db2212 mysqld[1526]: read_buffer_size=131072
|
2025-03-24T10:35:51.375774+00:00 db2212 mysqld[1526]: max_used_connections=699
|
2025-03-24T10:35:51.375836+00:00 db2212 mysqld[1526]: max_threads=2010
|
2025-03-24T10:35:51.375889+00:00 db2212 mysqld[1526]: thread_count=11
|
2025-03-24T10:35:51.375939+00:00 db2212 mysqld[1526]: It is possible that mysqld could use up to
|
2025-03-24T10:35:51.375994+00:00 db2212 mysqld[1526]: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 4769158 K bytes of memory
|
2025-03-24T10:35:51.376057+00:00 db2212 mysqld[1526]: Hope that's ok; if not, decrease some variables in the equation.
|
2025-03-24T10:35:51.376119+00:00 db2212 mysqld[1526]: Thread pointer: 0x7ec82c00e238
|
2025-03-24T10:35:51.376202+00:00 db2212 mysqld[1526]: Attempting backtrace. You can use the following information to find out
|
2025-03-24T10:35:51.376256+00:00 db2212 mysqld[1526]: where mysqld died. If you see no messages after this, something went
|
2025-03-24T10:35:51.376322+00:00 db2212 mysqld[1526]: terribly wrong...
|
2025-03-24T10:35:51.380230+00:00 db2212 mysqld[1526]: stack_bottom = 0x7ecb8de72000 thread_stack 0x30000
|
2025-03-24T10:36:01.700320+00:00 db2212 mysqld[1526]: 2025-03-24 10:36:01 0 [Note] /opt/wmf-mariadb106/bin/mysqld (initiated by: unknown): Normal shutdown
|
2025-03-24T10:36:01.701111+00:00 db2212 mysqld[1526]: 2025-03-24 10:36:01 0 [Note] Event Scheduler: Killing the scheduler thread, thread id 291205340
|
2025-03-24T10:36:01.701219+00:00 db2212 mysqld[1526]: 2025-03-24 10:36:01 0 [Note] Event Scheduler: Waiting for the scheduler thread to reply
|
2025-03-24T10:36:01.702047+00:00 db2212 mysqld[1526]: 2025-03-24 10:36:01 0 [Note] Event Scheduler: Stopped
|
2025-03-24T10:36:01.717413+00:00 db2212 mysqld[1526]: 2025-03-24 10:36:01 723714124 [Note] Slave SQL thread exiting, replication stopped in log 'db1163-bin.009028' at position 138781314; GTID position '0-171970637-5484646134,171970572-171970572-3935877275,171970637-171970637-2116621969,171970661-171970661-3655324752,171970745-171970745-4202012784,171974720-171974720-2572451842,171978826-171978826-1257998662,180355171-180355171-176295269,180355190-180355190-1378262411,180357895-180357895-1629141008,180359172-180359172-49714369,180363268-180363268-3447080256,180363372-180363372-23365506,180367477-180367477-1469247192', master: db1163.eqiad.wmnet:3306
|
2025-03-24T10:36:01.736722+00:00 db2212 mysqld[1526]: 2025-03-24 10:36:01 723714123 [Note] Slave I/O thread exiting, read up to log 'db1163-bin.009028', position 139284569; GTID position 180355171-180355171-176295269,180359172-180359172-49714369,171970637-171970637-2116621969,171978826-171978826-1257998662,180363268-180363268-3447080256,171974720-171974720-2572451842,171970745-171970745-4202014259,180355190-180355190-1378262411,171970661-171970661-3655324752,171970572-171970572-3935877275,180367477-180367477-1469247192,0-171970637-5484646134,180363372-180363372-23365506,180357895-180357895-1629141008, master db1163.eqiad.wmnet:3306
|
2025-03-24T10:36:11.713422+00:00 db2212 mysqld[1526]: /opt/wmf-mariadb106/bin/mysqld(my_print_stacktrace+0x2e)[0x557de866bbbe]
|
2025-03-24T10:36:12.522007+00:00 db2212 mysqld[1526]: /opt/wmf-mariadb106/bin/mysqld(handle_fatal_signal+0x2c3)[0x557de80f8123]
|
2025-03-24T10:36:25.145569+00:00 db2212 mysqld[1526]: /lib/x86_64-linux-gnu/libc.so.6(+0x3c050)[0x7f2bfec5b050]
|
2025-03-24T10:36:48.624619+00:00 db2212 mysqld[1526]: /opt/wmf-mariadb106/bin/mysqld(_ZN12Active_tranx24clear_active_tranx_nodesEPKcyPFiP3THDS1_yE+0x14)[0x557de8043524]
|
2025-03-24T10:36:49.913415+00:00 db2212 mysqld[1526]: /opt/wmf-mariadb106/bin/mysqld(_ZN21Repl_semi_sync_master12remove_slaveEv+0x54)[0x557de8043c54]
|
2025-03-24T10:36:50.039922+00:00 db2212 mysqld[1526]: sql/semisync_master.cc:260(Active_tranx::clear_active_tranx_nodes(char const*, unsigned long long, int (*)(THD*, char const*, unsigned long long)))[0x557de8043caa]
|
2025-03-24T10:36:50.055759+00:00 db2212 mysqld[1526]: sql/sql_repl.cc:3003(mysql_binlog_send(THD*, char*, unsigned long long, unsigned short))[0x557de7ed2e08]
|
2025-03-24T10:36:50.077184+00:00 db2212 mysqld[1526]: sql/sql_parse.cc:2177(dispatch_command(enum_server_command, THD*, char*, unsigned int, bool))[0x557de7ebb94b]
|
2025-03-24T10:36:50.078047+00:00 db2212 mysqld[1526]: sql/sql_parse.cc:1421(do_command(THD*, bool))[0x557de7ebc2df]
|
2025-03-24T10:36:50.087705+00:00 db2212 mysqld[1526]: sql/threadpool_common.cc:436(tp_callback(TP_connection*))[0x557de808947c]
|
2025-03-24T10:36:50.108982+00:00 db2212 mysqld[1526]: sql/threadpool_generic.cc:1551(worker_main(void*))[0x557de82afd9d]
|
2025-03-24T10:36:50.118522+00:00 db2212 mysqld[1526]: perfschema/pfs.cc:2204(pfs_spawn_thread)[0x557de837d770]
|
2025-03-24T10:37:02.172915+00:00 db2212 mysqld[1526]: /lib/x86_64-linux-gnu/libc.so.6(+0x891c4)[0x7f2bfeca81c4]
|
2025-03-24T10:37:07.178284+00:00 db2212 mysqld[1526]: /lib/x86_64-linux-gnu/libc.so.6(+0x10985c)[0x7f2bfed2885c]
|
2025-03-24T10:37:07.178664+00:00 db2212 mysqld[1526]: Trying to get some variables.
|
2025-03-24T10:37:07.178744+00:00 db2212 mysqld[1526]: Some pointers may be invalid and cause the dump to abort.
|
2025-03-24T10:37:07.178798+00:00 db2212 mysqld[1526]: Query (0x0): (null)
|
2025-03-24T10:37:07.178871+00:00 db2212 mysqld[1526]: Connection ID (thread ID): 405022204
|
2025-03-24T10:37:07.179010+00:00 db2212 mysqld[1526]: Status: KILL_SERVER
|
2025-03-24T10:37:07.179259+00:00 db2212 mysqld[1526]: Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=on,mrr_cost_based=on,mrr_sort_keys=on,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=off,condition_pushdown_from_having=on,not_null_range_scan=off,hash_join_cardinality=off,cset_narrowing=off
|
2025-03-24T10:37:07.179406+00:00 db2212 mysqld[1526]: The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mariadbd/ contains
|
2025-03-24T10:37:07.179466+00:00 db2212 mysqld[1526]: information that should help you find out what is causing the crash.
|
2025-03-24T10:37:07.179514+00:00 db2212 mysqld[1526]: We think the query pointer is invalid, but we will try to print it anyway.
|
2025-03-24T10:37:07.179570+00:00 db2212 mysqld[1526]: Query:
|
2025-03-24T10:37:07.179628+00:00 db2212 mysqld[1526]: Writing a core file...
|
2025-03-24T10:37:07.179690+00:00 db2212 mysqld[1526]: Working directory at /srv/sqldata
|
2025-03-24T10:37:07.179750+00:00 db2212 mysqld[1526]: Resource Limits:
|
2025-03-24T10:37:07.179806+00:00 db2212 mysqld[1526]: Limit Soft Limit Hard Limit Units
|
2025-03-24T10:37:07.179856+00:00 db2212 mysqld[1526]: Max cpu time unlimited unlimited seconds
|
2025-03-24T10:37:07.179902+00:00 db2212 mysqld[1526]: Max file size unlimited unlimited bytes
|
2025-03-24T10:37:07.179949+00:00 db2212 mysqld[1526]: Max data size unlimited unlimited bytes
|
2025-03-24T10:37:07.179995+00:00 db2212 mysqld[1526]: Max stack size 8388608 unlimited bytes
|
2025-03-24T10:37:07.180045+00:00 db2212 mysqld[1526]: Max core file size 0 0 bytes
|
2025-03-24T10:37:07.180086+00:00 db2212 mysqld[1526]: Max resident set unlimited unlimited bytes
|
2025-03-24T10:37:07.180160+00:00 db2212 mysqld[1526]: Max processes 2061516 2061516 processes
|
2025-03-24T10:37:07.180205+00:00 db2212 kernel: [6050990.143150] mysqld[3595690]: segfault at 30 ip 0000557de8043524 sp 00007ecb8de6eff0 error 4 in mariadbd[557de7d12000+a16000] likely on CPU 0 (core 0, socket 0)
|
2025-03-24T10:37:07.180238+00:00 db2212 mysqld[1526]: Max open files 200001 200001 files
|
2025-03-24T10:37:07.180296+00:00 db2212 mysqld[1526]: Max locked memory 8388608 8388608 bytes
|
2025-03-24T10:37:07.180537+00:00 db2212 mysqld[1526]: Max address space unlimited unlimited bytes
|
2025-03-24T10:37:07.180800+00:00 db2212 mysqld[1526]: Max file locks unlimited unlimited locks
|
2025-03-24T10:37:07.180849+00:00 db2212 mysqld[1526]: Max pending signals 2061516 2061516 signals
|
2025-03-24T10:37:07.180895+00:00 db2212 mysqld[1526]: Max msgqueue size 819200 819200 bytes
|
2025-03-24T10:37:07.180942+00:00 db2212 mysqld[1526]: Max nice priority 0 0
|
2025-03-24T10:37:07.180987+00:00 db2212 mysqld[1526]: Max realtime priority 0 0
|
2025-03-24T10:37:07.181033+00:00 db2212 mysqld[1526]: Max realtime timeout unlimited unlimited us
|
2025-03-24T10:37:07.181079+00:00 db2212 mysqld[1526]: Core pattern: /var/tmp/core/core.%h.%e.%p.%t
|
2025-03-24T10:37:07.181142+00:00 db2212 mysqld[1526]: Kernel version: Linux version 6.1.0-29-amd64 (debian-kernel@lists.debian.org) (gcc-12 (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40) #1 SMP PREEMPT_DYNAMIC Debian 6.1.123-1 (2025-01-02)
|
|
|
|
db2205
2025-03-24T07:28:01.156463+00:00 db2205 mysqld[1671]: 2025-03-24 7:28:01 399762200 [Note] Semi-sync replication switched OFF.
|
2025-03-24T07:28:01.157429+00:00 db2205 mysqld[1671]: 2025-03-24 7:28:01 87294697 [Note] Stopping ack receiver thread
|
2025-03-24T07:28:02.874218+00:00 db2205 mysqld[1671]: 2025-03-24 7:28:02 399376958 [Note] Slave SQL thread exiting, replication stopped in log 'db1223-bin.001015' at position 367839530; GTID position '0-171966669-4075095254,171966508-171966508-4344329680,171966669-171966669-4196523483,171974792-171974792-378345284,171974852-171974852-250551112,171978787-171978787-4497205829,171978912-171978912-496125857,180355192-180355192-3051044179,180358407-180358407-638907919,180359174-180359174-94172731,180363271-180363271-245332,180363280-180363280-1764603957,180363367-180363367-159599574,180363447-180363447-380873392', master: db1223.eqiad.wmnet:3306
|
2025-03-24T07:28:02.875049+00:00 db2205 mysqld[1671]: 2025-03-24 7:28:02 399376957 [Note] Slave I/O thread exiting, read up to log 'db1223-bin.001015', position 367839530; GTID position 171978787-171978787-4497205829,0-171966669-4075095254,171966669-171966669-4196523483,180358407-180358407-638907919,180363271-180363271-245332,171974792-171974792-378345284,180355192-180355192-3051044179,180359174-180359174-94172731,171978912-171978912-496125857,171966508-171966508-4344329680,171974852-171974852-250551112,180363447-180363447-380873392,180363367-180363367-159599574,180363280-180363280-1764603957, master db1223.eqiad.wmnet:3306
|
2025-03-24T07:28:02.966235+00:00 db2205 mysqld[1671]: 2025-03-24 7:28:02 399762212 [Note] Slave I/O thread: Start asynchronous replication to master 'repl2024@db1223.eqiad.wmnet:3306' in log 'db1223-bin.001015' at position 367839530
|
2025-03-24T07:28:03.162383+00:00 db2205 mysqld[1671]: 2025-03-24 7:28:03 399762212 [Note] Slave I/O thread: connected to master 'repl2024@db1223.eqiad.wmnet:3306',replication starts at GTID position '171978787-171978787-4497205829,0-171966669-4075095254,171966669-171966669-4196523483,180358407-180358407-638907919,180363271-180363271-245332,171974792-171974792-378345284,180355192-180355192-3051044179,180359174-180359174-94172731,171978912-171978912-496125857,171966508-171966508-4344329680,171974852-171974852-250551112,180363447-180363447-380873392,180363367-180363367-159599574,180363280-180363280-1764603957'
|
2025-03-24T07:28:11.605552+00:00 db2205 mysqld[1671]: 2025-03-24 7:28:11 94570071 [Note] Stop semi-sync binlog_dump to slave (server_id: 180363280)
|
2025-03-24T11:56:56.339801+00:00 db2205 mysqld[1671]: 2025-03-24 11:56:56 94572212 [Note] Stop semi-sync binlog_dump to slave (server_id: 180357898)
|
2025-03-24T11:56:56.906077+00:00 db2205 mysqld[1671]: 2025-03-24 11:56:56 399785474 [Note] Start semi-sync binlog_dump to slave (server_id: 180357898), pos(./db2205-bin.000932, 196173510)
|
2025-03-24T11:57:13.612173+00:00 db2205 mysqld[1671]: 2025-03-24 11:57:13 399785474 [Note] Stop semi-sync binlog_dump to slave (server_id: 180357898)
|
2025-03-24T11:57:17.325314+00:00 db2205 mysqld[1671]: 2025-03-24 11:57:17 87294924 [Note] Stop semi-sync binlog_dump to slave (server_id: 180363295)
|
2025-03-24T11:57:17.880496+00:00 db2205 mysqld[1671]: 2025-03-24 11:57:17 399785517 [Note] Start semi-sync binlog_dump to slave (server_id: 180363295), pos(./db2205-bin.000932, 196986742)
|
2025-03-24T11:57:34.656115+00:00 db2205 mysqld[1671]: 2025-03-24 11:57:34 399785517 [Note] Stop semi-sync binlog_dump to slave (server_id: 180363295)
|
2025-03-24T11:57:38.360667+00:00 db2205 mysqld[1671]: 2025-03-24 11:57:38 87294893 [Note] Stop semi-sync binlog_dump to slave (server_id: 180355172)
|
2025-03-24T11:57:38.925709+00:00 db2205 mysqld[1671]: 2025-03-24 11:57:38 399785563 [Note] Start semi-sync binlog_dump to slave (server_id: 180355172), pos(./db2205-bin.000932, 197954868)
|
2025-03-24T11:57:55.654232+00:00 db2205 mysqld[1671]: 2025-03-24 11:57:55 399785563 [Note] Stop semi-sync binlog_dump to slave (server_id: 180355172)
|
2025-03-24T11:57:59.356843+00:00 db2205 mysqld[1671]: 2025-03-24 11:57:59 87294859 [Note] Stop semi-sync binlog_dump to slave (server_id: 180359209)
|
2025-03-24T11:57:59.927540+00:00 db2205 mysqld[1671]: 2025-03-24 11:57:59 399785609 [Note] Start semi-sync binlog_dump to slave (server_id: 180359209), pos(./db2205-bin.000932, 199189935)
|
2025-03-24T11:58:16.682972+00:00 db2205 mysqld[1671]: 2025-03-24 11:58:16 399785609 [Note] Stop semi-sync binlog_dump to slave (server_id: 180359209)
|
2025-03-24T11:58:20.403266+00:00 db2205 mysqld[1671]: 2025-03-24 11:58:20 87294829 [Note] Stop semi-sync binlog_dump to slave (server_id: 180363425)
|
2025-03-24T11:58:20.982299+00:00 db2205 mysqld[1671]: 2025-03-24 11:58:20 399785643 [Note] Start semi-sync binlog_dump to slave (server_id: 180363425), pos(./db2205-bin.000932, 199867772)
|
2025-03-24T11:58:37.718810+00:00 db2205 mysqld[1671]: 2025-03-24 11:58:37 399785643 [Note] Stop semi-sync binlog_dump to slave (server_id: 180363425)
|
2025-03-24T11:58:41.446050+00:00 db2205 mysqld[1671]: 2025-03-24 11:58:41 87294795 [Note] Stop semi-sync binlog_dump to slave (server_id: 180367577)
|
2025-03-24T11:58:41.446229+00:00 db2205 mysqld[1671]: 250324 11:58:41 [ERROR] mysqld got signal 11 ;
|
2025-03-24T11:58:41.446540+00:00 db2205 mysqld[1671]: Sorry, we probably made a mistake, and this is a bug.
|
2025-03-24T11:58:41.446770+00:00 db2205 mysqld[1671]: Your assistance in bug reporting will enable us to fix this for the next release.
|
2025-03-24T11:58:41.446824+00:00 db2205 mysqld[1671]: To report this bug, see https://mariadb.com/kb/en/reporting-bugs
|
2025-03-24T11:58:41.446880+00:00 db2205 mysqld[1671]: We will try our best to scrape up some info that will hopefully help
|
2025-03-24T11:58:41.446928+00:00 db2205 mysqld[1671]: diagnose the problem, but since we have already crashed,
|
2025-03-24T11:58:41.446978+00:00 db2205 mysqld[1671]: something is definitely wrong and this may fail.
|
2025-03-24T11:58:41.447022+00:00 db2205 mysqld[1671]: Server version: 10.6.20-MariaDB-log source revision: f00711bba2cd383825d0be1867f7d7d7f641c9e4
|
2025-03-24T11:58:41.447078+00:00 db2205 mysqld[1671]: key_buffer_size=134217728
|
2025-03-24T11:58:41.447137+00:00 db2205 mysqld[1671]: read_buffer_size=131072
|
2025-03-24T11:58:41.447191+00:00 db2205 mysqld[1671]: max_used_connections=414
|
2025-03-24T11:58:41.447261+00:00 db2205 mysqld[1671]: max_threads=2010
|
2025-03-24T11:58:41.447309+00:00 db2205 mysqld[1671]: thread_count=11
|
2025-03-24T11:58:41.447366+00:00 db2205 mysqld[1671]: It is possible that mysqld could use up to
|
2025-03-24T11:58:41.447414+00:00 db2205 mysqld[1671]: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 4769158 K bytes of memory
|
2025-03-24T11:58:41.447468+00:00 db2205 mysqld[1671]: Hope that's ok; if not, decrease some variables in the equation.
|
2025-03-24T11:58:41.447526+00:00 db2205 mysqld[1671]: Thread pointer: 0x7f7cec467818
|
2025-03-24T11:58:41.447610+00:00 db2205 mysqld[1671]: Attempting backtrace. You can use the following information to find out
|
2025-03-24T11:58:41.447672+00:00 db2205 mysqld[1671]: where mysqld died. If you see no messages after this, something went
|
2025-03-24T11:58:41.447731+00:00 db2205 mysqld[1671]: terribly wrong...
|
2025-03-24T11:58:41.449777+00:00 db2205 mysqld[1671]: stack_bottom = 0x7f83fb523000 thread_stack 0x30000
|
2025-03-24T11:58:52.580529+00:00 db2205 mysqld[1671]: 2025-03-24 11:58:52 0 [Note] /opt/wmf-mariadb106/bin/mysqld (initiated by: unknown): Normal shutdown
|
2025-03-24T11:58:52.581484+00:00 db2205 mysqld[1671]: 2025-03-24 11:58:52 0 [Note] Event Scheduler: Killing the scheduler thread, thread id 87295409
|
2025-03-24T11:58:52.581551+00:00 db2205 mysqld[1671]: 2025-03-24 11:58:52 0 [Note] Event Scheduler: Waiting for the scheduler thread to reply
|
2025-03-24T11:58:52.581613+00:00 db2205 mysqld[1671]: 2025-03-24 11:58:52 0 [Note] Event Scheduler: Stopped
|
2025-03-24T11:58:52.590451+00:00 db2205 mysqld[1671]: 2025-03-24 11:58:52 399762213 [Note] Slave SQL thread exiting, replication stopped in log 'db1223-bin.001016' at position 8414818; GTID position '0-171966669-4075095254,171966508-171966508-4344329680,171966669-171966669-4196523483,171974792-171974792-378345284,171974852-171974852-250551112,171978787-171978787-4497205829,171978912-171978912-497393663,180355192-180355192-3051044179,180358407-180358407-638907919,180359174-180359174-94172731,180363271-180363271-245332,180363280-180363280-1764603957,180363367-180363367-159599574,180363447-180363447-380873392', master: db1223.eqiad.wmnet:3306
|
2025-03-24T11:58:52.593073+00:00 db2205 mysqld[1671]: 2025-03-24 11:58:52 399762212 [Note] Slave I/O thread exiting, read up to log 'db1223-bin.001016', position 8509585; GTID position 171978787-171978787-4497205829,0-171966669-4075095254,171966669-171966669-4196523483,180358407-180358407-638907919,180363271-180363271-245332,171974792-171974792-378345284,180355192-180355192-3051044179,180359174-180359174-94172731,171978912-171978912-497393832,171966508-171966508-4344329680,171974852-171974852-250551112,180363447-180363447-380873392,180363367-180363367-159599574,180363280-180363280-1764603957, master db1223.eqiad.wmnet:3306
|
2025-03-24T11:59:02.594148+00:00 db2205 mysqld[1671]: /opt/wmf-mariadb106/bin/mysqld(my_print_stacktrace+0x2e)[0x56465319fbbe]
|
2025-03-24T11:59:03.808704+00:00 db2205 mysqld[1671]: /opt/wmf-mariadb106/bin/mysqld(handle_fatal_signal+0x2c3)[0x564652c2c123]
|
2025-03-24T11:59:15.792461+00:00 db2205 mysqld[1671]: /lib/x86_64-linux-gnu/libc.so.6(+0x3c050)[0x7fe3d685b050]
|
2025-03-24T11:59:38.182744+00:00 db2205 mysqld[1671]: /opt/wmf-mariadb106/bin/mysqld(_ZN12Active_tranx24clear_active_tranx_nodesEPKcyPFiP3THDS1_yE+0x14)[0x564652b77524]
|
2025-03-24T11:59:39.029371+00:00 db2205 mysqld[1671]: /opt/wmf-mariadb106/bin/mysqld(_ZN21Repl_semi_sync_master12remove_slaveEv+0x54)[0x564652b77c54]
|
2025-03-24T11:59:39.139093+00:00 db2205 mysqld[1671]: sql/semisync_master.cc:260(Active_tranx::clear_active_tranx_nodes(char const*, unsigned long long, int (*)(THD*, char const*, unsigned long long)))[0x564652b77caa]
|
2025-03-24T11:59:39.154329+00:00 db2205 mysqld[1671]: sql/sql_repl.cc:3003(mysql_binlog_send(THD*, char*, unsigned long long, unsigned short))[0x564652a06e08]
|
2025-03-24T11:59:39.175376+00:00 db2205 mysqld[1671]: sql/sql_parse.cc:2177(dispatch_command(enum_server_command, THD*, char*, unsigned int, bool))[0x5646529ef94b]
|
2025-03-24T11:59:39.176265+00:00 db2205 mysqld[1671]: sql/sql_parse.cc:1421(do_command(THD*, bool))[0x5646529f02df]
|
2025-03-24T11:59:39.186407+00:00 db2205 mysqld[1671]: sql/threadpool_common.cc:436(tp_callback(TP_connection*))[0x564652bbd47c]
|
2025-03-24T11:59:39.207521+00:00 db2205 mysqld[1671]: sql/threadpool_generic.cc:1551(worker_main(void*))[0x564652de3d9d]
|
2025-03-24T11:59:39.217059+00:00 db2205 mysqld[1671]: perfschema/pfs.cc:2204(pfs_spawn_thread)[0x564652eb1770]
|
2025-03-24T11:59:51.032378+00:00 db2205 mysqld[1671]: /lib/x86_64-linux-gnu/libc.so.6(+0x891c4)[0x7fe3d68a81c4]
|
2025-03-24T11:59:56.037752+00:00 db2205 mysqld[1671]: /lib/x86_64-linux-gnu/libc.so.6(+0x10985c)[0x7fe3d692885c]
|
2025-03-24T11:59:56.038050+00:00 db2205 mysqld[1671]: Trying to get some variables.
|
2025-03-24T11:59:56.038294+00:00 db2205 mysqld[1671]: Some pointers may be invalid and cause the dump to abort.
|
2025-03-24T11:59:56.038374+00:00 db2205 mysqld[1671]: Query (0x0): (null)
|
2025-03-24T11:59:56.038427+00:00 db2205 mysqld[1671]: Connection ID (thread ID): 87294795
|
2025-03-24T11:59:56.038476+00:00 db2205 mysqld[1671]: Status: KILL_SERVER
|
2025-03-24T11:59:56.038686+00:00 db2205 mysqld[1671]: Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=on,mrr_cost_based=on,mrr_sort_keys=on,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=off,condition_pushdown_from_having=on,not_null_range_scan=off,hash_join_cardinality=off,cset_narrowing=off
|
2025-03-24T11:59:56.038879+00:00 db2205 mysqld[1671]: The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mariadbd/ contains
|
2025-03-24T11:59:56.038949+00:00 db2205 mysqld[1671]: information that should help you find out what is causing the crash.
|
2025-03-24T11:59:56.038999+00:00 db2205 mysqld[1671]: We think the query pointer is invalid, but we will try to print it anyway.
|
2025-03-24T11:59:56.039044+00:00 db2205 mysqld[1671]: Query:
|
2025-03-24T11:59:56.039119+00:00 db2205 mysqld[1671]: Writing a core file...
|
2025-03-24T11:59:56.039182+00:00 db2205 mysqld[1671]: Working directory at /srv/sqldata
|
2025-03-24T11:59:56.039228+00:00 db2205 mysqld[1671]: Resource Limits:
|
2025-03-24T11:59:56.039290+00:00 db2205 mysqld[1671]: Limit Soft Limit Hard Limit Units
|
2025-03-24T11:59:56.039368+00:00 db2205 mysqld[1671]: Max cpu time unlimited unlimited seconds
|
2025-03-24T11:59:56.039415+00:00 db2205 mysqld[1671]: Max file size unlimited unlimited bytes
|
2025-03-24T11:59:56.039462+00:00 db2205 mysqld[1671]: Max data size unlimited unlimited bytes
|
2025-03-24T11:59:56.039536+00:00 db2205 mysqld[1671]: Max stack size 8388608 unlimited bytes
|
2025-03-24T11:59:56.039588+00:00 db2205 mysqld[1671]: Max core file size 0 0 bytes
|
2025-03-24T11:59:56.039634+00:00 db2205 mysqld[1671]: Max resident set unlimited unlimited bytes
|
2025-03-24T11:59:56.039686+00:00 db2205 mysqld[1671]: Max processes 2061516 2061516 processes
|
2025-03-24T11:59:56.039749+00:00 db2205 mysqld[1671]: Max open files 200001 200001 files
|
2025-03-24T11:59:56.039808+00:00 db2205 mysqld[1671]: Max locked memory 8388608 8388608 bytes
|
2025-03-24T11:59:56.039890+00:00 db2205 mysqld[1671]: Max address space unlimited unlimited bytes
|
2025-03-24T11:59:56.039963+00:00 db2205 mysqld[1671]: Max file locks unlimited unlimited locks
|
2025-03-24T11:59:56.040012+00:00 db2205 mysqld[1671]: Max pending signals 2061516 2061516 signals
|
2025-03-24T11:59:56.040103+00:00 db2205 mysqld[1671]: Max msgqueue size 819200 819200 bytes
|
2025-03-24T11:59:56.040166+00:00 db2205 mysqld[1671]: Max nice priority 0 0
|
2025-03-24T11:59:56.040230+00:00 db2205 mysqld[1671]: Max realtime priority 0 0
|
2025-03-24T11:59:56.040467+00:00 db2205 mysqld[1671]: Max realtime timeout unlimited unlimited us
|
2025-03-24T11:59:56.040528+00:00 db2205 mysqld[1671]: Core pattern: /var/tmp/core/core.%h.%e.%p.%t
|
2025-03-24T11:59:56.040559+00:00 db2205 kernel: [4772097.639927] mysqld[2629267]: segfault at 30 ip 0000564652b77524 sp 00007f83fb51fff0 error 4 in mariadbd[564652846000+a16000] likely on CPU 4 (core 1, socket 0)
|
Attachments
Issue Links
- is caused by
-
-
- Closed
-
Activity
Is this being fixed on other versions too? I'm especially interested on 10.11
Thank you all!
(since I just learned that Management is still figuring out how to track release notes)
Release Notes
Fix primary crash when it reverts to async replication after its Semi-Synchronous Replication disabled while still connected to replica(s).
https://github.com/MariaDB/server/pull/3931