Details
-
Bug
-
Status: In Progress (View Workflow)
-
Major
-
Resolution: Unresolved
-
10.11.15, 10.11.16
-
None
-
debian 12, baremetal
-
Q2/2026 Server Maintenance
Description
We've upgraded to 10.11.16 from 10.11.13 and we've started to see SEGV crashes. We downgraded to 10.11.15 and the crashes are still there. 10.11.13 seems to be the only stable version.
These crashes aren't producing any traces on mariadb logs or on journalctl either, which is odd. Just:
Mar 30 10:08:59 clouddb1022 systemd[1]: mariadb@s3.service: Main process exited, code=killed, status=11/SEGV
|
And
Mar 30 10:08:59 clouddb1022 systemd[1]: mariadb@s3.service: Main process exited, code=killed, status=11/SEGV
|
Mar 30 10:08:59 clouddb1022 systemd[1]: mariadb@s3.service: Failed with result 'signal'.
|
Mar 30 10:08:59 clouddb1022 systemd[1]: mariadb@s3.service: Consumed 34min 19.184s CPU time.
|
Mar 30 10:09:05 clouddb1022 systemd[1]: mariadb@s3.service: Scheduled restart job, restart counter is at 3.
|
Mar 30 10:09:05 clouddb1022 systemd[1]: Stopped mariadb@s3.service - mariadb database server.
|
Mar 30 10:09:05 clouddb1022 systemd[1]: mariadb@s3.service: Consumed 34min 19.184s CPU time.
|
Mar 30 10:09:05 clouddb1022 systemd[1]: Starting mariadb@s3.service - mariadb database server...
|
Mar 30 10:09:05 clouddb1022 mysqld[2586624]: 2026-03-30 10:09:05 0 [Warning] Could not increase number of max_open_files to more than 200001 (request: 800329)
|
Mar 30 10:09:05 clouddb1022 mysqld[2586624]: 2026-03-30 10:09:05 0 [Note] Starting MariaDB 10.11.16-MariaDB source revision 3218602d3100db9ce7a875511a591cddc173cc16 server_uid s47KRb1UmTUPdWrEfBCAd7qcmkI= as process 2586624
|
Mar 30 10:09:05 clouddb1022 mysqld[2586624]: 2026-03-30 10:09:05 0 [Note] mysqld: Aria engine: starting recovery
|
Mar 30 10:09:05 clouddb1022 mysqld[2586624]: recovered pages: 0% 24% 35% 52% 100% (0.0 seconds); tables to flush: 2 1 0
|
Mar 30 10:09:05 clouddb1022 mysqld[2586624]: (0.0 seconds);
|
Mar 30 10:09:05 clouddb1022 mysqld[2586624]: 2026-03-30 10:09:05 0 [Note] mysqld: Aria engine: recovery done
|
Mar 30 10:09:05 clouddb1022 mysqld[2586624]: 2026-03-30 10:09:05 0 [Note] InnoDB: Compressed tables use zlib 1.3.1
|
Mar 30 10:09:05 clouddb1022 mysqld[2586624]: 2026-03-30 10:09:05 0 [Note] InnoDB: Number of transaction pools: 1
|
Mar 30 10:09:05 clouddb1022 mysqld[2586624]: 2026-03-30 10:09:05 0 [Note] InnoDB: Using AVX512 instructions
|
Mar 30 10:09:05 clouddb1022 mysqld[2586624]: 2026-03-30 10:09:05 0 [Note] InnoDB: Using Linux native AIO
|
Mar 30 10:09:05 clouddb1022 mysqld[2586624]: 2026-03-30 10:09:05 0 [Note] InnoDB: innodb_buffer_pool_size_max=102400m, innodb_buffer_pool_size=102400m
|
Mar 30 10:09:10 clouddb1022 mysqld[2586624]: 2026-03-30 10:09:10 0 [Note] InnoDB: Completed initialization of buffer pool
|
Mar 30 10:09:10 clouddb1022 mysqld[2586624]: 2026-03-30 10:09:10 0 [Note] InnoDB: File system buffers for log disabled (block size=4096 bytes)
|
Mar 30 10:09:10 clouddb1022 mysqld[2586624]: 2026-03-30 10:09:10 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=65148004503307
|
Mar 30 10:09:19 clouddb1022 mysqld[2586624]: 2026-03-30 10:09:19 0 [Note] InnoDB: End of log at LSN=65149039161580
|
Mar 30 10:09:19 clouddb1022 mysqld[2586624]: 2026-03-30 10:09:19 0 [Note] InnoDB: 1 transaction(s) which must be rolled back or cleaned up in total 1 row operations to undo
|
Mar 30 10:09:19 clouddb1022 mysqld[2586624]: 2026-03-30 10:09:19 0 [Note] InnoDB: Trx id counter is 165692918816
|
Mar 30 10:09:19 clouddb1022 mysqld[2586624]: 2026-03-30 10:09:19 0 [Note] InnoDB: To recover: 676145 pages
|
Mar 30 10:09:31 clouddb1022 mysqld[2586624]: 2026-03-30 10:09:31 0 [Note] InnoDB: Last binlog file './db1154-bin.000003', position 590001970
|
Mar 30 10:09:31 clouddb1022 mysqld[2586624]: 2026-03-30 10:09:31 0 [Note] InnoDB: 128 rollback segments are active.
|
Mar 30 10:09:31 clouddb1022 mysqld[2586624]: 2026-03-30 10:09:31 0 [Note] InnoDB: Starting in background the rollback of recovered transactions
|
Mar 30 10:09:31 clouddb1022 mysqld[2586624]: 2026-03-30 10:09:31 0 [Note] InnoDB: Removed temporary tablespace data file: "./ibtmp1"
|
Mar 30 10:09:31 clouddb1022 mysqld[2586624]: 2026-03-30 10:09:31 0 [Note] InnoDB: Setting file './ibtmp1' size to 12.000MiB. Physically writing the file full; Please wait ...
|
Mar 30 10:09:31 clouddb1022 mysqld[2586624]: 2026-03-30 10:09:31 0 [Note] InnoDB: File './ibtmp1' size is now 12.000MiB.
|
Mar 30 10:09:31 clouddb1022 mysqld[2586624]: 2026-03-30 10:09:31 0 [Note] InnoDB: log sequence number 65149039161580; transaction id 165692918817
|
Mar 30 10:09:31 clouddb1022 mysqld[2586624]: 2026-03-30 10:09:31 0 [Note] Plugin 'FEEDBACK' is disabled.
|
Mar 30 10:09:31 clouddb1022 mysqld[2586624]: 2026-03-30 10:09:31 0 [Note] InnoDB: Loading buffer pool(s) from /srv/sqldata.s3/ib_buffer_pool
|
Mar 30 10:09:31 clouddb1022 mysqld[2586624]: 2026-03-30 10:09:31 0 [Note] InnoDB: Rolled back recovered transaction 165692918814
|
Mar 30 10:09:31 clouddb1022 mysqld[2586624]: 2026-03-30 10:09:31 0 [Note] InnoDB: Rollback of non-prepared transactions completed
|
Mar 30 10:09:31 clouddb1022 mysqld[2586624]: 2026-03-30 10:09:31 0 [Note] Server socket created on IP: '0.0.0.0', port: '3313'.
|
Mar 30 10:09:31 clouddb1022 mysqld[2586624]: 2026-03-30 10:09:31 0 [Note] Server socket created on IP: '::', port: '3313'.
|
Mar 30 10:09:31 clouddb1022 mysqld[2586624]: 2026-03-30 10:09:31 0 [Note] Server socket created on IP: '0.0.0.0', port: '3333'.
|
Mar 30 10:09:31 clouddb1022 mysqld[2586624]: 2026-03-30 10:09:31 0 [Note] Server socket created on IP: '::', port: '3333'.
|
Mar 30 10:09:31 clouddb1022 mysqld[2586624]: 2026-03-30 10:09:31 0 [Warning] Neither --relay-log nor --relay-log-index were used; so replication may break when this MariaDB server acts as a replica and has its hostname changed. Please use '--log-basename=#' or '--relay-log=clouddb1022-relay-bin' to avoid this problem.
|
Mar 30 10:09:31 clouddb1022 mysqld[2586624]: 2026-03-30 10:09:31 0 [Note] /opt/wmf-mariadb1011/bin/mysqld: ready for connections.
|
Mar 30 10:09:31 clouddb1022 mysqld[2586624]: Version: '10.11.16-MariaDB' socket: '/run/mysqld/mysqld.s3.sock' port: 3313 MariaDB Server
|
Mar 30 10:09:31 clouddb1022 systemd[1]: Started mariadb@s3.service - mariadb database server.
|
I've attached a gdb and got some traces which I am attaching here.
This seems to be what is being run right before the crash:
"SET STATEMENT max_statement_time = 900 FOR\nSELECT * FROM ((\n\n", ' ' <repeats 20 times>, "SELECT\n", ' ' <repeats 24 times>, "'abwiki' AS dbName,\n", ' ' <repeats 24 times>, "revs.rev_id AS id,\n", ' ' <repeats 24 times>, "r"..., m_buf_length = 1077516, m_echo = true, m_echo_saved = false, m_cpp_buf = 0x7fc0c84a5760 "SET STATEMENT max_statement_time = 900 FOR\nSELECT * FROM ((\n\n", ' ' <repeats 20 times>, "SELECT\n", ' ' <repeats 24 times>, "'abwiki' AS dbName,\n", ' ' <repeats 24 times>, "revs.rev_id AS id,\n", ' ' <repeats 24 times>, "r"...
|
I am attaching a new gdb to try to capture the exact query so we can try to reproduce and investigate.
Internally being tracked at T420177
Attachments
Issue Links
- relates to
-
MDEV-35816 ASAN: use-after-poison in st_select_lex::print
-
- Closed
-