Details
-
Bug
-
Status: Closed (View Workflow)
-
Critical
-
Resolution: Fixed
-
10.6.13, 10.6.14, 10.11.5, 10.4(EOL), 10.5, 10.6, 10.9(EOL), 10.10(EOL), 10.11, 11.0(EOL)
-
Debian 11 (fresh install) + JeMalloc + MariaDB 10.6.15 or 10.11.5
Description
OS: Debian 11
Server version : MariaDB 10.6.14 (packaged by MariaDB) on tom of ZFS & on test environement MariaDB 10.6.15 (fresh OS install + fresh MariaDB install) without ZFS
Tested version : MariaDB 10.11.5 packaged by MariaDB (upgrade from 10.6.15)
Some queries in production generate a segfault.
Into a lab environement, we can reproduce the segfault, even with an explain of this request.
Both production & debug environment are using jemalloc.
Stacktrace for MariaDB 10.6.15 (test server)
230901 7:21:32 [ERROR] mysqld got signal 11 ;
|
This could be because you hit a bug. It is also possible that this binary
|
or one of the libraries it was linked against is corrupt, improperly built,
|
or misconfigured. This error can also be caused by malfunctioning hardware.
|
|
|
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
|
|
|
We will try our best to scrape up some info that will hopefully help
|
diagnose the problem, but since we have already crashed,
|
something is definitely wrong and this may fail.
|
|
|
Server version: 10.6.15-MariaDB-1:10.6.15+maria~deb11 source revision: 0d16eb35bc981023ce2f4912e8ecde68ca381f4e
|
key_buffer_size=134217728
|
read_buffer_size=131072
|
max_used_connections=1
|
max_threads=153
|
thread_count=1
|
It is possible that mysqld could use up to
|
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 467985 K bytes of memory
|
Hope that's ok; if not, decrease some variables in the equation.
|
|
|
Thread pointer: 0x7f71c96252d8
|
Attempting backtrace. You can use the following information to find out
|
where mysqld died. If you see no messages after this, something went
|
terribly wrong...
|
stack_bottom = 0x7f71ea252498 thread_stack 0x30000
|
/usr/sbin/mariadbd(my_print_stacktrace+0x2e)[0x56488f4912ee]
|
/usr/sbin/mariadbd(handle_fatal_signal+0x485)[0x56488ef56825]
|
/lib/x86_64-linux-gnu/libpthread.so.0(+0x13140)[0x7f71eab0e140]
|
/usr/sbin/mariadbd(_ZN13st_join_table21choose_best_splittingEjyPK8POSITIONPy+0x574)[0x56488eecf504]
|
/usr/sbin/mariadbd(_Z16best_access_pathP4JOINP13st_join_tableyPK8POSITIONjbdPS3_S6_+0x14f)[0x56488ed503ff]
|
/usr/sbin/mariadbd(+0x7f41bf)[0x56488ed541bf]
|
/usr/sbin/mariadbd(+0x7f447e)[0x56488ed5447e]
|
/usr/sbin/mariadbd(+0x7f447e)[0x56488ed5447e]
|
/usr/sbin/mariadbd(_Z11choose_planP4JOINy+0x2a4)[0x56488ed54fd4]
|
/usr/sbin/mariadbd(+0x81f3a3)[0x56488ed7f3a3]
|
/usr/sbin/mariadbd(_ZN4JOIN14optimize_innerEv+0x12ec)[0x56488ed8408c]
|
/usr/sbin/mariadbd(_ZN4JOIN8optimizeEv+0xa0)[0x56488ed846a0]
|
/usr/sbin/mariadbd(_Z12mysql_selectP3THDP10TABLE_LISTR4ListI4ItemEPS4_jP8st_orderS9_S7_S9_yP13select_resultP18st_select_lex_unitP13st_select_lex+0xb3)[0x56488ed84773]
|
/usr/sbin/mariadbd(_Z13handle_selectP3THDP3LEXP13select_resultm+0x14f)[0x56488ed84f7f]
|
/usr/sbin/mariadbd(+0x7aa641)[0x56488ed0a641]
|
/usr/sbin/mariadbd(_Z21mysql_execute_commandP3THDb+0x46f4)[0x56488ed197b4]
|
/usr/sbin/mariadbd(_Z11mysql_parseP3THDPcjP12Parser_state+0x1e7)[0x56488ed1ad67]
|
/usr/sbin/mariadbd(_Z16dispatch_command19enum_server_commandP3THDPcjb+0x156d)[0x56488ed1d5fd]
|
/usr/sbin/mariadbd(_Z10do_commandP3THDb+0x138)[0x56488ed1f328]
|
/usr/sbin/mariadbd(_Z24do_handle_one_connectionP7CONNECTb+0x3af)[0x56488ee2e9af]
|
/usr/sbin/mariadbd(handle_one_connection+0x5d)[0x56488ee2ecfd]
|
/usr/sbin/mariadbd(+0xc43ce2)[0x56488f1a3ce2]
|
/lib/x86_64-linux-gnu/libpthread.so.0(+0x7ea7)[0x7f71eab02ea7]
|
/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7f71ea709a2f]
|
|
|
Trying to get some variables.
|
Some pointers may be invalid and cause the dump to abort.
|
Query (0x7f71c967bb30): SELECT test.valdouble as measuredPower, test.valint1 as csLinkId
|
FROM tbl1 test,
|
(SELECT MAX(test.valdate) as maxDate, test.valint1
|
FROM tbl1 test
|
group by test.valint1) as last_entry
|
WHERE test.valint1 = last_entry.valint1
|
and test.valdate = last_entry.maxDate
|
and test.valint1 in (
|
3289475,
|
3289496,
|
3289562,
|
3289593,
|
3289594,
|
3289595,
|
3289626,
|
3289642,
|
3289658,
|
3289739,
|
3289750,
|
3289761,
|
3289762,
|
3289763,
|
3289774,
|
3289780,
|
3289951,
|
3289957,
|
3289988,
|
3290034,
|
1231562 )
|
|
|
Connection ID (thread ID): 31
|
Status: NOT_KILLED
|
|
|
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=on,condition_pushdown_from_having=on,not_null_range_scan=off,hash_join_cardinality=off
|
|
|
The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains
|
information that should help you find out what is causing the crash.
|
Writing a core file...
|
Working directory at /var/lib/mysql
|
Resource Limits:
|
Limit Soft Limit Hard Limit Units
|
Max cpu time unlimited unlimited seconds
|
Max file size unlimited unlimited bytes
|
Max data size unlimited unlimited bytes
|
Max stack size 8388608 unlimited bytes
|
Max core file size 0 unlimited bytes
|
Max resident set unlimited unlimited bytes
|
Max processes 15527 15527 processes
|
Max open files 102400 102400 files
|
Max locked memory 524288 524288 bytes
|
Max address space unlimited unlimited bytes
|
Max file locks unlimited unlimited locks
|
Max pending signals 15527 15527 signals
|
Max msgqueue size 819200 819200 bytes
|
Max nice priority 0 0
|
Max realtime priority 0 0
|
Max realtime timeout unlimited unlimited us
|
Core pattern: core
|
|
|
Kernel version: Linux version 5.10.0-25-amd64 (debian-kernel@lists.debian.org) (gcc-10 (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2) #1 SMP Debian 5.10.191-1 (2023-08-16)
|
Stacktrace for MariaDB 10.11.5 (also test server) :
230901 7:44:32 [ERROR] mysqld got signal 11 ;
|
This could be because you hit a bug. It is also possible that this binary
|
or one of the libraries it was linked against is corrupt, improperly built,
|
or misconfigured. This error can also be caused by malfunctioning hardware.
|
|
|
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
|
|
|
We will try our best to scrape up some info that will hopefully help
|
diagnose the problem, but since we have already crashed,
|
something is definitely wrong and this may fail.
|
|
|
Server version: 10.11.5-MariaDB-1:10.11.5+maria~deb11 source revision: 7875294b6b74b53dd3aaa723e6cc103d2bb47b2c
|
key_buffer_size=134217728
|
read_buffer_size=131072
|
max_used_connections=2
|
max_threads=153
|
thread_count=2
|
It is possible that mysqld could use up to
|
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 468037 K bytes of memory
|
Hope that's ok; if not, decrease some variables in the equation.
|
|
|
Thread pointer: 0x7f350c00e018
|
Attempting backtrace. You can use the following information to find out
|
where mysqld died. If you see no messages after this, something went
|
terribly wrong...
|
stack_bottom = 0x7f352ccc7498 thread_stack 0x30000
|
/usr/sbin/mariadbd(my_print_stacktrace+0x2e)[0x55a29f356eee]
|
/usr/sbin/mariadbd(handle_fatal_signal+0x485)[0x55a29ee34365]
|
/lib/x86_64-linux-gnu/libpthread.so.0(+0x13140)[0x7f352e06d140]
|
/usr/sbin/mariadbd(_ZN13st_join_table21choose_best_splittingEjyPK8POSITIONPy+0x57c)[0x55a29edab55c]
|
/usr/sbin/mariadbd(_Z16best_access_pathP4JOINP13st_join_tableyPK8POSITIONjbdPS3_S6_+0x177)[0x55a29ec21087]
|
/usr/sbin/mariadbd(+0x85fcc3)[0x55a29ec24cc3]
|
/usr/sbin/mariadbd(+0x864b90)[0x55a29ec29b90]
|
/usr/sbin/mariadbd(+0x8654b3)[0x55a29ec2a4b3]
|
/usr/sbin/mariadbd(+0x8654b3)[0x55a29ec2a4b3]
|
/usr/sbin/mariadbd(_Z11choose_planP4JOINy+0x2f4)[0x55a29ec2b0a4]
|
/usr/sbin/mariadbd(+0x88c082)[0x55a29ec51082]
|
/usr/sbin/mariadbd(_ZN4JOIN14optimize_innerEv+0x134c)[0x55a29ec55a5c]
|
/usr/sbin/mariadbd(_ZN4JOIN8optimizeEv+0xa0)[0x55a29ec56070]
|
/usr/sbin/mariadbd(_Z12mysql_selectP3THDP10TABLE_LISTR4ListI4ItemEPS4_jP8st_orderS9_S7_S9_yP13select_resultP18st_select_lex_unitP13st_select_lex+0xca)[0x55a29ec5615a]
|
/usr/sbin/mariadbd(_Z13handle_selectP3THDP3LEXP13select_resulty+0x14f)[0x55a29ec5695f]
|
/usr/sbin/mariadbd(+0x807131)[0x55a29ebcc131]
|
/usr/sbin/mariadbd(_Z21mysql_execute_commandP3THDb+0x486e)[0x55a29ebdb2de]
|
/usr/sbin/mariadbd(_Z11mysql_parseP3THDPcjP12Parser_state+0x1e7)[0x55a29ebdc987]
|
/usr/sbin/mariadbd(_Z16dispatch_command19enum_server_commandP3THDPcjb+0x15dd)[0x55a29ebdf28d]
|
/usr/sbin/mariadbd(_Z10do_commandP3THDb+0x138)[0x55a29ebe0f28]
|
/usr/sbin/mariadbd(_Z24do_handle_one_connectionP7CONNECTb+0x3af)[0x55a29ed02e1f]
|
/usr/sbin/mariadbd(handle_one_connection+0x5d)[0x55a29ed0316d]
|
/usr/sbin/mariadbd(+0xc95022)[0x55a29f05a022]
|
/lib/x86_64-linux-gnu/libpthread.so.0(+0x7ea7)[0x7f352e061ea7]
|
/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7f352dc68a2f]
|
|
|
Trying to get some variables.
|
Some pointers may be invalid and cause the dump to abort.
|
Query (0x7f350c042d30): SELECT test.valdouble as measuredPower, test.valint1 as csLinkId
|
FROM tbl1 test,
|
(SELECT MAX(test.valdate) as maxDate, test.valint1
|
FROM tbl1 test
|
group by test.valint1) as last_entry
|
WHERE test.valint1 = last_entry.valint1
|
and test.valdate = last_entry.maxDate
|
and test.valint1 in (
|
3289475,
|
3289496,
|
3289562,
|
3289593,
|
3289594,
|
3289595,
|
3289626,
|
3289642,
|
3289658,
|
3289739,
|
3289750,
|
3289761,
|
3289762,
|
3289763,
|
3289774,
|
3289780,
|
3289951,
|
3289957,
|
3289988,
|
3290034,
|
1231562 )
|
|
|
Connection ID (thread ID): 57
|
Status: NOT_KILLED
|
|
|
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=on,condition_pushdown_from_having=on,not_null_range_scan=off,hash_join_cardinality=off
|
|
|
The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains
|
information that should help you find out what is causing the crash.
|
Writing a core file...
|
Working directory at /var/lib/mysql
|
Resource Limits:
|
Limit Soft Limit Hard Limit Units
|
Max cpu time unlimited unlimited seconds
|
Max file size unlimited unlimited bytes
|
Max data size unlimited unlimited bytes
|
Max stack size 8388608 unlimited bytes
|
Max core file size 0 unlimited bytes
|
Max resident set unlimited unlimited bytes
|
Max processes 15527 15527 processes
|
Max open files 102400 102400 files
|
Max locked memory 524288 524288 bytes
|
Max address space unlimited unlimited bytes
|
Max file locks unlimited unlimited locks
|
Max pending signals 15527 15527 signals
|
Max msgqueue size 819200 819200 bytes
|
Max nice priority 0 0
|
Max realtime priority 0 0
|
Max realtime timeout unlimited unlimited us
|
Core pattern: core
|
|
|
Kernel version: Linux version 5.10.0-25-amd64 (debian-kernel@lists.debian.org) (gcc-10 (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2) #1 SMP Debian 5.10.191-1 (2023-08-16)
|
bdd-test-incident:/tmp# lsb_release -a
|
No LSB modules are available.
|
Distributor ID: Debian
|
Description: Debian GNU/Linux 11 (bullseye)
|
Release: 11
|
Codename: bullseye
|
MariaDB [CONF_V3]> show variables like '%malloc%';
|
+------------------------+------------------------------------------------------------+
|
| Variable_name | Value |
|
+------------------------+------------------------------------------------------------+
|
| version_malloc_library | jemalloc 5.2.1-0-gea6b3e973b477b8061e0076bb257dbd7f3faa756 |
|
+------------------------+------------------------------------------------------------+
|
1 row in set (0.002 sec)
|
|
|
MariaDB [CONF_V3]>
|
This issue seems to be related to MDEV-31403 - mark as fixed into 10.6.14 .
Please find attached the small dump file; and a request who crash.
Usefull information :
Bug did not occurs when items into the IN are smaller than `in_predicate_conversion_threshold` ;
==> Set the `in_predicate_conversion_threshold` to 20 for reproduce it easily.
Bug did not occurs when we have no indexes on the table (tempoary tests are `create table tbl1 select * from realtable limit 1000000` => no issues)
In some case (don't know why), first request (source the text file) generate a segfault, second one (after server restart) not. In this case, juste re-import data and crash occurs again.
Bug occurs with engine InnoDB (as into production) or MyISAM.
How to reproduce :
install a fresh MariaDB 10.6.15
|
create database test
|
use test
|
load mysqldump
|
set session in_predicate_conversion_threshold=20;
|
source request.sql
|
Attachments
Issue Links
- is caused by
-
-
- Closed
-
- is duplicated by
-
-
- Closed
-
- relates to
-
-
- Confirmed
-
-
-
- Open
-
-
-
- Closed
-
Activity
As mentioned above,
The failure started happening after this commit in 10.4.29:
commit ce7ffe61d836fe9f0cfc1087f058bc40d66e5cfb
Author: Igor Babaev
Date: Tue May 2 23:17:07 2023 -0700
The following query causes the same kind of crash. This query does not require conversion from IN predicate into IN subquery, rather it uses IN subquery directly:
SELECT test.valdouble, test.valint1 |
FROM tbl1 test, |
(SELECT max(test.valdate) AS maxdate, test.valint1 FROM tbl1 test GROUP BY test.valint1) AS last_entry WHERE test.valint1 = last_entry.valint1 AND test.valdate = last_entry.maxdate AND |
test.valint1 IN (select a from t0); |
Here t0 has been defined and filled by the commands:
create table t0 (a int); |
insert into t0 values |
(3289475),(3289496),(3289562),(3289593),(3289594),(3289595),(3289626),
|
(3289642),(3289658),(3289739),(3289750),(3289761),(3289762),(3289763),
|
(3289774),(3289780),(3289951),(3289957),(3289988),(3290034),(1231562);
|
A fix for this bug was pushed into 10.4. It should be merged upstream as it is.
The failure started happening after this commit in 10.4.29:
commit ce7ffe61d836fe9f0cfc1087f058bc40d66e5cfbAuthor: Igor BabaevDate: Tue May 2 23:17:07 2023 -0700MDEV-26301 Split optimization refills temporary table too many timescoping the test case from added files here:
valdate datetime,);test.valint1