[MDEV-31151] Inaccurate stack size caculation caused stack overflow in pinbox allocator Created: 2023-04-28  Updated: 2024-01-30

Status: Open
Project: MariaDB Server
Component/s: Server
Affects Version/s: 10.4, 10.5, 10.6, 10.7, 10.8, 10.9, 10.10, 10.11, 11.0
Fix Version/s: 10.5, 10.6, 10.11, 11.0, 11.1, 11.2

Type: Bug Priority: Critical
Reporter: Hugo Wen Assignee: Sergei Golubchik
Resolution: Unresolved Votes: 1
Labels: None

Issue Links:
Relates
relates to MDEV-28430 lf_alloc isn't safe on aarch64 (or pp... In Review

 Description   

Reuse the description from https://github.com/MariaDB/server/pull/2541:

MariaDB supports a "wait-free concurrent allocator based on pinning addresses". In lf_pinbox_real_free() it tries to sort the pinned addresses for better performance to use binary search during "real free". alloca() was used to allocate stack memory and copy addresses.

To prevent a stack overflow when allocating the stack memory the function checks if there's enough stack space. However, the available stack size was calculated inaccurately which eventually caused database crash due to stack overflow.

The crash was seen on MariaDB 10.6.11 but the same code defect exists on all MariaDB versions.

Crash stack trace:

#0  msort_with_tmp (p=0x40333f0790a0, b=0x40333f0790d0, n=3) at msort.c:40
 
#1  0x000040000456c86c in msort_with_tmp (n=3, b=0x40333f0790d0, p=0x40333f079100) at msort.c:45
 
#2  __GI___qsort_r (b=b@entry=0x40333f0790d0, n=n@entry=3, s=s@entry=8, cmp=cmp@entry=0xaaaac50af0e0 <ptr_cmp>, arg=arg@entry=0x0) at msort.c:297
 
#3  0x000040000456c968 in __GI_qsort (b=b@entry=0x40333f0790d0, n=n@entry=3, s=s@entry=8, cmp=cmp@entry=0xaaaac50af0e0 <ptr_cmp>) at msort.c:308
 
#4  0x0000aaaac50af498 in lf_pinbox_real_free (pins=0x4032bb19b610) at /local/p4clients/pkgbuild-QlPbg/workspace/src/MariaDB/mysys/lf_alloc-pin.c:358
 
#5  0x0000aaaac50af74c in lf_pinbox_free (pins=pins@entry=0x4032bb19b610, addr=<optimized out>) at /local/p4clients/pkgbuild-QlPbg/workspace/src/MariaDB/mysys/lf_alloc-pin.c:271
 
#6  0x0000aaaac50b0fc4 in l_delete (pins=0x4032bb19b610, keylen=<optimized out>, key=0x4033257b3b40 "\002dlrtms_lt", hashnr=4146604099, cs=0xaaaac5aeb5e0 <my_charset_bin>, head=0x400014a82098) at /local/p4clients/pkgbuild-QlPbg/workspace/src/MariaDB/mysys/lf_hash.cc:258
 
#7  lf_hash_delete (hash=0xaaaac5b74500 <mdl_locks>, pins=0x4032bb19b610, key=0x4033257b3b40, keylen=<optimized out>) at /local/p4clients/pkgbuild-QlPbg/workspace/src/MariaDB/mysys/lf_hash.cc:467
 
#8  0x0000aaaac48577c4 in MDL_context::release_lock (this=<optimized out>, duration=<optimized out>, ticket=0x403289fb0620) at /local/p4clients/pkgbuild-QlPbg/workspace/src/MariaDB/sql/mdl.cc:2886
 
#9  0x0000aaaac4857848 in MDL_context::release_locks_stored_before (this=this@entry=0x40332803d370, duration=duration@entry=MDL_TRANSACTION, sentinel=sentinel@entry=0x0) at /local/p4clients/pkgbuild-QlPbg/workspace/src/MariaDB/sql/mdl.cc:2936
 
#10 0x0000aaaac4857d24 in MDL_context::release_transactional_locks (this=this@entry=0x40332803d370, thd=thd@entry=0x40332803d218) at /local/p4clients/pkgbuild-QlPbg/workspace/src/MariaDB/sql/mdl.cc:3122
 
#11 0x0000aaaac476fa00 in THD::release_transactional_locks (this=<optimized out>) at /local/p4clients/pkgbuild-QlPbg/workspace/src/MariaDB/sql/sql_class.h:5071
 
#12 mysql_execute_command (thd=thd@entry=0x40332803d218, is_called_from_prepared_stmt=is_called_from_prepared_stmt@entry=false) at /local/p4clients/pkgbuild-QlPbg/workspace/src/MariaDB/sql/sql_parse.cc:6150
 
#13 0x0000aaaac4774f0c in mysql_parse (thd=0x40332803d218, rawbuf=<optimized out>, length=<optimized out>, parser_state=<optimized out>) at /local/p4clients/pkgbuild-QlPbg/workspace/src/MariaDB/sql/sql_parse.cc:8123
 
#14 0x0000aaaac476d050 in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x40332803d218, packet=packet@entry=0x403328054359 "", packet_length=packet_length@entry=2581, blocking=125, blocking@entry=true)
 
    at /local/p4clients/pkgbuild-QlPbg/workspace/src/MariaDB/sql/sql_parse.cc:1907
 
#15 0x0000aaaac476c244 in do_command (thd=0x40332803d218, blocking=blocking@entry=true) at /local/p4clients/pkgbuild-QlPbg/workspace/src/MariaDB/sql/sql_parse.cc:1417
 
#16 0x0000aaaac484eb44 in do_handle_one_connection (connect=<optimized out>, connect@entry=0x40330f279dd8, put_in_cache=put_in_cache@entry=true) at /local/p4clients/pkgbuild-QlPbg/workspace/src/MariaDB/sql/sql_connect.cc:1459
 
#17 0x0000aaaac484eec8 in handle_one_connection (arg=arg@entry=0x40330f279dd8) at /local/p4clients/pkgbuild-QlPbg/workspace/src/MariaDB/sql/sql_connect.cc:1361
 
#18 0x0000aaaac4dad8f0 in pfs_spawn_thread (arg=0x40330f367018) at /local/p4clients/pkgbuild-QlPbg/workspace/src/MariaDB/storage/perfschema/pfs.cc:2201
 
#19 0x000040000450a22c in start_thread (arg=0x400004533000) at pthread_create.c:465
 
#20 0x000040000460ca1c in thread_start () at ../sysdeps/unix/sysv/linux/aarch64/clone.S:80

I had created a PR https://github.com/MariaDB/server/pull/2541 for this issue by porting the fix from MySQL. Please see more detailed explanation in the PR description and comments.
Create this Jira as suggested by Daniel Black to better track this issue.
Generated at Thu Feb 08 10:21:40 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.