[MDEV-27955] main.func_json_notembedded test fails on out-of-memory on MariaDB 10.6.7 Created: 2022-02-26  Updated: 2023-02-05  Resolved: 2022-03-17

Status: Closed
Project: MariaDB Server
Component/s: Tests
Affects Version/s: 10.6.7
Fix Version/s: 10.6.8, 10.7.4, 10.8.3

Type: Bug Priority: Minor
Reporter: Otto Kekäläinen Assignee: Daniel Black
Resolution: Fixed Votes: 0
Labels: not-10.5

Issue Links:
Duplicate
is duplicated by MDEV-28251 too small max_statement_time is ignored Closed
Problem/Incident
is caused by MDEV-24909 JSON functions don't respect KILL QUE... Closed
Relates
relates to MDEV-28714 JSON functions still don't respect KI... Open
relates to MDEV-30518 main.func_json_notembedded fails on s... Open

 Description   

I notice that builds of MariaDB 10.6.7 in Debian was failing due to multiple reasons. One failure not reported before was main.func_json_notembedded on arch mipsel/s390x/powerpc:

main.func_json_notembedded               w3 [ fail ]
 
        Test ended at 2022-02-25 14:49:46
 
 
 
CURRENT_TEST: main.func_json_notembedded
 
mysqltest: At line 12: query 'set @obj=concat_ws('','{', repeat('"a":"b",', 125000000/2), '"c":"d"}')' failed: <Unknown> (5): Out of memory (Needed 500000016 bytes)
 
 
 
The result from queries just before the failure was:
 
set global max_allowed_packet=1073741824;
 
connect u,localhost,root;
 
#
 
# MDEV-24909 JSON functions don't respect KILL QUERY / max_statement_time limit
 
#
 
set group_concat_max_len= 4294967295;
 
set @obj=concat_ws('','{', repeat('"a":"b",', 125000000/2), '"c":"d"}');

Full logs at

 Comments   
Comment by Daniel Black [ 2022-02-27 ]

8 bytes repeated 62.5 million times seems to need more than 500M. So its not an overallocation, its a normal lack of resources.

I had noticed it failing on bb on s390x, and its not marked as a big test, so mark it as such.

Thanks for identifying the OOM cause.

Comment by Michael Widenius [ 2022-03-15 ]

The idea with the test is to check if json_array_append(@arr, '$[0]', 1) can be interrupted.
Making the strings to small will defeat the purpose of the test, which is to ensure that commands that takes more than a second are interrupted
I did some checking on my desktop (new,reasonable powerful cpu) and one can on that one decrease the repeats down to 1250 and still get
the statement interrupted.

That said, I am fine with the suggested patch, but better to mark the test 'big' too, to ensure it is not run on machines with little resources.

Comment by Michael Widenius [ 2022-03-15 ]

Ok to push

Comment by Otto Kekäläinen [ 2023-01-21 ]

main.func_json_notembedded still failed in 10.6.11 and 10.11.1 in Debian, see e.g. https://buildd.debian.org/status/fetch.php?pkg=mariadb-10.6&arch=ppc64&ver=1%3A10.6.11-2&stamp=1672785257&raw=0

However the test does not run out of memory now but crashes:

mysqltest: Could not open connection 'default' after 500 attempts: 2002 Can't connect to local server through socket '/<<PKGBUILDDIR>>/builddir/mysql-t' (111)
 
 
 
 
 
 - found 'core' (1/5)
 
Core generated by '/<<PKGBUILDDIR>>/builddir/sql/mariadbd'
 
Output from gdb follows. The first stack trace is from the failing thread.
 
The following stack traces are from all threads (so the failing one is
 
duplicated).
 
--------------------------
 
[New LWP 645951]
 
[New LWP 645958]
 
[New LWP 645971]
 
[New LWP 645960]
 
[New LWP 645961]
 
[New LWP 645968]
 
[New LWP 645959]
 
[Thread debugging using libthread_db enabled]
 
Using host libthread_db library "/lib/powerpc64-linux-gnu/libthread_db.so.1".
 
Core was generated by `/<<PKGBUILDDIR>>/builddir/sql/mariadbd --default'.
 
Program terminated with signal SIGABRT, Aborted.
 
#0  0x00007fff8e95da5c in ?? () from /lib/powerpc64-linux-gnu/libc.so.6
 
[Current thread is 1 (Thread 0x7fff8fa7b900 (LWP 645951))]
 
#0  0x00007fff8e95da5c in ?? () from /lib/powerpc64-linux-gnu/libc.so.6
 
#1  0x0000000134204998 in my_write_core (sig=<optimized out>) at ./mysys/stacktrace.c:424
 
#2  0x0000000133b8c1e8 in handle_fatal_signal (sig=<optimized out>) at ./sql/signal_handler.cc:355
 
#3  <signal handler called>
 
#4  0x00007fff8e9edaf0 in .__poll () from /lib/powerpc64-linux-gnu/libc.so.6
 
#5  0x00000001337bc5dc in poll (__timeout=-1, __nfds=<optimized out>, __fds=0x1000725e508) at /usr/include/powerpc64-linux-gnu/bits/poll2.h:39
 
#6  handle_connections_sockets () at ./sql/mysqld.cc:6182
 
#7  0x00000001337be01c in mysqld_main (argc=<optimized out>, argv=<optimized out>) at ./sql/mysqld.cc:5887
 
#8  0x0000000133765438 in main (argc=<optimized out>, argv=<optimized out>) at ./sql/main.cc:34

Comment by Daniel Black [ 2023-01-21 ]

Abort is from MTR as its took too long. There was some guidance somewhere on improving the test.

Generated at Thu Feb 08 09:56:54 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.