[MDEV-16390] SELECT with text columns lasts forever and can not be killed Created: 2018-06-04  Updated: 2019-08-06  Resolved: 2019-02-17

Status: Closed
Project: MariaDB Server
Component/s: Data Manipulation - Subquery, Data types, Storage Engine - InnoDB
Affects Version/s: 10.1.20
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: Guillermo Schulman Assignee: Unassigned
Resolution: Duplicate Votes: 2
Labels: blob, need_feedback
Environment:

Ubuntu 16.04

Attachments: File analyze-query.json     File last-t.sql     File my-10-1.cnf     File stack-trace.output     File tiny-tc.sql.gz    
Issue Links:
Duplicate
is duplicated by MDEV-15206 Server hangs with small aria_pagecach... Open

 Description   

Under a couple of circumstances a SELECT statement makes the mysql to behave really weird, the query takes forever and can not be killed (it gets locked in status killed forever if you kill it). The only way to get out of these unhealthy situation is to kill -9 the mysqld process. The statement must be run several times simultaneously to reproduce the problem.
This is the simplest version of my query which made it happen:

SELECT 
  t2.`id`,  
  t11_sf1.html,
 
  t11_sf13076.text,
 
  GROUP_CONCAT( CONCAT(IFNULL(t12_dataSetTableN31727_sf31612.`f2`,""), ",", IFNULL(t12_dataSetTableN31727_sf31613.`f2`,""), ",", IFNULL(t12_dataSetTableN31727_sf31931.`f2`,"")) SEPARATOR "\n") AS `AdditionalPostingLocation` 
FROM t2
 
JOIN t4 AS t44305 ON (t44305.`jobId` = t2.`id` AND t44305.`schemaDataId` IN (SELECT id FROM t5  WHERE (schemaSpecId = 3))) 
JOIN t6 AS t11 ON (t11.`schemaDataId` = t44305.`schemaDataId`) 
LEFT JOIN t1 AS t11_sf1 ON (t11_sf1.`id` = t11.`htmlContentIdf1`) 
LEFT JOIN t3 AS t11_sf13076 ON (t11_sf13076.`id` = t11.`multiLineTextIdf114`) 
LEFT JOIN t11 AS t12 ON (t12.`schemaDataId` = t44305.`schemaDataId` AND t12.`schemaFieldId` = 31727) 
LEFT JOIN t9 AS t12_dataSetTableN31727 ON (t12_dataSetTableN31727.`schemaDataId` = t12.`sdie`) 
LEFT JOIN t7 AS t12_dataSetTableN31727_sf31612 ON (t12_dataSetTableN31727_sf31612.`schemaDataId` = t12_dataSetTableN31727.`schemaData322Idf1`) 
LEFT JOIN t8 AS t12_dataSetTableN31727_sf31613 ON (t12_dataSetTableN31727_sf31613.`schemaDataId` = t12_dataSetTableN31727.`schemaData323Idf2`) 
LEFT JOIN t10 AS t12_dataSetTableN31727_sf31931 ON (t12_dataSetTableN31727_sf31931.`schemaDataId` = t12_dataSetTableN31727.`schemaData326Idf4`) 
LEFT JOIN t12 AS t11_sf22401 ON (t11_sf22401.`schemaDataId` = t44305.`schemaDataId` AND t11_sf22401.`schemaFieldId` = 22401) 
LEFT JOIN t13 AS t11_sf22401_sf22401 ON (t11_sf22401_sf22401.`id` = t11_sf22401.`schemaFieldOptionId`) 
WHERE 
  t11_sf22401_sf22401.`id` = 1245 
GROUP BY t2.`id`  
ORDER BY t2.`id` LIMIT 0, 20 
;

To make it fail I just run it simultaneously, I use mysqlslap to achieve it:
mysqlslap --concurrency=11 --query=last-t.sql --no-drop -uusername --create-schema=testcrash --delimiter=";"
Where last-t.sql is the attached file containing the above mentioned query, and testcrash is the DB schema which can be restored using the attached dump file (tiny-tc.sql.gz).

Here are some elements I have already tested which could help to understand the causes:

  • mysqlslap makes it fails with 11 as concurrency and, of course higher numbers. It doesn't with lower values. Most probably it depends on the hardware set, I tested it mainly in 36 cores envir.
  • all the parts of the query must be present to reproduce it.
  • one of the mandatory parts are the "text" columns included in the select list clause. If you just replace them with the id (integer) column of the same table it just stops failing.
  • it fails also replacing the concat_group function with another aggregate function such as count(1).
  • it doesn't fail if you force any other plan, for example by adding straight_join hint.
  • The contents of t1 table, the largest table because of the html text column must be as large as they are in the dump. Running exactly the same mysqlslap test with lighter contents prevents the crash.

This is the processlist you get when the behaviour is already happening:

MariaDB [testcrash]> show processlist;
 
+-----+-----------+-----------+-----------------+---------+------+----------------------+------------------------------------------------------------------------------------------------------+----------+
 
| Id  | User      | Host      | db              | Command | Time | State                | Info                                                                                                 | Progress |
 
+-----+-----------+-----------+-----------------+---------+------+----------------------+------------------------------------------------------------------------------------------------------+----------+
 
|   4 | mataquery | localhost | NULL            | Sleep   |    0 |                      | NULL                                                                                                 |    0.000 |
 
|   6 | root      | localhost | genesysloadtest | Sleep   |  603 |                      | NULL                                                                                                 |    0.000 |
 
|   9 | root      | localhost | testcrash       | Query   |    0 | init                 | show processlist                                                                                     |    0.000 |
 
| 155 | root      | localhost | testcrash       | Query   |   33 | Copying to tmp table | SELECT 
  t2.`id`,  
  t11_sf1.html AS `Description`, 
  t11_sf13076.text AS `JobLocation`, 
  GROUP |    0.000 |
 
| 156 | root      | localhost | testcrash       | Query   |   33 | Copying to tmp table | SELECT 
  t2.`id`,  
  t11_sf1.html AS `Description`, 
  t11_sf13076.text AS `JobLocation`, 
  GROUP |    0.000 |
 
| 157 | root      | localhost | testcrash       | Query   |   33 | Copying to tmp table | SELECT 
  t2.`id`,  
  t11_sf1.html AS `Description`, 
  t11_sf13076.text AS `JobLocation`, 
  GROUP |    0.000 |
 
| 158 | root      | localhost | testcrash       | Query   |   33 | Copying to tmp table | SELECT 
  t2.`id`,  
  t11_sf1.html AS `Description`, 
  t11_sf13076.text AS `JobLocation`, 
  GROUP |    0.000 |
 
| 159 | root      | localhost | testcrash       | Query   |   33 | Copying to tmp table | SELECT 
  t2.`id`,  
  t11_sf1.html AS `Description`, 
  t11_sf13076.text AS `JobLocation`, 
  GROUP |    0.000 |
 
| 160 | root      | localhost | testcrash       | Query   |   33 | Copying to tmp table | SELECT 
  t2.`id`,  
  t11_sf1.html AS `Description`, 
  t11_sf13076.text AS `JobLocation`, 
  GROUP |    0.000 |
 
| 161 | root      | localhost | testcrash       | Query   |   33 | Copying to tmp table | SELECT 
  t2.`id`,  
  t11_sf1.html AS `Description`, 
  t11_sf13076.text AS `JobLocation`, 
  GROUP |    0.000 |
 
| 162 | root      | localhost | testcrash       | Query   |   33 | Copying to tmp table | SELECT 
  t2.`id`,  
  t11_sf1.html AS `Description`, 
  t11_sf13076.text AS `JobLocation`, 
  GROUP |    0.000 |
 
| 163 | root      | localhost | testcrash       | Query   |   33 | Copying to tmp table | SELECT 
  t2.`id`,  
  t11_sf1.html AS `Description`, 
  t11_sf13076.text AS `JobLocation`, 
  GROUP |    0.000 |
 
| 164 | root      | localhost | testcrash       | Query   |   33 | Copying to tmp table | SELECT 
  t2.`id`,  
  t11_sf1.html AS `Description`, 
  t11_sf13076.text AS `JobLocation`, 
  GROUP |    0.000 |
 
| 165 | root      | localhost | testcrash       | Killed  |   33 | Copying to tmp table | SELECT 
  t2.`id`,  
  t11_sf1.html AS `Description`, 
  t11_sf13076.text AS `JobLocation`, 
  GROUP |    0.000 |
 
+-----+-----------+-----------+-----------------+---------+------+----------------------+------------------------------------------------------------------------------------------------------+----------+
 
14 rows in set (0.00 sec)

The last one (query 165) was killed but it gets hang on "killed" status for ever.

 Comments   
Comment by Alice Sherepa [ 2018-06-06 ]

please add your .cnf file(s)
I can not reproduce it so far. 

alice@henial:~/AM/_depot/m-tar/10.1.20/bin$ mysqlslap --concurrency=150 --query=/home/alice/Desktop/last-t.sql --no-drop -uroot --create-schema=test --delimiter=";" --socket=/home/alice/AM/m0-10.1.20/dt/my.sock --port=3306 --iterations=10
 
Benchmark
 
	Average number of seconds to run all queries: 1.613 seconds
 
	Minimum number of seconds to run all queries: 1.387 seconds
 
	Maximum number of seconds to run all queries: 1.837 seconds
 
	Number of clients running queries: 150
 
	Average number of queries per client: 1

Comment by Guillermo Schulman [ 2018-06-08 ]

Just uploaded the cnf.

Comment by Alice Sherepa [ 2018-06-11 ]

Could you please add the output of ANALYZE format=json of this query. Thanks!

Comment by Guillermo Schulman [ 2018-06-11 ]

Hi Alice, just uploaded the file with the requested content.

Comment by Alice Sherepa [ 2018-06-13 ]

Sorry, still not reproducible for me.
Please add SHOW EXPLAIN FOR <thread_id>; for connection, that hangs.

Comment by Guillermo Schulman [ 2018-06-13 ]

Once it is hanged you can not obtain its explain, it just gets hanged there:
MariaDB [(none)]> SHOW EXPLAIN FOR 151;

And after the lock wait timeout you get:
ERROR 1205 (HY000): Lock wait timeout exceeded; try restarting transaction

May I help in any other way?

Comment by Guillermo Schulman [ 2018-06-27 ]

Just to collaborate, I see the mysql tmp dir containing aria files (mad and mai extensions) during the proeblem. Just wondering if the problem might be related to it, as the queries state when the problem is going on is "Copying to tmp table".
The internal temporary tables are created with Aria in mariaDb and I can not set it to anything else because it is a read only setting (aria_used_for_temp_tables). This is just changeable at compiling time.
Here's how the mysql tmp dir looks like during the problem: 

total 7.2M
 
-rw-rw---- 1 mysql mysql 608K Jun 27 13:24 #sql_43c8_0.MAD
 
-rw-rw---- 1 mysql mysql 8.0K Jun 27 13:24 #sql_43c8_0.MAI
 
-rw-rw---- 1 mysql mysql 592K Jun 27 13:24 #sql_43c8_10.MAD
 
-rw-rw---- 1 mysql mysql 8.0K Jun 27 13:24 #sql_43c8_10.MAI
 
-rw-rw---- 1 mysql mysql 8.0K Jun 27 13:24 #sql_43c8_11.MAD
 
-rw-rw---- 1 mysql mysql 8.0K Jun 27 13:24 #sql_43c8_11.MAI
 
-rw-rw---- 1 mysql mysql 8.0K Jun 27 13:24 #sql_43c8_12.MAD
 
-rw-rw---- 1 mysql mysql 8.0K Jun 27 13:24 #sql_43c8_12.MAI
 
-rw-rw---- 1 mysql mysql 8.0K Jun 27 13:24 #sql_43c8_13.MAD
 
-rw-rw---- 1 mysql mysql 8.0K Jun 27 13:24 #sql_43c8_13.MAI
 
-rw-rw---- 1 mysql mysql 8.0K Jun 27 13:24 #sql_43c8_14.MAD
 
-rw-rw---- 1 mysql mysql 8.0K Jun 27 13:24 #sql_43c8_14.MAI
 
-rw-rw---- 1 mysql mysql 8.0K Jun 27 13:24 #sql_43c8_15.MAD
 
-rw-rw---- 1 mysql mysql 8.0K Jun 27 13:24 #sql_43c8_15.MAI
 
-rw-rw---- 1 mysql mysql 8.0K Jun 27 13:24 #sql_43c8_16.MAD
 
-rw-rw---- 1 mysql mysql 8.0K Jun 27 13:24 #sql_43c8_16.MAI
 
-rw-rw---- 1 mysql mysql 8.0K Jun 27 13:24 #sql_43c8_17.MAD
 
-rw-rw---- 1 mysql mysql 8.0K Jun 27 13:24 #sql_43c8_17.MAI
 
-rw-rw---- 1 mysql mysql 8.0K Jun 27 13:24 #sql_43c8_19.MAD
 
-rw-rw---- 1 mysql mysql 8.0K Jun 27 13:24 #sql_43c8_19.MAI
 
-rw-rw---- 1 mysql mysql 608K Jun 27 13:24 #sql_43c8_1.MAD
 
-rw-rw---- 1 mysql mysql 8.0K Jun 27 13:24 #sql_43c8_1.MAI
 
-rw-rw---- 1 mysql mysql 8.0K Jun 27 13:24 #sql_43c8_22.MAD
 
-rw-rw---- 1 mysql mysql 8.0K Jun 27 13:24 #sql_43c8_22.MAI
 
-rw-rw---- 1 mysql mysql 8.0K Jun 27 13:24 #sql_43c8_24.MAD
 
-rw-rw---- 1 mysql mysql 8.0K Jun 27 13:24 #sql_43c8_24.MAI
 
-rw-rw---- 1 mysql mysql 8.0K Jun 27 13:24 #sql_43c8_26.MAD
 
-rw-rw---- 1 mysql mysql 8.0K Jun 27 13:24 #sql_43c8_26.MAI
 
-rw-rw---- 1 mysql mysql 608K Jun 27 13:24 #sql_43c8_2.MAD
 
-rw-rw---- 1 mysql mysql 8.0K Jun 27 13:24 #sql_43c8_2.MAI
 
-rw-rw---- 1 mysql mysql 608K Jun 27 13:24 #sql_43c8_3.MAD
 
-rw-rw---- 1 mysql mysql 8.0K Jun 27 13:24 #sql_43c8_3.MAI
 
-rw-rw---- 1 mysql mysql 592K Jun 27 13:24 #sql_43c8_4.MAD
 
-rw-rw---- 1 mysql mysql 8.0K Jun 27 13:24 #sql_43c8_4.MAI
 
-rw-rw---- 1 mysql mysql 608K Jun 27 13:24 #sql_43c8_5.MAD
 
-rw-rw---- 1 mysql mysql 8.0K Jun 27 13:24 #sql_43c8_5.MAI
 
-rw-rw---- 1 mysql mysql 592K Jun 27 13:24 #sql_43c8_6.MAD
 
-rw-rw---- 1 mysql mysql 8.0K Jun 27 13:24 #sql_43c8_6.MAI
 
-rw-rw---- 1 mysql mysql 592K Jun 27 13:24 #sql_43c8_7.MAD
 
-rw-rw---- 1 mysql mysql 8.0K Jun 27 13:24 #sql_43c8_7.MAI
 
-rw-rw---- 1 mysql mysql 608K Jun 27 13:24 #sql_43c8_8.MAD
 
-rw-rw---- 1 mysql mysql 8.0K Jun 27 13:24 #sql_43c8_8.MAI
 
-rw-rw---- 1 mysql mysql 592K Jun 27 13:24 #sql_43c8_9.MAD
 
-rw-rw---- 1 mysql mysql 8.0K Jun 27 13:24 #sql_43c8_9.MAI

And it remains like that (same files, same size) until killing mysql service.
Just my two cents.

Comment by Elena Stepanova [ 2018-07-17 ]

Hi gschulman,

A few more questions.

  • The Killed connection in the processlist, the one about which you said "The last one (query 165) was killed but it gets hang on "killed" status for ever." – what killed it? Did you do it manually after things already went bad, or is there always some mysteriously "killed" connection when you encounter the problem?
  • Regarding temporary tables – it's natural that they are there, if the process hangs when it's created them before it had a chance to delete them; but did you check that the temporary directory hasn't run out of the disk space or isn't very close to it? That could certainly be related to the problem.
  • Would you be able maybe to get a stack trace from the hanging (but still alive) process when the problem occurs? You'd need to run 

    gdb --batch --eval-command="thread apply all bt" <path to mysqld binary> <mysqld pid>

    You might also need to install dbgsym packages to produce a meaningful output.

Comment by Guillermo Schulman [ 2018-07-23 ]

Hi Elena,
thanks for your feedback. Here are the answers for your questions:

. The Killed connection in the processlist, the one about which you said "The last one (query 165) was killed but it gets hang on "killed" status for ever." – what killed it? Did you do it manually after things already went bad, or is there always some mysteriously "killed" connection when you encounter the problem?

I kill it manually when things are still ok and no problem is noticeable. There are no mysterously triggered "kills", I run them manually.

. Regarding temporary tables – it's natural that they are there, if the process hangs when it's created them before it had a chance to delete them; but did you check that the temporary directory hasn't run out of the disk space or isn't very close to it? That could certainly be related to the problem.

I had checked it and it is not an out of disk space problem. Actually where I am able to reproduce the problem right now there are 40 Gb available in the mariaDb tmp disk and during the problem there are still 40 Gb available.

. Would you be able maybe to get a stack trace from the hanging (but still alive) process when the problem occurs? You'd need to run
gdb --batch --eval-command="thread apply all bt" /usr/sbin/mysqld 15208 > /tmp/stack-trace.output
You might also need to install dbgsym packages to produce a meaningful output.

I am attaching the obtained output file. I got this error message when I ran the command though:

[ root@test-server:~# ] gdb --batch --eval-command="thread apply all bt" /usr/sbin/mysqld 15208 > /tmp/stack-trace.output
84 ../sysdeps/unix/syscall-template.S: No such file or directory.

Comment by Elena Stepanova [ 2019-01-19 ]

gschulman,

Sorry for the delay.
Are you still facing this issue? If so, could you please try to increase aria-pagecache-buffer-size and see if the problem goes away?
If so, it would be likely a duplicate of MDEV-15206.
I can reproduce the effect on your test case easily if I use a lower value of aria-pagecache-buffer-size, with stack traces somewhat resembling what you observe, e.g.

10.1.20 release bintar

 
Thread 9 (Thread 0x7f0e0e774700 (LWP 21975)):
 
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
 
#1  0x0000562bc55d009b in inline_mysql_cond_wait (mutex=0x562bc6813008 <maria_pagecache_var+200>, that=0x7f0e58c1a1e8, src_file=<optimized out>, src_line=<optimized out>) at /build/include/mysql/psi/mysql_thread.h:1165
 
#2  inline_mysql_cond_wait (src_line=1816, mutex=0x562bc6813008 <maria_pagecache_var+200>, that=0x7f0e58c1a1e8, src_file=<optimized out>) at /build/storage/maria/ma_pagecache.c:1777
 
#3  get_hash_link (pagecache=pagecache@entry=0x562bc6812f40 <maria_pagecache_var>, file=file@entry=0x7f0e58cc0308, pageno=pageno@entry=0) at /build/storage/maria/ma_pagecache.c:1815
 
#4  0x0000562bc55d3c3e in find_block (page_st=<synthetic pointer>, reg_req=1 '\001', block_is_copied=1 '\001', wrmode=0 '\000', init_hits_left=0, pageno=0, file=0x7f0e58cc0308, pagecache=0x562bc6812f40 <maria_pagecache_var>) at /build/storage/maria/ma_pagecache.c:1912
 
#5  pagecache_read (pagecache=0x562bc6812f40 <maria_pagecache_var>, file=file@entry=0x7f0e58cc0308, pageno=pageno@entry=0, level=level@entry=0, buff=0x7f0e58d3a008 '\377' <repeats 200 times>..., type=type@entry=PAGECACHE_PLAIN_PAGE, lock=lock@entry=PAGECACHE_LOCK_LEFT_UNLOCKED, page_link=0x7f0e0e770890, page_link@entry=0x0) at /build/storage/maria/ma_pagecache.c:3393
 
#6  0x0000562bc560344c in _ma_read_bitmap_page (page=0, bitmap=0x7f0e58cc02d0, info=0x7f0e58c47008) at /build/storage/maria/ma_bitmap.c:1054
 
#7  _ma_change_bitmap_page (page=0, bitmap=0x7f0e58cc02d0, info=0x7f0e58c47008) at /build/storage/maria/ma_bitmap.c:1113
 
#8  move_to_next_bitmap (bitmap=0x7f0e58cc02d0, info=0x7f0e58c47008) at /build/storage/maria/ma_bitmap.c:1154
 
#9  find_head (info=info@entry=0x7f0e58c47008, length=3254, position=position@entry=3) at /build/storage/maria/ma_bitmap.c:1633
 
#10 0x0000562bc5604d8d in _ma_bitmap_find_place (info=info@entry=0x7f0e58c47008, row=row@entry=0x7f0e58c47060, blocks=blocks@entry=0x7f0e58c47060) at /build/storage/maria/ma_bitmap.c:2040
 
#11 0x0000562bc55fc182 in allocate_and_write_block_record (row=0x7f0e58c47060, record=0x7f0e58c3c020 "\372\r", info=0x7f0e58c47008, undo_lsn=<optimized out>) at /build/storage/maria/ma_blockrec.c:3533
 
#12 _ma_write_init_block_record (info=0x7f0e58c47008, record=0x7f0e58c3c020 "\372\r") at /build/storage/maria/ma_blockrec.c:3604
 
#13 0x0000562bc5608b1b in maria_write (info=0x7f0e58c47008, record=0x7f0e58c3c020 "\372\r") at /build/storage/maria/ma_write.c:156
 
#14 0x0000562bc50d8c98 in ha_write_tmp_row (buf=<optimized out>, this=0x7f0e58ce0d20) at /build/sql/sql_class.h:5448
 
#15 end_write (join=0x7f0e58c9b888, join_tab=<optimized out>, end_of_records=<optimized out>) at /build/sql/sql_select.cc:19668
 
#16 0x0000562bc50c8c62 in evaluate_null_complemented_join_record (join_tab=0x7f0e58c64438, join=<optimized out>) at /build/sql/sql_select.cc:18652
 
#17 sub_select (join=0x7f0e58c9b888, join_tab=<optimized out>, end_of_records=<optimized out>) at /build/sql/sql_select.cc:18348
 
#18 0x0000562bc50be7e0 in evaluate_join_record (join=join@entry=0x7f0e58c9b888, join_tab=join_tab@entry=0x7f0e58c640f0, error=<optimized out>) at /build/sql/sql_select.cc:18529
 
#19 0x0000562bc50c895e in sub_select (join=0x7f0e58c9b888, join_tab=0x7f0e58c640f0, end_of_records=<optimized out>) at /build/sql/sql_select.cc:18304
 
#20 0x0000562bc50c8c62 in evaluate_null_complemented_join_record (join_tab=0x7f0e58c63da8, join=<optimized out>) at /build/sql/sql_select.cc:18652
 
#21 sub_select (join=0x7f0e58c9b888, join_tab=<optimized out>, end_of_records=<optimized out>) at /build/sql/sql_select.cc:18348
 
#22 0x0000562bc50c8c62 in evaluate_null_complemented_join_record (join_tab=0x7f0e58c63a60, join=<optimized out>) at /build/sql/sql_select.cc:18652
 
#23 sub_select (join=0x7f0e58c9b888, join_tab=<optimized out>, end_of_records=<optimized out>) at /build/sql/sql_select.cc:18348
 
#24 0x0000562bc50c8c62 in evaluate_null_complemented_join_record (join_tab=0x7f0e58c63718, join=<optimized out>) at /build/sql/sql_select.cc:18652
 
#25 sub_select (join=0x7f0e58c9b888, join_tab=<optimized out>, end_of_records=<optimized out>) at /build/sql/sql_select.cc:18348
 
#26 0x0000562bc50c8c62 in evaluate_null_complemented_join_record (join_tab=0x7f0e58c633d0, join=<optimized out>) at /build/sql/sql_select.cc:18652
 
#27 sub_select (join=0x7f0e58c9b888, join_tab=<optimized out>, end_of_records=<optimized out>) at /build/sql/sql_select.cc:18348
 
#28 0x0000562bc50be7e0 in evaluate_join_record (join=join@entry=0x7f0e58c9b888, join_tab=join_tab@entry=0x7f0e58c63088, error=<optimized out>) at /build/sql/sql_select.cc:18529
 
#29 0x0000562bc50c895e in sub_select (join=0x7f0e58c9b888, join_tab=0x7f0e58c63088, end_of_records=<optimized out>) at /build/sql/sql_select.cc:18304
 
#30 0x0000562bc50c8c62 in evaluate_null_complemented_join_record (join_tab=0x7f0e58c62d40, join=<optimized out>) at /build/sql/sql_select.cc:18652
 
#31 sub_select (join=0x7f0e58c9b888, join_tab=<optimized out>, end_of_records=<optimized out>) at /build/sql/sql_select.cc:18348
 
#32 0x0000562bc50be7e0 in evaluate_join_record (join=join@entry=0x7f0e58c9b888, join_tab=join_tab@entry=0x7f0e58c629f8, error=<optimized out>) at /build/sql/sql_select.cc:18529
 
#33 0x0000562bc50c895e in sub_select (join=0x7f0e58c9b888, join_tab=0x7f0e58c629f8, end_of_records=<optimized out>) at /build/sql/sql_select.cc:18304
 
#34 0x0000562bc50be7e0 in evaluate_join_record (join=join@entry=0x7f0e58c9b888, join_tab=join_tab@entry=0x7f0e58c626b0, error=<optimized out>) at /build/sql/sql_select.cc:18529
 
#35 0x0000562bc50c895e in sub_select (join=0x7f0e58c9b888, join_tab=0x7f0e58c626b0, end_of_records=<optimized out>) at /build/sql/sql_select.cc:18304
 
#36 0x0000562bc50be7e0 in evaluate_join_record (join=join@entry=0x7f0e58c9b888, join_tab=join_tab@entry=0x7f0e58c62368, error=<optimized out>) at /build/sql/sql_select.cc:18529
 
#37 0x0000562bc50c895e in sub_select (join=0x7f0e58c9b888, join_tab=0x7f0e58c62368, end_of_records=<optimized out>) at /build/sql/sql_select.cc:18304
 
#38 0x0000562bc50d66bd in do_select (join=join@entry=0x7f0e58c9b888, fields=fields@entry=0x0, table=table@entry=0x7f0e58d38020, procedure=procedure@entry=0x0) at /build/sql/sql_select.cc:17959
 
#39 0x0000562bc50e9294 in JOIN::exec_inner (this=this@entry=0x7f0e58c9b888) at /build/sql/sql_select.cc:2759
 
#40 0x0000562bc50eb54d in JOIN::exec (this=this@entry=0x7f0e58c9b888) at /build/sql/sql_select.cc:2505
 
#41 0x0000562bc50e7f3a in mysql_select (thd=thd@entry=0x7f0e13a9f008, rref_pointer_array=rref_pointer_array@entry=0x7f0e13aa3288, tables=<optimized out>, wild_num=<optimized out>, fields=..., conds=<optimized out>, og_num=2, order=0x7f0e58c9b540, group=0x7f0e58c9b3e0, having=0x0, proc_param=0x0, select_options=2147781376, result=result@entry=0x7f0e58c9b868, unit=unit@entry=0x7f0e13aa2910, select_lex=select_lex@entry=0x7f0e13aa3010) at /build/sql/sql_select.cc:3439
 
#42 0x0000562bc50eb82d in handle_select (thd=thd@entry=0x7f0e13a9f008, lex=lex@entry=0x7f0e13aa2848, result=result@entry=0x7f0e58c9b868, setup_tables_done_option=setup_tables_done_option@entry=0) at /build/sql/sql_select.cc:384
 
#43 0x0000562bc508b272 in execute_sqlcom_select (thd=thd@entry=0x7f0e13a9f008, all_tables=0x7f0e58c2e440) at /build/sql/sql_parse.cc:5896
 
#44 0x0000562bc5097530 in mysql_execute_command (thd=thd@entry=0x7f0e13a9f008) at /build/sql/sql_parse.cc:2968
 
#45 0x0000562bc509aaf4 in mysql_parse (thd=0x7f0e13a9f008, rawbuf=<optimized out>, length=<optimized out>, parser_state=<optimized out>) at /build/sql/sql_parse.cc:7319
 
#46 0x0000562bc509d5c3 in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x7f0e13a9f008, packet=0x7f0e58c2c020 "SELECT t2.`id`, t11_sf1.html, t11_sf13076.text, GROUP_CONCAT( CONCAT(IFNULL(t12_dataSetTableN31727_sf31612.`f2`,\"\"), \",\", IFNULL(t12_dataSetTableN31727_sf31613.`f2`,\"\"), \",\", IFNULL(t12_dataSetTableN3"..., packet_length=1566, packet_length@entry=1567) at /build/sql/sql_parse.cc:1487
 
#47 0x0000562bc509db5b in do_command (thd=thd@entry=0x7f0e13a9f008) at /build/sql/sql_parse.cc:1108
 
#48 0x0000562bc5196177 in threadpool_process_request (thd=0x7f0e13a9f008) at /build/sql/threadpool_common.cc:271
 
#49 0x0000562bc51e0f75 in handle_event (connection=0x7f0e64c3ca08) at /build/sql/threadpool_unix.cc:1450
 
#50 worker_main (param=param@entry=0x7f0e59ea8608) at /build/sql/threadpool_unix.cc:1498
 
#51 0x0000562bc559bc2d in pfs_spawn_thread (arg=0x7f0e05012c08) at /build/storage/perfschema/pfs.cc:1860
 
#52 0x00007f0e6631e494 in start_thread (arg=0x7f0e0e774700) at pthread_create.c:333
 
#53 0x00007f0e659db93f in clone () from /lib/x86_64-linux-gnu/libc.so.6

Comment by Elena Stepanova [ 2019-02-17 ]

Assuming for now it is indeed a duplicate of MDEV-15206.

Comment by Guillermo Schulman [ 2019-08-06 ]

Hi Elena, sorry for the delay.
Yes, it was indeed the low value for aria-pagecache-buffer-size variable which we set to a small value as we don't use explicitly aria storage engine but mariaDb uses it internally. Thanks for your help!

Generated at Thu Feb 08 08:28:34 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.