[MDEV-32643] "[ERROR] mysqld got signal 11" with one SQL Created: 2023-11-01  Updated: 2023-12-15  Resolved: 2023-12-15

Status: Closed
Project: MariaDB Server
Component/s: Server
Affects Version/s: 10.6.14
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: William Wong Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: None
Environment:

redhat 7 on VMware
Galera


Issue Links:
Duplicate
duplicates MDEV-31440 choose_best_splitting: crash on updat... Confirmed

 Description   

Hi ,

Our environment is 3 nodes Galera clusters - 2 DB nodes + 1 arbitrator

One of our DB encounter signal 11 when application runs a particular SQL. Below is the stack trace from DB log. Kindly advise what we can do to troubleshoot this case

231020 11:03:00 [ERROR] mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
 
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
 
Server version: 10.6.14-MariaDB-log source revision: c93754d45e5d9379e3e23d7ada1d5f21d2711f66
key_buffer_size=20971520
read_buffer_size=1048576
max_used_connections=348
max_threads=511
thread_count=477
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1606203 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
 
Thread pointer: 0x7f48e03e29b8
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x7f49a5eddc80 thread_stack 0x49000
/usr/sbin/mariadbd(my_print_stacktrace+0x2e)[0x55f5501f2cde]
/usr/sbin/mariadbd(handle_fatal_signal+0x307)[0x55f54fc45c27]
sigaction.c:0(__restore_rt)[0x7f4c38ff0630]
/usr/sbin/mariadbd(_ZN13st_join_table21choose_best_splittingEjyPK8POSITIONPy+0x9ac)[0x55f54fbb8e9c]
/usr/sbin/mariadbd(_Z16best_access_pathP4JOINP13st_join_tableyPK8POSITIONjbdPS3_S6_+0x139)[0x55f54fa72629]
/usr/sbin/mariadbd(+0x81bd48)[0x55f54fa75d48]
/usr/sbin/mariadbd(+0x81c42a)[0x55f54fa7642a]
/usr/sbin/mariadbd(+0x81c42a)[0x55f54fa7642a]
/usr/sbin/mariadbd(+0x81c42a)[0x55f54fa7642a]
/usr/sbin/mariadbd(+0x81c42a)[0x55f54fa7642a]
/usr/sbin/mariadbd(+0x81c42a)[0x55f54fa7642a]
/usr/sbin/mariadbd(_Z11choose_planP4JOINy+0x257)[0x55f54fa76977]
/usr/sbin/mariadbd(_ZN4JOIN14optimize_innerEv+0x3119)[0x55f54fa7f829]
/usr/sbin/mariadbd(_ZN4JOIN8optimizeEv+0xb2)[0x55f54fa7ff72]
/usr/sbin/mariadbd(_Z12mysql_selectP3THDP10TABLE_LISTR4ListI4ItemEPS4_jP8st_orderS9_S7_S9_yP13select_resultP18st_select_lex_unitP13st_select_lex+0xb8)[0x55f54fa80048]
/usr/sbin/mariadbd(_Z13handle_selectP3THDP3LEXP13select_resultm+0x1e4)[0x55f54fa80954]
/usr/sbin/mariadbd(+0x67a619)[0x55f54f8d4619]
/usr/sbin/mariadbd(_Z21mysql_execute_commandP3THDb+0x34c3)[0x55f54fa21043]
/usr/sbin/mariadbd(_Z11mysql_parseP3THDPcjP12Parser_state+0x20b)[0x55f54fa233eb]
/usr/sbin/mariadbd(+0x7c9c01)[0x55f54fa23c01]
/usr/sbin/mariadbd(_Z16dispatch_command19enum_server_commandP3THDPcjb+0x1dd3)[0x55f54fa26153]
/usr/sbin/mariadbd(_Z10do_commandP3THDb+0x123)[0x55f54fa26d53]
/usr/sbin/mariadbd(_Z24do_handle_one_connectionP7CONNECTb+0x187)[0x55f54fb1f877]
/usr/sbin/mariadbd(handle_one_connection+0x34)[0x55f54fb1fb14]
/usr/sbin/mariadbd(+0xc381dc)[0x55f54fe921dc]
pthread_create.c:0(start_thread)[0x7f4c38fe8ea5]
/lib64/libc.so.6(clone+0x6d)[0x7f4c3850396d]
 
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
 
Query (0x7f48e03fefa0): SELECT rch_lic_no, t1.res_name,t1.age, t1.infected_date, t1.test_type, ifnull(t1.no_of_vacc,0), t1.type_of_vacc  ,name_chi, rch_service_cd, c3.desc_eng as rch_type_cd, c4.desc_eng as care_lvl_cd, ifnull(capacity,0), ifnull(res.totalResident,0), case when (capacity <= 60 )then 'A'   when ( capacity >= 61 and capacity  <= 100 )then 'B'  when ( capacity >= 101 and  capacity  <= 150)then 'C'  when ( capacity > 150)then 'D' end as capacityType  ,ifnull(area,0.0), c2.desc_eng, rch_cgat_cd , IFNULL(t1.holdctr_cd, '') , IFNULL(t1.holdctr_admit_date, '') FROM rch_organization o LEFT JOIN (               SELECT o.org_id, rrh.name AS res_name , case when (rrh.dob = '') then NULL ELSE  (2023 - if(date_format(rrh.dob, '%Y') IS NULL, rrh.dob, DATE_FORMAT(rrh.dob, '%Y'))) END AS age ,  rrh.infected_date AS infected_date, c1.desc_chi as test_type                               , case when (rrh.vaccine_brand1 IS not null and rrh.vaccine_brand2 IS null) then '1'                              when (rrh.vaccine_brand2 IS NOT NULL AND rrh.vaccine_brand3 IS NULL) then '2'                                   when (rrh.vaccine_brand3 IS NOT NULL AND rrh.vaccine_brand4 IS NULL) then '3'                                   when (rrh.vaccine_brand4 IS NOT NULL AND rrh.vaccine_brand5 IS NULL) then '4'                                   when (rrh.vaccine_brand5 IS NOT NULL) then '5'                                 ELSE '0' END AS no_of_vacc,                             case when (rrh.vaccine_brand1 IS not null and rrh.vaccine_brand2 IS null) then rrh.vaccine_brand1                                when (rrh.vaccine_brand2 IS NOT NULL AND rrh.vaccine_brand3 IS NULL) then rrh.vaccine_brand2                                    when (rrh.vaccine_brand3 IS NOT NULL and rrh.vaccine_brand4 IS NULL) then rrh.vaccine_brand3                                    when (rrh.vaccine_brand4 IS NOT NULL and rrh.vaccine_brand5 IS NULL) then rrh.vaccine_brand4                                    when (rrh.vaccine_brand5 IS NOT NULL) then rrh.vaccine_brand5                                  ELSE '' END AS type_of_vacc, holdctr_cd, holdctr_admit_date                             FROM  rch_resident_history rrh                                        LEFT JOIN  (SELECT * from rch_organization WHERE status = 'A' and effective_date <= '2023/10/19' ) o                                    ON o.org_id = rrh.org_id                                    LEFT JOIN (SELECT * from rch_code_table WHERE code_type like 'TEST_TYPE%' )  c1 on c1.code_value = rrh.infected_test_type_cd                                    WHERE rrh.snapshot_date= (SELECT  MAX(snapshot_date)AS snapshot_date FROM rch_resident_history) and rrh.status = 'A'        )t1 ON t1.org_id = o.org_id  LEFT JOIN (SELECT org_id, COUNT(resident_id) AS totalResident FROM rch_resident GROUP BY org_id) res ON o.org_id = res.org_id   LEFT JOIN (SELECT * from rch_code_table WHERE code_type = 'DISTRICT' )  c2 on c2.code_value = o.district_cd  LEFT JOIN (SELECT * FROM rch_code_table WHERE code_type = 'HOME_TYPE') c3 on c3.code_value = o.rch_type_cd  LEFT JOIN (SELECT * FROM rch_code_table WHERE code_type = 'CARE_LEVEL')c4 on c4.code_value = o.care_lvl_cd  WHERE  org_type = 'RCH'       AND (t1.infected_date >= '2022-01-01'  AND t1.infected_date  <= '2023/10/19' AND t1.infected_date >= '2000-01-01') and o.status = 'A' and o.effective_date <= '2023/10/19' and o.org_id in (79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 237, 238, 239, 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260, 261, 262, 263, 264, 265, 266, 267, 268, 269, 270, 271, 272, 273, 274, 275, 276, 277, 278, 279, 280, 281, 282, 283, 284, 285, 286, 287, 288, 289, 290, 291, 292, 293, 294, 295, 296, 297, 298, 299, 300, 301, 302, 303, 304, 305, 306, 307, 308, 309, 310, 311, 312, 313, 314, 315, 316, 317, 318, 319, 320, 321, 322, 323, 324, 325, 326, 327, 328, 329, 330, 331, 332, 333, 334, 335, 336, 337, 338, 339, 340, 341, 342, 343, 344, 345, 346, 347, 348, 349, 350, 351, 352, 353, 354, 355, 356, 357, 358, 359, 360, 361, 362, 363, 364, 365, 366, 367, 368, 369, 370, 371, 372, 373, 374, 375, 377, 379, 380, 381, 382, 383, 384, 385, 386, 387, 388, 389, 390, 391, 392, 393, 394, 395, 396, 397, 398, 399, 400, 401, 402, 403, 404, 405, 406, 407, 408, 409, 410, 411, 412, 413, 414, 415, 416, 417, 418, 419, 420, 421, 422, 423, 424, 425, 426, 427, 428, 429, 430, 431, 432, 433, 434, 435, 436, 437, 438, 439, 440, 441, 442, 443, 444, 445, 446, 447, 448, 449, 450, 451, 452, 453, 454, 455, 456, 457, 458, 459, 460, 461, 462, 463, 464, 465, 466, 467, 468, 469, 470, 471, 472, 473, 474, 475, 477, 478, 479, 480, 481, 482, 483, 484, 485, 486, 487, 488, 489, 490, 491, 492, 493, 494, 495, 496, 497, 498, 499, 500, 501, 502, 503, 504, 505, 506, 507, 508, 509, 510, 511, 512, 513, 514, 515, 516, 517, 518, 519, 520, 521, 522, 523, 524, 525, 526, 527, 528, 529, 530, 531, 532, 533, 534, 535, 536, 537, 538, 539, 540, 541, 542, 543, 544, 545, 546, 547, 548, 549, 550, 551, 552, 553, 554, 555, 557, 558, 559, 560, 561, 562, 563, 564, 565, 566, 567, 568, 569, 570, 571, 572, 573, 574, 575, 576, 577, 578, 579, 580, 581, 582, 583, 584, 585, 586, 587, 588, 589, 590, 591, 592, 593, 594, 595, 596, 597, 598, 599, 600, 601, 602, 603, 604, 605, 606, 607, 608, 609, 610, 611, 612, 613, 614, 615, 616, 617, 618, 619, 620, 621, 622, 623, 624, 625, 626, 627, 628, 629, 630, 631, 632, 633, 634, 635, 636, 637, 638, 639, 640, 641, 642, 643, 644, 645, 646, 647, 649, 650, 651, 652, 653, 654, 655, 656, 657, 658, 659, 660, 661, 662, 663, 664, 665, 666, 667, 668, 669, 670, 671, 672, 673, 674, 675, 676, 677, 679, 681, 682, 683, 684, 685, 687, 688, 690, 692, 693, 694, 695, 696, 697, 698, 699, 700, 701, 702, 703, 704, 705, 707, 708, 709, 710, 711, 712, 713, 714, 715, 717, 718, 719, 720, 721, 722, 723, 724, 725, 726, 727, 728, 729, 730, 731, 732, 733, 735, 736, 737, 738, 739, 740, 741, 742, 744, 745, 746, 747, 748, 750, 751, 752, 753, 754, 755, 756, 757, 758, 762, 763, 764, 765, 766, 767, 768, 769, 770, 771, 772, 773, 774, 775, 777, 778, 779, 780, 781, 782, 783, 784, 785, 786, 787, 788, 789, 790, 791, 792, 793, 794, 795, 796, 797, 798, 799, 800, 801, 802, 803, 804, 805, 806, 807, 808, 809, 810, 811, 812, 813, 814, 815, 816, 817, 818, 819, 820, 821, 822, 823, 824, 825, 826, 827, 828, 830, 831, 832, 834, 835, 836, 837, 838, 839, 840, 841, 842, 843, 844, 845, 846, 847, 848, 849, 850, 851, 852, 853, 854, 855, 856, 857, 858, 859, 860, 861, 862, 863, 864, 865, 866, 867, 868, 869, 870, 871, 872, 873, 874, 875, 876, 877, 878, 879, 880, 881, 882, 883, 884, 885, 886, 887, 888, 889, 890, 891, 892, 893, 894, 895, 896, 897, 898, 899, 900, 901, 902, 903, 905, 906, 907, 908, 909, 910, 911, 912, 913, 914, 915, 916, 917, 918, 919, 920, 921, 922, 923, 924, 925, 926, 927, 928, 929, 930, 931, 932, 933, 934, 935, 936, 937, 938, 939, 940, 941, 942, 943, 944, 945, 946, 947, 948, 949, 950, 951, 952, 953, 954, 955, 956, 957, 958, 959, 960, 961, 962, 963, 964, 965, 966, 967, 968, 969, 970, 971, 972, 974, 975, 976, 977, 978, 979, 980, 981, 982, 983, 984, 985, 986, 987, 988, 989, 991, 992, 993, 994, 995, 996, 997, 998, 999, 1000, 1001, 1002, 1003, 1004, 1005, 1006, 1007, 1008, 1009, 1010, 1011, 1012, 1013, 1014, 1015, 1016, 1017, 1018, 1019, 1020, 1021, 1022, 1023, 1025, 1026, 1027, 1028, 1030, 1031, 1032, 1033, 1034, 1035, 1036, 1037, 1038, 1039, 1040, 1041, 1042, 1043, 1044, 1045, 1046, 1047, 1048, 1049, 1051, 1052, 1053, 1054, 1055, 1056, 1057, 1058, 1059, 1060, 1061, 1062, 1063, 1064, 1065, 1066, 1067, 1068, 1069, 1070, 1071, 1072, 1073, 1074, 1075, 1076, 1077, 1078, 1079, 1080, 1081, 1082, 1083, 1085, 1086, 1087, 1088, 1089, 1090, 1091, 1092, 1093, 1094, 1095, 1096, 1097, 1098, 1099, 1100, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, 1112, 1113, 1115, 1116, 1117, 1118, 1119, 1120, 1121, 1122, 1123, 1124, 1125, 1126, 1127, 1128, 1129, 1130, 1131, 1132, 1133, 1134, 1135, 1136, 1137, 1138, 1139, 1140, 1141, 1142, 1143, 1144, 1145, 1146, 1149, 1150, 1151, 1152, 1153, 1154, 1155, 1156, 1157, 1158, 1159, 1160, 1161, 1162, 1163, 1164, 1165, 1166, 1167, 1168, 1169, 1170, 1171, 1172, 1173, 1174, 1175, 1176, 1177, 1178, 1179, 1180, 1181, 1182, 1183, 1184, 2146, 2149, 2151, 2177, 2179, 2182, 2185, 2188, 2194, 2197, 2200, 2203, 2206, 2209, 2212, 2215, 2218, 2221, 2224, 2227, 2230, 2233, 2236, 2239, 2242, 2245, 2248, 2251, 2254, 2257, 2260, 2263, 2266, 2269, 2272, 2275, 2278, 2281, 2284, 2287, 2290, 2296, 2299, 2302, 2305, 2308, 2311, 2314) order by rch_lic_no, t1.infected_date
 
Connection ID (thread ID): 4884498
Status: NOT_KILLED
 
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=on,condition_pushdown_from_having=on,not_null_range_scan=off,hash_join_cardinality=off

Regards,
William Wong



 Comments   
Comment by Alice Sherepa [ 2023-12-15 ]

This is most likely the same problem as MDEV-31440.
Maybe try to set optimizer_switch='split_materialized=off' as a temporary workaround.
If after the patch of MDEV-31440 the bug will appear again - please comment here and the report will be reopened and investigated further.

Generated at Thu Feb 08 10:32:54 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.