messages Feb 22 09:53:13 redcap-dbserver-prod-02 mariadbd: 220222 9:53:13 [ERROR] mysqld got signal 11 ; Feb 22 09:53:13 redcap-dbserver-prod-02 mariadbd: This could be because you hit a bug. It is also possible that this binary Feb 22 09:53:13 redcap-dbserver-prod-02 mariadbd: or one of the libraries it was linked against is corrupt, improperly built, Feb 22 09:53:13 redcap-dbserver-prod-02 mariadbd: or misconfigured. This error can also be caused by malfunctioning hardware. Feb 22 09:53:13 redcap-dbserver-prod-02 mariadbd: To report this bug, see https://mariadb.com/kb/en/reporting-bugs Feb 22 09:53:13 redcap-dbserver-prod-02 mariadbd: We will try our best to scrape up some info that will hopefully help Feb 22 09:53:13 redcap-dbserver-prod-02 mariadbd: diagnose the problem, but since we have already crashed, Feb 22 09:53:13 redcap-dbserver-prod-02 mariadbd: something is definitely wrong and this may fail. Feb 22 09:53:13 redcap-dbserver-prod-02 mariadbd: Server version: 10.5.13-MariaDB Feb 22 09:53:13 redcap-dbserver-prod-02 mariadbd: key_buffer_size=134217728 Feb 22 09:53:13 redcap-dbserver-prod-02 mariadbd: read_buffer_size=131072 Feb 22 09:53:13 redcap-dbserver-prod-02 mariadbd: max_used_connections=173 Feb 22 09:53:13 redcap-dbserver-prod-02 mariadbd: max_threads=302 Feb 22 09:53:13 redcap-dbserver-prod-02 mariadbd: thread_count=14 Feb 22 09:53:13 redcap-dbserver-prod-02 mariadbd: It is possible that mysqld could use up to Feb 22 09:53:13 redcap-dbserver-prod-02 mariadbd: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 795892 K bytes of memory Feb 22 09:53:13 redcap-dbserver-prod-02 mariadbd: Hope that's ok; if not, decrease some variables in the equation. Feb 22 09:53:13 redcap-dbserver-prod-02 mariadbd: Thread pointer: 0x7fbafc0008d8 Feb 22 09:53:13 redcap-dbserver-prod-02 mariadbd: Attempting backtrace. You can use the following information to find out Feb 22 09:53:13 redcap-dbserver-prod-02 mariadbd: where mysqld died. If you see no messages after this, something went Feb 22 09:53:13 redcap-dbserver-prod-02 mariadbd: terribly wrong... Feb 22 09:53:13 redcap-dbserver-prod-02 mariadbd: stack_bottom = 0x7fbb4d7f9c90 thread_stack 0x49000 Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: ??:0(my_print_stacktrace)[0x55d161720d6e] Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: ??:0(handle_fatal_signal)[0x55d161123367] Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: sigaction.c:0(__restore_rt)[0x7fbe193aa630] Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: ??:0(st_join_table::save_explain_data(Explain_table_access*, unsigned long long, bool, st_join_table*))[0x55d160f64c4f] Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: ??:0(JOIN::save_explain_data_intern(Explain_query*, bool, bool, bool, char const*))[0x55d160f6678b] Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: ??:0(JOIN::save_explain_data(Explain_query*, bool, bool, bool, bool))[0x55d160f668f0] Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: ??:0(JOIN::build_explain())[0x55d160f66b4c] Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: ??:0(JOIN::optimize())[0x55d160f7c0b1] Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: ??:0(mysql_select(THD*, TABLE_LIST*, List&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*))[0x55d160f7c1c7] Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: ??:0(handle_select(THD*, LEX*, select_result*, unsigned long))[0x55d160f7ce34] Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: /usr/sbin/mariadbd(+0x63b749)[0x55d160ddb749] Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: ??:0(mysql_execute_command(THD*))[0x55d160f2087d] Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: ??:0(mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool))[0x55d160f22d15] Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: ??:0(dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool))[0x55d160f255e5] Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: ??:0(do_command(THD*))[0x55d160f26d7f] Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: ??:0(do_handle_one_connection(CONNECT*, bool))[0x55d161013662] Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: ??:0(handle_one_connection)[0x55d161013934] Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: ??:0(MyCTX_nopad::finish(unsigned char*, unsigned int*))[0x55d16137291d] Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: pthread_create.c:0(start_thread)[0x7fbe193a2ea5] Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: ??:0(__clone)[0x7fbe188bdb0d] Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: Trying to get some variables. Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: Some pointers may be invalid and cause the dump to abort. Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: Query (0x7fbafc010510): select distinct event_id, field_name, value, instance, record Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: from redcap_data where project_id = 7827 and field_name in ('record_id', 'notes', 'email', 'date_consent', 'consent_person', 'recruit_md', 'date_start', 'date_45d', 'date_end', 'vis_id', 'fitbit_id', 'date_fitbit_mailed', 'fitbit_mail_track', 'date_fitbit_delivered', 'date_fitbit_data_start', 'return_fitbit_date', 'payment_comments', 'payment_date', 'vincent_number', 'date_payment_card_confirmed', 'date_payment_card_loaded', 'payment_amount', 'payment_partial_explain', 'return_phone_date', 'android_version', 'iphone_version', 'aware_android_version', 'aware_iphone_version', 'uuid', 'smartphone_storage_total', 'smartphone_storage_free', 'fitbit_email', 'fitbit_password', 'fitbit_firmware', 'fitbit_android_version', 'fitbit_iphone_version', 'uuid_end_date', 'uuid2', 'uuid2_start_date', 'fitbit2_email', 'flag_date', 'aware_lastsync', 'fitbit_lastsync', 'daily_lastsync', 'weekly_lastsync', 'flag_notes', 'flag_troubleshoot', 'com_notes', 'data_notes', 'opioid1_other_spec_45d', 'notes_45d', 'notes_90d', 'name', 'phone_number', 'number_type_other', 'phone_number_2', 'number_type_other_2', 'contact_when', 'address_street', 'address_city', 'address_state', 'address_zip', 'demo_datetime', 'dob', 'age', 'height_ft', 'height_inch', 'height', 'weight', 'bmi', 'race_other', 'race_multiple', 'living_other', 'employment_other', 'smoker_years', 'smoker_cigarettesdaily', 'wifi_note', 'own_comp_note', 'own_tablet_note', 'own_smartphone_model', 'own_smartphone_note', 'access_internet_note', 'social_media_note', 'own_tracker_note', 'notes_bl', 'cancer_type', 'cancer_dx_date', 'cancer_stage', 'comorbid_dx', 'chemo_start', 'emr_height_in', 'emr_weight_lb', 'date_death', 'chemo1_date', 'chemo1_cycle', 'chemo1_day', 'chemo1_med', 'chemo1_notes', 'chemo2_date', 'chemo2_cycle', 'chemo2_day', 'chemo2_med', 'chemo2_notes', 'chemo3_date', 'chemo3_cycle', 'chemo3_day', 'chemo3_med', 'chemo3_notes', 'chemo4_date', 'chemo4_cycle', 'chemo4_day', 'chemo4_med', 'chemo4_notes', 'chemo5_date', 'chemo5_cycle', 'chemo5_day', 'chemo5_med', 'chemo5_notes', 'chemo6_date', 'chemo6_cycle', 'chemo6_day', 'chemo6_med', 'chemo6_notes', 'chemo7_date', 'chemo7_cycle', 'chemo7_day', 'chemo7_med', 'chemo7_notes', 'chemo8_date', 'chemo8_cycle', 'chemo8_day', 'chemo8_med', 'chemo8_notes', 'chemo9_date', 'chemo9_cycle', 'chemo9_day', 'chemo9_med', 'chemo9_notes', 'chemo10_date', 'chemo10_cycle', 'chemo10_day', 'chemo10_med', 'chemo10_notes', 'chemo11_date', 'chemo11_cycle', 'chemo11_day', 'chemo11_med', 'chemo11_notes', 'chemo12_date', 'chemo12_cycle', 'chemo12_day', 'chemo12_med', 'chemo12_notes', 'chemo13_date', 'chemo13_cycle', 'chemo13_day', 'chemo13_med', 'chemo13_notes', 'chemo14_date', 'chemo14_cycle', 'chemo14_day', 'chemo14_med', 'chemo14_notes', 'chemo15_date', 'chemo15_cycle', 'chemo15_day', 'chemo15_med', 'chemo15_notes', 'chemo16_date', 'chemo16_cycle', 'chemo16_day', 'chemo16_med', 'chemo16_notes', 'chemo17_date', 'chemo17_cycle', 'chemo17_day', 'chemo17_med', 'chemo17_notes', 'chemo18_date', 'chemo18_cycle', 'chemo18_day', 'chemo18_med', 'chemo18_notes', 'chemo19_date', 'chemo19_cycle', 'chemo19_day', 'chemo19_med', 'chemo19_notes', 'chemo20_date', 'chemo20_cycle', 'chemo20_day', 'chemo20_med', 'chemo20_notes', 'chemo21_date', 'chemo21_cycle', 'chemo21_day', 'chemo21_med', 'chemo21_notes', 'chemo22_date', 'chemo22_cycle', 'chemo22_day', 'chemo22_med', 'chemo22_notes', 'chemo23_date', 'chemo23_cycle', 'chemo23_day', 'chemo23_med', 'chemo23_notes', 'chemo24_date', 'chemo24_cycle', 'chemo24_day', 'chemo24_med', 'chemo24_notes', 'chemo25_date', 'chemo25_cycle', 'chemo25_day', 'chemo25_med', 'chemo25_notes', 'chemo26_date', 'chemo26_cycle', 'chemo26_day', 'chemo26_med', 'chemo26_notes', 'chemo27_date', 'chemo27_cycle', 'chemo27_day', 'chemo27_med', 'chemo27_notes', 'chemo28_date', 'chemo28_cycle', 'chemo28_day', 'chemo28_med', 'chemo28_notes', 'chemo29_date', 'chemo29_cycle', 'chemo29_day', 'chemo29_med', 'chemo29_notes', 'chemo30_date', 'chemo30_cycle', 'chemo30_day', 'chemo30_med', 'chemo30_notes', 'chemo31_date', 'chemo31_cycle', 'chemo31_day', 'chemo31_med', 'chemo31_notes', 'chemo32_date', 'chemo32_cycle', 'chemo32_day', 'chemo32_med', 'chemo32_notes', 'chemo33_date', 'chemo33_cycle', 'chemo33_day', 'chemo33_med', 'chemo33_notes', 'chemo34_date', 'chemo34_cycle', 'chemo34_day', 'chemo34_med', 'chemo34_notes', 'chemo35_date', 'chemo35_cycle', 'chemo35_day', 'chemo35_med', 'chemo35_notes', 'chemo36_date', 'chemo36_cycle', 'chemo36_day', 'chemo36_med', 'chemo36_notes', 'chemo37_date', 'chemo37_cycle', 'chemo37_day', 'chemo37_med', 'chemo37_notes', 'chemo38_date', 'chemo38_cycle', 'chemo38_day', 'chemo38_med', 'chemo38_notes', 'chemo39_date', 'chemo39_cycle', 'chemo39_day', 'chemo39_med', 'chemo39_notes', 'chemo40_date', 'chemo40_cycle', 'chemo40_day', 'chemo40_med', 'chemo40_notes', 'chemo41_date', 'chemo41_cycle', 'chemo41_day', 'chemo41_med', 'chemo41_notes', 'chemo42_date', 'chemo42_cycle', 'chemo42_day', 'chemo42_med', 'chemo42_notes', 'chemo43_date', 'chemo43_cycle', 'chemo43_day', 'chemo43_med', 'chemo43_notes', 'chemo44_date', 'chemo44_cycle', 'chemo44_day', 'chemo44_med', 'chemo44_notes', 'chemo45_date', 'chemo45_cycle', 'chemo45_day', 'chemo45_med', 'chemo45_notes', 'hosp1_admit_date', 'hosp1_discharge_date', 'hosp1_reason', 'hosp1_notes', 'hosp2_admit_date', 'hosp2_discharge_date', 'hosp2_reason', 'hosp2_notes', 'hosp3_admit_date', 'hosp3_discharge_date', 'hosp3_reason', 'hosp3_notes', 'surgery', 'ed') Feb 22 09:53:14 redcap-dbserver-prod-02 kernel: mariadbd[24765]: segfault at 0 ip 000055d160f64c4f sp 00007fbb4d7f8430 error 4 in mariadbd[55d1607a0000+1613000] Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: and (value like '%1%') and event_id = 36032 Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: order by abs(value), value Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: Connection ID (thread ID): 473355 Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: Status: KILL_CONNECTION Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=on,condition_pushdown_from_having=on,not_null_range_scan=off Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: information that should help you find out what is causing the crash. Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: Writing a core file... Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: Working directory at /data/mysql Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: Resource Limits: Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: Limit Soft Limit Hard Limit Units Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: Max cpu time unlimited unlimited seconds Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: Max file size unlimited unlimited bytes Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: Max data size unlimited unlimited bytes Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: Max stack size 8388608 unlimited bytes Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: Max core file size 0 unlimited bytes Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: Max resident set unlimited unlimited bytes Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: Max processes 95700 95700 processes Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: Max open files 32768 32768 files Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: Max locked memory 65536 65536 bytes Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: Max address space unlimited unlimited bytes Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: Max file locks unlimited unlimited locks Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: Max pending signals 95700 95700 signals Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: Max msgqueue size 819200 819200 bytes Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: Max nice priority 0 0 Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: Max realtime priority 0 0 Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: Max realtime timeout unlimited unlimited us Feb 22 09:53:14 redcap-dbserver-prod-02 mariadbd: Core pattern: core Feb 22 09:53:15 redcap-dbserver-prod-02 systemd: mariadb.service: main process exited, code=killed, status=11/SEGV Feb 22 09:53:16 redcap-dbserver-prod-02 systemd: Unit mariadb.service entered failed state. Feb 22 09:53:16 redcap-dbserver-prod-02 systemd: mariadb.service failed. Feb 22 10:43:58 redcap-dbserver-prod-02 mariadbd: 220222 10:43:58 [ERROR] mysqld got signal 11 ; Feb 22 10:43:58 redcap-dbserver-prod-02 mariadbd: This could be because you hit a bug. It is also possible that this binary Feb 22 10:43:58 redcap-dbserver-prod-02 mariadbd: or one of the libraries it was linked against is corrupt, improperly built, Feb 22 10:43:58 redcap-dbserver-prod-02 mariadbd: or misconfigured. This error can also be caused by malfunctioning hardware. Feb 22 10:43:58 redcap-dbserver-prod-02 mariadbd: To report this bug, see https://mariadb.com/kb/en/reporting-bugs Feb 22 10:43:58 redcap-dbserver-prod-02 mariadbd: We will try our best to scrape up some info that will hopefully help Feb 22 10:43:58 redcap-dbserver-prod-02 mariadbd: diagnose the problem, but since we have already crashed, Feb 22 10:43:58 redcap-dbserver-prod-02 mariadbd: something is definitely wrong and this may fail. Feb 22 10:43:58 redcap-dbserver-prod-02 mariadbd: Server version: 10.5.13-MariaDB Feb 22 10:43:58 redcap-dbserver-prod-02 mariadbd: key_buffer_size=134217728 Feb 22 10:43:58 redcap-dbserver-prod-02 mariadbd: read_buffer_size=131072 Feb 22 10:43:58 redcap-dbserver-prod-02 mariadbd: max_used_connections=14 Feb 22 10:43:58 redcap-dbserver-prod-02 mariadbd: max_threads=302 Feb 22 10:43:58 redcap-dbserver-prod-02 mariadbd: thread_count=14 Feb 22 10:43:58 redcap-dbserver-prod-02 mariadbd: It is possible that mysqld could use up to Feb 22 10:43:58 redcap-dbserver-prod-02 mariadbd: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 795892 K bytes of memory Feb 22 10:43:58 redcap-dbserver-prod-02 mariadbd: Hope that's ok; if not, decrease some variables in the equation. Feb 22 10:43:58 redcap-dbserver-prod-02 mariadbd: Thread pointer: 0x7f5b040009b8 Feb 22 10:43:58 redcap-dbserver-prod-02 mariadbd: Attempting backtrace. You can use the following information to find out Feb 22 10:43:58 redcap-dbserver-prod-02 mariadbd: where mysqld died. If you see no messages after this, something went Feb 22 10:43:58 redcap-dbserver-prod-02 mariadbd: terribly wrong... Feb 22 10:43:58 redcap-dbserver-prod-02 mariadbd: stack_bottom = 0x7f5de4084c90 thread_stack 0x49000 Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: ??:0(my_print_stacktrace)[0x563c8c8b1d6e] Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: ??:0(handle_fatal_signal)[0x563c8c2b4367] Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: sigaction.c:0(__restore_rt)[0x7f5e033ad630] Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: ??:0(st_join_table::save_explain_data(Explain_table_access*, unsigned long long, bool, st_join_table*))[0x563c8c0f5c4f] Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: ??:0(JOIN::save_explain_data_intern(Explain_query*, bool, bool, bool, char const*))[0x563c8c0f778b] Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: ??:0(JOIN::save_explain_data(Explain_query*, bool, bool, bool, bool))[0x563c8c0f78f0] Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: ??:0(JOIN::build_explain())[0x563c8c0f7b4c] Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: ??:0(JOIN::optimize())[0x563c8c10d0b1] Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: ??:0(mysql_select(THD*, TABLE_LIST*, List&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*))[0x563c8c10d1c7] Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: ??:0(handle_select(THD*, LEX*, select_result*, unsigned long))[0x563c8c10de34] Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: /usr/sbin/mariadbd(+0x63b749)[0x563c8bf6c749] Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: ??:0(mysql_execute_command(THD*))[0x563c8c0b187d] Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: ??:0(mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool))[0x563c8c0b3d15] Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: ??:0(dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool))[0x563c8c0b65e5] Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: ??:0(do_command(THD*))[0x563c8c0b7d7f] Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: ??:0(do_handle_one_connection(CONNECT*, bool))[0x563c8c1a4662] Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: ??:0(handle_one_connection)[0x563c8c1a4934] Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: ??:0(MyCTX_nopad::finish(unsigned char*, unsigned int*))[0x563c8c50391d] Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: pthread_create.c:0(start_thread)[0x7f5e033a5ea5] Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: ??:0(__clone)[0x7f5e028c0b0d] Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: Trying to get some variables. Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: Some pointers may be invalid and cause the dump to abort. Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: Query (0x7f5b040305d0): select distinct event_id, field_name, value, instance, record Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: from redcap_data where project_id = 7827 and field_name in ('record_id', 'notes', 'email', 'date_consent', 'consent_person', 'recruit_md', 'date_start', 'date_45d', 'date_end', 'vis_id', 'fitbit_id', 'date_fitbit_mailed', 'fitbit_mail_track', 'date_fitbit_delivered', 'date_fitbit_data_start', 'return_fitbit_date', 'payment_comments', 'payment_date', 'vincent_number', 'date_payment_card_confirmed', 'date_payment_card_loaded', 'payment_amount', 'payment_partial_explain', 'return_phone_date', 'android_version', 'iphone_version', 'aware_android_version', 'aware_iphone_version', 'uuid', 'smartphone_storage_total', 'smartphone_storage_free', 'fitbit_email', 'fitbit_password', 'fitbit_firmware', 'fitbit_android_version', 'fitbit_iphone_version', 'uuid_end_date', 'uuid2', 'uuid2_start_date', 'fitbit2_email', 'flag_date', 'aware_lastsync', 'fitbit_lastsync', 'daily_lastsync', 'weekly_lastsync', 'flag_notes', 'flag_troubleshoot', 'com_notes', 'data_notes', 'opioid1_other_spec_45d', 'notes_45d', 'notes_90d', 'name', 'phone_number', 'number_type_other', 'phone_number_2', 'number_type_other_2', 'contact_when', 'address_street', 'address_city', 'address_state', 'address_zip', 'demo_datetime', 'dob', 'age', 'height_ft', 'height_inch', 'height', 'weight', 'bmi', 'race_other', 'race_multiple', 'living_other', 'employment_other', 'smoker_years', 'smoker_cigarettesdaily', 'wifi_note', 'own_comp_note', 'own_tablet_note', 'own_smartphone_model', 'own_smartphone_note', 'access_internet_note', 'social_media_note', 'own_tracker_note', 'notes_bl', 'cancer_type', 'cancer_dx_date', 'cancer_stage', 'comorbid_dx', 'chemo_start', 'emr_height_in', 'emr_weight_lb', 'date_death', 'chemo1_date', 'chemo1_cycle', 'chemo1_day', 'chemo1_med', 'chemo1_notes', 'chemo2_date', 'chemo2_cycle', 'chemo2_day', 'chemo2_med', 'chemo2_notes', 'chemo3_date', 'chemo3_cycle', 'chemo3_day', 'chemo3_med', 'chemo3_notes', 'chemo4_date', 'chemo4_cycle', 'chemo4_day', 'chemo4_med', 'chemo4_notes', 'chemo5_date', 'chemo5_cycle', 'chemo5_day', 'chemo5_med', 'chemo5_notes', 'chemo6_date', 'chemo6_cycle', 'chemo6_day', 'chemo6_med', 'chemo6_notes', 'chemo7_date', 'chemo7_cycle', 'chemo7_day', 'chemo7_med', 'chemo7_notes', 'chemo8_date', 'chemo8_cycle', 'chemo8_day', 'chemo8_med', 'chemo8_notes', 'chemo9_date', 'chemo9_cycle', 'chemo9_day', 'chemo9_med', 'chemo9_notes', 'chemo10_date', 'chemo10_cycle', 'chemo10_day', 'chemo10_med', 'chemo10_notes', 'chemo11_date', 'chemo11_cycle', 'chemo11_day', 'chemo11_med', 'chemo11_notes', 'chemo12_date', 'chemo12_cycle', 'chemo12_day', 'chemo12_med', 'chemo12_notes', 'chemo13_date', 'chemo13_cycle', 'chemo13_day', 'chemo13_med', 'chemo13_notes', 'chemo14_date', 'chemo14_cycle', 'chemo14_day', 'chemo14_med', 'chemo14_notes', 'chemo15_date', 'chemo15_cycle', 'chemo15_day', 'chemo15_med', 'chemo15_notes', 'chemo16_date', 'chemo16_cycle', 'chemo16_day', 'chemo16_med', 'chemo16_notes', 'chemo17_date', 'chemo17_cycle', 'chemo17_day', 'chemo17_med', 'chemo17_notes', 'chemo18_date', 'chemo18_cycle', 'chemo18_day', 'chemo18_med', 'chemo18_notes', 'chemo19_date', 'chemo19_cycle', 'chemo19_day', 'chemo19_med', 'chemo19_notes', 'chemo20_date', 'chemo20_cycle', 'chemo20_day', 'chemo20_med', 'chemo20_notes', 'chemo21_date', 'chemo21_cycle', 'chemo21_day', 'chemo21_med', 'chemo21_notes', 'chemo22_date', 'chemo22_cycle', 'chemo22_day', 'chemo22_med', 'chemo22_notes', 'chemo23_date', 'chemo23_cycle', 'chemo23_day', 'chemo23_med', 'chemo23_notes', 'chemo24_date', 'chemo24_cycle', 'chemo24_day', 'chemo24_med', 'chemo24_notes', 'chemo25_date', 'chemo25_cycle', 'chemo25_day', 'chemo25_med', 'chemo25_notes', 'chemo26_date', 'chemo26_cycle', 'chemo26_day', 'chemo26_med', 'chemo26_notes', 'chemo27_date', 'chemo27_cycle', 'chemo27_day', 'chemo27_med', 'chemo27_notes', 'chemo28_date', 'chemo28_cycle', 'chemo28_day', 'chemo28_med', 'chemo28_notes', 'chemo29_date', 'chemo29_cycle', 'chemo29_day', 'chemo29_med', 'chemo29_notes', 'chemo30_date', 'chemo30_cycle', 'chemo30_day', 'chemo30_med', 'chemo30_notes', 'chemo31_date', 'chemo31_cycle', 'chemo31_day', 'chemo31_med', 'chemo31_notes', 'chemo32_date', 'chemo32_cycle', 'chemo32_day', 'chemo32_med', 'chemo32_notes', 'chemo33_date', 'chemo33_cycle', 'chemo33_day', 'chemo33_med', 'chemo33_notes', 'chemo34_date', 'chemo34_cycle', 'chemo34_day', 'chemo34_med', 'chemo34_notes', 'chemo35_date', 'chemo35_cycle', 'chemo35_day', 'chemo35_med', 'chemo35_notes', 'chemo36_date', 'chemo36_cycle', 'chemo36_day', 'chemo36_med', 'chemo36_notes', 'chemo37_date', 'chemo37_cycle', 'chemo37_day', 'chemo37_med', 'chemo37_notes', 'chemo38_date', 'chemo38_cycle', 'chemo38_day', 'chemo38_med', 'chemo38_notes', 'chemo39_date', 'chemo39_cycle', 'chemo39_day', 'chemo39_med', 'chemo39_notes', 'chemo40_date', 'chemo40_cycle', 'chemo40_day', 'chemo40_med', 'chemo40_notes', 'chemo41_date', 'chemo41_cycle', 'chemo41_day', 'chemo41_med', 'chemo41_notes', 'chemo42_date', 'chemo42_cycle', 'chemo42_day', 'chemo42_med', 'chemo42_notes', 'chemo43_date', 'chemo43_cycle', 'chemo43_day', 'chemo43_med', 'chemo43_notes', 'chemo44_date', 'chemo44_cycle', 'chemo44_day', 'chemo44_med', 'chemo44_notes', 'chemo45_date', 'chemo45_cycle', 'chemo45_day', 'chemo45_med', 'chemo45_notes', 'hosp1_admit_date', 'hosp1_discharge_date', 'hosp1_reason', 'hosp1_notes', 'hosp2_admit_date', 'hosp2_discharge_date', 'hosp2_reason', 'hosp2_notes', 'hosp3_admit_date', 'hosp3_discharge_date', 'hosp3_reason', 'hosp3_notes', 'surgery', 'ed') Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: and (value like '%1%') and event_id = 36032 Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: order by abs(value), value Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: Connection ID (thread ID): 12519 Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: Status: KILL_CONNECTION Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=on,condition_pushdown_from_having=on,not_null_range_scan=off Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: information that should help you find out what is causing the crash. Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: Writing a core file... Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: Working directory at /data/mysql Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: Resource Limits: Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: Limit Soft Limit Hard Limit Units Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: Max cpu time unlimited unlimited seconds Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: Max file size unlimited unlimited bytes Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: Max data size unlimited unlimited bytes Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: Max stack size 8388608 unlimited bytes Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: Max core file size 0 unlimited bytes Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: Max resident set unlimited unlimited bytes Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: Max processes 95700 95700 processes Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: Max open files 32768 32768 files Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: Max locked memory 65536 65536 bytes Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: Max address space unlimited unlimited bytes Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: Max file locks unlimited unlimited locks Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: Max pending signals 95700 95700 signals Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: Max msgqueue size 819200 819200 bytes Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: Max nice priority 0 0 Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: Max realtime priority 0 0 Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: Max realtime timeout unlimited unlimited us Feb 22 10:43:59 redcap-dbserver-prod-02 mariadbd: Core pattern: core Feb 22 10:44:00 redcap-dbserver-prod-02 systemd: mariadb.service: main process exited, code=killed, status=11/SEGV Feb 22 10:44:01 redcap-dbserver-prod-02 systemd: Unit mariadb.service entered failed state. Feb 22 10:44:01 redcap-dbserver-prod-02 systemd: mariadb.service failed. messages-20220220 Feb 14 10:15:33 redcap-dbserver-prod-02 mariadbd: 220214 10:15:33 [ERROR] mysqld got signal 11 ; Feb 14 10:15:33 redcap-dbserver-prod-02 mariadbd: This could be because you hit a bug. It is also possible that this binary Feb 14 10:15:33 redcap-dbserver-prod-02 mariadbd: or one of the libraries it was linked against is corrupt, improperly built, Feb 14 10:15:33 redcap-dbserver-prod-02 mariadbd: or misconfigured. This error can also be caused by malfunctioning hardware. Feb 14 10:15:33 redcap-dbserver-prod-02 mariadbd: To report this bug, see https://mariadb.com/kb/en/reporting-bugs Feb 14 10:15:33 redcap-dbserver-prod-02 mariadbd: We will try our best to scrape up some info that will hopefully help Feb 14 10:15:33 redcap-dbserver-prod-02 mariadbd: diagnose the problem, but since we have already crashed, Feb 14 10:15:33 redcap-dbserver-prod-02 mariadbd: something is definitely wrong and this may fail. Feb 14 10:15:33 redcap-dbserver-prod-02 mariadbd: Server version: 10.5.13-MariaDB Feb 14 10:15:33 redcap-dbserver-prod-02 mariadbd: key_buffer_size=134217728 Feb 14 10:15:33 redcap-dbserver-prod-02 mariadbd: read_buffer_size=131072 Feb 14 10:15:33 redcap-dbserver-prod-02 mariadbd: max_used_connections=57 Feb 14 10:15:33 redcap-dbserver-prod-02 mariadbd: max_threads=252 Feb 14 10:15:33 redcap-dbserver-prod-02 mariadbd: thread_count=13 Feb 14 10:15:33 redcap-dbserver-prod-02 mariadbd: It is possible that mysqld could use up to Feb 14 10:15:33 redcap-dbserver-prod-02 mariadbd: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 685818 K bytes of memory Feb 14 10:15:33 redcap-dbserver-prod-02 mariadbd: Hope that's ok; if not, decrease some variables in the equation. Feb 14 10:15:33 redcap-dbserver-prod-02 mariadbd: Thread pointer: 0x7f8244035b58 Feb 14 10:15:33 redcap-dbserver-prod-02 mariadbd: Attempting backtrace. You can use the following information to find out Feb 14 10:15:33 redcap-dbserver-prod-02 mariadbd: where mysqld died. If you see no messages after this, something went Feb 14 10:15:33 redcap-dbserver-prod-02 mariadbd: terribly wrong... Feb 14 10:15:33 redcap-dbserver-prod-02 mariadbd: stack_bottom = 0x7f825ef68c90 thread_stack 0x49000 Feb 14 10:15:34 redcap-dbserver-prod-02 mariadbd: ??:0(my_print_stacktrace)[0x559a7f86bd6e] Feb 14 10:15:34 redcap-dbserver-prod-02 mariadbd: ??:0(handle_fatal_signal)[0x559a7f26e367] Feb 14 10:15:34 redcap-dbserver-prod-02 mariadbd: sigaction.c:0(__restore_rt)[0x7f8504c04630] Feb 14 10:15:34 redcap-dbserver-prod-02 mariadbd: ??:0(st_join_table::save_explain_data(Explain_table_access*, unsigned long long, bool, st_join_table*))[0x559a7f0afc4f] Feb 14 10:15:34 redcap-dbserver-prod-02 mariadbd: ??:0(JOIN::save_explain_data_intern(Explain_query*, bool, bool, bool, char const*))[0x559a7f0b178b] Feb 14 10:15:34 redcap-dbserver-prod-02 mariadbd: ??:0(JOIN::save_explain_data(Explain_query*, bool, bool, bool, bool))[0x559a7f0b18f0] Feb 14 10:15:34 redcap-dbserver-prod-02 mariadbd: ??:0(JOIN::build_explain())[0x559a7f0b1b4c] Feb 14 10:15:34 redcap-dbserver-prod-02 mariadbd: ??:0(JOIN::optimize())[0x559a7f0c70b1] Feb 14 10:15:34 redcap-dbserver-prod-02 mariadbd: ??:0(mysql_select(THD*, TABLE_LIST*, List&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*))[0x559a7f0c71c7] Feb 14 10:15:34 redcap-dbserver-prod-02 mariadbd: ??:0(handle_select(THD*, LEX*, select_result*, unsigned long))[0x559a7f0c7e34] Feb 14 10:15:34 redcap-dbserver-prod-02 mariadbd: /usr/sbin/mariadbd(+0x63b749)[0x559a7ef26749] Feb 14 10:15:34 redcap-dbserver-prod-02 mariadbd: ??:0(mysql_execute_command(THD*))[0x559a7f06b87d] Feb 14 10:15:34 redcap-dbserver-prod-02 mariadbd: ??:0(mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool))[0x559a7f06dd15] Feb 14 10:15:34 redcap-dbserver-prod-02 mariadbd: ??:0(dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool))[0x559a7f0705e5] Feb 14 10:15:34 redcap-dbserver-prod-02 mariadbd: ??:0(do_command(THD*))[0x559a7f071d7f] Feb 14 10:15:34 redcap-dbserver-prod-02 mariadbd: ??:0(do_handle_one_connection(CONNECT*, bool))[0x559a7f15e662] Feb 14 10:15:34 redcap-dbserver-prod-02 mariadbd: ??:0(handle_one_connection)[0x559a7f15e934] Feb 14 10:15:34 redcap-dbserver-prod-02 mariadbd: ??:0(MyCTX_nopad::finish(unsigned char*, unsigned int*))[0x559a7f4bd91d] Feb 14 10:15:35 redcap-dbserver-prod-02 mariadbd: pthread_create.c:0(start_thread)[0x7f8504bfcea5] Feb 14 10:15:35 redcap-dbserver-prod-02 mariadbd: ??:0(__clone)[0x7f8504117b0d] Feb 14 10:15:35 redcap-dbserver-prod-02 mariadbd: Trying to get some variables. Feb 14 10:15:35 redcap-dbserver-prod-02 mariadbd: Some pointers may be invalid and cause the dump to abort. Feb 14 10:15:35 redcap-dbserver-prod-02 mariadbd: Query (0x7f8244098fb0): select distinct event_id, field_name, value, instance, record Feb 14 10:15:35 redcap-dbserver-prod-02 mariadbd: from redcap_data where project_id = 7827 and field_name in ('record_id', 'notes', 'email', 'date_consent', 'consent_person', 'recruit_md', 'date_start', 'date_45d', 'date_end', 'vis_id', 'fitbit_id', 'date_fitbit_mailed', 'fitbit_mail_track', 'date_fitbit_delivered', 'date_fitbit_data_start', 'return_fitbit_date', 'payment_comments', 'payment_date', 'vincent_number', 'date_payment_card_confirmed', 'date_payment_card_loaded', 'payment_amount', 'payment_partial_explain', 'return_phone_date', 'android_version', 'iphone_version', 'aware_android_version', 'aware_iphone_version', 'uuid', 'smartphone_storage_total', 'smartphone_storage_free', 'fitbit_email', 'fitbit_password', 'fitbit_firmware', 'fitbit_android_version', 'fitbit_iphone_version', 'uuid_end_date', 'uuid2', 'uuid2_start_date', 'fitbit2_email', 'flag_date', 'aware_lastsync', 'fitbit_lastsync', 'daily_lastsync', 'weekly_lastsync', 'flag_notes', 'flag_troubleshoot', 'com_notes', 'data_notes', 'opioid1_other_spec_45d', 'notes_45d', 'notes_90d', 'name', 'phone_number', 'number_type_other', 'phone_number_2', 'number_type_other_2', 'contact_when', 'address_street', 'address_city', 'address_state', 'address_zip', 'demo_datetime', 'dob', 'age', 'height_ft', 'height_inch', 'height', 'weight', 'bmi', 'race_other', 'race_multiple', 'living_other', 'employment_other', 'smoker_years', 'smoker_cigarettesdaily', 'wifi_note', 'own_comp_note', 'own_tablet_note', 'own_smartphone_model', 'own_smartphone_note', 'access_internet_note', 'social_media_note', 'own_tracker_note', 'notes_bl', 'cancer_type', 'cancer_dx_date', 'cancer_stage', 'comorbid_dx', 'chemo_start', 'emr_height_in', 'emr_weight_lb', 'date_death', 'chemo1_date', 'chemo1_cycle', 'chemo1_day', 'chemo1_med', 'chemo1_notes', 'chemo2_date', 'chemo2_cycle', 'chemo2_day', 'chemo2_med', 'chemo2_notes', 'chemo3_date', 'chemo3_cycle', 'chemo3_day', 'chemo3_med', 'chemo3_notes', 'chemo4_date', 'chemo4_cycle', 'chemo4_day', 'chemo4_med', 'chemo4_notes', 'chemo5_date', 'chemo5_cycle', 'chemo5_day', 'chemo5_med', 'chemo5_notes', 'chemo6_date', 'chemo6_cycle', 'chemo6_day', 'chemo6_med', 'chemo6_notes', 'chemo7_date', 'chemo7_cycle', 'chemo7_day', 'chemo7_med', 'chemo7_notes', 'chemo8_date', 'chemo8_cycle', 'chemo8_day', 'chemo8_med', 'chemo8_notes', 'chemo9_date', 'chemo9_cycle', 'chemo9_day', 'chemo9_med', 'chemo9_notes', 'chemo10_date', 'chemo10_cycle', 'chemo10_day', 'chemo10_med', 'chemo10_notes', 'chemo11_date', 'chemo11_cycle', 'chemo11_day', 'chemo11_med', 'chemo11_notes', 'chemo12_date', 'chemo12_cycle', 'chemo12_day', 'chemo12_med', 'chemo12_notes', 'chemo13_date', 'chemo13_cycle', 'chemo13_day', 'chemo13_med', 'chemo13_notes', 'chemo14_date', 'chemo14_cycle', 'chemo14_day', 'chemo14_med', 'chemo14_notes', 'chemo15_date', 'chemo15_cycle', 'chemo15_day', 'chemo15_med', 'chemo15_notes', 'chemo16_date', 'chemo16_cycle', 'chemo16_day', 'chemo16_med', 'chemo16_notes', 'chemo17_date', 'chemo17_cycle', 'chemo17_day', 'chemo17_med', 'chemo17_notes', 'chemo18_date', 'chemo18_cycle', 'chemo18_day', 'chemo18_med', 'chemo18_notes', 'chemo19_date', 'chemo19_cycle', 'chemo19_day', 'chemo19_med', 'chemo19_notes', 'chemo20_date', 'chemo20_cycle', 'chemo20_day', 'chemo20_med', 'chemo20_notes', 'chemo21_date', 'chemo21_cycle', 'chemo21_day', 'chemo21_med', 'chemo21_notes', 'chemo22_date', 'chemo22_cycle', 'chemo22_day', 'chemo22_med', 'chemo22_notes', 'chemo23_date', 'chemo23_cycle', 'chemo23_day', 'chemo23_med', 'chemo23_notes', 'chemo24_date', 'chemo24_cycle', 'chemo24_day', 'chemo24_med', 'chemo24_notes', 'chemo25_date', 'chemo25_cycle', 'chemo25_day', 'chemo25_med', 'chemo25_notes', 'chemo26_date', 'chemo26_cycle', 'chemo26_day', 'chemo26_med', 'chemo26_notes', 'chemo27_date', 'chemo27_cycle', 'chemo27_day', 'chemo27_med', 'chemo27_notes', 'chemo28_date', 'chemo28_cycle', 'chemo28_day', 'chemo28_med', 'chemo28_notes', 'chemo29_date', 'chemo29_cycle', 'chemo29_day', 'chemo29_med', 'chemo29_notes', 'chemo30_date', 'chemo30_cycle', 'chemo30_day', 'chemo30_med', 'chemo30_notes', 'chemo31_date', 'chemo31_cycle', 'chemo31_day', 'chemo31_med', 'chemo31_notes', 'chemo32_date', 'chemo32_cycle', 'chemo32_day', 'chemo32_med', 'chemo32_notes', 'chemo33_date', 'chemo33_cycle', 'chemo33_day', 'chemo33_med', 'chemo33_notes', 'chemo34_date', 'chemo34_cycle', 'chemo34_day', 'chemo34_med', 'chemo34_notes', 'chemo35_date', 'chemo35_cycle', 'chemo35_day', 'chemo35_med', 'chemo35_notes', 'chemo36_date', 'chemo36_cycle', 'chemo36_day', 'chemo36_med', 'chemo36_notes', 'chemo37_date', 'chemo37_cycle', 'chemo37_day', 'chemo37_med', 'chemo37_notes', 'chemo38_date', 'chemo38_cycle', 'chemo38_day', 'chemo38_med', 'chemo38_notes', 'chemo39_date', 'chemo39_cycle', 'chemo39_day', 'chemo39_med', 'chemo39_notes', 'chemo40_date', 'chemo40_cycle', 'chemo40_day', 'chemo40_med', 'chemo40_notes', 'chemo41_date', 'chemo41_cycle', 'chemo41_day', 'chemo41_med', 'chemo41_notes', 'chemo42_date', 'chemo42_cycle', 'chemo42_day', 'chemo42_med', 'chemo42_notes', 'chemo43_date', 'chemo43_cycle', 'chemo43_day', 'chemo43_med', 'chemo43_notes', 'chemo44_date', 'chemo44_cycle', 'chemo44_day', 'chemo44_med', 'chemo44_notes', 'chemo45_date', 'chemo45_cycle', 'chemo45_day', 'chemo45_med', 'chemo45_notes', 'hosp1_admit_date', 'hosp1_discharge_date', 'hosp1_reason', 'hosp1_notes', 'hosp2_admit_date', 'hosp2_discharge_date', 'hosp2_reason', 'hosp2_notes', 'hosp3_admit_date', 'hosp3_discharge_date', 'hosp3_reason', 'hosp3_notes', 'surgery', 'ed') Feb 14 10:15:35 redcap-dbserver-prod-02 mariadbd: and (value like '%zebri%') and event_id = 36032 Feb 14 10:15:35 redcap-dbserver-prod-02 mariadbd: order by abs(value), value Feb 14 10:15:35 redcap-dbserver-prod-02 mariadbd: Connection ID (thread ID): 1208141 Feb 14 10:15:35 redcap-dbserver-prod-02 mariadbd: Status: KILL_CONNECTION Feb 14 10:15:35 redcap-dbserver-prod-02 mariadbd: Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=on,condition_pushdown_from_having=on,not_null_range_scan=off Feb 14 10:15:35 redcap-dbserver-prod-02 mariadbd: The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains Feb 14 10:15:35 redcap-dbserver-prod-02 mariadbd: information that should help you find out what is causing the crash. Feb 14 10:15:35 redcap-dbserver-prod-02 mariadbd: Writing a core file... Feb 14 10:15:35 redcap-dbserver-prod-02 mariadbd: Working directory at /data/mysql Feb 14 10:15:35 redcap-dbserver-prod-02 mariadbd: Resource Limits: Feb 14 10:15:35 redcap-dbserver-prod-02 mariadbd: Limit Soft Limit Hard Limit Units Feb 14 10:15:35 redcap-dbserver-prod-02 mariadbd: Max cpu time unlimited unlimited seconds Feb 14 10:15:35 redcap-dbserver-prod-02 mariadbd: Max file size unlimited unlimited bytes Feb 14 10:15:35 redcap-dbserver-prod-02 mariadbd: Max data size unlimited unlimited bytes Feb 14 10:15:35 redcap-dbserver-prod-02 mariadbd: Max stack size 8388608 unlimited bytes Feb 14 10:15:35 redcap-dbserver-prod-02 mariadbd: Max core file size 0 unlimited bytes Feb 14 10:15:35 redcap-dbserver-prod-02 mariadbd: Max resident set unlimited unlimited bytes Feb 14 10:15:35 redcap-dbserver-prod-02 mariadbd: Max processes 95700 95700 processes Feb 14 10:15:35 redcap-dbserver-prod-02 mariadbd: Max open files 32768 32768 files Feb 14 10:15:35 redcap-dbserver-prod-02 mariadbd: Max locked memory 65536 65536 bytes Feb 14 10:15:35 redcap-dbserver-prod-02 mariadbd: Max address space unlimited unlimited bytes Feb 14 10:15:35 redcap-dbserver-prod-02 mariadbd: Max file locks unlimited unlimited locks Feb 14 10:15:35 redcap-dbserver-prod-02 mariadbd: Max pending signals 95700 95700 signals Feb 14 10:15:35 redcap-dbserver-prod-02 mariadbd: Max msgqueue size 819200 819200 bytes Feb 14 10:15:35 redcap-dbserver-prod-02 mariadbd: Max nice priority 0 0 Feb 14 10:15:35 redcap-dbserver-prod-02 mariadbd: Max realtime priority 0 0 Feb 14 10:15:35 redcap-dbserver-prod-02 mariadbd: Max realtime timeout unlimited unlimited us Feb 14 10:15:35 redcap-dbserver-prod-02 mariadbd: Core pattern: core Feb 14 10:15:35 redcap-dbserver-prod-02 kernel: mariadbd[4257]: segfault at 0 ip 0000559a7f0afc4f sp 00007f825ef67430 error 4 in mariadbd[559a7e8eb000+1613000] Feb 14 10:15:36 redcap-dbserver-prod-02 systemd: mariadb.service: main process exited, code=killed, status=11/SEGV Feb 14 10:15:37 redcap-dbserver-prod-02 systemd: Unit mariadb.service entered failed state. Feb 14 10:15:37 redcap-dbserver-prod-02 systemd: mariadb.service failed. Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: 220214 10:17:00 [ERROR] mysqld got signal 11 ; Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: This could be because you hit a bug. It is also possible that this binary Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: or one of the libraries it was linked against is corrupt, improperly built, Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: or misconfigured. This error can also be caused by malfunctioning hardware. Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: To report this bug, see https://mariadb.com/kb/en/reporting-bugs Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: We will try our best to scrape up some info that will hopefully help Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: diagnose the problem, but since we have already crashed, Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: something is definitely wrong and this may fail. Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: Server version: 10.5.13-MariaDB Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: key_buffer_size=134217728 Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: read_buffer_size=131072 Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: max_used_connections=9 Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: max_threads=252 Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: thread_count=9 Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: It is possible that mysqld could use up to Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 685818 K bytes of memory Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: Hope that's ok; if not, decrease some variables in the equation. Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: Thread pointer: 0x7f7f400009b8 Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: Attempting backtrace. You can use the following information to find out Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: where mysqld died. If you see no messages after this, something went Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: terribly wrong... Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: stack_bottom = 0x7f7fb0926c90 thread_stack 0x49000 Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: ??:0(my_print_stacktrace)[0x55799b21ad6e] Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: ??:0(handle_fatal_signal)[0x55799ac1d367] Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: sigaction.c:0(__restore_rt)[0x7f825651c630] Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: ??:0(st_join_table::save_explain_data(Explain_table_access*, unsigned long long, bool, st_join_table*))[0x55799aa5ec4f] Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: ??:0(JOIN::save_explain_data_intern(Explain_query*, bool, bool, bool, char const*))[0x55799aa6078b] Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: ??:0(JOIN::save_explain_data(Explain_query*, bool, bool, bool, bool))[0x55799aa608f0] Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: ??:0(JOIN::build_explain())[0x55799aa60b4c] Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: ??:0(JOIN::optimize())[0x55799aa760b1] Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: ??:0(mysql_select(THD*, TABLE_LIST*, List&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*))[0x55799aa761c7] Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: ??:0(handle_select(THD*, LEX*, select_result*, unsigned long))[0x55799aa76e34] Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: /usr/sbin/mariadbd(+0x63b749)[0x55799a8d5749] Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: ??:0(mysql_execute_command(THD*))[0x55799aa1a87d] Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: ??:0(mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool))[0x55799aa1cd15] Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: ??:0(dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool))[0x55799aa1f5e5] Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: ??:0(do_command(THD*))[0x55799aa20d7f] Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: ??:0(do_handle_one_connection(CONNECT*, bool))[0x55799ab0d662] Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: ??:0(handle_one_connection)[0x55799ab0d934] Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: ??:0(MyCTX_nopad::finish(unsigned char*, unsigned int*))[0x55799ae6c91d] Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: pthread_create.c:0(start_thread)[0x7f8256514ea5] Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: ??:0(__clone)[0x7f8255a2fb0d] Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: Trying to get some variables. Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: Some pointers may be invalid and cause the dump to abort. Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: Query (0x7f7f40010510): select distinct event_id, field_name, value, instance, record Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: from redcap_data where project_id = 7827 and field_name in ('record_id', 'notes', 'email', 'date_consent', 'consent_person', 'recruit_md', 'date_start', 'date_45d', 'date_end', 'vis_id', 'fitbit_id', 'date_fitbit_mailed', 'fitbit_mail_track', 'date_fitbit_delivered', 'date_fitbit_data_start', 'return_fitbit_date', 'payment_comments', 'payment_date', 'vincent_number', 'date_payment_card_confirmed', 'date_payment_card_loaded', 'payment_amount', 'payment_partial_explain', 'return_phone_date', 'android_version', 'iphone_version', 'aware_android_version', 'aware_iphone_version', 'uuid', 'smartphone_storage_total', 'smartphone_storage_free', 'fitbit_email', 'fitbit_password', 'fitbit_firmware', 'fitbit_android_version', 'fitbit_iphone_version', 'uuid_end_date', 'uuid2', 'uuid2_start_date', 'fitbit2_email', 'flag_date', 'aware_lastsync', 'fitbit_lastsync', 'daily_lastsync', 'weekly_lastsync', 'flag_notes', 'flag_troubleshoot', 'com_notes', 'data_notes', 'opioid1_other_spec_45d', 'notes_45d', 'notes_90d', 'name', 'phone_number', 'number_type_other', 'phone_number_2', 'number_type_other_2', 'contact_when', 'address_street', 'address_city', 'address_state', 'address_zip', 'demo_datetime', 'dob', 'age', 'height_ft', 'height_inch', 'height', 'weight', 'bmi', 'race_other', 'race_multiple', 'living_other', 'employment_other', 'smoker_years', 'smoker_cigarettesdaily', 'wifi_note', 'own_comp_note', 'own_tablet_note', 'own_smartphone_model', 'own_smartphone_note', 'access_internet_note', 'social_media_note', 'own_tracker_note', 'notes_bl', 'cancer_type', 'cancer_dx_date', 'cancer_stage', 'comorbid_dx', 'chemo_start', 'emr_height_in', 'emr_weight_lb', 'date_death', 'chemo1_date', 'chemo1_cycle', 'chemo1_day', 'chemo1_med', 'chemo1_notes', 'chemo2_date', 'chemo2_cycle', 'chemo2_day', 'chemo2_med', 'chemo2_notes', 'chemo3_date', 'chemo3_cycle', 'chemo3_day', 'chemo3_med', 'chemo3_notes', 'chemo4_date', 'chemo4_cycle', 'chemo4_day', 'chemo4_med', 'chemo4_notes', 'chemo5_date', 'chemo5_cycle', 'chemo5_day', 'chemo5_med', 'chemo5_notes', 'chemo6_date', 'chemo6_cycle', 'chemo6_day', 'chemo6_med', 'chemo6_notes', 'chemo7_date', 'chemo7_cycle', 'chemo7_day', 'chemo7_med', 'chemo7_notes', 'chemo8_date', 'chemo8_cycle', 'chemo8_day', 'chemo8_med', 'chemo8_notes', 'chemo9_date', 'chemo9_cycle', 'chemo9_day', 'chemo9_med', 'chemo9_notes', 'chemo10_date', 'chemo10_cycle', 'chemo10_day', 'chemo10_med', 'chemo10_notes', 'chemo11_date', 'chemo11_cycle', 'chemo11_day', 'chemo11_med', 'chemo11_notes', 'chemo12_date', 'chemo12_cycle', 'chemo12_day', 'chemo12_med', 'chemo12_notes', 'chemo13_date', 'chemo13_cycle', 'chemo13_day', 'chemo13_med', 'chemo13_notes', 'chemo14_date', 'chemo14_cycle', 'chemo14_day', 'chemo14_med', 'chemo14_notes', 'chemo15_date', 'chemo15_cycle', 'chemo15_day', 'chemo15_med', 'chemo15_notes', 'chemo16_date', 'chemo16_cycle', 'chemo16_day', 'chemo16_med', 'chemo16_notes', 'chemo17_date', 'chemo17_cycle', 'chemo17_day', 'chemo17_med', 'chemo17_notes', 'chemo18_date', 'chemo18_cycle', 'chemo18_day', 'chemo18_med', 'chemo18_notes', 'chemo19_date', 'chemo19_cycle', 'chemo19_day', 'chemo19_med', 'chemo19_notes', 'chemo20_date', 'chemo20_cycle', 'chemo20_day', 'chemo20_med', 'chemo20_notes', 'chemo21_date', 'chemo21_cycle', 'chemo21_day', 'chemo21_med', 'chemo21_notes', 'chemo22_date', 'chemo22_cycle', 'chemo22_day', 'chemo22_med', 'chemo22_notes', 'chemo23_date', 'chemo23_cycle', 'chemo23_day', 'chemo23_med', 'chemo23_notes', 'chemo24_date', 'chemo24_cycle', 'chemo24_day', 'chemo24_med', 'chemo24_notes', 'chemo25_date', 'chemo25_cycle', 'chemo25_day', 'chemo25_med', 'chemo25_notes', 'chemo26_date', 'chemo26_cycle', 'chemo26_day', 'chemo26_med', 'chemo26_notes', 'chemo27_date', 'chemo27_cycle', 'chemo27_day', 'chemo27_med', 'chemo27_notes', 'chemo28_date', 'chemo28_cycle', 'chemo28_day', 'chemo28_med', 'chemo28_notes', 'chemo29_date', 'chemo29_cycle', 'chemo29_day', 'chemo29_med', 'chemo29_notes', 'chemo30_date', 'chemo30_cycle', 'chemo30_day', 'chemo30_med', 'chemo30_notes', 'chemo31_date', 'chemo31_cycle', 'chemo31_day', 'chemo31_med', 'chemo31_notes', 'chemo32_date', 'chemo32_cycle', 'chemo32_day', 'chemo32_med', 'chemo32_notes', 'chemo33_date', 'chemo33_cycle', 'chemo33_day', 'chemo33_med', 'chemo33_notes', 'chemo34_date', 'chemo34_cycle', 'chemo34_day', 'chemo34_med', 'chemo34_notes', 'chemo35_date', 'chemo35_cycle', 'chemo35_day', 'chemo35_med', 'chemo35_notes', 'chemo36_date', 'chemo36_cycle', 'chemo36_day', 'chemo36_med', 'chemo36_notes', 'chemo37_date', 'chemo37_cycle', 'chemo37_day', 'chemo37_med', 'chemo37_notes', 'chemo38_date', 'chemo38_cycle', 'chemo38_day', 'chemo38_med', 'chemo38_notes', 'chemo39_date', 'chemo39_cycle', 'chemo39_day', 'chemo39_med', 'chemo39_notes', 'chemo40_date', 'chemo40_cycle', 'chemo40_day', 'chemo40_med', 'chemo40_notes', 'chemo41_date', 'chemo41_cycle', 'chemo41_day', 'chemo41_med', 'chemo41_notes', 'chemo42_date', 'chemo42_cycle', 'chemo42_day', 'chemo42_med', 'chemo42_notes', 'chemo43_date', 'chemo43_cycle', 'chemo43_day', 'chemo43_med', 'chemo43_notes', 'chemo44_date', 'chemo44_cycle', 'chemo44_day', 'chemo44_med', 'chemo44_notes', 'chemo45_date', 'chemo45_cycle', 'chemo45_day', 'chemo45_med', 'chemo45_notes', 'hosp1_admit_date', 'hosp1_discharge_date', 'hosp1_reason', 'hosp1_notes', 'hosp2_admit_date', 'hosp2_discharge_date', 'hosp2_reason', 'hosp2_notes', 'hosp3_admit_date', 'hosp3_discharge_date', 'hosp3_reason', 'hosp3_notes', 'surgery', 'ed') Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: and (value like '%z%') and event_id = 36032 Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: order by abs(value), value Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: Connection ID (thread ID): 269 Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: Status: KILL_CONNECTION Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=on,condition_pushdown_from_having=on,not_null_range_scan=off Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: information that should help you find out what is causing the crash. Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: Writing a core file... Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: Working directory at /data/mysql Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: Resource Limits: Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: Limit Soft Limit Hard Limit Units Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: Max cpu time unlimited unlimited seconds Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: Max file size unlimited unlimited bytes Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: Max data size unlimited unlimited bytes Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: Max stack size 8388608 unlimited bytes Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: Max core file size 0 unlimited bytes Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: Max resident set unlimited unlimited bytes Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: Max processes 95700 95700 processes Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: Max open files 32768 32768 files Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: Max locked memory 65536 65536 bytes Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: Max address space unlimited unlimited bytes Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: Max file locks unlimited unlimited locks Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: Max pending signals 95700 95700 signals Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: Max msgqueue size 819200 819200 bytes Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: Max nice priority 0 0 Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: Max realtime priority 0 0 Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: Max realtime timeout unlimited unlimited us Feb 14 10:17:00 redcap-dbserver-prod-02 mariadbd: Core pattern: core Feb 14 10:17:00 redcap-dbserver-prod-02 kernel: mariadbd[9723]: segfault at 0 ip 000055799aa5ec4f sp 00007f7fb0925430 error 4 in mariadbd[55799a29a000+1613000] Feb 14 10:17:00 redcap-dbserver-prod-02 systemd: mariadb.service: main process exited, code=killed, status=11/SEGV Feb 14 10:17:00 redcap-dbserver-prod-02 systemd: Unit mariadb.service entered failed state. Feb 14 10:17:00 redcap-dbserver-prod-02 systemd: mariadb.service failed. messages-20220206 Feb 2 15:02:22 redcap-dbserver-prod-02 mariadbd: 220202 15:02:22 [ERROR] mysqld got signal 11 ; Feb 2 15:02:22 redcap-dbserver-prod-02 mariadbd: This could be because you hit a bug. It is also possible that this binary Feb 2 15:02:22 redcap-dbserver-prod-02 mariadbd: or one of the libraries it was linked against is corrupt, improperly built, Feb 2 15:02:22 redcap-dbserver-prod-02 mariadbd: or misconfigured. This error can also be caused by malfunctioning hardware. Feb 2 15:02:22 redcap-dbserver-prod-02 mariadbd: To report this bug, see https://mariadb.com/kb/en/reporting-bugs Feb 2 15:02:22 redcap-dbserver-prod-02 mariadbd: We will try our best to scrape up some info that will hopefully help Feb 2 15:02:22 redcap-dbserver-prod-02 mariadbd: diagnose the problem, but since we have already crashed, Feb 2 15:02:22 redcap-dbserver-prod-02 mariadbd: something is definitely wrong and this may fail. Feb 2 15:02:22 redcap-dbserver-prod-02 mariadbd: Server version: 10.5.13-MariaDB Feb 2 15:02:22 redcap-dbserver-prod-02 mariadbd: key_buffer_size=134217728 Feb 2 15:02:22 redcap-dbserver-prod-02 mariadbd: read_buffer_size=131072 Feb 2 15:02:22 redcap-dbserver-prod-02 mariadbd: max_used_connections=43 Feb 2 15:02:22 redcap-dbserver-prod-02 mariadbd: max_threads=252 Feb 2 15:02:22 redcap-dbserver-prod-02 mariadbd: thread_count=15 Feb 2 15:02:22 redcap-dbserver-prod-02 mariadbd: It is possible that mysqld could use up to Feb 2 15:02:22 redcap-dbserver-prod-02 mariadbd: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 685818 K bytes of memory Feb 2 15:02:22 redcap-dbserver-prod-02 mariadbd: Hope that's ok; if not, decrease some variables in the equation. Feb 2 15:02:22 redcap-dbserver-prod-02 mariadbd: Thread pointer: 0x7f1db00009b8 Feb 2 15:02:22 redcap-dbserver-prod-02 mariadbd: Attempting backtrace. You can use the following information to find out Feb 2 15:02:22 redcap-dbserver-prod-02 mariadbd: where mysqld died. If you see no messages after this, something went Feb 2 15:02:22 redcap-dbserver-prod-02 mariadbd: terribly wrong... Feb 2 15:02:22 redcap-dbserver-prod-02 mariadbd: stack_bottom = 0x7f1e1c0d4c90 thread_stack 0x49000 Feb 2 15:02:22 redcap-dbserver-prod-02 mariadbd: ??:0(my_print_stacktrace)[0x5597b1c9dd6e] Feb 2 15:02:22 redcap-dbserver-prod-02 mariadbd: ??:0(handle_fatal_signal)[0x5597b16a0367] Feb 2 15:02:22 redcap-dbserver-prod-02 mariadbd: sigaction.c:0(__restore_rt)[0x7f20d0c7a630] Feb 2 15:02:22 redcap-dbserver-prod-02 mariadbd: ??:0(st_join_table::save_explain_data(Explain_table_access*, unsigned long long, bool, st_join_table*))[0x5597b14e1c4f] Feb 2 15:02:22 redcap-dbserver-prod-02 mariadbd: ??:0(JOIN::save_explain_data_intern(Explain_query*, bool, bool, bool, char const*))[0x5597b14e378b] Feb 2 15:02:22 redcap-dbserver-prod-02 mariadbd: ??:0(JOIN::save_explain_data(Explain_query*, bool, bool, bool, bool))[0x5597b14e38f0] Feb 2 15:02:22 redcap-dbserver-prod-02 mariadbd: ??:0(JOIN::build_explain())[0x5597b14e3b4c] Feb 2 15:02:22 redcap-dbserver-prod-02 mariadbd: ??:0(JOIN::optimize())[0x5597b14f90b1] Feb 2 15:02:22 redcap-dbserver-prod-02 mariadbd: ??:0(mysql_select(THD*, TABLE_LIST*, List&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*))[0x5597b14f91c7] Feb 2 15:02:22 redcap-dbserver-prod-02 mariadbd: ??:0(handle_select(THD*, LEX*, select_result*, unsigned long))[0x5597b14f9e34] Feb 2 15:02:22 redcap-dbserver-prod-02 mariadbd: /usr/sbin/mariadbd(+0x63b749)[0x5597b1358749] Feb 2 15:02:22 redcap-dbserver-prod-02 mariadbd: ??:0(mysql_execute_command(THD*))[0x5597b149d87d] Feb 2 15:02:22 redcap-dbserver-prod-02 mariadbd: ??:0(mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool))[0x5597b149fd15] Feb 2 15:02:22 redcap-dbserver-prod-02 mariadbd: ??:0(dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool))[0x5597b14a25e5] Feb 2 15:02:22 redcap-dbserver-prod-02 mariadbd: ??:0(do_command(THD*))[0x5597b14a3d7f] Feb 2 15:02:22 redcap-dbserver-prod-02 mariadbd: ??:0(do_handle_one_connection(CONNECT*, bool))[0x5597b1590662] Feb 2 15:02:22 redcap-dbserver-prod-02 mariadbd: ??:0(handle_one_connection)[0x5597b1590934] Feb 2 15:02:22 redcap-dbserver-prod-02 mariadbd: ??:0(MyCTX_nopad::finish(unsigned char*, unsigned int*))[0x5597b18ef91d] Feb 2 15:02:23 redcap-dbserver-prod-02 mariadbd: pthread_create.c:0(start_thread)[0x7f20d0c72ea5] Feb 2 15:02:23 redcap-dbserver-prod-02 mariadbd: ??:0(__clone)[0x7f20d018db0d] Feb 2 15:02:23 redcap-dbserver-prod-02 mariadbd: Trying to get some variables. Feb 2 15:02:23 redcap-dbserver-prod-02 mariadbd: Some pointers may be invalid and cause the dump to abort. Feb 2 15:02:23 redcap-dbserver-prod-02 mariadbd: Query (0x7f1db0010510): select distinct event_id, field_name, value, instance, record Feb 2 15:02:23 redcap-dbserver-prod-02 mariadbd: from redcap_data where project_id = 7827 and field_name in ('record_id', 'notes', 'email', 'date_consent', 'consent_person', 'recruit_md', 'date_start', 'date_45d', 'date_end', 'vis_id', 'fitbit_id', 'date_fitbit_mailed', 'fitbit_mail_track', 'date_fitbit_delivered', 'date_fitbit_data_start', 'return_fitbit_date', 'payment_comments', 'payment_date', 'vincent_number', 'date_payment_card_confirmed', 'date_payment_card_loaded', 'payment_amount', 'payment_partial_explain', 'return_phone_date', 'android_version', 'iphone_version', 'aware_android_version', 'aware_iphone_version', 'uuid', 'smartphone_storage_total', 'smartphone_storage_free', 'fitbit_email', 'fitbit_password', 'fitbit_firmware', 'fitbit_android_version', 'fitbit_iphone_version', 'uuid_end_date', 'uuid2', 'uuid2_start_date', 'fitbit2_email', 'flag_date', 'aware_lastsync', 'fitbit_lastsync', 'daily_lastsync', 'weekly_lastsync', 'flag_notes', 'flag_troubleshoot', 'com_notes', 'data_notes', 'opioid1_other_spec_45d', 'notes_45d', 'notes_90d', 'name', 'phone_number', 'number_type_other', 'phone_number_2', 'number_type_other_2', 'contact_when', 'address_street', 'address_city', 'address_state', 'address_zip', 'demo_datetime', 'dob', 'age', 'height_ft', 'height_inch', 'height', 'weight', 'bmi', 'race_other', 'race_multiple', 'living_other', 'employment_other', 'smoker_years', 'smoker_cigarettesdaily', 'wifi_note', 'own_comp_note', 'own_tablet_note', 'own_smartphone_model', 'own_smartphone_note', 'access_internet_note', 'social_media_note', 'own_tracker_note', 'notes_bl', 'cancer_type', 'cancer_dx_date', 'cancer_stage', 'comorbid_dx', 'chemo_start', 'emr_height_in', 'emr_weight_lb', 'date_death', 'chemo1_date', 'chemo1_cycle', 'chemo1_day', 'chemo1_med', 'chemo1_notes', 'chemo2_date', 'chemo2_cycle', 'chemo2_day', 'chemo2_med', 'chemo2_notes', 'chemo3_date', 'chemo3_cycle', 'chemo3_day', 'chemo3_med', 'chemo3_notes', 'chemo4_date', 'chemo4_cycle', 'chemo4_day', 'chemo4_med', 'chemo4_notes', 'chemo5_date', 'chemo5_cycle', 'chemo5_day', 'chemo5_med', 'chemo5_notes', 'chemo6_date', 'chemo6_cycle', 'chemo6_day', 'chemo6_med', 'chemo6_notes', 'chemo7_date', 'chemo7_cycle', 'chemo7_day', 'chemo7_med', 'chemo7_notes', 'chemo8_date', 'chemo8_cycle', 'chemo8_day', 'chemo8_med', 'chemo8_notes', 'chemo9_date', 'chemo9_cycle', 'chemo9_day', 'chemo9_med', 'chemo9_notes', 'chemo10_date', 'chemo10_cycle', 'chemo10_day', 'chemo10_med', 'chemo10_notes', 'chemo11_date', 'chemo11_cycle', 'chemo11_day', 'chemo11_med', 'chemo11_notes', 'chemo12_date', 'chemo12_cycle', 'chemo12_day', 'chemo12_med', 'chemo12_notes', 'chemo13_date', 'chemo13_cycle', 'chemo13_day', 'chemo13_med', 'chemo13_notes', 'chemo14_date', 'chemo14_cycle', 'chemo14_day', 'chemo14_med', 'chemo14_notes', 'chemo15_date', 'chemo15_cycle', 'chemo15_day', 'chemo15_med', 'chemo15_notes', 'chemo16_date', 'chemo16_cycle', 'chemo16_day', 'chemo16_med', 'chemo16_notes', 'chemo17_date', 'chemo17_cycle', 'chemo17_day', 'chemo17_med', 'chemo17_notes', 'chemo18_date', 'chemo18_cycle', 'chemo18_day', 'chemo18_med', 'chemo18_notes', 'chemo19_date', 'chemo19_cycle', 'chemo19_day', 'chemo19_med', 'chemo19_notes', 'chemo20_date', 'chemo20_cycle', 'chemo20_day', 'chemo20_med', 'chemo20_notes', 'chemo21_date', 'chemo21_cycle', 'chemo21_day', 'chemo21_med', 'chemo21_notes', 'chemo22_date', 'chemo22_cycle', 'chemo22_day', 'chemo22_med', 'chemo22_notes', 'chemo23_date', 'chemo23_cycle', 'chemo23_day', 'chemo23_med', 'chemo23_notes', 'chemo24_date', 'chemo24_cycle', 'chemo24_day', 'chemo24_med', 'chemo24_notes', 'chemo25_date', 'chemo25_cycle', 'chemo25_day', 'chemo25_med', 'chemo25_notes', 'chemo26_date', 'chemo26_cycle', 'chemo26_day', 'chemo26_med', 'chemo26_notes', 'chemo27_date', 'chemo27_cycle', 'chemo27_day', 'chemo27_med', 'chemo27_notes', 'chemo28_date', 'chemo28_cycle', 'chemo28_day', 'chemo28_med', 'chemo28_notes', 'chemo29_date', 'chemo29_cycle', 'chemo29_day', 'chemo29_med', 'chemo29_notes', 'chemo30_date', 'chemo30_cycle', 'chemo30_day', 'chemo30_med', 'chemo30_notes', 'chemo31_date', 'chemo31_cycle', 'chemo31_day', 'chemo31_med', 'chemo31_notes', 'chemo32_date', 'chemo32_cycle', 'chemo32_day', 'chemo32_med', 'chemo32_notes', 'chemo33_date', 'chemo33_cycle', 'chemo33_day', 'chemo33_med', 'chemo33_notes', 'chemo34_date', 'chemo34_cycle', 'chemo34_day', 'chemo34_med', 'chemo34_notes', 'chemo35_date', 'chemo35_cycle', 'chemo35_day', 'chemo35_med', 'chemo35_notes', 'chemo36_date', 'chemo36_cycle', 'chemo36_day', 'chemo36_med', 'chemo36_notes', 'chemo37_date', 'chemo37_cycle', 'chemo37_day', 'chemo37_med', 'chemo37_notes', 'chemo38_date', 'chemo38_cycle', 'chemo38_day', 'chemo38_med', 'chemo38_notes', 'chemo39_date', 'chemo39_cycle', 'chemo39_day', 'chemo39_med', 'chemo39_notes', 'chemo40_date', 'chemo40_cycle', 'chemo40_day', 'chemo40_med', 'chemo40_notes', 'chemo41_date', 'chemo41_cycle', 'chemo41_day', 'chemo41_med', 'chemo41_notes', 'chemo42_date', 'chemo42_cycle', 'chemo42_day', 'chemo42_med', 'chemo42_notes', 'chemo43_date', 'chemo43_cycle', 'chemo43_day', 'chemo43_med', 'chemo43_notes', 'chemo44_date', 'chemo44_cycle', 'chemo44_day', 'chemo44_med', 'chemo44_notes', 'chemo45_date', 'chemo45_cycle', 'chemo45_day', 'chemo45_med', 'chemo45_notes', 'hosp1_admit_date', 'hosp1_discharge_date', 'hosp1_reason', 'hosp1_notes', 'hosp2_admit_date', 'hosp2_discharge_date', 'hosp2_reason', 'hosp2_notes', 'hosp3_admit_date', 'hosp3_discharge_date', 'hosp3_reason', 'hosp3_notes', 'surgery', 'ed') Feb 2 15:02:23 redcap-dbserver-prod-02 kernel: mariadbd[7340]: segfault at 0 ip 00005597b14e1c4f sp 00007f1e1c0d3430 error 4 Feb 2 15:02:23 redcap-dbserver-prod-02 mariadbd: and (value like '%1%') and event_id = 36032 Feb 2 15:02:23 redcap-dbserver-prod-02 mariadbd: order by abs(value), value Feb 2 15:02:23 redcap-dbserver-prod-02 mariadbd: Connection ID (thread ID): 863207 Feb 2 15:02:23 redcap-dbserver-prod-02 mariadbd: Status: KILL_CONNECTION Feb 2 15:02:23 redcap-dbserver-prod-02 mariadbd: Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=on,condition_pushdown_from_having=on,not_null_range_scan=off Feb 2 15:02:23 redcap-dbserver-prod-02 mariadbd: The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains Feb 2 15:02:23 redcap-dbserver-prod-02 mariadbd: information that should help you find out what is causing the crash. Feb 2 15:02:23 redcap-dbserver-prod-02 mariadbd: Writing a core file... Feb 2 15:02:23 redcap-dbserver-prod-02 mariadbd: Working directory at /data/mysql Feb 2 15:02:23 redcap-dbserver-prod-02 mariadbd: Resource Limits: Feb 2 15:02:23 redcap-dbserver-prod-02 mariadbd: Limit Soft Limit Hard Limit Units Feb 2 15:02:23 redcap-dbserver-prod-02 mariadbd: Max cpu time unlimited unlimited seconds Feb 2 15:02:23 redcap-dbserver-prod-02 mariadbd: Max file size unlimited unlimited bytes Feb 2 15:02:23 redcap-dbserver-prod-02 mariadbd: Max data size unlimited unlimited bytes Feb 2 15:02:23 redcap-dbserver-prod-02 mariadbd: Max stack size 8388608 unlimited bytes Feb 2 15:02:23 redcap-dbserver-prod-02 mariadbd: Max core file size 0 unlimited bytes Feb 2 15:02:23 redcap-dbserver-prod-02 mariadbd: Max resident set unlimited unlimited bytes Feb 2 15:02:23 redcap-dbserver-prod-02 mariadbd: Max processes 95700 95700 processes Feb 2 15:02:23 redcap-dbserver-prod-02 mariadbd: Max open files 32768 32768 files Feb 2 15:02:23 redcap-dbserver-prod-02 mariadbd: Max locked memory 65536 65536 bytes Feb 2 15:02:23 redcap-dbserver-prod-02 mariadbd: Max address space unlimited unlimited bytes Feb 2 15:02:23 redcap-dbserver-prod-02 mariadbd: Max file locks unlimited unlimited locks Feb 2 15:02:23 redcap-dbserver-prod-02 mariadbd: Max pending signals 95700 95700 signals Feb 2 15:02:23 redcap-dbserver-prod-02 mariadbd: Max msgqueue size 819200 819200 bytes Feb 2 15:02:23 redcap-dbserver-prod-02 mariadbd: Max nice priority 0 0 Feb 2 15:02:23 redcap-dbserver-prod-02 mariadbd: Max realtime priority 0 0 Feb 2 15:02:23 redcap-dbserver-prod-02 mariadbd: Max realtime timeout unlimited unlimited us Feb 2 15:02:23 redcap-dbserver-prod-02 mariadbd: Core pattern: core Feb 2 15:02:23 redcap-dbserver-prod-02 systemd: mariadb.service: main process exited, code=killed, status=11/SEGV Feb 2 15:02:24 redcap-dbserver-prod-02 systemd: Unit mariadb.service entered failed state. Feb 2 15:02:24 redcap-dbserver-prod-02 systemd: mariadb.service failed. messages-20220130 Jan 26 13:45:12 redcap-dbserver-prod-02 mariadbd: 220126 13:45:12 [ERROR] mysqld got signal 11 ; Jan 26 13:45:12 redcap-dbserver-prod-02 mariadbd: This could be because you hit a bug. It is also possible that this binary Jan 26 13:45:12 redcap-dbserver-prod-02 mariadbd: or one of the libraries it was linked against is corrupt, improperly built, Jan 26 13:45:12 redcap-dbserver-prod-02 mariadbd: or misconfigured. This error can also be caused by malfunctioning hardware. Jan 26 13:45:12 redcap-dbserver-prod-02 mariadbd: To report this bug, see https://mariadb.com/kb/en/reporting-bugs Jan 26 13:45:12 redcap-dbserver-prod-02 mariadbd: We will try our best to scrape up some info that will hopefully help Jan 26 13:45:12 redcap-dbserver-prod-02 mariadbd: diagnose the problem, but since we have already crashed, Jan 26 13:45:12 redcap-dbserver-prod-02 mariadbd: something is definitely wrong and this may fail. Jan 26 13:45:12 redcap-dbserver-prod-02 mariadbd: Server version: 10.5.13-MariaDB Jan 26 13:45:12 redcap-dbserver-prod-02 mariadbd: key_buffer_size=134217728 Jan 26 13:45:12 redcap-dbserver-prod-02 mariadbd: read_buffer_size=131072 Jan 26 13:45:12 redcap-dbserver-prod-02 mariadbd: max_used_connections=92 Jan 26 13:45:12 redcap-dbserver-prod-02 mariadbd: max_threads=252 Jan 26 13:45:12 redcap-dbserver-prod-02 mariadbd: thread_count=18 Jan 26 13:45:12 redcap-dbserver-prod-02 mariadbd: It is possible that mysqld could use up to Jan 26 13:45:12 redcap-dbserver-prod-02 mariadbd: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 685818 K bytes of memory Jan 26 13:45:12 redcap-dbserver-prod-02 mariadbd: Hope that's ok; if not, decrease some variables in the equation. Jan 26 13:45:12 redcap-dbserver-prod-02 mariadbd: Thread pointer: 0x7fc3890f1d58 Jan 26 13:45:12 redcap-dbserver-prod-02 mariadbd: Attempting backtrace. You can use the following information to find out Jan 26 13:45:12 redcap-dbserver-prod-02 mariadbd: where mysqld died. If you see no messages after this, something went Jan 26 13:45:12 redcap-dbserver-prod-02 mariadbd: terribly wrong... Jan 26 13:45:12 redcap-dbserver-prod-02 mariadbd: stack_bottom = 0x7fc3dc1b6c90 thread_stack 0x49000 Jan 26 13:45:12 redcap-dbserver-prod-02 mariadbd: ??:0(my_print_stacktrace)[0x5600ebf77d6e] Jan 26 13:45:12 redcap-dbserver-prod-02 mariadbd: ??:0(handle_fatal_signal)[0x5600eb97a367] Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: sigaction.c:0(__restore_rt)[0x7fc68786d630] Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: ??:0(st_join_table::save_explain_data(Explain_table_access*, unsigned long long, bool, st_join_table*))[0x5600eb7bbc4f] Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: ??:0(JOIN::save_explain_data_intern(Explain_query*, bool, bool, bool, char const*))[0x5600eb7bd78b] Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: ??:0(JOIN::save_explain_data(Explain_query*, bool, bool, bool, bool))[0x5600eb7bd8f0] Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: ??:0(JOIN::build_explain())[0x5600eb7bdb4c] Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: ??:0(JOIN::optimize())[0x5600eb7d30b1] Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: ??:0(mysql_select(THD*, TABLE_LIST*, List&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*))[0x5600eb7d31c7] Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: ??:0(handle_select(THD*, LEX*, select_result*, unsigned long))[0x5600eb7d3e34] Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: /usr/sbin/mariadbd(+0x63b749)[0x5600eb632749] Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: ??:0(mysql_execute_command(THD*))[0x5600eb77787d] Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: ??:0(mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool))[0x5600eb779d15] Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: ??:0(dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool))[0x5600eb77c5e5] Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: ??:0(do_command(THD*))[0x5600eb77dd7f] Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: ??:0(do_handle_one_connection(CONNECT*, bool))[0x5600eb86a662] Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: ??:0(handle_one_connection)[0x5600eb86a934] Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: ??:0(MyCTX_nopad::finish(unsigned char*, unsigned int*))[0x5600ebbc991d] Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: pthread_create.c:0(start_thread)[0x7fc687865ea5] Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: ??:0(__clone)[0x7fc686d80b0d] Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: Trying to get some variables. Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: Some pointers may be invalid and cause the dump to abort. Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: Query (0x7fc389115d60): select distinct event_id, field_name, value, instance, record Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: from redcap_data where project_id = 2195 and field_name in ('record_id', 'subject_id', 'mrn', 'lastname', 'firstname', 'birth_date', 'health_care_utilization_id', 'subject', 'consent_date', 'consent_time', 'person_oc', 'consent_comment', 'context_date', 'sample_subject_id', 'sample_id', 'draw_date', 'draw_time', 'total_volume', 'handoff_to_other', 'handoff_time', 'comment', 'subject_unique_id', 'data_sample_id', 'cbc_lab_date', 'hgb', 'hct', 'wbc', 'mcv', 'mchc', 'hep_date', 'hb_s', 'hb_f', 'hb_a', 'hb_a2', 'data_comment', 'handoff_subject_id', 'handoff_sample_id', 'pickup_ts', 'pickup_comment', 'samples_receipt_timestamp', 'receipt_comment', 'sample002_subject_id', 'sample002_id', 'draw002_date', 'draw002_time', 'total_volume002', 'handoff002_subject_id', 'handoff002_sample_id', 'pickup002_ts', 'pickup002_comment', 'samples002_receipt_time', 'receipt002_comment', 'sample_repeated_subject_id', 'sample_repeated_id', 'sample_drawn_on', 'sample_total_volume', 'sample_received_subject_id', 'sample_received_id', 'sample_received_date', 'sample_received_comment') Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: and (value like '%am%') and event_id in (9787,9790,9784,9785,9782,9783,13318,13319,13320,13321,13322,30992,34034,40408) Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: order by abs(value), value Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: Connection ID (thread ID): 3718363 Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: Status: KILL_CONNECTION Jan 26 13:45:13 redcap-dbserver-prod-02 kernel: mariadbd[1267]: segfault at 0 ip 00005600eb7bbc4f sp 00007fc3dc1b5430 error 4 in mariadbd[5600eaff7000+1613000] Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=on,condition_pushdown_from_having=on,not_null_range_scan=off Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: information that should help you find out what is causing the crash. Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: Writing a core file... Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: Working directory at /data/mysql Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: Resource Limits: Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: Limit Soft Limit Hard Limit Units Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: Max cpu time unlimited unlimited seconds Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: Max file size unlimited unlimited bytes Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: Max data size unlimited unlimited bytes Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: Max stack size 8388608 unlimited bytes Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: Max core file size 0 unlimited bytes Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: Max resident set unlimited unlimited bytes Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: Max processes 95700 95700 processes Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: Max open files 32768 32768 files Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: Max locked memory 65536 65536 bytes Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: Max address space unlimited unlimited bytes Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: Max file locks unlimited unlimited locks Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: Max pending signals 95700 95700 signals Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: Max msgqueue size 819200 819200 bytes Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: Max nice priority 0 0 Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: Max realtime priority 0 0 Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: Max realtime timeout unlimited unlimited us Jan 26 13:45:13 redcap-dbserver-prod-02 mariadbd: Core pattern: core Jan 26 13:45:14 redcap-dbserver-prod-02 systemd: mariadb.service: main process exited, code=killed, status=11/SEGV Jan 26 13:45:14 redcap-dbserver-prod-02 systemd: Unit mariadb.service entered failed state. Jan 26 13:45:14 redcap-dbserver-prod-02 systemd: mariadb.service failed. messages-20211219 Dec 15 08:50:09 redcap-dbserver-prod-02 mariadbd: 211215 8:50:09 [ERROR] mysqld got signal 11 ; Dec 15 08:50:09 redcap-dbserver-prod-02 mariadbd: This could be because you hit a bug. It is also possible that this binary Dec 15 08:50:09 redcap-dbserver-prod-02 mariadbd: or one of the libraries it was linked against is corrupt, improperly built, Dec 15 08:50:09 redcap-dbserver-prod-02 mariadbd: or misconfigured. This error can also be caused by malfunctioning hardware. Dec 15 08:50:09 redcap-dbserver-prod-02 mariadbd: To report this bug, see https://mariadb.com/kb/en/reporting-bugs Dec 15 08:50:09 redcap-dbserver-prod-02 mariadbd: We will try our best to scrape up some info that will hopefully help Dec 15 08:50:09 redcap-dbserver-prod-02 mariadbd: diagnose the problem, but since we have already crashed, Dec 15 08:50:09 redcap-dbserver-prod-02 mariadbd: something is definitely wrong and this may fail. Dec 15 08:50:09 redcap-dbserver-prod-02 mariadbd: Server version: 10.5.13-MariaDB Dec 15 08:50:09 redcap-dbserver-prod-02 mariadbd: key_buffer_size=134217728 Dec 15 08:50:09 redcap-dbserver-prod-02 mariadbd: read_buffer_size=131072 Dec 15 08:50:09 redcap-dbserver-prod-02 mariadbd: max_used_connections=83 Dec 15 08:50:09 redcap-dbserver-prod-02 mariadbd: max_threads=252 Dec 15 08:50:09 redcap-dbserver-prod-02 mariadbd: thread_count=15 Dec 15 08:50:09 redcap-dbserver-prod-02 mariadbd: It is possible that mysqld could use up to Dec 15 08:50:09 redcap-dbserver-prod-02 mariadbd: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 685818 K bytes of memory Dec 15 08:50:09 redcap-dbserver-prod-02 mariadbd: Hope that's ok; if not, decrease some variables in the equation. Dec 15 08:50:09 redcap-dbserver-prod-02 mariadbd: Thread pointer: 0x7fc2380544c8 Dec 15 08:50:09 redcap-dbserver-prod-02 mariadbd: Attempting backtrace. You can use the following information to find out Dec 15 08:50:09 redcap-dbserver-prod-02 mariadbd: where mysqld died. If you see no messages after this, something went Dec 15 08:50:09 redcap-dbserver-prod-02 mariadbd: terribly wrong... Dec 15 08:50:09 redcap-dbserver-prod-02 mariadbd: stack_bottom = 0x7fc550820c90 thread_stack 0x49000 Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: ??:0(my_print_stacktrace)[0x561256d02d6e] Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: ??:0(handle_fatal_signal)[0x561256705367] Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: sigaction.c:0(__restore_rt)[0x7fc55ca3b630] Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: ??:0(st_join_table::save_explain_data(Explain_table_access*, unsigned long long, bool, st_join_table*))[0x561256546c4f] Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: ??:0(JOIN::save_explain_data_intern(Explain_query*, bool, bool, bool, char const*))[0x56125654878b] Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: ??:0(JOIN::save_explain_data(Explain_query*, bool, bool, bool, bool))[0x5612565488f0] Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: ??:0(JOIN::build_explain())[0x561256548b4c] Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: ??:0(JOIN::optimize())[0x56125655e0b1] Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: ??:0(mysql_select(THD*, TABLE_LIST*, List&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*))[0x56125655e1c7] Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: ??:0(handle_select(THD*, LEX*, select_result*, unsigned long))[0x56125655ee34] Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: /usr/sbin/mariadbd(+0x63b749)[0x5612563bd749] Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: ??:0(mysql_execute_command(THD*))[0x56125650287d] Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: ??:0(mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool))[0x561256504d15] Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: ??:0(dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool))[0x5612565075e5] Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: ??:0(do_command(THD*))[0x561256508d7f] Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: ??:0(do_handle_one_connection(CONNECT*, bool))[0x5612565f5662] Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: ??:0(handle_one_connection)[0x5612565f5934] Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: ??:0(MyCTX_nopad::finish(unsigned char*, unsigned int*))[0x56125695491d] Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: pthread_create.c:0(start_thread)[0x7fc55ca33ea5] Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: ??:0(__clone)[0x7fc55bf4eb0d] Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: Trying to get some variables. Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: Some pointers may be invalid and cause the dump to abort. Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: Query (0x7fc2380bda30): select distinct event_id, field_name, value, instance, record Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: from redcap_data where project_id = 11645 and field_name in ('record_id', 'status_pid', 'status_studyid', 'status_consent_date', 'status_reconsent_date', 'status_consent_inelgible_note', 'status_consent_other_note', 'status_lts_other', 'status_lts_withdrew', 'status_notes', 'status_flag_notes', 'status_cr_year', 'visit_date', 'visit_notes', 'visit_date_2', 'visit_notes_2', 'visit_date_3', 'visit_notes_3', 'visit_date_4', 'visit_notes_4', 'visit_date_5', 'visit_notes_5', 'visit_date_6', 'visit_notes_6', 'visit_date_7', 'visit_notes_7', 'visit_date_8', 'visit_notes_8', 'visit_date_9', 'visit_notes_9', 'visit_date_10', 'visit_notes_10', 'visit_date_11', 'visit_notes_11', 'visit_date_12', 'visit_notes_12', 'visit_date_13', 'visit_notes_13', 'visit_date_14', 'visit_notes_14', 'visit_date_15', 'visit_notes_15', 'pals_id', 'first_name', 'last_name', 'initials', 'dob', 'email', 'covid19_prior_subjectid', 'covid19_prior_date', 'covid19_prior_scomments', 'covid19_prior_ecomments', 'covid19_prior_tcomments', 'covid19_prior_qcomments', 'covid19_prior_screenedby', 'covid19_prior_sigdate', 'covid19_prior_sigtime', 'covid19_dayof_subjectid', 'covid19_dayof_date', 'covid19_dayof_scomments', 'covid19_dayof_ecomments', 'covid19_dayof_tcomments', 'covid19_dayof_qcomments', 'covid19_dayof_screenedby', 'covid19_dayof_sigdate', 'covid19_dayof_sigtime', 'ultrasound_covid19_screening_date', 'ultrasound_covid19_screening_diagnosis_comments', 'ultrasound_covid19_screening_symptoms_comments', 'ultrasound_covid19_screening_exposure_comments', 'ultrasound_covid19_screening_travel_comments', 'ultrasound_covid19_signature_date', 'pals_age', 'pals_staff', 'pals_eligchecklist_date', 'pals_notes', 'pals_doc_staff', 'pals_doc_persons_other', 'pals_doc_explain', 'pals_doc_time', 'pals_doc_add_staff', 'pals_doc_comp', 'pals_doc_coor_date', 'pals_doc_con_date', 'pals_doc_con_age', 'pals_doc_comments', 'hars_date', 'hars_14_behavior_other', 'hars_score', 'hars_notes', 'ms_date', 'ms_q1d', 'ms_q1e', 'ms_col1_total', 'ms_q3a', 'ms_q3b', 'ms_q4', 'ms_q4a', 'ms_q5_response', 'ms_col2_total', 'ms_q6b', 'ms_q6c', 'ms_q6_subscore', 'ms_q7a', 'ms_q7b', 'ms_q7c', 'ms_q7d', 'ms_q7_subscore', 'ms_col3_total', 'ms_q9_1', 'ms_q9_2', 'ms_q9_3', 'ms_q9_4', 'ms_q9_5', 'ms_q9_6', 'ms_q9_7', 'ms_q9_8', 'ms_q9_9', 'ms_q9_10', 'ms_q9_11', 'ms_q9_12', 'ms_q9_13', 'ms_q9_14', 'ms_q9_15', 'ms_q9_16', 'ms_q9_17', 'ms_q9_18', 'ms_q9_19', 'ms_q9_20', 'ms_q9_21', 'ms_q9_22', 'ms_q9_23', 'ms_q9_24', 'ms_col4_total', 'ms_q10a_notes', 'ms_q10b_notes', 'ms_q10c_notes', 'ms_q12', 'ms_col5_total', 'ms_q14', 'ms_col6_total', 'ms_col7_total', 'ms_q18d', 'ms_q18e', 'ms_col8_total', 'ms_total', 'ms_notes', 'tmoca_date', 'tmoca_education', 'moled', 'mads', 'malet', 'attention_subtraction_step1', 'maser', 'mlrep', 'tmoca_language_fluency_numwords', 'mlfluen', 'mlabst', 'mdrnc', 'morientation', 'mtotal', 'tmoca_administeredby', 'tmoca_notes', 'madrs_date', 'madrs_rater_initial', 'madrs_total', 'pswq_date', 'pswq_total', 'berlindate', 'berlinheight_in', 'berlinweight_lb', 'berlinage', 'item1_score', 'item2_score', 'item3_score', 'item4_score', 'item5_score', 'category1_score', 'category1_pos', 'item6_score', 'item7_score', 'item8_score', 'category2_score', 'category2_pos', 'berlinheight_m', 'berlinweight_kg', 'berlin_bmi', 'category3_pos', 'berlin_risk', 'psqi_date', 'psqi_1', 'psqi_2', 'psqi_3', 'psqi_4', 'psqi_5other', 'psqi_10e', 'psqi_comp1', 'psqi_comp2q2', 'psqi_comp2sum', 'psqi_comp2', 'psqi_comp3', 'psqi_comp4hours', 'psqi_comp4hse', 'psqi_comp4', 'psqi_comp5sum', 'psqi_comp5', 'psqi_comp6', 'psqi_comp7sum', 'psqi_comp7', 'psqi_global', 'mh_date', 'highbloodpressure_comment', 'heartmurmur_comment', 'angina_comment', 'heartattack_comment', 'coronaryarterydisease_comment', 'bypasssurgerystents_comment', 'angioplasty_comment', 'heartfailure_comment', 'arrhythmia_comment', 'heartsurgery_comment', 'endarterectomy_comment', 'raynauds_comment', 'cardiomyopathy_comment', 'asthma_comment', 'bronchitis_comment', 'chroniccough_comment', 'emphysema_comment', 'pneumonia_comment', 'sleepapnea_comment', 'tuberculosis_comment', 'kidneyfailure_comment', 'hysterectomy_comment', 'oophorectomy_comment', 'currentpregnancy_comment', 'endometriosis_comment', 'diabetes_comment', 'thyroiddisease_comment', 'polycysticovarionsyndrome_comment', 'parathyroiddisease_comment', 'addisonsdisease_comment', 'gravesdisease_comment', 'bleedingdisorder_comment', 'anemia_comment', 'osteoarthritis_comment', 'osteoporosis_comment', 'rheumatoidarthritis_comment', 'fibromyalgia_comment', 'psoriaticarthritis_comment', 'seizures_comment', 'migraines_comment', 'otherheadaches_comment', 'parkinsons_comment', 'stroke_comment', 'tia_comment', 'epilepsy_comment', 'dementiaalzheimers_comment', 'uncounsiousminutes', 'breastcancer_comment', 'reproductivecancer_comment', 'lymphoma_comment', 'leukemia_comment', 'pheochromocytoma_comment', 'pancreatictumor_comment', 'othercancer_comment', 'depression_comment', 'bipolardisorder_comment', 'panicattacks_comment', 'obsessivecompulsivedisorder_comment', 'otheranxietydisorder_comment', 'posttraumaticdisorder_comment', 'schizophrenia_comment', 'eatingdisorder_comment', 'substanceabuseaddiction_comment', 'temporalarteritis_comment', 'lupus_comment', 'sjogrenssyndrome_comment', 'celiacdisease_comment', 'chronicfatiguesyndrome_comment', 'inflammatoryboweldisease_comment', 'lichenslerosis_comment', 'multiplesclerosis_comment', 'scleroderma_comment', 'psoriasis_comment', 'allergiesspecified', 'genderidentityother', 'agedaytimehotflashesstarted', 'hotflashesnumberperoccurence', 'agedaytimeflashesstopped', 'hotflashesnumberperoccurence_2', 'daytimehotflashcomments', 'agenightsweatsstarted', 'nightsweatsnumberperoccurence', 'agenightsweatsstopped', 'nightsweatsnumberperoccurence_2', 'nightsweatscomments', 'hysterectomydate', 'ovariesremoveddate', 'menstrualbleedingstart', 'uterineabalationdate', 'endometrialablationprocedure', 'hormonesdatelasttaken', 'oralcontraceptiveyearsused', 'hormonetherapyyearsused', 'hormonepatchesyearsused', 'hormoneringsyearsused', 'hormoneimplantsyearsused', 'hormoneinjectionsyearsused', 'hormoneotheryearsused', 'hormoneyear', 'hormonesprayscreamstype', 'hormonesprayscreamsyearsused', 'antidepressantsdatelasttaken', 'antidepressantsssri_snri_specified', 'antidepressantsyearsused', 'cancermedicationspecified', 'cancermedicationdatelasttaken', 'cancermedicationyearsused', 'sermspecified', 'sermdatelasttaken', 'sermyearsused', 'aromataseinhibitorspecified', 'aromataseinhibitordatelastused', 'aromataseinhibitoryearsused', 'mh_notes', 'swanss_date', 'swanss_1a_01', 'swanss_2a_01', 'med_med', 'med_start', 'med_end', 'med_dose', 'med_reason', 'med_comment', 'med_med_2', 'med_start_2', 'med_end_2', 'med_dose_2', 'med_reason_2', 'med_comment_2', 'med_med_3', 'med_start_3', 'med_end_3', 'med_dose_3', 'med_reason_3', 'med_comment_3', 'med_med_4', 'med_start_4', 'med_end_4', 'med_dose_4', 'med_reason_4', 'med_comment_4', 'med_med_5', 'med_start_5', 'med_end_5', 'med_dose_5', 'med_reason_5', 'med_comment_5', 'med_med_6', 'med_start_6', 'med_end_6', 'med_dose_6', 'med_reason_6', 'med_comment_6', 'med_med_7', 'med_start_7', 'med_end_7', 'med_dose_7', 'med_reason_7', 'med_comment_7', 'med_med_8', 'med_start_8', 'med_end_8', 'med_dose_8', 'med_reason_8', 'med_comment_8', 'med_med_9', 'med_start_9', 'med_end_9', 'med_dose_9', 'med_reason_9', 'med_comment_9', 'med_med_10', 'med_start_10', 'med_end_10', 'med_dose_10', 'med_reason_10', 'med_comment_10', 'med_med_11', 'med_start_11', 'med_end_11', 'med_dose_11', 'med_reason_11', 'med_comment_11', 'med_med_12', 'med_start_12', 'med_end_12', 'med_dose_12', 'med_reason_12', 'med_comment_12', 'med_med_13', 'med_start_13', 'med_end_13', 'med_dose_13', 'med_reason_13', 'med_comment_13', 'med_med_14', 'med_start_14', 'med_end_14', 'med_dose_14', 'med_reason_14', 'med_comment_14', 'med_med_15', 'med_start_15', 'med_end_15', 'med_dose_15', 'med_reason_15', 'med_comment_15', 'med_med_16', 'med_start_16', 'med_end_16', 'med_dose_16', 'med_reason_16', 'med_comment_16', 'med_med_17', 'med_start_17', 'med_end_17', 'med_dose_17', 'med_reason_17', 'med_comment_17', 'med_med_18', 'med_start_18', 'med_end_18', 'med_dose_18', 'med_reason_18', 'med_comment_18', 'med_med_19', 'med_start_19', 'med_end_19', 'med_dose_19', 'med_reason_19', 'med_comment_19', 'med_med_20', 'med_start_20', 'med_end_20', 'med_dose_20', 'med_reason_20', 'med_comment_20', 'med_med_21', 'med_start_21', 'med_end_21', 'med_dose_21', 'med_reason_21', 'med_comment_21', 'med_med_22', 'med_start_22', 'med_end_22', 'med_dose_22', 'med_reason_22', 'med_comment_22', 'med_med_23', 'med_start_23', 'med_end_23', 'med_dose_23', 'med_reason_23', 'med_comment_23', 'med_med_24', 'med_start_24', 'med_end_24', 'med_dose_24', 'med_reason_24', 'med_comment_24', 'med_med_25', 'med_start_25', 'med_end_25', 'med_dose_25', 'med_reason_25', 'med_comment_25', 'mr_studyid', 'mr_scancode', 'mr_scanid', 'mr_filemaker_scanid', 'mr_sdate1', 'mr_sdate2', 'mr_sdate3', 'mr_sdate4', 'mr_sdate5', 'mr_notes', 'mr_collected', 'mr_entered', 'mr_scan_date', 'mr_tech1', 'mr_tech2', 'mr_start', 'mr_stop', 'mr_hires_notes', 'mr_hires_if_date', 'mr_hires_if_notes', 'mr_hires_qc_mnotes', 'mr_hires_qc_inotes', 'mr_hires_qc_gnotes', 'mr_hires_qc_csnotes', 'mr_dtit2_notes', 'mr_dtit2_qc_mnotes', 'mr_dtit2_qc_inotes', 'mr_dtit2_qc_gnotes', 'mr_dtit2_qc_csnotes', 'mr_lhippo_notes', 'mr_lhippo_qc_mnotes', 'mr_lhippo_qc_inotes', 'mr_lhippo_qc_gnotes', 'mr_lhippo_qc_cfnotes', 'mr_rhippo_notes', 'mr_rhippo_qc_mnotes', 'mr_rhippo_qc_inotes', 'mr_rhippo_qc_gnotes', 'mr_rhippo_qc_cfnotes', 'mr_tof_notes', 'mr_tof_qc_mnotes', 'mr_tof_qc_gnotes', 'mr_tof_qc_cfnotes', 'mr_swi_notes', 'mr_swi_qc_mnotes', 'mr_swi_qc_inotes', 'mr_swi_qc_gnotes', 'mr_swi_qc_cfnotes', 'mr_scan_notes', 'mr_qc_verfied_by', 'mr_qc_verified_date', 'mr_if_notes', 'mr_if_initial', 'mr_if_initial_date', 'mr_if_second', 'mr_if_second_date', 'mr_status2', 'mr_scan_pulledby', 'mr_scan_pull_date', 'mr_mrid', 'mr_vauliting', 'mr_status3', 'mr_vault_path', 'mr_vault_notes', 'arsq_date', 'arsq_dom', 'arsq_weighted_dom', 'arsq_tom', 'arsq_weighted_tom', 'arsq_self', 'arsq_weighted_self', 'arsq_plan', 'arsq_weighted_plan', 'arsq_sleep', 'arsq_weighted_sleep', 'arsq_comfort', 'arsq_weighted_comfort', 'arsq_soma', 'arsq_weighted_soma', 'head_circ_cm', 'head_circ_mm', 'height_in', 'weight_lb', 'vitals_notes', 'ae_event1', 'ae_start1', 'ae_stop1', 'ae_event2', 'ae_start2', 'ae_stop2', 'ae_event3', 'ae_start3', 'ae_stop3', 'ae_event4', 'ae_start4', 'ae_stop4', 'ae_event5', 'ae_start5', 'ae_stop5', 'lab_results_date', 'glucose_mg_dl', 'ldl_mg_dl', 'vldl_mg_dl', 'cholesterol_mg_dl', 'triglycerides_mg_dl', 'hdl_mg_dl', 'insulin_mg_dl', 'lab_results_notes', 'pregnancyt_date', 'pregnancyt_comment', 'u_covid19_screen_date', 'urlpre_urlid', 'urlpre_date', 'urlpre_time', 'urlpre_q2_yesa', 'urlpre_q2_yesb', 'urlpre_q2_yesc1', 'urlpre_q2_yesc2', 'urlpre_q2_yesd1', 'urlpre_q2_noa5other', 'urlpre_q2_nob1', 'urlpre_q2_nob2', 'urlpre_q3a', 'urlpre_q3b', 'urlpre_q4a', 'urlpre_q4b', 'urlpre_q5a', 'urlpre_q6', 'urlpre_q6a', 'urlpre_q6b', 'ucaritod_date', 'tech', 'urlid', 'scale', 'patid', 'reader', 'tcafr_automan', 'tcanr_automan', 'tcafl_automan', 'tcanl_automan', 'tcafr1', 'tcafr2', 'tcafl1', 'tcafl2', 'tcmfr1', 'tcmfr2', 'tcmfl1', 'tcmfl2', 'tcxfr1', 'tcxfr2', 'tcxfl1', 'tcxfl2', 'tcanr1', 'tcanr2', 'tcanl1', 'tcanl2', 'tcmnr1', 'tcmnr2', 'tcmnl1', 'tcmnl2', 'tcxnr1', 'tcxnr2', 'tcxnl1', 'tcxnl2', 'l41ar', 'l42ar', 'l41al', 'l42al', 'l41mr', 'l42mr', 'l41ml', 'l42ml', 'l41xr', 'l42xr', 'l41xl', 'l42xl', 'tcsdnr1', 'tcsdnr2', 'tcsdnl1', 'tcsdnl2', 'tclennr1', 'tclennr2', 'tclennl1', 'tclennl2', 'tcsdfr1', 'tcsdfr2', 'tcsdfl1', 'tcsdfl2', 'tclenfr1', 'tclenfr2', 'tclenfl1', 'tclenfl2', 'l4sdr1', 'l4sdr2', 'l4sdl1', 'l4sdl2', 'l4lenr1', 'l4lenr2', 'l4lenl1', 'l4lenl2', 'adavgr1', 'adavgr2', 'adavgl1', 'adavgl2', 'adminr1', 'adminr2', 'adminl1', 'adminl2', 'admaxr1', 'admaxr2', 'admaxl1', 'admaxl2', 'adsdr1', 'adsdr2', 'adsdl1', 'adsdl2', 'adlenr1', 'adlenr2', 'adlenl1', 'adlenl2', 'tcafr_comment', 'tcanr_comment', 'tcafl_comment', 'tcanl_comment', 'project', 'tiafr_automan', 'tiafl_automan', 'tiafr', 'tiafl', 'timfr', 'timfl', 'tixfr', 'tixfl', 'tisdfr', 'tisdfl', 'tilenfr', 'tilenfl', 'tiafr_comment', 'tiafl_comment', 'tbafr_automan', 'tbafl_automan', 'tbafr', 'tbafl', 'tbmfr', 'tbmfl', 'tbxfr', 'tbxfl', 'tbsdfr', 'tbsdfl', 'tblenfr', 'tblenfl', 'tbafr_comment', 'tbafl_comment', 'editflag', 'i', 'tcafr', 'tcanr', 'tcafl', 'tcanl', 'tcmfr', 'tcxfr', 'tcmnr', 'tcxnr', 'tcmfl', 'tcxfl', 'tcmnl', 'tcxnl', 'adavgr', 'adavgl', 'adminr', 'adminl', 'admaxr', 'admaxl', 'mavg', 'mavgr', 'mavgl', 'mmaxr', 'mmaxl', 'mmax', 'mminr', 'mminl', 'mmin', 'aad1r', 'aad2r', 'aadr', 'aldr', 'mad1r', 'mad2r', 'madr', 'mldr', 'mxad1r', 'mxad2r', 'mmaxadr', 'mxldr', 'aad1l', 'aad2l', 'aadl', 'aldl', 'mad1l', 'mad2l', 'madl', 'mldl', 'mxad1l', 'mxad2l', 'mmaxadl', 'mxldl', 'aad', 'mad', 'mmaxad', 'ald', 'mld', 'mxld', 'adavg', 'admin', 'admax', 'mavg_cca', 'mavg_ica', 'mavg_bulb', 'mavgr_cca', 'mavgl_cca', 'mmaxr_cca', 'mmaxl_cca', 'mmax_cca', 'mminr_cca', 'mminl_cca', 'mmin_cca', 'dos', 'dor', 'uimt_study', 'urlvisit', 'uimt_labid', 'uimt_id', 'uimt_tech', 'uimt_dos', 'room', 'cd', 'scan', 'les1r', 'les2r', 'les3r', 'lesir', 'leser', 'gra1r', 'gra2r', 'gra3r', 'grair', 'graer', 'reasonr', 'commentr', 'les1l', 'les2l', 'les3l', 'lesil', 'lesel', 'gra1l', 'gra2l', 'gra3l', 'grail', 'grael', 'reasonl', 'commentl', 'comp_patid', 'comp_labid', 'comp_study', 'comp_time', 'comp_tech', 'bsbp2', 'bdbp2', 'bpp2', 'height', 'weight', 'csbp1', 'csbp2', 'csbp3', 'csbpavg', 'cdbp1', 'cdbp2', 'cdbp3', 'cdbpavg', 'cmap1', 'cmap2', 'cmap3', 'cmapavg', 'cpp1', 'cpp2', 'cpp3', 'cppavg', 'bhr1', 'bhr2', 'bhr3', 'hr', 'aix1', 'aix2', 'aix3', 'aixavg', 'ap1', 'ap2', 'ap3', 'apavg', 'sp_amp1', 'sp_amp2', 'sp_amp3', 'spamp', 'pp_ratio1', 'pp_ratio2', 'pp_ratio3', 'ppr', 'cftime1', 'cftime2', 'cftime3', 'cftime', 'cfdis1', 'cfdis2', 'cfdis3', 'cfdis', 'cfpwv1', 'cfpwv2', 'cfpwv3', 'cfpwv', 'cftol1', 'cftol2', 'cftol3', 'cftol', 'comp_dob', 'comp_dos', 'pwvsd', 'pcntdev', 'aisd', 'aidev', 'apsd', 'apdev', 'ampsd', 'ampdev', 'covid19_screen_date2', 'covid19_screen_fever_date_v2', 'covid19_screen_breath_date_v2', 'covid19_screen_cough_date_v2', 'covid19_screen_fatigue_date_v2', 'covid19_screen_nose_date_v2', 'covid19_screen_smell_date_v2', 'covid19_screen_taste_date_v2', 'covid19_screen_diarrhea_date_v2', 'covid19_screen_chest_date_v2', 'covid19_screen_aches_date_v2', 'covid19_screen_headache_date_v2', 'covid19_screen_throat_date_v2', 'covid19_screen_congestion_date_v2', 'covid19_screen_nausea_date_v2', 'covid19_screen_exposure_date_v2', 'covid19_screen_diagnosed_date_v2', 'covid19_screen_fever_date_2_v2', 'covid19_screen_breath_date_2_v2', 'covid19_screen_cough_date_2_v2', 'covid19_screen_fatigue_date_2_v2', 'covid19_screen_nose_date_2_v2', 'covid19_screen_smell_date_2_v2', 'covid19_screen_taste_date_2_v2', 'covid19_screen_diarrhea_date_2_v2', 'covid19_screen_chest_date_2_v2', 'covid19_screen_aches_date_2_v2', 'covid19_screen_headache_date_2_v2', 'covid19_screen_throat_date_2_v2', 'covid19_screen_congestion_date_2_v2', 'covid19_screen_nausea_date_2_v2', 'covid19_screen_exposure_date_2_v2', 'covid19_screen_diagnosed_date_2_v2', 'covid19_screen_contact_date_v2', 'covid19_screen_2_formversion', 'covid19_screen_2_formversion_date', 'covid19_screen_date', 'covid19_screen_fever_date', 'covid19_screen_breath_date', 'covid19_screen_cough_date', 'covid19_screen_fatigue_date', 'covid19_screen_nose_date', 'covid19_screen_smell_date', 'covid19_screen_taste_date', 'covid19_screen_diarrhea_date', 'covid19_screen_chest_date', 'covid19_screen_aches_date', 'covid19_screen_headache_date', 'covid19_screen_throat_date', 'covid19_screen_congestion_date', 'covid19_screen_nausea_date', 'covid19_screen_exposure_date', 'covid19_screen_diagnosed_date', 'covid19_screen_fever_date_2', 'covid19_screen_breath_date_2', 'covid19_screen_cough_date_2', 'covid19_screen_fatigue_date_2', 'covid19_screen_nose_date_2', 'covid19_screen_smell_date_2', 'covid19_screen_taste_date_2', 'covid19_screen_diarrhea_date_2', 'covid19_screen_chest_date_2', 'covid19_screen_aches_date_2', 'covid19_screen_headache_date_2', 'covid19_screen_throat_date_2', 'covid19_screen_congestion_date_2', 'covid19_screen_nausea_date_2', 'covid19_screen_exposure_date_2', 'covid19_screen_diagnosed_date_2', 'covid19_screen_contact_date', 'covid19_prior_formversion', 'covid19_prior_formversion_date') Dec 15 08:50:10 redcap-dbserver-prod-02 kernel: mariadbd[30264]: segfault at 0 ip 0000561256546c4f sp 00007fc55081f430 error 4 in mariadbd[561255d82000+1613000] Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: and (value like '%co%') and event_id = 49594 Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: order by abs(value), value Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: Connection ID (thread ID): 1593582 Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: Status: KILL_CONNECTION Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=on,condition_pushdown_from_having=on,not_null_range_scan=off Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: information that should help you find out what is causing the crash. Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: Writing a core file... Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: Working directory at /data/mysql Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: Resource Limits: Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: Limit Soft Limit Hard Limit Units Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: Max cpu time unlimited unlimited seconds Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: Max file size unlimited unlimited bytes Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: Max data size unlimited unlimited bytes Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: Max stack size 8388608 unlimited bytes Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: Max core file size 0 unlimited bytes Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: Max resident set unlimited unlimited bytes Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: Max processes 95700 95700 processes Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: Max open files 32768 32768 files Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: Max locked memory 65536 65536 bytes Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: Max address space unlimited unlimited bytes Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: Max file locks unlimited unlimited locks Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: Max pending signals 95700 95700 signals Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: Max msgqueue size 819200 819200 bytes Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: Max nice priority 0 0 Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: Max realtime priority 0 0 Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: Max realtime timeout unlimited unlimited us Dec 15 08:50:10 redcap-dbserver-prod-02 mariadbd: Core pattern: core Dec 15 08:50:10 redcap-dbserver-prod-02 systemd: mariadb.service: main process exited, code=killed, status=11/SEGV Dec 15 08:50:11 redcap-dbserver-prod-02 systemd: Unit mariadb.service entered failed state. Dec 15 08:50:11 redcap-dbserver-prod-02 systemd: mariadb.service failed. messages-20211017 Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: 211011 9:39:32 [ERROR] mysqld got signal 11 ; Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: This could be because you hit a bug. It is also possible that this binary Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: or one of the libraries it was linked against is corrupt, improperly built, Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: or misconfigured. This error can also be caused by malfunctioning hardware. Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: To report this bug, see https://mariadb.com/kb/en/reporting-bugs Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: We will try our best to scrape up some info that will hopefully help Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: diagnose the problem, but since we have already crashed, Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: something is definitely wrong and this may fail. Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: Server version: 10.5.10-MariaDB-log Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: key_buffer_size=134217728 Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: read_buffer_size=131072 Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: max_used_connections=85 Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: max_threads=252 Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: thread_count=11 Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: It is possible that mysqld could use up to Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 685816 K bytes of memory Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: Hope that's ok; if not, decrease some variables in the equation. Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: Thread pointer: 0x7f506403ad58 Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: Attempting backtrace. You can use the following information to find out Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: where mysqld died. If you see no messages after this, something went Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: terribly wrong... Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: stack_bottom = 0x7f50d461fc90 thread_stack 0x49000 Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: ??:0(my_print_stacktrace)[0x555757412dfe] Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: ??:0(handle_fatal_signal)[0x555756e133b7] Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: sigaction.c:0(__restore_rt)[0x7f538071c630] Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: ??:0(st_join_table::save_explain_data(Explain_table_access*, unsigned long long, bool, st_join_table*))[0x555756c5412f] Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: ??:0(JOIN::save_explain_data_intern(Explain_query*, bool, bool, bool, char const*))[0x555756c55cab] Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: ??:0(JOIN::save_explain_data(Explain_query*, bool, bool, bool, bool))[0x555756c55e30] Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: ??:0(JOIN::build_explain())[0x555756c5608c] Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: ??:0(JOIN::optimize())[0x555756c6abc1] Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: ??:0(mysql_select(THD*, TABLE_LIST*, List&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*))[0x555756c6acc8] Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: ??:0(handle_select(THD*, LEX*, select_result*, unsigned long))[0x555756c6b8d4] Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: /usr/sbin/mariadbd(+0x63962c)[0x555756acd62c] Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: ??:0(mysql_execute_command(THD*))[0x555756c0ee13] Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: ??:0(mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool))[0x555756c114fa] Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: ??:0(dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool))[0x555756c14223] Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: ??:0(do_command(THD*))[0x555756c1595f] Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: ??:0(do_handle_one_connection(CONNECT*, bool))[0x555756d00c02] Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: ??:0(handle_one_connection)[0x555756d00ec4] Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: ??:0(MyCTX_nopad::finish(unsigned char*, unsigned int*))[0x555757064f3d] Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: 2021-10-11 9:39:32 15695971 [Warning] Sort aborted, host: redcap-web02.dbmi.pitt.edu, user: RedcapWeb, thread: 15695971, query: select * from redcap_data_access_groups where project_id = 2195 order by trim(group_name) Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: pthread_create.c:0(start_thread)[0x7f5380714ea5] Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: ??:0(__clone)[0x7f537fc2f9fd] Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: Trying to get some variables. Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: Some pointers may be invalid and cause the dump to abort. Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: Query (0x7f50641c46c0): select record, event_id, field_name, value, instance from redcap_data Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: where project_id = 2195 and record != '' and event_id in ('9787', '9790', '9784', '9785', '9782', '9783', '13318', '13319', '13320', '13321', '13322', '30992', '34034', '40408') and field_name in ('record_id', 'subject_id', 'lastname', 'firstname', 'gender', 'genotype') Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: Connection ID (thread ID): 15695969 Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: Status: KILL_CONNECTION Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=on,condition_pushdown_from_having=on,not_null_range_scan=off Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: information that should help you find out what is causing the crash. Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: Writing a core file... Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: Working directory at /data/mysql Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: Resource Limits: Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: Limit Soft Limit Hard Limit Units Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: Max cpu time unlimited unlimited seconds Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: Max file size unlimited unlimited bytes Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: Max data size unlimited unlimited bytes Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: Max stack size 8388608 unlimited bytes Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: Max core file size 0 unlimited bytes Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: Max resident set unlimited unlimited bytes Oct 11 09:39:32 redcap-dbserver-prod-02 kernel: mariadbd[26826]: segfault at 0 ip 0000555756c5412f sp 00007f50d461e380 error 4 in mariadbd[555756494000+1607000] Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: Max processes 95700 95700 processes Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: Max open files 32768 32768 files Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: Max locked memory 65536 65536 bytes Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: Max address space unlimited unlimited bytes Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: Max file locks unlimited unlimited locks Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: Max pending signals 95700 95700 signals Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: Max msgqueue size 819200 819200 bytes Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: Max nice priority 0 0 Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: Max realtime priority 0 0 Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: Max realtime timeout unlimited unlimited us Oct 11 09:39:32 redcap-dbserver-prod-02 mariadbd: Core pattern: core Oct 11 09:39:32 redcap-dbserver-prod-02 systemd: mariadb.service: main process exited, code=killed, status=11/SEGV Oct 11 09:39:33 redcap-dbserver-prod-02 systemd: Unit mariadb.service entered failed state. Oct 11 09:39:33 redcap-dbserver-prod-02 systemd: mariadb.service failed.