2025-05-13 11:35:46 89581134 [Warning] Aborted connection 89581134 to db: 'aaa_reporting' user: 'alepo' host: '10.76.25.19' (Got timeout reading communication packets) 250513 13:20:17 [ERROR] mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. To report this bug, see https://mariadb.com/kb/en/reporting-bugs We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. Server version: 10.11.5-MariaDB-log source revision: 7875294b6b74b53dd3aaa723e6cc103d2bb47b2c key_buffer_size=67108864 read_buffer_size=131072 max_used_connections=23 max_threads=802 thread_count=37 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1831964 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x7f30dc000c58 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x7f325c280b98 thread_stack 0x49000 /usr/sbin/mariadbd(my_print_stacktrace+0x2e)[0x55d98923612e] /usr/sbin/mariadbd(handle_fatal_signal+0x485)[0x55d988d2f5b5] /lib64/libpthread.so.0(+0x12d40)[0x7f585b34bd40] /usr/sbin/mariadbd(_ZN7handler17ha_index_read_mapEPhPKhm16ha_rkey_function+0x2f0)[0x55d988d36890] /usr/sbin/mariadbd(+0xc37600)[0x55d988f2b600] /usr/sbin/mariadbd(_ZN7handler17ha_index_read_mapEPhPKhm16ha_rkey_function+0x298)[0x55d988d36838] /usr/sbin/mariadbd(+0x854578)[0x55d988b48578] /usr/sbin/mariadbd(_Z10sub_selectP4JOINP13st_join_tableb+0x1a2)[0x55d988b33322] /usr/sbin/mariadbd(_ZN4JOIN10exec_innerEv+0xe59)[0x55d988b5f579] /usr/sbin/mariadbd(_ZN4JOIN4execEv+0x37)[0x55d988b5f837] /usr/sbin/mariadbd(_ZN18st_select_lex_unit4execEv+0x474)[0x55d988baa8a4] /usr/sbin/mariadbd(+0x7b6396)[0x55d988aaa396] /usr/sbin/mariadbd(_Z27mysql_handle_single_derivedP3LEXP10TABLE_LISTj+0x99)[0x55d988aa9d69] /usr/sbin/mariadbd(_ZN13st_join_table12preread_initEv+0x74)[0x55d988b33064] /usr/sbin/mariadbd(_Z10sub_selectP4JOINP13st_join_tableb+0xa3)[0x55d988b33223] /usr/sbin/mariadbd(_ZN4JOIN10exec_innerEv+0xe59)[0x55d988b5f579] /usr/sbin/mariadbd(_ZN4JOIN4execEv+0x37)[0x55d988b5f837] /usr/sbin/mariadbd(_Z12mysql_selectP3THDP10TABLE_LISTR4ListI4ItemEPS4_jP8st_orderS9_S7_S9_yP13select_resultP18st_select_lex_unitP13st_select_lex+0x11f)[0x55d988b5db8f] /usr/sbin/mariadbd(_Z13handle_selectP3THDP3LEXP13select_resulty+0x14b)[0x55d988b5e31b] /usr/sbin/mariadbd(+0x7f26d2)[0x55d988ae66d2] /usr/sbin/mariadbd(_Z21mysql_execute_commandP3THDb+0x126d)[0x55d988aeebca] /usr/sbin/mariadbd(_Z11mysql_parseP3THDPcjP12Parser_state+0x25c)[0x55d988ae1b87] /usr/sbin/mariadbd(_Z16dispatch_command19enum_server_commandP3THDPcjb+0x1080)[0x55d988aeb6e6] /usr/sbin/mariadbd(_Z10do_commandP3THDb+0x12f)[0x55d988aecdab] /usr/sbin/mariadbd(_Z24do_handle_one_connectionP7CONNECTb+0x3f7)[0x55d988c00977] /usr/sbin/mariadbd(handle_one_connection+0x5d)[0x55d988c00cbd] /usr/sbin/mariadbd(+0xc42c9d)[0x55d988f36c9d] /lib64/libpthread.so.0(+0x81da)[0x7f585b3411da] /lib64/libc.so.6(clone+0x43)[0x7f585a692e73] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query (0x7f30dc012ee0): SELECT USER_NAME,NAS_IP,ERROR_TIME,PASSWORDTYPED,case when ERROR_MSG='USER_NOT_FOUND' then 'Invalid User' when ERROR_MSG='USER_BARRED' then 'User Suspended' when ERROR_MSG='INVALID_PASSWORD' then 'Invalid Password' else ERROR_MSG end as ERROR_MSG from vw_aaa_auth_errorlogsmp where USER_NAME='100323179535' LIMIT 10 Connection ID (thread ID): 89778131 Status: NOT_KILLED Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=on,condition_pushdown_from_having=on,not_null_range_scan=off,hash_join_cardinality=off The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains information that should help you find out what is causing the crash. Writing a core file... Working directory at /data/mysql Resource Limits: Limit Soft Limit Hard Limit Units Max cpu time unlimited unlimited seconds Max file size unlimited unlimited bytes Max data size unlimited unlimited bytes Max stack size 8388608 unlimited bytes Max core file size 0 unlimited bytes Max resident set unlimited unlimited bytes Max processes 2061666 2061666 processes Max open files 32768 32768 files Max locked memory 65536 65536 bytes Max address space unlimited unlimited bytes Max file locks unlimited unlimited locks Max pending signals 2061666 2061666 signals Max msgqueue size 819200 819200 bytes Max nice priority 0 0 Max realtime priority 0 0 Max realtime timeout unlimited unlimited us Core pattern: |/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h %e Kernel version: Linux version 4.18.0-513.18.1.el8_9.x86_64 (mockbuild@x86-vm-08.build.eng.bos.redhat.com) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-20) (GCC)) #1 SMP Thu Feb 1 03:51:05 EST 2024 2025-05-13 13:20:54 0 [Note] Starting MariaDB 10.11.5-MariaDB-log source revision 7875294b6b74b53dd3aaa723e6cc103d2bb47b2c as process 2562071 2025-05-13 13:20:54 0 [Warning] You need to use --log-bin to make --binlog-format work. 2025-05-13 13:20:54 0 [Note] InnoDB: Compressed tables use zlib 1.2.11 2025-05-13 13:20:54 0 [Note] InnoDB: Number of transaction pools: 1 2025-05-13 13:20:54 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions 2025-05-13 13:20:54 0 [Note] InnoDB: Using Linux native AIO 2025-05-13 13:20:54 0 [Note] InnoDB: Initializing buffer pool, total size = 150.000GiB, chunk size = 2.344GiB 2025-05-13 13:20:55 0 [Note] InnoDB: Completed initialization of buffer pool 2025-05-13 13:20:55 0 [Note] InnoDB: Buffered log writes (block size=512 bytes) 2025-05-13 13:20:55 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=73021844230723 2025-05-13 13:21:10 0 [Note] InnoDB: Read redo log up to LSN=73023296986153 2025-05-13 13:21:25 0 [Note] InnoDB: Parsed redo log up to LSN=73024635950228; to recover: 2291955 pages 2025-05-13 13:21:26 0 [Note] InnoDB: End of log at LSN=73024793958166 2025-05-13 13:21:27 0 [Note] InnoDB: To recover: 2345640 pages 2025-05-13 13:21:42 0 [Note] InnoDB: To recover: 2345640 pages 2025-05-13 13:21:57 0 [Note] InnoDB: To recover: 2345640 pages 2025-05-13 13:22:07 0 [Note] InnoDB: 128 rollback segments are active. 2025-05-13 13:22:08 0 [Note] InnoDB: Removed temporary tablespace data file: "./ibtmp1" 2025-05-13 13:22:08 0 [Note] InnoDB: Setting file './ibtmp1' size to 12.000MiB. Physically writing the file full; Please wait ... 2025-05-13 13:22:08 0 [Note] InnoDB: File './ibtmp1' size is now 12.000MiB. 2025-05-13 13:22:08 0 [Note] InnoDB: log sequence number 73024793958166; transaction id 72035731579 2025-05-13 13:22:08 0 [Note] InnoDB: Loading buffer pool(s) from /data/mysql/ib_buffer_pool 2025-05-13 13:22:08 0 [Note] Plugin 'FEEDBACK' is disabled. 2025-05-13 13:22:08 0 [Warning] 'thread-concurrency' was removed. It does nothing now and exists only for compatibility with old my.cnf files. 2025-05-13 13:22:08 0 [Warning] 'innodb-thread-concurrency' was removed. It does nothing now and exists only for compatibility with old my.cnf files. 2025-05-13 13:22:08 0 [Warning] 'innodb-log-files-in-group' was removed. It does nothing now and exists only for compatibility with old my.cnf files. 2025-05-13 13:22:08 0 [Warning] You need to use --log-bin to make --expire-logs-days or --binlog-expire-logs-seconds work. 2025-05-13 13:22:08 0 [Note] Server socket created on IP: '0.0.0.0'. 2025-05-13 13:22:08 0 [Note] Server socket created on IP: '::'. 2025-05-13 13:22:08 0 [Warning] 'user' entry '@alepo-db2-ilb' ignored in --skip-name-resolve mode. 2025-05-13 13:22:08 0 [Warning] 'user' entry 'root@localhost.localdomain' ignored in --skip-name-resolve mode. 2025-05-13 13:22:08 0 [Warning] 'proxies_priv' entry '@% root@alepo-db2-ilb' ignored in --skip-name-resolve mode. 2025-05-13 13:22:11 4 [Note] Slave I/O thread: Start asynchronous replication to master 'gtid@10.76.24.71:3306' in log 'db1-binary.390487' at position 26261022 2025-05-13 13:22:11 4 [Note] Slave I/O thread: connected to master 'gtid@10.76.24.71:3306',replication started in log 'db1-binary.390487' at position 26261022 2025-05-13 13:22:11 5 [Note] Slave SQL thread initialized, starting replication in log 'db1-binary.390487' at position 26261022, relay log './db3-relay.618849' position: 15138444 2025-05-13 13:22:11 0 [Note] /usr/sbin/mariadbd: ready for connections. Version: '10.11.5-MariaDB-log' socket: '/var/lib/mysql/mysql.sock' port: 3306 MariaDB Server 2025-05-13 13:22:14 0 [Note] InnoDB: Buffer pool(s) load completed at 250513 13:22:14 2025-05-13 14:16:02 38547 [Warning] Aborted connection 38547 to db: 'aaa_reporting' user: 'alepo' host: '10.76.25.19' (Got timeout reading communication packets) 2025-05-13 14:41:38 78418 [Warning] Aborted connection 78418 to db: 'aaa_reporting' user: 'alepo' host: '10.76.25.19' (Got timeout reading communication packets) 2025-05-13 15:45:08 171872 [Warning] Aborted connection 171872 to db: 'aaa_reporting' user: 'alepo' host: '10.76.25.19' (Got timeout reading communication packets) 2025-05-13 16:10:38 210888 [Warning] Aborted connection 210888 to db: 'aaa_reporting' user: 'alepo' host: '10.76.25.19' (Got timeout reading communication packets) 2025-05-13 16:15:06 216168 [Warning] Aborted connection 216168 to db: 'aaa_reporting' user: 'alepo' host: '10.76.25.19' (Got timeout reading communication packets) 2025-05-13 18:09:09 387314 [Warning] Aborted connection 387314 to db: 'aaa_reporting' user: 'alepo' host: '10.76.25.19' (Got timeout reading communication packets) 2025-05-13 18:43:00 436434 [Warning] Aborted connection 436434 to db: 'aaa_reporting' user: 'alepo' host: '10.76.25.19' (Got timeout reading communication packets) 2025-05-13 19:43:00 524306 [Warning] Aborted connection 524306 to db: 'aaa_reporting' user: 'alepo' host: '10.76.25.19' (Got timeout reading communication packets) 2025-05-13 20:41:03 609744 [Warning] Aborted connection 609744 to db: 'aaa_reporting' user: 'alepo' host: '10.76.25.19' (Got timeout reading communication packets) 2025-05-13 22:05:32 731360 [Warning] Aborted connection 731360 to db: 'aaa_reporting' user: 'alepo' host: '10.76.25.19' (Got timeout reading communication packets) 2025-05-14 0:09:52 899236 [Warning] Aborted connection 899236 to db: 'aaa_reporting' user: 'alepo' host: '10.76.25.19' (Got timeout reading communication packets)