[MDEV-31716] My server got crashed Created: 2023-07-17 Updated: 2023-07-17 |
|
| Status: | Open |
| Project: | MariaDB Server |
| Component/s: | Storage Engine - InnoDB |
| Affects Version/s: | 10.9.7 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major |
| Reporter: | Kevin Nguyen | Assignee: | Unassigned |
| Resolution: | Unresolved | Votes: | 0 |
| Labels: | assertion, crash | ||
| Issue Links: |
|
||||||||
| Description |
|
My server is hosting WooCommerce stores. It looks like the thumnail generator is running background and causing the issue with the pics. Jul 17 01:48:02 server postfix/qmgr[1057313]: 7826A4217C: removed Jul 17 01:48:18 server mariadbd[3185478]: 2023-07-17 01:48:18 0xffe97a443090 InnoDB: Assertion failure in file ./storage/innobase/row/row0ins.cc line 222 Jul 17 01:48:18 server mariadbd[3185478]: InnoDB: Failing assertion: !cursor->index()->is_committed() Jul 17 01:48:18 server mariadbd[3185478]: InnoDB: We intentionally generate a memory trap. Jul 17 01:48:18 server mariadbd[3185478]: InnoDB: Submit a detailed bug report to https://jira.mariadb.org/ Jul 17 01:48:18 server mariadbd[3185478]: InnoDB: If you get repeated assertion failures or crashes, even Jul 17 01:48:18 server mariadbd[3185478]: InnoDB: immediately after the mariadbd startup, there may be Jul 17 01:48:18 server mariadbd[3185478]: InnoDB: corruption in the InnoDB tablespace. Please refer to Jul 17 01:48:18 server mariadbd[3185478]: InnoDB: https://mariadb.com/kb/en/library/innodb-recovery-modes/ Jul 17 01:48:18 server mariadbd[3185478]: InnoDB: about forcing recovery. Jul 17 01:48:18 server mariadbd[3185478]: 230717 1:48:18 [ERROR] mysqld got signal 6 ; Jul 17 01:48:18 server mariadbd[3185478]: This could be because you hit a bug. It is also possible that this binary Jul 17 01:48:18 server mariadbd[3185478]: or one of the libraries it was linked against is corrupt, improperly built, Jul 17 01:48:18 server mariadbd[3185478]: or misconfigured. This error can also be caused by malfunctioning hardware. Jul 17 01:48:18 server mariadbd[3185478]: To report this bug, see https://mariadb.com/kb/en/reporting-bugs Jul 17 01:48:18 server mariadbd[3185478]: We will try our best to scrape up some info that will hopefully help Jul 17 01:48:18 server mariadbd[3185478]: diagnose the problem, but since we have already crashed, Jul 17 01:48:18 server mariadbd[3185478]: something is definitely wrong and this may fail. Jul 17 01:48:18 server mariadbd[3185478]: Server version: 10.9.7-MariaDB-1:10.9.7+maria~deb11 source revision: 33fd519ca7318cd85bee56e8f79df4608ead194e Jul 17 01:48:18 server mariadbd[3185478]: key_buffer_size=134217728 Jul 17 01:48:18 server mariadbd[3185478]: read_buffer_size=131072 Jul 17 01:48:18 server mariadbd[3185478]: max_used_connections=19 Jul 17 01:48:18 server mariadbd[3185478]: max_threads=1002 Jul 17 01:48:18 server mariadbd[3185478]: thread_count=19 Jul 17 01:48:18 server mariadbd[3185478]: It is possible that mysqld could use up to Jul 17 01:48:18 server mariadbd[3185478]: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 2337934 K bytes of memory Jul 17 01:48:18 server mariadbd[3185478]: Hope that's ok; if not, decrease some variables in the equation. Jul 17 01:48:18 server mariadbd[3185478]: Thread pointer: 0xffe930000f48 Jul 17 01:48:18 server mariadbd[3185478]: Attempting backtrace. You can use the following information to find out Jul 17 01:48:18 server mariadbd[3185478]: where mysqld died. If you see no messages after this, something went Jul 17 01:48:18 server mariadbd[3185478]: terribly wrong... Jul 17 01:48:18 server mariadbd[3185478]: stack_bottom = 0xffe97a442818 thread_stack 0x49000 Jul 17 01:48:18 server mariadbd[3185478]: /usr/sbin/mariadbd(my_print_stacktrace+0x30)[0xaaaad670c1a0] Jul 17 01:48:18 server mariadbd[3185478]: /usr/sbin/mariadbd(handle_fatal_signal+0x464)[0xaaaad6210534] Jul 17 01:48:18 server mariadbd[3186319]: addr2line: 'linux-vdso.so.1': No such file Jul 17 01:48:18 server mariadbd[3185478]: linux-vdso.so.1(__kernel_rt_sigreturn+0x0)[0xffffb417d7d0] Jul 17 01:48:18 server mariadbd[3185478]: linux/raise.c:51(__GI_raise)[0xffffb37f4eac] Jul 17 01:48:18 server mariadbd[3185478]: stdlib/abort.c:81(__GI_abort)[0xffffb37e1aa0] Jul 17 01:48:18 server mariadbd[3185478]: /usr/sbin/mariadbd(+0x6677d8)[0xaaaad5ec77d8] Jul 17 01:48:18 server mariadbd[3185478]: /usr/sbin/mariadbd(+0xd0c350)[0xaaaad656c350] Jul 17 01:48:18 server mariadbd[3185478]: /usr/sbin/mariadbd(+0xd0e600)[0xaaaad656e600] Jul 17 01:48:18 server mariadbd[3185478]: /usr/sbin/mariadbd(+0xd3ef54)[0xaaaad659ef54] Jul 17 01:48:18 server mariadbd[3185478]: /usr/sbin/mariadbd(+0xd3f2e8)[0xaaaad659f2e8] Jul 17 01:48:18 server mariadbd[3185478]: /usr/sbin/mariadbd(+0xd1ed48)[0xaaaad657ed48] Jul 17 01:48:18 server mariadbd[3185478]: /usr/sbin/mariadbd(+0xc69ba4)[0xaaaad64c9ba4] Jul 17 01:48:18 server mariadbd[3185478]: /usr/sbin/mariadbd(ZN7handler13ha_update_rowEPKhS1+0x348)[0xaaaad621ee28] Jul 17 01:48:18 server mariadbd[3185478]: /usr/sbin/mariadbd(Z12mysql_updateP3THDP10TABLE_LISTR4ListI4ItemES6_PS4_jP8st_orderybPySA+0x1eac)[0xaaaad60b6aa0] Jul 17 01:48:18 server mariadbd[3185478]: /usr/sbin/mariadbd(_Z21mysql_execute_commandP3THDb+0x1b74)[0xaaaad5ff3d68] Jul 17 01:48:18 server mariadbd[3185478]: /usr/sbin/mariadbd(_Z11mysql_parseP3THDPcjP12Parser_state+0x1c4)[0xaaaad5ff73b4] Jul 17 01:48:18 server mariadbd[3185478]: /usr/sbin/mariadbd(_Z16dispatch_command19enum_server_commandP3THDPcjb+0xee4)[0xaaaad5ff9284] Jul 17 01:48:18 server mariadbd[3185478]: /usr/sbin/mariadbd(_Z10do_commandP3THDb+0x110)[0xaaaad5ffad20] Jul 17 01:48:18 server mariadbd[3185478]: /usr/sbin/mariadbd(_Z24do_handle_one_connectionP7CONNECTb+0x388)[0xaaaad60f63a8] Jul 17 01:48:18 server mariadbd[3185478]: /usr/sbin/mariadbd(handle_one_connection+0x60)[0xaaaad60f6794] Jul 17 01:48:18 server mariadbd[3185478]: /usr/sbin/mariadbd(+0xbc058c)[0xaaaad642058c] Jul 17 01:48:18 server mariadbd[3185478]: nptl/pthread_create.c:477(start_thread)[0xffffb3bd5648] Jul 17 01:48:18 server mariadbd[3185478]: aarch64/clone.S:81(thread_start)[0xffffb3892fdc] Jul 17 01:48:18 server mariadbd[3185478]: Trying to get some variables. Jul 17 01:48:18 server mariadbd[3185478]: Some pointers may be invalid and cause the dump to abort. Jul 17 01:48:18 server mariadbd[3185478]: Query (0xffe930010990): UPDATE `wp_postmeta` SET `meta_value` = 'a:6:{s:5:\"width\";i:1155;s:6:\"height\";i:1155;s:4:\"file\";s:20:\"2016/07/image-44.png\";s:8:\"filesize\";i:336148;s:5:\"sizes\";a:0:{}s:10:\"image_meta\";a:12:{s:8:\"aperture\";s:1:\"0\";s:6:\"credit\";s:0:\"\";s:6:\"camera\";s:0:\"\";s:7:\"caption\";s:0:\"\";s:17:\"created_timestamp\";s:1:\"0\";s:9:\"copyright\";s:0:\"\";s:12:\"focal_length\";s:1:\"0\";s:3:\"iso\";s:1:\"0\";s:13:\"shutter_speed\";s:1:\"0\";s:5:\"title\";s:0:\"\";s:11:\"orientation\";s:1:\"0\";s:8:\"keywords\";a:0:{}}}' WHERE `post_id` = 516 AND `meta_key` = '_wp_attachment_metadata' Jul 17 01:48:18 server mariadbd[3185478]: Connection ID (thread ID): 1236 Jul 17 01:48:18 server mariadbd[3185478]: Status: NOT_KILLED Jul 17 01:48:18 server mariadbd[3185478]: Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=on,condition_pushdown_from_having=on,not_null_range_scan=off,hash_join_cardinality=off Jul 17 01:48:18 server mariadbd[3185478]: The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains Jul 17 01:48:18 server mariadbd[3185478]: information that should help you find out what is causing the crash. Jul 17 01:48:18 server mariadbd[3185478]: Writing a core file... Jul 17 01:48:18 server mariadbd[3185478]: Working directory at /mnt/home/system/mysql Jul 17 01:48:18 server mariadbd[3185478]: Resource Limits: Jul 17 01:48:18 server mariadbd[3185478]: Limit Soft Limit Hard Limit Units Jul 17 01:48:18 server mariadbd[3185478]: Max cpu time unlimited unlimited seconds Jul 17 01:48:18 server mariadbd[3185478]: Max file size unlimited unlimited bytes Jul 17 01:48:18 server mariadbd[3185478]: Max data size unlimited unlimited bytes Jul 17 01:48:18 server mariadbd[3185478]: Max stack size 8388608 unlimited bytes Jul 17 01:48:18 server mariadbd[3185478]: Max core file size 0 unlimited bytes Jul 17 01:48:18 server mariadbd[3185478]: Max resident set unlimited unlimited bytes Jul 17 01:48:18 server mariadbd[3185478]: Max processes 508832 508832 processes Jul 17 01:48:18 server mariadbd[3185478]: Max open files 1048576 1048576 files Jul 17 01:48:18 server mariadbd[3185478]: Max locked memory 524288 524288 bytes Jul 17 01:48:18 server mariadbd[3185478]: Max address space unlimited unlimited bytes Jul 17 01:48:18 server mariadbd[3185478]: Max file locks unlimited unlimited locks Jul 17 01:48:18 server mariadbd[3185478]: Max pending signals 508832 508832 signals Jul 17 01:48:18 server mariadbd[3185478]: Max msgqueue size 819200 819200 bytes Jul 17 01:48:18 server mariadbd[3185478]: Max nice priority 0 0 Jul 17 01:48:18 server mariadbd[3185478]: Max realtime priority 0 0 Jul 17 01:48:18 server mariadbd[3185478]: Max realtime timeout unlimited unlimited us Jul 17 01:48:18 server mariadbd[3185478]: Core pattern: core Jul 17 01:48:18 server mariadbd[3185478]: Kernel version: Linux version 5.10.0-14-cloud-arm64 (debian-kernel@lists.debian.org) (gcc-10 (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2) #1 SMP Debian 5.10.113-1 (2022-04-29) Jul 17 01:48:18 server systemd[1]: mariadb.service: Main process exited, code=killed, status=6/ABRT Jul 17 01:48:18 server systemd[1]: mariadb.service: Failed with result 'signal'. Jul 17 01:48:18 server systemd[1]: mariadb.service: Consumed 4min 59.196s CPU time. Jul 17 01:48:23 server systemd[1]: mariadb.service: Scheduled restart job, restart counter is at 28. Jul 17 01:48:23 server systemd[1]: Stopped MariaDB 10.9.7 database server. Jul 17 01:48:23 server systemd[1]: mariadb.service: Consumed 4min 59.196s CPU time. Jul 17 01:48:23 server systemd[1]: Starting MariaDB 10.9.7 database server... Jul 17 01:48:23 server mariadbd[3186420]: 2023-07-17 1:48:23 0 [Warning] Could not increase number of max_open_files to more than 1048576 (request: 4294967295) Jul 17 01:48:23 server mariadbd[3186420]: 2023-07-17 1:48:23 0 [Note] Starting MariaDB 10.9.7-MariaDB-1:10.9.7+maria~deb11 source revision 33fd519ca7318cd85bee56e8f79df4608ead194e as process 3186420 Jul 17 01:48:23 server mariadbd[3186420]: 2023-07-17 1:48:23 0 [Note] InnoDB: Compressed tables use zlib 1.2.11 Jul 17 01:48:23 server mariadbd[3186420]: 2023-07-17 1:48:23 0 [Note] InnoDB: Number of transaction pools: 1 Jul 17 01:48:23 server mariadbd[3186420]: 2023-07-17 1:48:23 0 [Note] InnoDB: Using ARMv8 crc32 + pmull instructions Jul 17 01:48:23 server mariadbd[3186420]: 2023-07-17 1:48:23 0 [Note] InnoDB: Using liburing Jul 17 01:48:23 server mariadbd[3186420]: 2023-07-17 1:48:23 0 [Note] InnoDB: Initializing buffer pool, total size = 86.000GiB, chunk size = 1.344GiB Jul 17 01:48:23 server mariadbd[3186420]: 2023-07-17 1:48:23 0 [Note] InnoDB: Completed initialization of buffer pool Jul 17 01:48:24 server mariadbd[3186420]: 2023-07-17 1:48:24 0 [Note] InnoDB: File system buffers for log disabled (block size=512 bytes) Jul 17 01:48:24 server mariadbd[3186420]: 2023-07-17 1:48:24 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=4075773707138 Jul 17 01:48:32 server dhclient[664]: XMT: Solicit on ens5, interval 124210ms. Jul 17 01:48:38 server mariadbd[3186420]: 2023-07-17 1:48:38 0 [Note] InnoDB: Read redo log up to LSN=4087187758714 Jul 17 01:48:47 server mariadbd[3186420]: 2023-07-17 1:48:47 0 [Note] InnoDB: 1 transaction(s) which must be rolled back or cleaned up in total 1 row operations to undo Jul 17 01:48:47 server mariadbd[3186420]: 2023-07-17 1:48:47 0 [Note] InnoDB: Trx id counter is 974819294 Jul 17 01:48:47 server mariadbd[3186420]: 2023-07-17 1:48:47 0 [Note] InnoDB: Starting final batch to recover 213044 pages from redo log. Jul 17 01:48:50 server mariadbd[3186420]: 2023-07-17 1:48:50 0 [Note] InnoDB: 128 rollback segments are active. Jul 17 01:48:50 server mariadbd[3186420]: 2023-07-17 1:48:50 0 [Note] InnoDB: Starting in background the rollback of recovered transactions Jul 17 01:48:50 server mariadbd[3186420]: 2023-07-17 1:48:50 0 [Warning] InnoDB: Record in index `meta_key` of table `saleone4`.`wp_postmeta` was not found on rollback, trying to insert: TUPLE (info_bits=0, 4 fields): {[23]_wp_attachment_metadata(0x5F77705F6174746163686D656E745F6D65746164617461),[32] (0x0000000000000000000000000000000000000000000000000000000000000000),[8] (0x0000000000000204),[8] (0x0000000000001B01)} at: COMPACT RECORD(info_bits=0, 4 fields): {[24]_wp_attachment_image_alt(0x5F77705F6174746163686D656E745F696D6167655F616C74),[32]Zero Lucks Given St. Patrick's D(0x5A65726F204C75636B7320476976656E2053742E205061747269636B27732044),[8] (0x000000000004E2C0),[8] .(0x000000000082D02E)}Jul 17 01:48:50 server mariadbd[3186420]: 2023-07-17 1:48:50 0 [Note] InnoDB: Removed temporary tablespace data file: "./ibtmp1" at: COMPACT RECORD(info_bits=0, 4 fields): {[0](0x),[8] f(0x0000000000001966),[8] (0x0000000000000200),[19]_downloadable_files(0x5F646F776E6C6F616461626C655F66696C6573)}Jul 17 01:48:50 server mariadbd[3186420]: 2023-07-17 1:48:50 0 [Note] InnoDB: Rolled back recovered transaction 974819263 |