Details
-
Bug
-
Status: Open (View Workflow)
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
-
None
Description
Dec 10 00:15:46 main mariadbd[1435]: 2022-12-10 0:15:46 0 [ERROR] RocksDB: Error detected in background, Status Code: 5, Status: IO error: No space left on deviceWhile appending to file: ./#rocksdb/581104.sst: No space left on device
|
Dec 10 00:15:46 main mariadbd[1435]: 2022-12-10 0:15:46 0 [ERROR] RocksDB: BackgroundErrorReason: 0
|
Dec 10 00:15:46 main mariadbd[1435]: 2022-12-10 0:15:46 0 [ERROR] LibRocksDB:[db_impl/db_impl_compaction_flush.cc:2249] Waiting after background flush error: IO error: No space left on deviceWhile appending to file: ./#rocksdb/581104.sst: No space left on deviceAccumulated background error counts: 1
|
Dec 10 00:15:46 main mariadbd[1435]: 2022-12-10 0:15:46 1156551 [ERROR] RocksDB: Error detected in background, Status Code: 5, Status: IO error: No space left on deviceWhile appending to file: ./#rocksdb/581104.sst: No space left on device
|
Dec 10 00:15:46 main mariadbd[1435]: 2022-12-10 0:15:46 1156551 [ERROR] RocksDB: BackgroundErrorReason: 2
|
Dec 10 00:15:46 main mariadbd[1435]: 2022-12-10 0:15:46 1156551 [ERROR] RocksDB: failed to write to WAL, Status Code: 5, Status: IO error: No space left on deviceWhile appending to file: ./#rocksdb/581104.sst: No space left on device
|
Dec 10 00:15:46 main mariadbd[1435]: 2022-12-10 0:15:46 1156551 [ERROR] MyRocks: aborting on WAL write error.
|
Dec 10 00:15:46 main mariadbd[1435]: 221210 0:15:46 [ERROR] mysqld got signal 6 ;
|
Dec 10 00:15:46 main mariadbd[1435]: This could be because you hit a bug. It is also possible that this binary
|
Dec 10 00:15:46 main mariadbd[1435]: or one of the libraries it was linked against is corrupt, improperly built,
|
Dec 10 00:15:46 main mariadbd[1435]: or misconfigured. This error can also be caused by malfunctioning hardware.
|
Dec 10 00:15:46 main mariadbd[1435]: To report this bug, see https://mariadb.com/kb/en/reporting-bugs
|
Dec 10 00:15:46 main mariadbd[1435]: We will try our best to scrape up some info that will hopefully help
|
Dec 10 00:15:46 main mariadbd[1435]: diagnose the problem, but since we have already crashed,
|
Dec 10 00:15:46 main mariadbd[1435]: something is definitely wrong and this may fail.
|
Dec 10 00:15:46 main mariadbd[1435]: Server version: 10.7.7-MariaDB-1:10.7.7+maria~deb11-log
|
Dec 10 00:15:46 main mariadbd[1435]: key_buffer_size=33554432
|
Dec 10 00:15:46 main mariadbd[1435]: read_buffer_size=131072
|
Dec 10 00:15:46 main mariadbd[1435]: max_used_connections=27
|
Dec 10 00:15:46 main mariadbd[1435]: max_threads=102
|
Dec 10 00:15:46 main mariadbd[1435]: thread_count=27
|
Dec 10 00:15:46 main mariadbd[1435]: It is possible that mysqld could use up to
|
Dec 10 00:15:46 main mariadbd[1435]: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 257379 K bytes of memory
|
Dec 10 00:15:46 main mariadbd[1435]: Hope that's ok; if not, decrease some variables in the equation.
|
Dec 10 00:15:46 main mariadbd[1435]: Thread pointer: 0x7febb4000c58
|
Dec 10 00:15:46 main mariadbd[1435]: Attempting backtrace. You can use the following information to find out
|
Dec 10 00:15:46 main mariadbd[1435]: where mysqld died. If you see no messages after this, something went
|
Dec 10 00:15:46 main mariadbd[1435]: terribly wrong...
|
Dec 10 00:15:46 main mariadbd[1435]: stack_bottom = 0x7ff4a00d8d18 thread_stack 0x49000
|
Dec 10 00:15:46 main mariadbd[1435]: ??:0(my_print_stacktrace)[0x55c48e11dc9e]
|
Dec 10 00:15:46 main mariadbd[1435]: ??:0(handle_fatal_signal)[0x55c48dc04145]
|
Dec 10 00:15:46 main mariadbd[1435]: ??:0(__restore_rt)[0x7ff4c2a47140]
|
Dec 10 00:15:46 main mariadbd[1435]: ??:0(gsignal)[0x7ff4c257ece1]
|
Dec 10 00:15:46 main mariadbd[1435]: ??:0(abort)[0x7ff4c2568537]
|
Dec 10 00:15:46 main mariadbd[1435]: ??:0(myrocks::rdb_handle_io_error(rocksdb::Status, myrocks::RDB_IO_ERROR_TYPE))[0x7ff4bbb02d8d]
|
Dec 10 00:15:46 main mariadbd[1435]: ??:0(myrocks::Rdb_transaction_impl::commit_no_binlog())[0x7ff4bbb2d812]
|
Dec 10 00:15:46 main mariadbd[1435]: ??:0(myrocks::timeout_message(char const*, char const*, char const*))[0x7ff4bbaffd68]
|
Dec 10 00:15:46 main mariadbd[1435]: ??:0(ha_check_and_coalesce_trx_read_only(THD*, Ha_trx_info*, bool))[0x55c48dc07a29]
|
Dec 10 00:15:46 main mariadbd[1435]: ??:0(ha_commit_trans(THD*, bool))[0x55c48dc14b36]
|
Dec 10 00:15:46 main mariadbd[1435]: ??:0(trans_commit(THD*))[0x55c48daf7183]
|
Dec 10 00:15:46 main mariadbd[1435]: ??:0(mysql_execute_command(THD*, bool))[0x55c48d9c535d]
|
Dec 10 00:15:46 main mariadbd[1435]: ??:0(mysql_parse(THD*, char*, unsigned int, Parser_state*))[0x55c48d9c8197]
|
Dec 10 00:15:46 main mariadbd[1435]: ??:0(dispatch_command(enum_server_command, THD*, char*, unsigned int, bool))[0x55c48d9caa5d]
|
Dec 10 00:15:46 main mariadbd[1435]: ??:0(do_command(THD*, bool))[0x55c48d9cc5c8]
|
Dec 10 00:15:46 main mariadbd[1435]: ??:0(do_handle_one_connection(CONNECT*, bool))[0x55c48dae71ef]
|
Dec 10 00:15:46 main mariadbd[1435]: ??:0(handle_one_connection)[0x55c48dae753d]
|
Dec 10 00:15:46 main mariadbd[1435]: ??:0(MyCTX_nopad::finish(unsigned char*, unsigned int*))[0x55c48de24c22]
|
Dec 10 00:15:46 main mariadbd[1435]: ??:0(start_thread)[0x7ff4c2a3bea7]
|
Dec 10 00:15:46 main mariadbd[1435]: ??:0(clone)[0x7ff4c2642a2f]
|
Dec 10 00:15:46 main mariadbd[1435]: Trying to get some variables.
|
Dec 10 00:15:46 main mariadbd[1435]: Some pointers may be invalid and cause the dump to abort.
|
Dec 10 00:15:46 main mariadbd[1435]: Query (0x7febb4010920): COMMIT
|
Dec 10 00:15:46 main mariadbd[1435]: Connection ID (thread ID): 1156551
|
Dec 10 00:15:46 main mariadbd[1435]: Status: NOT_KILLED
|
Dec 10 00:15:46 main mariadbd[1435]: Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=on,condition_pushdown_from_having=on,not_null_range_scan=off
|
Dec 10 00:15:46 main mariadbd[1435]: The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains
|
Dec 10 00:15:46 main mariadbd[1435]: information that should help you find out what is causing the crash.
|
Dec 10 00:15:46 main mariadbd[1435]: Writing a core file...
|
Dec 10 00:15:46 main mariadbd[1435]: Working directory at /var/lib/mysql
|
Dec 10 00:15:46 main mariadbd[1435]: Resource Limits:
|
Dec 10 00:15:46 main mariadbd[1435]: Limit Soft Limit Hard Limit Units
|
Dec 10 00:15:46 main mariadbd[1435]: Max cpu time unlimited unlimited seconds
|
Dec 10 00:15:46 main mariadbd[1435]: Max file size unlimited unlimited bytes
|
Dec 10 00:15:46 main mariadbd[1435]: Max data size unlimited unlimited bytes
|
Dec 10 00:15:46 main mariadbd[1435]: Max stack size 8388608 unlimited bytes
|
Dec 10 00:15:46 main mariadbd[1435]: Max core file size 0 unlimited bytes
|
Dec 10 00:15:46 main mariadbd[1435]: Max resident set unlimited unlimited bytes
|
Dec 10 00:15:46 main mariadbd[1435]: Max processes 514767 514767 processes
|
Dec 10 00:15:46 main mariadbd[1435]: Max open files 32768 32768 files
|
Dec 10 00:15:46 main mariadbd[1435]: Max locked memory 524288 524288 bytes
|
Dec 10 00:15:46 main mariadbd[1435]: Max address space unlimited unlimited bytes
|
Dec 10 00:15:46 main mariadbd[1435]: Max file locks unlimited unlimited locks
|
Dec 10 00:15:46 main mariadbd[1435]: Max pending signals 514767 514767 signals
|
Dec 10 00:15:46 main mariadbd[1435]: Max msgqueue size 819200 819200 bytes
|
Dec 10 00:15:46 main mariadbd[1435]: Max nice priority 0 0
|
Dec 10 00:15:46 main mariadbd[1435]: Max realtime priority 0 0
|
Dec 10 00:15:46 main mariadbd[1435]: Max realtime timeout unlimited unlimited us
|
Dec 10 00:15:46 main mariadbd[1435]: Core pattern: core
|
Dec 10 00:15:46 main mariadbd[1435]: Kernel version: Linux version 5.10.0-19-amd64 (debian-kernel@lists.debian.org) (gcc-10 (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2) #1 SMP Debian 5.10.149-2 (2022-10-21)
|
Dec 10 00:15:51 main mariadbd[3482145]: 2022-12-10 0:15:51 0 [Note] /usr/sbin/mariadbd (server 10.7.7-MariaDB-1:10.7.7+maria~deb11-log) starting as process 3482145 ...
|
Dec 10 00:15:51 main mariadbd[3482145]: 2022-12-10 0:15:51 0 [Note] RocksDB: 2 column families found
|
Dec 10 00:15:51 main mariadbd[3482145]: 2022-12-10 0:15:51 0 [Note] RocksDB: Column Families at start:
|
Dec 10 00:15:51 main mariadbd[3482145]: 2022-12-10 0:15:51 0 [Note] cf=default
|
Dec 10 00:15:51 main mariadbd[3482145]: 2022-12-10 0:15:51 0 [Note] write_buffer_size=67108864
|
Dec 10 00:15:51 main mariadbd[3482145]: 2022-12-10 0:15:51 0 [Note] target_file_size_base=33554432
|
Dec 10 00:15:51 main mariadbd[3482145]: 2022-12-10 0:15:51 0 [Note] cf=__system__
|
Dec 10 00:15:51 main mariadbd[3482145]: 2022-12-10 0:15:51 0 [Note] write_buffer_size=67108864
|
Dec 10 00:15:51 main mariadbd[3482145]: 2022-12-10 0:15:51 0 [Note] target_file_size_base=67108864
|
Dec 10 00:15:52 main mariadbd[3482145]: 2022-12-10 0:15:52 0 [ERROR] RocksDB: Error opening instance, Status Code: 5, Status: IO error: No space left on deviceWhile appending to file: ./#rocksdb/581104.sst: No space left on device
|
Dec 10 00:15:52 main mariadbd[3482145]: 2022-12-10 0:15:52 0 [ERROR] Plugin 'ROCKSDB' init function returned error.
|
Dec 10 00:15:52 main mariadbd[3482145]: 2022-12-10 0:15:52 0 [ERROR] Plugin 'ROCKSDB' registration as a STORAGE ENGINE failed.
|
Dec 10 00:15:52 main mariadbd[3482145]: 2022-12-10 0:15:52 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
|
Dec 10 00:15:52 main mariadbd[3482145]: 2022-12-10 0:15:52 0 [Note] InnoDB: Number of transaction pools: 1
|
Dec 10 00:15:52 main mariadbd[3482145]: 2022-12-10 0:15:52 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
|
Dec 10 00:15:52 main mariadbd[3482145]: 2022-12-10 0:15:52 0 [Note] InnoDB: Using liburing
|
Dec 10 00:15:52 main mariadbd[3482145]: 2022-12-10 0:15:52 0 [Note] InnoDB: Initializing buffer pool, total size = 34359738368, chunk size = 134217728
|
Dec 10 00:15:52 main mariadbd[3482145]: 2022-12-10 0:15:52 0 [Note] InnoDB: Completed initialization of buffer pool
|
Dec 10 00:15:52 main mariadbd[3482145]: 2022-12-10 0:15:52 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=201143428594,201162725253
|
Dec 10 00:16:04 main mariadbd[3482145]: 2022-12-10 0:16:04 0 [Note] InnoDB: Starting final batch to recover 34907 pages from redo log.
|
Dec 10 00:16:07 main mariadbd[3482145]: 2022-12-10 0:16:07 0 [Note] InnoDB: To recover: 7866 pages from log
|
Dec 10 00:16:07 main mariadbd[3482145]: 2022-12-10 0:16:07 0 [Note] InnoDB: 128 rollback segments are active.
|
Dec 10 00:16:07 main mariadbd[3482145]: 2022-12-10 0:16:07 0 [Note] InnoDB: Removed temporary tablespace data file: "./ibtmp1"
|
Dec 10 00:16:07 main mariadbd[3482145]: 2022-12-10 0:16:07 0 [Note] InnoDB: Creating shared tablespace for temporary tables
|
Dec 10 00:16:07 main mariadbd[3482145]: 2022-12-10 0:16:07 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
|
Dec 10 00:16:07 main mariadbd[3482145]: 2022-12-10 0:16:07 0 [ERROR] InnoDB: preallocating 12582912 bytes for file ./ibtmp1 failed with error 28
|
Dec 10 00:16:07 main mariadbd[3482145]: 2022-12-10 0:16:07 0 [ERROR] InnoDB: Could not set the file size of './ibtmp1'. Probably out of disk space
|
Dec 10 00:16:07 main mariadbd[3482145]: 2022-12-10 0:16:07 0 [ERROR] InnoDB: Unable to create the shared innodb_temporary
|
Dec 10 00:16:07 main mariadbd[3482145]: 2022-12-10 0:16:07 0 [ERROR] InnoDB: Plugin initialization aborted with error Generic error
|
Dec 10 00:16:07 main mariadbd[3482145]: 2022-12-10 0:16:07 0 [Note] InnoDB: FTS optimize thread exiting.
|
Dec 10 00:16:07 main mariadbd[3482145]: 2022-12-10 0:16:07 0 [Note] InnoDB: Starting shutdown...
|
Dec 10 00:16:08 main mariadbd[3482145]: 2022-12-10 0:16:08 0 [Note] InnoDB: Removed temporary tablespace data file: "./ibtmp1"
|
Dec 10 00:16:08 main mariadbd[3482145]: 2022-12-10 0:16:08 0 [ERROR] Plugin 'InnoDB' init function returned error.
|
Dec 10 00:16:08 main mariadbd[3482145]: 2022-12-10 0:16:08 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
|
Dec 10 00:16:08 main mariadbd[3482145]: 2022-12-10 0:16:08 0 [Note] Plugin 'FEEDBACK' is disabled.
|
Dec 10 00:16:08 main mariadbd[3482145]: 2022-12-10 0:16:08 0 [ERROR] Unknown/unsupported storage engine: InnoDB
|
Dec 10 00:16:08 main mariadbd[3482145]: 2022-12-10 0:16:08 0 [ERROR] Aborting
|
After restart MariaDB
Dec 10 10:48:59 main mariadbd[3495662]: 2022-12-10 10:48:59 0 [Note] /usr/sbin/mariadbd (server 10.7.7-MariaDB-1:10.7.7+maria~deb11-log) starting as process 3495662 ...
|
Dec 10 10:48:59 main mariadbd[3495662]: 2022-12-10 10:48:59 0 [Note] RocksDB: 2 column families found
|
Dec 10 10:48:59 main mariadbd[3495662]: 2022-12-10 10:48:59 0 [Note] RocksDB: Column Families at start:
|
Dec 10 10:48:59 main mariadbd[3495662]: 2022-12-10 10:48:59 0 [Note] cf=default
|
Dec 10 10:48:59 main mariadbd[3495662]: 2022-12-10 10:48:59 0 [Note] write_buffer_size=67108864
|
Dec 10 10:48:59 main mariadbd[3495662]: 2022-12-10 10:48:59 0 [Note] target_file_size_base=33554432
|
Dec 10 10:48:59 main mariadbd[3495662]: 2022-12-10 10:48:59 0 [Note] cf=__system__
|
Dec 10 10:48:59 main mariadbd[3495662]: 2022-12-10 10:48:59 0 [Note] write_buffer_size=67108864
|
Dec 10 10:48:59 main mariadbd[3495662]: 2022-12-10 10:48:59 0 [Note] target_file_size_base=67108864
|
Dec 10 10:49:00 main mariadbd[3495662]: 2022-12-10 10:49:00 0 [Note] RocksDB: Table_store: loaded DDL data for 3 tables
|
Dec 10 10:49:00 main mariadbd[3495662]: 2022-12-10 10:49:00 0 [Note] RocksDB: global statistics using get_sched_indexer_t indexer
|
Dec 10 10:49:00 main mariadbd[3495662]: 2022-12-10 10:49:00 0 [Note] MyRocks storage engine plugin has been successfully initialized.
|
Dec 10 10:49:00 main mariadbd[3495662]: 2022-12-10 10:49:00 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
|
Dec 10 10:49:00 main mariadbd[3495662]: 2022-12-10 10:49:00 0 [Note] InnoDB: Number of transaction pools: 1
|
Dec 10 10:49:00 main mariadbd[3495662]: 2022-12-10 10:49:00 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
|
Dec 10 10:49:00 main mariadbd[3495662]: 2022-12-10 10:49:00 0 [Note] InnoDB: Using liburing
|
Dec 10 10:49:00 main mariadbd[3495662]: 2022-12-10 10:49:00 0 [Note] InnoDB: Initializing buffer pool, total size = 34359738368, chunk size = 134217728
|
Dec 10 10:49:00 main mariadbd[3495662]: 2022-12-10 10:49:00 0 [Note] InnoDB: Completed initialization of buffer pool
|
Dec 10 10:49:00 main mariadbd[3495662]: 2022-12-10 10:49:00 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=201143428594,201143428594
|
Dec 10 10:49:13 main mariadbd[3495662]: 2022-12-10 10:49:13 0 [Note] InnoDB: Starting final batch to recover 34907 pages from redo log.
|
Dec 10 10:49:15 main mariadbd[3495662]: 2022-12-10 10:49:15 0 [Note] InnoDB: To recover: 12912 pages from log
|
Dec 10 10:49:16 main mariadbd[3495662]: 2022-12-10 10:49:16 0 [Note] InnoDB: 128 rollback segments are active.
|
Dec 10 10:49:16 main mariadbd[3495662]: 2022-12-10 10:49:16 0 [Note] InnoDB: Creating shared tablespace for temporary tables
|
Dec 10 10:49:16 main mariadbd[3495662]: 2022-12-10 10:49:16 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
|
Dec 10 10:49:16 main mariadbd[3495662]: 2022-12-10 10:49:16 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
|
Dec 10 10:49:16 main mariadbd[3495662]: 2022-12-10 10:49:16 0 [Note] InnoDB: 10.7.7 started; log sequence number 202450436185; transaction id 554514390
|
Dec 10 10:49:16 main mariadbd[3495662]: 2022-12-10 10:49:16 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
|
Dec 10 10:49:16 main mariadbd[3495662]: 2022-12-10 10:49:16 0 [Note] Plugin 'FEEDBACK' is disabled.
|
Dec 10 10:49:16 main mariadbd[3495662]: 2022-12-10 10:49:16 0 [Note] Recovering after a crash using tc.log
|
Dec 10 10:49:16 main mariadbd[3495662]: 2022-12-10 10:49:16 0 [Note] Starting table crash recovery...
|
Dec 10 10:49:16 main mariadbd[3495662]: 2022-12-10 10:49:16 0 [Note] Crash table recovery finished.
|
Dec 10 10:49:16 main mariadbd[3495662]: 2022-12-10 10:49:16 0 [Note] Server socket created on IP: '127.0.0.1'.
|
Dec 10 10:49:16 main mariadbd[3495662]: 2022-12-10 10:49:16 0 [Note] /usr/sbin/mariadbd: ready for connections.
|
Dec 10 10:49:16 main mariadbd[3495662]: Version: '10.7.7-MariaDB-1:10.7.7+maria~deb11-log' socket: '/run/mysqld/mysqld.sock' port: 3306 mariadb.org binary distribution
|
Dec 10 10:49:18 main mariadbd[3495662]: 2022-12-10 10:49:18 0 [Note] InnoDB: Buffer pool(s) load completed at 221210 10:49:18
|