[MDEV-15625] Mariabackup cannot prepare backup Created: 2018-03-21  Updated: 2018-07-24  Resolved: 2018-03-21

Status: Closed
Project: MariaDB Server
Component/s: Backup
Affects Version/s: 10.2.13
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: Justin Desilets Assignee: Vladislav Vaintroub
Resolution: Duplicate Votes: 0
Labels: innodb

Issue Links:
Duplicate
duplicates MDEV-14545 Backup fails due to MLOG_INDEX_LOAD r... Closed
Relates
relates to MDEV-16809 Allow full redo logging for ALTER TABLE Closed

 Description   

We recently upgraded our maraidb from 5.5 to 10.2.13. I am able to run mariabackup-10.2.13:

mariabackup --backup --target-dir /raid/backup -u root -p 'xxx' --parallel=8

However when I try and run the prepare option against it I receive the following:

mariabackup --prepare --target-dir /raid/backup --parallel=8
mariabackup based on MariaDB server 10.2.13-MariaDB Linux (x86_64)
mariabackup: cd to /raid/backup/
mariabackup: This target seems to be not prepared yet.
mariabackup: using the following InnoDB configuration for recovery:
mariabackup:   innodb_data_home_dir = .
mariabackup:   innodb_data_file_path = ibdata1:12M:autoextend
mariabackup:   innodb_log_group_home_dir = .
mariabackup: Starting InnoDB instance for recovery.
mariabackup: Using 104857600 bytes for buffer pool (set by --use-memory parameter)
2018-03-21 14:31:58 140691870587008 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2018-03-21 14:31:58 140691870587008 [Note] InnoDB: Uses event mutexes
2018-03-21 14:31:58 140691870587008 [Note] InnoDB: Compressed tables use zlib 1.2.7
2018-03-21 14:31:58 140691870587008 [Note] InnoDB: Number of pools: 1
2018-03-21 14:31:58 140691870587008 [Note] InnoDB: Using SSE2 crc32 instructions
2018-03-21 14:31:58 140691870587008 [Note] InnoDB: Initializing buffer pool, total size = 100M, instances = 1, chunk size = 100M
2018-03-21 14:31:58 140691870587008 [Note] InnoDB: Completed initialization of buffer pool
2018-03-21 14:31:58 140691384563456 [Note] InnoDB: page_cleaner coordinator priority: -20
2018-03-21 14:31:58 140691870587008 [Note] InnoDB: Highest supported file format is Barracuda.
2018-03-21 14:31:58 140691870587008 [Note] InnoDB: Starting crash recovery from checkpoint LSN=1709592421615
2018-03-21 14:31:58 140691870587008 [Note] InnoDB: Ignoring data file 'skms/session.ibd' with space ID 2670, since the redo log references skms/session.ibd with space ID 2668.
2018-03-21 14:31:58 140691870587008 [Note] InnoDB: Ignoring data file './skms/#sql-ib2689-2006477425.ibd' with space ID 2670. Another data file called skms/session.ibd exists with the same space ID.
2018-03-21 14:31:58 140691870587008 [Note] InnoDB: Ignoring data file './skms/#sql-ib2689-2006477425.ibd' with space ID 2670. Another data file called skms/session.ibd exists with the same space ID.
2018-03-21 14:31:58 140691870587008 [Note] InnoDB: Ignoring data file './skms/#sql-ib2689-2006477425.ibd' with space ID 2670. Another data file called skms/session.ibd exists with the same space ID.
2018-03-21 14:31:58 140691870587008 [Note] InnoDB: Ignoring data file './skms/#sql-ib2689-2006477425.ibd' with space ID 2670. Another data file called skms/session.ibd exists with the same space ID.
2018-03-21 14:31:58 0x7ff560f81880  InnoDB: Assertion failure in file /home/buildbot/buildbot/padding_for_CPACK_RPM_BUILD_SOURCE_DIRS_PREFIX/mariadb-10.2.13/storage/innobase/log/log0recv.cc line 2467
InnoDB: Failing assertion: type != MLOG_INDEX_LOAD || srv_operation == SRV_OPERATION_NORMAL
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to https://jira.mariadb.org/
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: https://mariadb.com/kb/en/library/xtradbinnodb-recovery-modes/
InnoDB: about forcing recovery.
180321 14:31:58 [ERROR] mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
 
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
 
Server version: 10.2.13-MariaDB
key_buffer_size=0
read_buffer_size=131072
max_used_connections=0
max_threads=1
thread_count=0
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 5421 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
 
Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x0 thread_stack 0x49000
addr2line: 'mariabackup': No such file
mariabackup(my_print_stacktrace+0x2e)[0x555e0dfb54de]
mariabackup(handle_fatal_signal+0x355)[0x555e0daa0335]
sigaction.c:0(__restore_rt)[0x7ff560b9a5e0]
:0(__GI_raise)[0x7ff55f0a71f7]
:0(__GI_abort)[0x7ff55f0a88e8]
addr2line: 'mariabackup': No such file
mariabackup(+0x434758)[0x555e0d7d0758]
mariabackup(+0x99aacc)[0x555e0dd36acc]
mariabackup(+0x99af04)[0x555e0dd36f04]
mariabackup(+0x99c091)[0x555e0dd38091]
mariabackup(+0xa4b840)[0x555e0dde7840]
mariabackup(+0x457ce7)[0x555e0d7f3ce7]
mariabackup(main+0x185)[0x555e0d7d2795]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7ff55f093c05]
addr2line: 'mariabackup': No such file
mariabackup(+0x44fd6d)[0x555e0d7ebd6d]
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
Aborted (core dumped)

Earlier in the day I was able to run through this exact same process and things went fine. Now however this core dump is repeatable. I am attempting to run the backup process on a slave node, but I have seen the same behavior on our master node as well. Server is a brand new, clean build of CentOS 7.4. Please let me know what additional information I can gather to help resolve this problem.



 Comments   
Comment by Vladislav Vaintroub [ 2018-03-21 ]

Please run backup with --ddl-lock-per-table option, and also wait for the next version where MDEV-14545 will be fixed.

Generated at Thu Feb 08 08:22:48 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.