Details
-
Bug
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Won't Fix
-
10.1.11
-
Centos 7
Description
After SST, the receiver node is shutdown but never restarted
Is there a workaround and where exactly is that shutdown done a quick inspection of xtrabackup SST script does not look to trigger shutdown ?
ON DONOR (innobackup.backup.log) :
160204 11:53:07 innobackupex: Starting the backup operation
|
|
IMPORTANT: Please check that the backup run completes successfully.
|
At the end of a successful backup run innobackupex
|
prints "completed OK!".
|
|
160204 11:53:07 Connecting to MySQL server host: localhost, user: galerabkp, password: not set, port: 0, socket: /var/lib/mysql/mysql.sock
|
Using server version 10.1.11-MariaDB-log
|
innobackupex version 2.3.3 based on MySQL server 5.6.24 Linux (x86_64) (revision id: 525ca7d)
|
xtrabackup: uses posix_fadvise().
|
xtrabackup: cd to /home/mysql
|
xtrabackup: open files limit requested 0, set to 16364
|
xtrabackup: using the following InnoDB configuration:
|
xtrabackup: innodb_data_home_dir = ./
|
xtrabackup: innodb_data_file_path = ibdata1:10M:autoextend
|
xtrabackup: innodb_log_group_home_dir = ./
|
xtrabackup: innodb_log_files_in_group = 8
|
xtrabackup: innodb_log_file_size = 536870912
|
xtrabackup: using O_DIRECT
|
160204 11:53:07 >> log scanned up to (83463863966)
|
xtrabackup: Generating a list of tablespaces
|
160204 11:53:07 [01] Streaming ./ibdata1
|
160204 11:53:08 [01] ...done
|
160204 11:53:08 >> log scanned up to (83463879153)
|
160204 11:53:08 [01] Streaming ./mysql/innodb_table_stats.ibd
|
160204 11:53:08 [01] ...done
|
160204 11:53:08 [01] Streaming ./mysql/innodb_index_stats.ibd
|
160204 11:53:08 [01] ...done
|
160204 11:53:08 [01] Streaming ./mysql/gtid_slave_pos.ibd
|
160204 11:53:08 [01] ...done
|
160204 11:53:08 [01] Streaming ./db_calameo_v2/tbl_amazon_upload_queues.ibd
|
160204 11:53:08 [01] ...done
|
160204 11:53:08 [01] Streaming ./db_calameo_v2/tbl_annotations.ibd
|
…
|
|
160204 12:04:07 [01] ...done
|
160204 12:04:07 [01] Streaming ./db_calameo_v2/tbl_book_downloads.frm to <STDOUT>
|
160204 12:04:07 [01] ...done
|
160204 12:04:07 Finished backing up non-InnoDB tables and files
|
160204 12:04:07 [00] Streaming xtrabackup_galera_info
|
160204 12:04:07 [00] ...done
|
xtrabackup: The latest check point (for incremental): '83470194891'
|
xtrabackup: Stopping log copying thread.
|
.160204 12:04:07 >> log scanned up to (83470387726)
|
|
160204 12:04:07 Executing UNLOCK TABLES
|
160204 12:04:07 All tables unlocked
|
160204 12:04:07 [00] Streaming ib_buffer_pool to <STDOUT>
|
160204 12:04:07 [00] ...done
|
160204 12:04:07 Backup created in directory '/tmp/tmp.CrTCxlgvHm'
|
160204 12:04:07 [00] Streaming backup-my.cnf
|
160204 12:04:07 [00] ...done
|
160204 12:04:07 [00] Streaming xtrabackup_info
|
160204 12:04:07 [00] ...done
|
xtrabackup: Transaction log of lsn (83455063062) to (83470387726) was copied.
|
160204 12:04:07 completed OK!
|
|
ON JOINER (innobackup.prepare.log) :
|
160204 12:04:07 innobackupex: Starting the apply-log operation
|
|
IMPORTANT: Please check that the apply-log run completes successfully.
|
At the end of a successful apply-log run innobackupex
|
prints "completed OK!".
|
|
innobackupex version 2.3.3 based on MySQL server 5.6.24 Linux (x86_64) (revision id: 525ca7d)
|
xtrabackup: cd to /home/mysql//.sst
|
xtrabackup: This target seems to be not prepared yet.
|
xtrabackup: xtrabackup_logfile detected: size=17252352, start_lsn=(83455063062)
|
xtrabackup: using the following InnoDB configuration for recovery:
|
xtrabackup: innodb_data_home_dir = ./
|
xtrabackup: innodb_data_file_path = ibdata1:10M:autoextend
|
xtrabackup: innodb_log_group_home_dir = ./
|
xtrabackup: innodb_log_files_in_group = 1
|
xtrabackup: innodb_log_file_size = 17252352
|
xtrabackup: using the following InnoDB configuration for recovery:
|
xtrabackup: innodb_data_home_dir = ./
|
xtrabackup: innodb_data_file_path = ibdata1:10M:autoextend
|
xtrabackup: innodb_log_group_home_dir = ./
|
xtrabackup: innodb_log_files_in_group = 1
|
xtrabackup: innodb_log_file_size = 17252352
|
xtrabackup: Starting InnoDB instance for recovery.
|
xtrabackup: Using 104857600 bytes for buffer pool (set by --use-memory parameter)
|
InnoDB: Using atomics to ref count buffer pool pages
|
InnoDB: The InnoDB memory heap is disabled
|
InnoDB: Mutexes and rw_locks use GCC atomic builtins
|
InnoDB: Memory barrier is not used
|
InnoDB: Compressed tables use zlib 1.2.7
|
InnoDB: Using CPU crc32 instructions
|
InnoDB: Initializing buffer pool, size = 100.0M
|
InnoDB: Completed initialization of buffer pool
|
InnoDB: Highest supported file format is Barracuda.
|
InnoDB: Log scan progressed past the checkpoint lsn 83455063062
|
InnoDB: Database was not shutdown normally!
|
InnoDB: Starting crash recovery.
|
InnoDB: Reading tablespace information from the .ibd files...
|
InnoDB: Restoring possible half-written data pages
|
InnoDB: from the doublewrite buffer...
|
InnoDB: Doing recovery: scanned up to log sequence number 83460305920 (34%)
|
InnoDB: Doing recovery: scanned up to log sequence number 83465548800 (68%)
|
InnoDB: Doing recovery: scanned up to log sequence number 83470387726 (99%)
|
InnoDB: Starting an apply batch of log records to the database...
|
InnoDB: Progress in percent: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
|
InnoDB: Apply batch completed
|
InnoDB: 128 rollback segment(s) are active.
|
InnoDB: Waiting for purge to start
|
InnoDB: 5.6.24 started; log sequence number 83470387726
|
|
xtrabackup: starting shutdown with innodb_fast_shutdown = 1
|
InnoDB: FTS optimize thread exiting.
|
InnoDB: Starting shutdown...
|
InnoDB: Shutdown completed; log sequence number 83470390901
|
xtrabackup: using the following InnoDB configuration for recovery:
|
xtrabackup: innodb_data_home_dir = ./
|
xtrabackup: innodb_data_file_path = ibdata1:10M:autoextend
|
xtrabackup: innodb_log_group_home_dir = ./
|
xtrabackup: innodb_log_files_in_group = 8
|
xtrabackup: innodb_log_file_size = 536870912
|
InnoDB: Using atomics to ref count buffer pool pages
|
InnoDB: The InnoDB memory heap is disabled
|
InnoDB: Mutexes and rw_locks use GCC atomic builtins
|
InnoDB: Memory barrier is not used
|
InnoDB: Compressed tables use zlib 1.2.7
|
InnoDB: Using CPU crc32 instructions
|
InnoDB: Initializing buffer pool, size = 100.0M
|
InnoDB: Completed initialization of buffer pool
|
InnoDB: Setting log file ./ib_logfile101 size to 512 MB
|
InnoDB: Progress in MB: 100 200 300 400 500
|
InnoDB: Setting log file ./ib_logfile1 size to 512 MB
|
InnoDB: Progress in MB: 100 200 300 400 500
|
InnoDB: Setting log file ./ib_logfile2 size to 512 MB
|
InnoDB: Progress in MB: 100 200 300 400 500
|
InnoDB: Setting log file ./ib_logfile3 size to 512 MB
|
InnoDB: Progress in MB: 100 200 300 400 500
|
InnoDB: Setting log file ./ib_logfile4 size to 512 MB
|
InnoDB: Progress in MB: 100 200 300 400 500
|
InnoDB: Setting log file ./ib_logfile5 size to 512 MB
|
InnoDB: Progress in MB: 100 200 300 400 500
|
InnoDB: Setting log file ./ib_logfile6 size to 512 MB
|
InnoDB: Progress in MB: 100 200 300 400 500
|
InnoDB: Setting log file ./ib_logfile7 size to 512 MB
|
InnoDB: Progress in MB: 100 200 300 400 500
|
InnoDB: Renaming log file ./ib_logfile101 to ./ib_logfile0
|
InnoDB: New log files created, LSN=83470390901
|
InnoDB: Highest supported file format is Barracuda.
|
InnoDB: 128 rollback segment(s) are active.
|
InnoDB: Waiting for purge to start
|
InnoDB: 5.6.24 started; log sequence number 83470391308
|
xtrabackup: starting shutdown with innodb_fast_shutdown = 1
|
InnoDB: FTS optimize thread exiting.
|
InnoDB: Starting shutdown...
|
InnoDB: Shutdown completed; log sequence number 83470395168
|
160204 12:04:24 completed OK!
|
MARIADB.SERVICE
#
|
# /etc/systemd/system/mariadb.service
|
#
|
# This file is free software; you can redistribute it and/or modify it
|
# under the terms of the GNU Lesser General Public License as published by
|
# the Free Software Foundation; either version 2.1 of the License, or
|
# (at your option) any later version.
|
#
|
# Thanks to:
|
# Daniel Black
|
# Erkan Yanar
|
# David Strauss
|
# and probably others
|
|
[Unit]
|
Description=MariaDB database server
|
After=network.target
|
After=syslog.target
|
|
[Install]
|
WantedBy=multi-user.target
|
Alias=mysql.service
|
Alias=mysqld.service
|
|
|
[Service]
|
|
##############################################################################
|
## Core requirements
|
##
|
|
Type=notify
|
|
# Setting this to true can break replication and the Type=notify settings
|
# See also bind-address mysqld option.
|
PrivateNetwork=false
|
|
##############################################################################
|
## Package maintainers
|
##
|
|
User=mysql
|
Group=mysql
|
|
# To allow memlock to be used as non-root user if set in configuration
|
CapabilityBoundingSet=CAP_IPC_LOCK
|
|
# Execute pre and post scripts as root, otherwise it does it as User=
|
PermissionsStartOnly=true
|
|
# Needed to create system tables etc.
|
# ExecStartPre=/usr/bin/mysql_install_db -u mysql
|
|
# Start main service
|
# MYSQLD_OPTS here is for users to set in /etc/systemd/system/mariadb.service.d/MY_SPECIAL.conf
|
# Use the [service] section and Environment="MYSQLD_OPTS=...".
|
# This isn't a replacement for my.cnf.
|
# _WSREP_NEW_CLUSTER is for the exclusive use of the script galera_new_cluster
|
|
ExecStart=/usr/sbin/mysqld $MYSQLD_OPTS $_WSREP_NEW_CLUSTER
|
|
|
KillMode=process
|
KillSignal=SIGTERM
|
|
# Don't want to see an automated SIGKILL ever
|
SendSIGKILL=no
|
|
# Restart crashed server only, on-failure would also restart, for example, when
|
# my.cnf contains unknown option
|
Restart=on-abort
|
RestartSec=5s
|
|
UMask=007
|
|
##############################################################################
|
## USERs can override
|
##
|
##
|
## by creating a file in /etc/systemd/system/mariadb.service.d/MY_SPECIAL.conf
|
## and adding/setting the following will override this file's settings.
|
|
# Useful options not previously available in [mysqld_safe]
|
|
# Kernels like killing mysqld when out of memory because its big.
|
# Lets temper that preference a little.
|
# OOMScoreAdjust=-600
|
|
# Explicitly start with high IO priority
|
# BlockIOWeight=1000
|
|
# If you don't use the /tmp directory for SELECT ... OUTFILE and
|
# LOAD DATA INFILE you can enable PrivateTmp=true for a little more security.
|
PrivateTmp=false
|
|
##
|
## Options previously available to be set via [mysqld_safe]
|
## that now needs to be set by systemd config files as mysqld_safe
|
## isn't executed.
|
##
|
|
# Number of files limit. previously [mysqld_safe] open-file-limit
|
LimitNOFILE=16364
|
|
# Maximium core size. previously [mysqld_safe] core-file-size
|
# LimitCore=
|
|
# Nice priority. previously [mysqld_safe] nice
|
# Nice=-5
|
|
# Timezone. previously [mysqld_safe] timezone
|
# Environment="TZ=UTC"
|
|
# Library substitutions. previously [mysqld_safe] malloc-lib with explict paths
|
# (in LD_LIBRARY_PATH) and library name (in LD_PRELOAD).
|
# Environment="LD_LIBRARY_PATH=/path1 /path2" "LD_PRELOAD=
|
|
# Flush caches. previously [mysqld_safe] flush-caches=1
|
# ExecStartPre=sync
|
# ExecStartPre=sysctl -q -w vm.drop_caches=3
|
|
# numa-interleave=1 equalivant
|
# Change ExecStart=numactl --interleave=all /usr/sbin/mysqld......
|
|
# crash-script equalivent
|
# FailureAction=
|
Attachments
Issue Links
- is caused by
-
MDEV-9202 Systemd timeout is not sufficient for larger servers
- Closed
- relates to
-
MDEV-17571 Make systemd timeout behavior more compatible with long Galera SSTs
- Closed
-
MDEV-17934 Make systemd timeout behavior more compatible with longer Galera recovery times
- Closed