Type:
Bug
Priority:
Major
Resolution:
Fixed
Affects Version/s:
10.6.14 , 10.6.17 , 10.3(EOL) , 10.4(EOL) , 10.5 , 10.6
Environment:
OS Linux Gentoo
Hello,
in data center 1, there are 4 nodes:
3 node galera cluster -> async slave
From the async slave I'm trying to create a new slave in DC2 that will then form a new galera cluster with 2 more nodes.
cd /var/lib/mysql
ssh slave-server 'mariabackup --backup --stream=xbstream --parallel=4' | mbstream -x --parallel=4
...
[00] 2024-02-26 03:25:59 MySQL binlog position: filename 'mysqld-bin.003284', position '666302125', GTID of the last change '0-8-18213972007'
[00] 2024-02-26 03:25:59 Streaming backup-my.cnf
[00] 2024-02-26 03:25:59 ...done
[00] 2024-02-26 03:25:59 Streaming xtrabackup_info
[00] 2024-02-26 03:25:59 ...done
[00] 2024-02-26 03:25:59 Redo log (from LSN 31507622579405 to 31508457792375) was copied.
[00] 2024-02-26 03:25:59 completed OK!
# mariabackup --prepare --use-memory=40G --target-dir=/var/lib/mysql
mariabackup based on MariaDB server 10.6.14-MariaDB Linux (x86_64)
[00] 2024-02-26 08:34:36 cd to /var/lib/mysql/
[00] 2024-02-26 08:34:36 open files limit requested 0, set to 1024
[00] 2024-02-26 08:34:36 This target seems to be not prepared yet.
[00] 2024-02-26 08:34:36 mariabackup: using the following InnoDB configuration for recovery:
[00] 2024-02-26 08:34:36 innodb_data_home_dir = .
[00] 2024-02-26 08:34:36 innodb_data_file_path = ibdata1:12M:autoextend
[00] 2024-02-26 08:34:36 innodb_log_group_home_dir = .
[00] 2024-02-26 08:34:36 InnoDB: Using liburing
[00] 2024-02-26 08:34:36 Starting InnoDB instance for recovery.
[00] 2024-02-26 08:34:36 mariabackup: Using 42949672960 bytes for buffer pool (set by --use-memory parameter)
2024-02-26 8:34:36 0 [Note] InnoDB: Compressed tables use zlib 1.3
2024-02-26 8:34:36 0 [Note] InnoDB: Using transactional memory
2024-02-26 8:34:36 0 [Note] InnoDB: Number of pools: 1
2024-02-26 8:34:36 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
2024-02-26 8:34:36 0 [Note] InnoDB: Using liburing
2024-02-26 8:34:36 0 [Note] InnoDB: Initializing buffer pool, total size = 42949672960, chunk size = 42949672960
2024-02-26 8:34:36 0 [Note] InnoDB: Completed initialization of buffer pool
2024-02-26 8:34:36 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=31507622579405,31507622976800
2024-02-26 8:34:48 0 [ERROR] InnoDB: Crash recovery is broken due to insufficient innodb_log_file_size; last checkpoint LSN=31507622579405, current LSN=31508457793604. Shutdown is in progress.
2024-02-26 8:34:48 0 [Note] InnoDB: Starting final batch to recover 237985 pages from redo log.
2024-02-26 8:34:51 0 [Note] InnoDB: To recover: 174243 pages from log
2024-02-26 8:35:00 0 [Note] InnoDB: Last binlog file './mysqld-bin.003284', position 666209873
[00] 2024-02-26 08:35:01 Last binlog file ./mysqld-bin.003284, position 666209873
[00] 2024-02-26 08:35:01 mariabackup: Recovered WSREP position: 2430d56f-1879-11e8-9a58-defae87a2441:6397994274
[00] 2024-02-26 08:35:02 completed OK!
Then the node is started and it works ok, but this line troubles me:
2024-02-26 8:34:48 0 [ERROR] InnoDB: Crash recovery is broken due to insufficient innodb_log_file_size; last checkpoint LSN=31507622579405, current LSN=31508457793604. Shutdown is in progress.
Is this expected? All I could find to this was: https://jira.mariadb.org/browse/MDEV-29982
I've tried MariaDB 10.6.14 and 10.6.17, same error, multiple times.
Thank you!
is caused by
MDEV-17458
Unable to Start Galera Node
Closed
Transition
Time In Source Status
Execution Times
Open
Confirmed
3d 3h 57m
1
Confirmed
Closed
10d 18h 21m
1
{"report":{"fcp":3238.4000000953674,"ttfb":1706.7000002861023,"pageVisibility":"visible","entityId":128069,"key":"jira.project.issue.view-issue","isInitial":true,"threshold":1000,"elementTimings":{},"userDeviceMemory":8,"userDeviceProcessors":64,"apdex":0.5,"journeyId":"0bbff795-5355-432e-b475-580abcee2037","navigationType":0,"readyForUser":3423.5999999046326,"redirectCount":0,"resourceLoadedEnd":4665.5,"resourceLoadedStart":1711.5999999046326,"resourceTiming":[{"duration":668.1000003814697,"initiatorType":"link","name":"https://jira.mariadb.org/s/2c21342762a6a02add1c328bed317ffd-CDN/lu2cib/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/css/_super/batch.css","startTime":1711.5999999046326,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":1711.5999999046326,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":2379.7000002861023,"responseStart":0,"secureConnectionStart":0},{"duration":662.1999998092651,"initiatorType":"link","name":"https://jira.mariadb.org/s/7ebd35e77e471bc30ff0eba799ebc151-CDN/lu2cib/820016/12ta74/494e4c556ecbb29f90a3d3b4f09cb99c/_/download/contextbatch/css/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true&whisper-enabled=true","startTime":1717.8000001907349,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":1717.8000001907349,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":2380,"responseStart":0,"secureConnectionStart":0},{"duration":661.6999998092651,"initiatorType":"script","name":"https://jira.mariadb.org/s/0917945aaa57108d00c5076fea35e069-CDN/lu2cib/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/js/_super/batch.js?locale=en","startTime":1726.2000002861023,"connectEnd":1726.2000002861023,"connectStart":1726.2000002861023,"domainLookupEnd":1726.2000002861023,"domainLookupStart":1726.2000002861023,"fetchStart":1726.2000002861023,"redirectEnd":0,"redirectStart":0,"requestStart":1726.2000002861023,"responseEnd":2387.9000000953674,"responseStart":2387.9000000953674,"secureConnectionStart":1726.2000002861023},{"duration":951,"initiatorType":"script","name":"https://jira.mariadb.org/s/2d8175ec2fa4c816e8023260bd8c1786-CDN/lu2cib/820016/12ta74/494e4c556ecbb29f90a3d3b4f09cb99c/_/download/contextbatch/js/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true&whisper-enabled=true","startTime":1729.2000002861023,"connectEnd":1729.2000002861023,"connectStart":1729.2000002861023,"domainLookupEnd":1729.2000002861023,"domainLookupStart":1729.2000002861023,"fetchStart":1729.2000002861023,"redirectEnd":0,"redirectStart":0,"requestStart":1729.2000002861023,"responseEnd":2680.2000002861023,"responseStart":2680.2000002861023,"secureConnectionStart":1729.2000002861023},{"duration":987.4000000953674,"initiatorType":"script","name":"https://jira.mariadb.org/s/a9324d6758d385eb45c462685ad88f1d-CDN/lu2cib/820016/12ta74/c92c0caa9a024ae85b0ebdbed7fb4bd7/_/download/contextbatch/js/atl.global,-_super/batch.js?locale=en","startTime":1730.0999999046326,"connectEnd":1730.0999999046326,"connectStart":1730.0999999046326,"domainLookupEnd":1730.0999999046326,"domainLookupStart":1730.0999999046326,"fetchStart":1730.0999999046326,"redirectEnd":0,"redirectStart":0,"requestStart":1730.0999999046326,"responseEnd":2717.5,"responseStart":2717.5,"secureConnectionStart":1730.0999999046326},{"duration":987.2000002861023,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-en/jira.webresources:calendar-en.js","startTime":1731.5,"connectEnd":1731.5,"connectStart":1731.5,"domainLookupEnd":1731.5,"domainLookupStart":1731.5,"fetchStart":1731.5,"redirectEnd":0,"redirectStart":0,"requestStart":1731.5,"responseEnd":2718.7000002861023,"responseStart":2718.7000002861023,"secureConnectionStart":1731.5},{"duration":987.9000000953674,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-localisation-moment/jira.webresources:calendar-localisation-moment.js","startTime":1732.5,"connectEnd":1732.5,"connectStart":1732.5,"domainLookupEnd":1732.5,"domainLookupStart":1732.5,"fetchStart":1732.5,"redirectEnd":0,"redirectStart":0,"requestStart":1732.5,"responseEnd":2720.4000000953674,"responseStart":2720.4000000953674,"secureConnectionStart":1732.5},{"duration":990.5999999046326,"initiatorType":"link","name":"https://jira.mariadb.org/s/b04b06a02d1959df322d9cded3aeecc1-CDN/lu2cib/820016/12ta74/a2ff6aa845ffc9a1d22fe23d9ee791fc/_/download/contextbatch/css/jira.global.look-and-feel,-_super/batch.css","startTime":1733.4000000953674,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":1733.4000000953674,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":2724,"responseStart":0,"secureConnectionStart":0},{"duration":989.9000000953674,"initiatorType":"script","name":"https://jira.mariadb.org/rest/api/1.0/shortcuts/820016/47140b6e0a9bc2e4913da06536125810/shortcuts.js?context=issuenavigation&context=issueaction","startTime":1733.9000000953674,"connectEnd":1733.9000000953674,"connectStart":1733.9000000953674,"domainLookupEnd":1733.9000000953674,"domainLookupStart":1733.9000000953674,"fetchStart":1733.9000000953674,"redirectEnd":0,"redirectStart":0,"requestStart":1733.9000000953674,"responseEnd":2723.800000190735,"responseStart":2723.800000190735,"secureConnectionStart":1733.9000000953674},{"duration":991.2000002861023,"initiatorType":"link","name":"https://jira.mariadb.org/s/3ac36323ba5e4eb0af2aa7ac7211b4bb-CDN/lu2cib/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/css/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.css?jira.create.linked.issue=true","startTime":1734.5,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":1734.5,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":2725.7000002861023,"responseStart":0,"secureConnectionStart":0},{"duration":990.4000000953674,"initiatorType":"script","name":"https://jira.mariadb.org/s/5d5e8fe91fbc506585e83ea3b62ccc4b-CDN/lu2cib/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/js/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.js?jira.create.linked.issue=true&locale=en","startTime":1734.9000000953674,"connectEnd":1734.9000000953674,"connectStart":1734.9000000953674,"domainLookupEnd":1734.9000000953674,"domainLookupStart":1734.9000000953674,"fetchStart":1734.9000000953674,"redirectEnd":0,"redirectStart":0,"requestStart":1734.9000000953674,"responseEnd":2725.300000190735,"responseStart":2725.300000190735,"secureConnectionStart":1734.9000000953674},{"duration":2332.1000003814697,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-js/jira.webresources:bigpipe-js.js","startTime":1737.0999999046326,"connectEnd":1737.0999999046326,"connectStart":1737.0999999046326,"domainLookupEnd":1737.0999999046326,"domainLookupStart":1737.0999999046326,"fetchStart":1737.0999999046326,"redirectEnd":0,"redirectStart":0,"requestStart":1737.0999999046326,"responseEnd":4069.2000002861023,"responseStart":4069.2000002861023,"secureConnectionStart":1737.0999999046326},{"duration":2926.4000000953674,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-init/jira.webresources:bigpipe-init.js","startTime":1739.0999999046326,"connectEnd":1739.0999999046326,"connectStart":1739.0999999046326,"domainLookupEnd":1739.0999999046326,"domainLookupStart":1739.0999999046326,"fetchStart":1739.0999999046326,"redirectEnd":0,"redirectStart":0,"requestStart":1739.0999999046326,"responseEnd":4665.5,"responseStart":4665.5,"secureConnectionStart":1739.0999999046326},{"duration":1165.0999999046326,"initiatorType":"xmlhttprequest","name":"https://jira.mariadb.org/rest/webResources/1.0/resources","startTime":2600.4000000953674,"connectEnd":2600.4000000953674,"connectStart":2600.4000000953674,"domainLookupEnd":2600.4000000953674,"domainLookupStart":2600.4000000953674,"fetchStart":2600.4000000953674,"redirectEnd":0,"redirectStart":0,"requestStart":2600.4000000953674,"responseEnd":3765.5,"responseStart":3765.5,"secureConnectionStart":2600.4000000953674},{"duration":1539,"initiatorType":"script","name":"https://www.google-analytics.com/analytics.js","startTime":3223.4000000953674,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":3223.4000000953674,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":4762.400000095367,"responseStart":0,"secureConnectionStart":0}],"fetchStart":0,"domainLookupStart":0,"domainLookupEnd":0,"connectStart":0,"connectEnd":0,"requestStart":1528,"responseStart":1707,"responseEnd":1721,"domLoading":1711,"domInteractive":4712,"domContentLoadedEventStart":4712,"domContentLoadedEventEnd":4762,"domComplete":6911,"loadEventStart":6911,"loadEventEnd":6912,"userAgent":"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)","marks":[{"name":"bigPipe.sidebar-id.start","time":4670.300000190735},{"name":"bigPipe.sidebar-id.end","time":4671.200000286102},{"name":"bigPipe.activity-panel-pipe-id.start","time":4671.400000095367},{"name":"bigPipe.activity-panel-pipe-id.end","time":4672.300000190735},{"name":"activityTabFullyLoaded","time":4780.800000190735}],"measures":[],"correlationId":"ca9711c8b89ddd","effectiveType":"4g","downlink":9.4,"rtt":0,"serverDuration":114,"dbReadsTimeInMs":20,"dbConnsTimeInMs":31,"applicationHash":"9d11dbea5f4be3d4cc21f03a88dd11d8c8687422","experiments":[]}}