Hey there,
there is a reproduceable crash of MariaDB 10.3.x when running mariabackup on it. Reporting as advised in the resulting Error Log:
[00] 2020-03-30 12:39:07 Connecting to MySQL server host: localhost, user: root, password: set, port: 3306, socket: /var/lib/mysql/mysql.sock
|
[00] 2020-03-30 12:39:07 Using server version 10.3.21-MariaDB
|
mariabackup based on MariaDB server 10.3.21-MariaDB Linux (x86_64)
|
[00] 2020-03-30 12:39:07 uses posix_fadvise().
|
[00] 2020-03-30 12:39:07 cd to /var/lib/mysql/
|
[00] 2020-03-30 12:39:07 open files limit requested 0, set to 1024
|
[00] 2020-03-30 12:39:07 mariabackup: using the following InnoDB configuration:
|
[00] 2020-03-30 12:39:07 innodb_data_home_dir =
|
[00] 2020-03-30 12:39:07 innodb_data_file_path = ibdata1:12M:autoextend
|
[00] 2020-03-30 12:39:07 innodb_log_group_home_dir = ./
|
[00] 2020-03-30 12:39:07 InnoDB: Using Linux native AIO
|
2020-03-30 12:39:07 0 [Note] InnoDB: Number of pools: 1
|
[00] 2020-03-30 12:39:07 mariabackup: Generating a list of tablespaces
|
2020-03-30 12:39:07 0 [Warning] InnoDB: Allocated tablespace ID 2325 for sf_XXXXXX_relaunch/ezinfocollection_attribute, old maximum was 0
|
2020-03-30 12:39:07 0 [ERROR] InnoDB: Operating system error number 24 in a file operation.
|
2020-03-30 12:39:07 0 [ERROR] InnoDB: Error number 24 means 'Too many open files'
|
2020-03-30 12:39:07 0 [Note] InnoDB: Some operating system error numbers are described at https://mariadb.com/kb/en/library/operating-system-error-codes/
|
2020-03-30 12:39:07 0 [ERROR] InnoDB: File ./XXXXXX_magento2/salesrule.ibd: 'open' returned OS error 224.
|
2020-03-30 12:39:07 0x7ff942b7d8c0 InnoDB: Assertion failure in file /root/rpmbuild/BUILD/mariadb-10.3.21/storage/innobase/fil/fil0fil.cc line 677
|
InnoDB: Failing assertion: success
|
InnoDB: We intentionally generate a memory trap.
|
InnoDB: Submit a detailed bug report to https://jira.mariadb.org/
|
InnoDB: If you get repeated assertion failures or crashes, even
|
InnoDB: immediately after the mysqld startup, there may be
|
InnoDB: corruption in the InnoDB tablespace. Please refer to
|
InnoDB: https://mariadb.com/kb/en/library/innodb-recovery-modes/
|
InnoDB: about forcing recovery.
|
200330 12:39:07 [ERROR] mysqld got signal 6 ;
|
This could be because you hit a bug. It is also possible that this binary
|
or one of the libraries it was linked against is corrupt, improperly built,
|
or misconfigured. This error can also be caused by malfunctioning hardware.
|
|
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
|
|
We will try our best to scrape up some info that will hopefully help
|
diagnose the problem, but since we have already crashed,
|
something is definitely wrong and this may fail.
|
|
Server version: 10.3.21-MariaDB
|
key_buffer_size=0
|
read_buffer_size=131072
|
max_used_connections=0
|
max_threads=1
|
thread_count=0
|
It is possible that mysqld could use up to
|
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 5600 K bytes of memory
|
Hope that's ok; if not, decrease some variables in the equation.
|
|
Thread pointer: 0x0
|
Attempting backtrace. You can use the following information to find out
|
where mysqld died. If you see no messages after this, something went
|
terribly wrong...
|
stack_bottom = 0x0 thread_stack 0x49000
|
mariabackup(my_print_stacktrace+0x3d)[0x5570572ef63d]
|
mariabackup(handle_fatal_signal+0x327)[0x557056e2ce17]
|
/lib64/libpthread.so.0(+0xf5f0)[0x7ff9427685f0]
|
/lib64/libc.so.6(gsignal+0x37)[0x7ff940824337]
|
/lib64/libc.so.6(abort+0x148)[0x7ff940825a28]
|
mariabackup(+0x4e4616)[0x557056aec616]
|
mariabackup(+0xaa5b99)[0x5570570adb99]
|
mariabackup(+0xaa5c1f)[0x5570570adc1f]
|
mariabackup(+0x507b7e)[0x557056b0fb7e]
|
mariabackup(+0x5071bd)[0x557056b0f1bd]
|
mariabackup(+0x5076ad)[0x557056b0f6ad]
|
mariabackup(+0x50d0b6)[0x557056b150b6]
|
mariabackup(main+0x194)[0x557056aee5f4]
|
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7ff940810505]
|
mariabackup(+0x50073d)[0x557056b0873d]
|
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
|
information that should help you find out what is causing the crash.
|
Writing a core file...
|
Working directory at /var/lib/mysql
|
Max. Open Files should not be the Source I guess as it's raised to the limit already:
# cat /var/run/mariadb/mariadb.pid
|
2805
|
# cat /proc/2805/limits
|
Limit Soft Limit Hard Limit Units
|
Max cpu time unlimited unlimited seconds
|
Max file size unlimited unlimited bytes
|
Max data size unlimited unlimited bytes
|
Max stack size 8388608 unlimited bytes
|
Max core file size 0 unlimited bytes
|
Max resident set unlimited unlimited bytes
|
Max processes 62987 62987 processes
|
Max open files 1048576 1048576 files
|
Max locked memory unlimited unlimited bytes
|
Max address space unlimited unlimited bytes
|
Max file locks unlimited unlimited locks
|
Max pending signals 514290 514290 signals
|
Max msgqueue size 819200 819200 bytes
|
Max nice priority 0 0
|
Max realtime priority 0 0
|
Max realtime timeout unlimited unlimited us
|
Hope someone here can give me a helping hand on this! 
thx, bye from Austria
Andreas
{"report":{"fcp":734.5,"ttfb":167,"pageVisibility":"visible","entityId":84826,"key":"jira.project.issue.view-issue","isInitial":true,"threshold":1000,"elementTimings":{},"userDeviceMemory":8,"userDeviceProcessors":64,"apdex":1,"journeyId":"fb2c2994-940e-4849-b75b-4c9bcbc0cbfa","navigationType":0,"readyForUser":818.3000000715256,"redirectCount":0,"resourceLoadedEnd":844.2000000476837,"resourceLoadedStart":172.20000004768372,"resourceTiming":[{"duration":38.699999928474426,"initiatorType":"link","name":"https://jira.mariadb.org/s/2c21342762a6a02add1c328bed317ffd-CDN/lu2bu7/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/css/_super/batch.css","startTime":172.20000004768372,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":172.20000004768372,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":210.89999997615814,"responseStart":0,"secureConnectionStart":0},{"duration":38.39999997615814,"initiatorType":"link","name":"https://jira.mariadb.org/s/7ebd35e77e471bc30ff0eba799ebc151-CDN/lu2bu7/820016/12ta74/8679b4946efa1a0bb029a3a22206fb5d/_/download/contextbatch/css/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true","startTime":172.5,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":172.5,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":210.89999997615814,"responseStart":0,"secureConnectionStart":0},{"duration":96.10000002384186,"initiatorType":"script","name":"https://jira.mariadb.org/s/fbf975c0cce4b1abf04784eeae9ba1f4-CDN/lu2bu7/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/js/_super/batch.js?locale=en","startTime":172.60000002384186,"connectEnd":172.60000002384186,"connectStart":172.60000002384186,"domainLookupEnd":172.60000002384186,"domainLookupStart":172.60000002384186,"fetchStart":172.60000002384186,"redirectEnd":0,"redirectStart":0,"requestStart":172.60000002384186,"responseEnd":268.7000000476837,"responseStart":268.7000000476837,"secureConnectionStart":172.60000002384186},{"duration":175.5,"initiatorType":"script","name":"https://jira.mariadb.org/s/099b33461394b8015fc36c0a4b96e19f-CDN/lu2bu7/820016/12ta74/8679b4946efa1a0bb029a3a22206fb5d/_/download/contextbatch/js/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true","startTime":172.70000004768372,"connectEnd":172.70000004768372,"connectStart":172.70000004768372,"domainLookupEnd":172.70000004768372,"domainLookupStart":172.70000004768372,"fetchStart":172.70000004768372,"redirectEnd":0,"redirectStart":0,"requestStart":172.70000004768372,"responseEnd":348.2000000476837,"responseStart":348.2000000476837,"secureConnectionStart":172.70000004768372},{"duration":179.10000002384186,"initiatorType":"script","name":"https://jira.mariadb.org/s/94c15bff32baef80f4096a08aceae8bc-CDN/lu2bu7/820016/12ta74/c92c0caa9a024ae85b0ebdbed7fb4bd7/_/download/contextbatch/js/atl.global,-_super/batch.js?locale=en","startTime":172.89999997615814,"connectEnd":172.89999997615814,"connectStart":172.89999997615814,"domainLookupEnd":172.89999997615814,"domainLookupStart":172.89999997615814,"fetchStart":172.89999997615814,"redirectEnd":0,"redirectStart":0,"requestStart":172.89999997615814,"responseEnd":352,"responseStart":352,"secureConnectionStart":172.89999997615814},{"duration":179.39999997615814,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-en/jira.webresources:calendar-en.js","startTime":173,"connectEnd":173,"connectStart":173,"domainLookupEnd":173,"domainLookupStart":173,"fetchStart":173,"redirectEnd":0,"redirectStart":0,"requestStart":173,"responseEnd":352.39999997615814,"responseStart":352.39999997615814,"secureConnectionStart":173},{"duration":179.80000007152557,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-localisation-moment/jira.webresources:calendar-localisation-moment.js","startTime":173,"connectEnd":173,"connectStart":173,"domainLookupEnd":173,"domainLookupStart":173,"fetchStart":173,"redirectEnd":0,"redirectStart":0,"requestStart":173,"responseEnd":352.8000000715256,"responseStart":352.8000000715256,"secureConnectionStart":173},{"duration":180.10000002384186,"initiatorType":"link","name":"https://jira.mariadb.org/s/b04b06a02d1959df322d9cded3aeecc1-CDN/lu2bu7/820016/12ta74/a2ff6aa845ffc9a1d22fe23d9ee791fc/_/download/contextbatch/css/jira.global.look-and-feel,-_super/batch.css","startTime":173.20000004768372,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":173.20000004768372,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":353.3000000715256,"responseStart":0,"secureConnectionStart":0},{"duration":179.90000009536743,"initiatorType":"script","name":"https://jira.mariadb.org/rest/api/1.0/shortcuts/820016/47140b6e0a9bc2e4913da06536125810/shortcuts.js?context=issuenavigation&context=issueaction","startTime":173.39999997615814,"connectEnd":173.39999997615814,"connectStart":173.39999997615814,"domainLookupEnd":173.39999997615814,"domainLookupStart":173.39999997615814,"fetchStart":173.39999997615814,"redirectEnd":0,"redirectStart":0,"requestStart":173.39999997615814,"responseEnd":353.3000000715256,"responseStart":353.3000000715256,"secureConnectionStart":173.39999997615814},{"duration":180.20000004768372,"initiatorType":"link","name":"https://jira.mariadb.org/s/3ac36323ba5e4eb0af2aa7ac7211b4bb-CDN/lu2bu7/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/css/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.css?jira.create.linked.issue=true","startTime":173.5,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":173.5,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":353.7000000476837,"responseStart":0,"secureConnectionStart":0},{"duration":180.20000004768372,"initiatorType":"script","name":"https://jira.mariadb.org/s/3339d87fa2538a859872f2df449bf8d0-CDN/lu2bu7/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/js/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.js?jira.create.linked.issue=true&locale=en","startTime":173.60000002384186,"connectEnd":173.60000002384186,"connectStart":173.60000002384186,"domainLookupEnd":173.60000002384186,"domainLookupStart":173.60000002384186,"fetchStart":173.60000002384186,"redirectEnd":0,"redirectStart":0,"requestStart":173.60000002384186,"responseEnd":353.8000000715256,"responseStart":353.8000000715256,"secureConnectionStart":173.60000002384186},{"duration":378.3000000715256,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-js/jira.webresources:bigpipe-js.js","startTime":183.89999997615814,"connectEnd":183.89999997615814,"connectStart":183.89999997615814,"domainLookupEnd":183.89999997615814,"domainLookupStart":183.89999997615814,"fetchStart":183.89999997615814,"redirectEnd":0,"redirectStart":0,"requestStart":183.89999997615814,"responseEnd":562.2000000476837,"responseStart":562.2000000476837,"secureConnectionStart":183.89999997615814},{"duration":489.60000002384186,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-init/jira.webresources:bigpipe-init.js","startTime":185.60000002384186,"connectEnd":185.60000002384186,"connectStart":185.60000002384186,"domainLookupEnd":185.60000002384186,"domainLookupStart":185.60000002384186,"fetchStart":185.60000002384186,"redirectEnd":0,"redirectStart":0,"requestStart":185.60000002384186,"responseEnd":675.2000000476837,"responseStart":675.2000000476837,"secureConnectionStart":185.60000002384186},{"duration":148.39999997615814,"initiatorType":"xmlhttprequest","name":"https://jira.mariadb.org/rest/webResources/1.0/resources","startTime":485.2000000476837,"connectEnd":485.2000000476837,"connectStart":485.2000000476837,"domainLookupEnd":485.2000000476837,"domainLookupStart":485.2000000476837,"fetchStart":485.2000000476837,"redirectEnd":0,"redirectStart":0,"requestStart":485.2000000476837,"responseEnd":633.6000000238419,"responseStart":633.6000000238419,"secureConnectionStart":485.2000000476837},{"duration":125,"initiatorType":"script","name":"https://www.google-analytics.com/analytics.js","startTime":710.8999999761581,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":710.8999999761581,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":835.8999999761581,"responseStart":0,"secureConnectionStart":0},{"duration":113.20000004768372,"initiatorType":"link","name":"https://jira.mariadb.org/s/d5715adaadd168a9002b108b2b039b50-CDN/lu2bu7/820016/12ta74/be4b45e9cec53099498fa61c8b7acba4/_/download/contextbatch/css/jira.project.sidebar,-_super,-project.issue.navigator,-jira.general,-jira.browse.project,-jira.view.issue,-jira.global,-atl.general,-com.atlassian.jira.projects.sidebar.init/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true","startTime":723.5,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":723.5,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":836.7000000476837,"responseStart":0,"secureConnectionStart":0},{"duration":112.90000009536743,"initiatorType":"link","name":"https://jira.mariadb.org/s/50bc9be5bfead1a25e72c1a9338c94f6-CDN/lu2bu7/820016/12ta74/e108c7645258ccb43280ed3404e3e949/_/download/contextbatch/css/com.atlassian.jira.plugins.jira-development-integration-plugin:0,-_super,-jira.view.issue,-jira.global,-jira.general,-jira.browse.project,-project.issue.navigator,-atl.general/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true","startTime":723.8999999761581,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":723.8999999761581,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":836.8000000715256,"responseStart":0,"secureConnectionStart":0},{"duration":113,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/e65b778d185daf5aee24936755b43da6/_/download/contextbatch/js/browser-metrics-plugin.contrib,-_super,-project.issue.navigator,-jira.view.issue,-atl.general/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true","startTime":724.6000000238419,"connectEnd":724.6000000238419,"connectStart":724.6000000238419,"domainLookupEnd":724.6000000238419,"domainLookupStart":724.6000000238419,"fetchStart":724.6000000238419,"redirectEnd":0,"redirectStart":0,"requestStart":724.6000000238419,"responseEnd":837.6000000238419,"responseStart":837.6000000238419,"secureConnectionStart":724.6000000238419},{"duration":117.70000004768372,"initiatorType":"script","name":"https://jira.mariadb.org/s/f51ef5507eea4c158f257c66c93b2a3f-CDN/lu2bu7/820016/12ta74/be4b45e9cec53099498fa61c8b7acba4/_/download/contextbatch/js/jira.project.sidebar,-_super,-project.issue.navigator,-jira.general,-jira.browse.project,-jira.view.issue,-jira.global,-atl.general,-com.atlassian.jira.projects.sidebar.init/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true","startTime":725.1000000238419,"connectEnd":725.1000000238419,"connectStart":725.1000000238419,"domainLookupEnd":725.1000000238419,"domainLookupStart":725.1000000238419,"fetchStart":725.1000000238419,"redirectEnd":0,"redirectStart":0,"requestStart":725.1000000238419,"responseEnd":842.8000000715256,"responseStart":842.8000000715256,"secureConnectionStart":725.1000000238419},{"duration":118.80000007152557,"initiatorType":"script","name":"https://jira.mariadb.org/s/86ee9bbc76cd1bcd8556fcdcf46241c9-CDN/lu2bu7/820016/12ta74/e108c7645258ccb43280ed3404e3e949/_/download/contextbatch/js/com.atlassian.jira.plugins.jira-development-integration-plugin:0,-_super,-jira.view.issue,-jira.global,-jira.general,-jira.browse.project,-project.issue.navigator,-atl.general/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true","startTime":725.3999999761581,"connectEnd":725.3999999761581,"connectStart":725.3999999761581,"domainLookupEnd":725.3999999761581,"domainLookupStart":725.3999999761581,"fetchStart":725.3999999761581,"redirectEnd":0,"redirectStart":0,"requestStart":725.3999999761581,"responseEnd":844.2000000476837,"responseStart":844.2000000476837,"secureConnectionStart":725.3999999761581}],"fetchStart":1,"domainLookupStart":1,"domainLookupEnd":1,"connectStart":1,"connectEnd":1,"requestStart":28,"responseStart":167,"responseEnd":186,"domLoading":171,"domInteractive":903,"domContentLoadedEventStart":903,"domContentLoadedEventEnd":952,"domComplete":1768,"loadEventStart":1768,"loadEventEnd":1770,"userAgent":"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)","marks":[{"name":"bigPipe.sidebar-id.start","time":864.8000000715256},{"name":"bigPipe.sidebar-id.end","time":865.6000000238419},{"name":"bigPipe.activity-panel-pipe-id.start","time":865.6000000238419},{"name":"bigPipe.activity-panel-pipe-id.end","time":868.2000000476837},{"name":"activityTabFullyLoaded","time":966.3000000715256}],"measures":[],"correlationId":"1291e99aa827e","effectiveType":"4g","downlink":9.9,"rtt":0,"serverDuration":76,"dbReadsTimeInMs":13,"dbConnsTimeInMs":20,"applicationHash":"9d11dbea5f4be3d4cc21f03a88dd11d8c8687422","experiments":[]}}