Following error output is generated during execution of an "normal" update query and leads to a crash of mariadb master (master-master replication).
Occurrence: 3 times on two different database tables, not reproducible so far.
2019 - 01 - 29 13 : 24 : 41 434826568 [ERROR] InnoDB: Page old data size 15175 new data size 15391 , page old max ins size 1063 new max ins size 847
2019 - 01 - 29 13 : 24 : 41 434826568 [ERROR] InnoDB: Submit a detailed bug report to https: //jira.mariadb.org/
2019 - 01 - 29 13 : 24 : 41 0x7f0118195700 InnoDB: Assertion failure in file /home/buildbot/buildbot/build/mariadb- 10.3 . 7 /storage/innobase/btr/btr0cur.cc line 4308
InnoDB: Failing assertion: rec
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to https: //jira.mariadb.org/
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: https: //mariadb.com/kb/en/library/xtradbinnodb-recovery-modes/
InnoDB: about forcing recovery.
190129 13 : 24 : 41 [ERROR] mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
To report this bug, see https: //mariadb.com/kb/en/reporting-bugs
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
Server version: 10.3 . 7 -MariaDB- 1 : 10.3 . 7 +maria~xenial-log
key_buffer_size= 8589934592
read_buffer_size= 2097152
max_used_connections= 200
max_threads= 242
thread_count= 138
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 24749309 K bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
Thread pointer: 0x7efdec0009a8
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this , something went
terribly wrong...
stack_bottom = 0x7f0118194cf8 thread_stack 0x30000
/usr/sbin/mysqld(my_print_stacktrace+ 0x2e )[ 0x55f49174003e ]
/usr/sbin/mysqld(handle_fatal_signal+ 0x347 )[ 0x55f4911ec607 ]
/lib/x86_64-linux-gnu/libpthread.so. 0 (+ 0x11390 )[ 0x7f045d869390 ]
/lib/x86_64-linux-gnu/libc.so. 6 (gsignal+ 0x38 )[ 0x7f045ce38428 ]
/lib/x86_64-linux-gnu/libc.so. 6 (abort+ 0x16a )[ 0x7f045ce3a02a ]
/usr/sbin/mysqld(+ 0x49ee74 )[ 0x55f490f39e74 ]
/usr/sbin/mysqld(+ 0xb0a1aa )[ 0x55f4915a51aa ]
/usr/sbin/mysqld(+ 0xa8c949 )[ 0x55f491527949 ]
/usr/sbin/mysqld(+ 0xa91a1d )[ 0x55f49152ca1d ]
/usr/sbin/mysqld(+ 0xa92c4f )[ 0x55f49152dc4f ]
/usr/sbin/mysqld(+ 0xa5fed4 )[ 0x55f4914faed4 ]
/usr/sbin/mysqld(+ 0x99f3ff )[ 0x55f49143a3ff ]
/usr/sbin/mysqld(_ZN7handler13ha_update_rowEPKhS1_+ 0x122 )[ 0x55f4911f7072 ]
/usr/sbin/mysqld(_Z12mysql_updateP3THDP10TABLE_LISTR4ListI4ItemES6_PS4_jP8st_ordery15enum_duplicatesbPySB_+ 0x16b1 )[ 0x55f4910b05d1 ]
/usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+ 0x2478 )[ 0x55f491014058 ]
/usr/sbin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_statebb+ 0x21a )[ 0x55f49101a77a ]
/usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcjbb+ 0x1c23 )[ 0x55f49101d0c3 ]
/usr/sbin/mysqld(_Z10do_commandP3THD+ 0x13c )[ 0x55f49101db7c ]
/usr/sbin/mysqld(_Z24do_handle_one_connectionP7CONNECT+ 0x252 )[ 0x55f4910ebc92 ]
/usr/sbin/mysqld(handle_one_connection+ 0x3d )[ 0x55f4910ebe0d ]
/lib/x86_64-linux-gnu/libpthread.so. 0 (+ 0x76ba )[ 0x7f045d85f6ba ]
/lib/x86_64-linux-gnu/libc.so. 6 (clone+ 0x6d )[ 0x7f045cf0a41d ]
"Optimize table" does not help because the second crash occurred after an "optimize table run" on that database table.
Elena Stepanova
made changes -
2019-02-09 15:35
Field
Original Value
New Value
Labels
crash
crash need_feedback
Elena Stepanova
made changes -
2019-02-13 11:01
Labels
crash need_feedback
crash
Elena Stepanova
made changes -
2019-02-24 12:43
Labels
crash
crash need_feedback
Elena Stepanova
made changes -
2019-04-20 23:08
Labels
crash need_feedback
crash
Elena Stepanova
made changes -
2019-04-20 23:08
Labels
crash
crash need_feedback
Elena Stepanova
made changes -
2019-05-20 10:58
Component/s
Storage Engine - InnoDB
[ 10129
]
Fix Version/s
N/A
[ 14700
]
Resolution
Incomplete
[ 4
]
Status
Open
[ 1
]
Closed
[ 6
]
Christian Kueppers
made changes -
2019-08-15 10:01
Affects Version/s
10.3.17
[ 23411
]
Environment
1. environnement: Ubuntu 16.04.5, kernel: 4.4.0-141-generic x86_64, MariaDB version: 10.3.12
2. environnement: Ubuntu 16.04.4, kernel: 4.4.0-127-generic x86_64, MariaDB version: 10.3.7
1. environnement: Ubuntu 16.04.5, kernel: 4.4.0-141-generic x86_64, MariaDB version: 10.3.12
2. environnement: Ubuntu 16.04.4, kernel: 4.4.0-127-generic x86_64, MariaDB version: 10.3.7
3. environnement: Ubuntu 18.04.3, kernel: 4.15.0-46-generic, MariaDB version: 10.3.17
Elena Stepanova
made changes -
2019-08-15 10:58
Resolution
Incomplete
[ 4
]
Status
Closed
[ 6
]
Stalled
[ 10000
]
Elena Stepanova
made changes -
2019-08-15 10:58
Labels
crash need_feedback
crash
Elena Stepanova
made changes -
2019-08-15 10:59
Fix Version/s
10.3
[ 22126
]
Fix Version/s
N/A
[ 14700
]
Assignee
Marko Mäkelä
[ marko
]
Marko Mäkelä
made changes -
2019-08-23 10:04
Fix Version/s
10.4
[ 22408
]
Affects Version/s
10.4
[ 22408
]
Julien Fritsch
made changes -
2019-10-08 15:34
Labels
crash
crash need_feedback
Elena Stepanova
made changes -
2019-11-05 19:36
Labels
crash need_feedback
crash
Elena Stepanova
made changes -
2019-11-05 19:36
Labels
crash
crash need_feedback
Marko Mäkelä
made changes -
2019-11-14 19:38
Affects Version/s
10.3.7
[ 23005
]
Marko Mäkelä
made changes -
2019-11-14 19:39
Affects Version/s
10.3.7
[ 23005
]
Affects Version/s
10.3.17
[ 23411
]
Marko Mäkelä
made changes -
2019-11-14 20:04
Priority
Critical
[ 2
]
Major
[ 3
]
Sergei Golubchik
made changes -
2021-12-06 21:35
Workflow
MariaDB v3
[ 92391
]
MariaDB v4
[ 143576
]
Marko Mäkelä
made changes -
2022-06-07 15:05
Status
Stalled
[ 10000
]
In Progress
[ 3
]
Marko Mäkelä
made changes -
2022-06-07 15:06
Fix Version/s
10.6
[ 24028
]
Fix Version/s
10.7
[ 24805
]
Fix Version/s
10.8
[ 26121
]
Fix Version/s
10.9
[ 26905
]
Fix Version/s
10.3
[ 22126
]
Fix Version/s
10.4
[ 22408
]
Marko Mäkelä
made changes -
2022-06-08 07:08
issue.field.resolutiondate
2022-06-08 07:08:00.0
2022-06-08 07:08:00.482
Marko Mäkelä
made changes -
2022-06-08 07:08
Fix Version/s
10.6.9
[ 27507
]
Fix Version/s
10.7.5
[ 27505
]
Fix Version/s
10.8.4
[ 27503
]
Fix Version/s
10.9.2
[ 27115
]
Fix Version/s
10.6
[ 24028
]
Fix Version/s
10.7
[ 24805
]
Fix Version/s
10.8
[ 26121
]
Fix Version/s
10.9
[ 26905
]
Resolution
Fixed
[ 1
]
Status
In Progress
[ 3
]
Closed
[ 6
]
{"report":{"fcp":1444.7000007629395,"ttfb":537.8000001907349,"pageVisibility":"visible","entityId":72487,"key":"jira.project.issue.view-issue","isInitial":true,"threshold":1000,"elementTimings":{},"userDeviceMemory":8,"userDeviceProcessors":64,"apdex":0.5,"journeyId":"7bf2d047-5087-425d-b320-93190e869658","navigationType":0,"readyForUser":1549,"redirectCount":0,"resourceLoadedEnd":2024.3000001907349,"resourceLoadedStart":543.1000003814697,"resourceTiming":[{"duration":381.3999996185303,"initiatorType":"link","name":"https://jira.mariadb.org/s/2c21342762a6a02add1c328bed317ffd-CDN/lu2bv2/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/css/_super/batch.css","startTime":543.1000003814697,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":543.1000003814697,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":924.5,"responseStart":0,"secureConnectionStart":0},{"duration":381.6000003814697,"initiatorType":"link","name":"https://jira.mariadb.org/s/7ebd35e77e471bc30ff0eba799ebc151-CDN/lu2bv2/820016/12ta74/2380add21a9a1006587582385952de73/_/download/contextbatch/css/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true","startTime":543.5,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":543.5,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":925.1000003814697,"responseStart":0,"secureConnectionStart":0},{"duration":437.3999996185303,"initiatorType":"script","name":"https://jira.mariadb.org/s/e9b27a47da5fb0f74a35acd57e9847fb-CDN/lu2bv2/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/js/_super/batch.js?locale=en","startTime":543.6000003814697,"connectEnd":543.6000003814697,"connectStart":543.6000003814697,"domainLookupEnd":543.6000003814697,"domainLookupStart":543.6000003814697,"fetchStart":543.6000003814697,"redirectEnd":0,"redirectStart":0,"requestStart":543.6000003814697,"responseEnd":981,"responseStart":981,"secureConnectionStart":543.6000003814697},{"duration":495,"initiatorType":"script","name":"https://jira.mariadb.org/s/c32eb0da7ad9831253f8397e6cc26afd-CDN/lu2bv2/820016/12ta74/2380add21a9a1006587582385952de73/_/download/contextbatch/js/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true","startTime":543.8000001907349,"connectEnd":543.8000001907349,"connectStart":543.8000001907349,"domainLookupEnd":543.8000001907349,"domainLookupStart":543.8000001907349,"fetchStart":543.8000001907349,"redirectEnd":0,"redirectStart":0,"requestStart":543.8000001907349,"responseEnd":1038.8000001907349,"responseStart":1038.8000001907349,"secureConnectionStart":543.8000001907349},{"duration":498.69999980926514,"initiatorType":"script","name":"https://jira.mariadb.org/s/bc0bcb146314416123c992714ee00ff7-CDN/lu2bv2/820016/12ta74/c92c0caa9a024ae85b0ebdbed7fb4bd7/_/download/contextbatch/js/atl.global,-_super/batch.js?locale=en","startTime":543.9000005722046,"connectEnd":543.9000005722046,"connectStart":543.9000005722046,"domainLookupEnd":543.9000005722046,"domainLookupStart":543.9000005722046,"fetchStart":543.9000005722046,"redirectEnd":0,"redirectStart":0,"requestStart":543.9000005722046,"responseEnd":1042.6000003814697,"responseStart":1042.6000003814697,"secureConnectionStart":543.9000005722046},{"duration":498.8999996185303,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bv2/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-en/jira.webresources:calendar-en.js","startTime":544.2000007629395,"connectEnd":544.2000007629395,"connectStart":544.2000007629395,"domainLookupEnd":544.2000007629395,"domainLookupStart":544.2000007629395,"fetchStart":544.2000007629395,"redirectEnd":0,"redirectStart":0,"requestStart":544.2000007629395,"responseEnd":1043.1000003814697,"responseStart":1043.1000003814697,"secureConnectionStart":544.2000007629395},{"duration":499.19999980926514,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bv2/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-localisation-moment/jira.webresources:calendar-localisation-moment.js","startTime":544.3000001907349,"connectEnd":544.3000001907349,"connectStart":544.3000001907349,"domainLookupEnd":544.3000001907349,"domainLookupStart":544.3000001907349,"fetchStart":544.3000001907349,"redirectEnd":0,"redirectStart":0,"requestStart":544.3000001907349,"responseEnd":1043.5,"responseStart":1043.5,"secureConnectionStart":544.3000001907349},{"duration":581.7000007629395,"initiatorType":"link","name":"https://jira.mariadb.org/s/b04b06a02d1959df322d9cded3aeecc1-CDN/lu2bv2/820016/12ta74/a2ff6aa845ffc9a1d22fe23d9ee791fc/_/download/contextbatch/css/jira.global.look-and-feel,-_super/batch.css","startTime":544.5,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":544.5,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1126.2000007629395,"responseStart":0,"secureConnectionStart":0},{"duration":499.19999980926514,"initiatorType":"script","name":"https://jira.mariadb.org/rest/api/1.0/shortcuts/820016/47140b6e0a9bc2e4913da06536125810/shortcuts.js?context=issuenavigation&context=issueaction","startTime":544.7000007629395,"connectEnd":544.7000007629395,"connectStart":544.7000007629395,"domainLookupEnd":544.7000007629395,"domainLookupStart":544.7000007629395,"fetchStart":544.7000007629395,"redirectEnd":0,"redirectStart":0,"requestStart":544.7000007629395,"responseEnd":1043.9000005722046,"responseStart":1043.9000005722046,"secureConnectionStart":544.7000007629395},{"duration":581.5,"initiatorType":"link","name":"https://jira.mariadb.org/s/3ac36323ba5e4eb0af2aa7ac7211b4bb-CDN/lu2bv2/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/css/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.css?jira.create.linked.issue=true","startTime":544.8000001907349,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":544.8000001907349,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1126.3000001907349,"responseStart":0,"secureConnectionStart":0},{"duration":499.5999994277954,"initiatorType":"script","name":"https://jira.mariadb.org/s/719848dd97ebe0663199f49a3936487a-CDN/lu2bv2/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/js/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.js?jira.create.linked.issue=true&locale=en","startTime":544.9000005722046,"connectEnd":544.9000005722046,"connectStart":544.9000005722046,"domainLookupEnd":544.9000005722046,"domainLookupStart":544.9000005722046,"fetchStart":544.9000005722046,"redirectEnd":0,"redirectStart":0,"requestStart":544.9000005722046,"responseEnd":1044.5,"responseStart":1044.5,"secureConnectionStart":544.9000005722046},{"duration":1276,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bv2/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-js/jira.webresources:bigpipe-js.js","startTime":551.4000005722046,"connectEnd":551.4000005722046,"connectStart":551.4000005722046,"domainLookupEnd":551.4000005722046,"domainLookupStart":551.4000005722046,"fetchStart":551.4000005722046,"redirectEnd":0,"redirectStart":0,"requestStart":551.4000005722046,"responseEnd":1827.4000005722046,"responseStart":1827.4000005722046,"secureConnectionStart":551.4000005722046},{"duration":1470.0999994277954,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bv2/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-init/jira.webresources:bigpipe-init.js","startTime":554.2000007629395,"connectEnd":554.2000007629395,"connectStart":554.2000007629395,"domainLookupEnd":554.2000007629395,"domainLookupStart":554.2000007629395,"fetchStart":554.2000007629395,"redirectEnd":0,"redirectStart":0,"requestStart":554.2000007629395,"responseEnd":2024.3000001907349,"responseStart":2024.3000001907349,"secureConnectionStart":554.2000007629395},{"duration":702.1999998092651,"initiatorType":"xmlhttprequest","name":"https://jira.mariadb.org/rest/webResources/1.0/resources","startTime":1139.2000007629395,"connectEnd":1139.2000007629395,"connectStart":1139.2000007629395,"domainLookupEnd":1139.2000007629395,"domainLookupStart":1139.2000007629395,"fetchStart":1139.2000007629395,"redirectEnd":0,"redirectStart":0,"requestStart":1139.2000007629395,"responseEnd":1841.4000005722046,"responseStart":1841.4000005722046,"secureConnectionStart":1139.2000007629395},{"duration":623.4000005722046,"initiatorType":"script","name":"https://www.google-analytics.com/analytics.js","startTime":1437.8000001907349,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":1437.8000001907349,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":2061.2000007629395,"responseStart":0,"secureConnectionStart":0}],"fetchStart":0,"domainLookupStart":0,"domainLookupEnd":0,"connectStart":0,"connectEnd":0,"requestStart":226,"responseStart":538,"responseEnd":554,"domLoading":541,"domInteractive":2071,"domContentLoadedEventStart":2071,"domContentLoadedEventEnd":2133,"domComplete":2971,"loadEventStart":2971,"loadEventEnd":2971,"userAgent":"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)","marks":[{"name":"bigPipe.sidebar-id.start","time":2030.4000005722046},{"name":"bigPipe.sidebar-id.end","time":2031.4000005722046},{"name":"bigPipe.activity-panel-pipe-id.start","time":2031.6000003814697},{"name":"bigPipe.activity-panel-pipe-id.end","time":2038.9000005722046},{"name":"activityTabFullyLoaded","time":2158.5}],"measures":[],"correlationId":"7353c8e5c7690c","effectiveType":"4g","downlink":10,"rtt":0,"serverDuration":226,"dbReadsTimeInMs":46,"dbConnsTimeInMs":58,"applicationHash":"9d11dbea5f4be3d4cc21f03a88dd11d8c8687422","experiments":[]}}