Details
-
Bug
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Cannot Reproduce
-
10.2.21, 10.3.25
-
Docker version 18.09.1, build 4c52b90
Container version: 10.2.21-MariaDB-1:10.2.21
Description
We have recently updated a mariadb 10.2.17 to 10.2.21.
The update was very smooth without any errors. At least once a day since we did the minor version update we have a crash. The crash happens mostly on our main write node, the other two are read nodes.
The differences in the logs, i see with the new version is this Warning multiple times throughout the day:
{"log":"2019-01-29 0:14:00 139950395401984 [Warning] WSREP: SQL statement was ineffective thd: 2929856 buf: 226\n","stream":"stderr","time":"2019-01-28T22:14:00.682735534Z"} |
{"log":"schema: xxxxxx \n","stream":"stderr","time":"2019-01-28T22:14:00.682766101Z"} |
{"log":"QUERY: commit\n","stream":"stderr","time":"2019-01-28T22:14:00.682771205Z"} |
{"log":" =\u003e Skipping replication\n","stream":"stderr","time":"2019-01-28T22:14:00.68277568Z"} |
Now the crash happens after multiple of these warnings in this manner:
{"log":"2019-01-29 0:14:00 139950395401984 [Warning] WSREP: SQL statement was ineffective thd: 2929856 buf: 226\n","stream":"stderr","time":"2019-01-28T22:14:00.682735534Z"} |
{"log":"schema: XXXXXX \n","stream":"stderr","time":"2019-01-28T22:14:00.682766101Z"} |
{"log":"QUERY: commit\n","stream":"stderr","time":"2019-01-28T22:14:00.682771205Z"} |
{"log":" =\u003e Skipping replication\n","stream":"stderr","time":"2019-01-28T22:14:00.68277568Z"} |
{"log":"2019-01-29 0:14:00 139950400808704 [Warning] WSREP: SQL statement was ineffective thd: 2929858 buf: 226\n","stream":"stderr","time":"2019-01-28T22:14:00.692498864Z"} |
{"log":"schema: XXXXXX \n","stream":"stderr","time":"2019-01-28T22:14:00.692525167Z"} |
{"log":"QUERY: commit\n","stream":"stderr","time":"2019-01-28T22:14:00.692534055Z"} |
{"log":" =\u003e Skipping replication\n","stream":"stderr","time":"2019-01-28T22:14:00.692541387Z"} |
{"log":"2019-01-29 0:15:03 139950402430720 [Warning] WSREP: SQL statement was ineffective thd: 2931591 buf: 214\n","stream":"stderr","time":"2019-01-28T22:15:03.76604226Z"} |
{"log":"schema: XXXXXX \n","stream":"stderr","time":"2019-01-28T22:15:03.766070584Z"} |
{"log":"QUERY: commit\n","stream":"stderr","time":"2019-01-28T22:15:03.766076151Z"} |
{"log":" =\u003e Skipping replication\n","stream":"stderr","time":"2019-01-28T22:15:03.76608061Z"} |
{"log":"2019-01-29 0:15:03 139950402430720 [ERROR] WSREP: FSM: no such a transition ROLLED_BACK -\u003e ROLLED_BACK\n","stream":"stderr","time":"2019-01-28T22:15:03.766085945Z"} |
{"log":"190129 0:15:03 [ERROR] mysqld got signal 6 ;\n","stream":"stderr","time":"2019-01-28T22:15:03.766242364Z"} |
After these error the line before the crash is :
{"log":"Fatal signal 11 while backtracing\n","stream":"stderr","time":"2019-01-28T22:15:14.47965349Z"} |
The node later recovers connecting back to the cluster.
I have looked up this error:
WSREP: FSM: no such a transition ROLLED_BACK
|
in issue: https://mariadb.atlassian.net/browse/MDEV-7217
It seems pretty old and i shouldn't have it in this up to date version of mariadb. Any suggestions are welcome. Any more information i would be glad to provide.