Details
-
Bug
-
Status: Confirmed (View Workflow)
-
Major
-
Resolution: Unresolved
-
10.8.8, 10.6.14, 10.11.4, 10.7(EOL)
-
None
-
Amazon Linux using centos7-amd64 MariaDB server builds
Description
We are running a 5-node MariaDB Galera cluster and are unable to adapt the latest LTS 10.11. As a general approach we have completely automated database node setup and have only made changes to the cluster by replacing existing nodes with new ones with a changed configuration or later database server version. Here is how we're trying to update the cluster:
- Configure & launch a new node to the existing cluster
- As a part of its launch operations, the new node makes a state transfer from one node in the existing cluster (automatically; this is a standard operation)
- After successful launch of a new node, I desync & terminate the oldest node of cluster
- (repeat until all cluster nodes are of new version)
In the beginning our cluster was of version 10.6.14 and we tried to start updating directly to 10.11.4 by configuring & launching a node to the cluster. MariaDB server startup failed and reported the following error:
WSREP: Failed to start mysqld for wsrep recovery: '[Note] Starting MariaDB 10.11.4-MariaDB-log source revision 4e2b93dffef2414a11ca5edc8d215f57ee5010e5 as process 5688
[Note] InnoDB: Compressed tables use zlib 1.2.7
[Note] InnoDB: Number of transaction pools: 1
[Note] InnoDB: Using crc32 + pclmulqdq instructions
[Note] InnoDB: Using Linux native AIO
[Note] InnoDB: Initializing buffer pool, total size = XGiB, chunk size = YMiB
[Note] InnoDB: Completed initialization of buffer pool
[Note] InnoDB: File system buffers for log disabled (block size=512 bytes)
[ERROR] InnoDB: Upgrade after a crash is not supported. The redo log was created with MariaDB 10.5.10. You must start up and shut down MariaDB 10.7 or earlier.
[ERROR] InnoDB: Plugin initialization aborted with error Generic error
[Note] InnoDB: Starting shutdown...
[ERROR] Plugin 'InnoDB' init function returned error.
[ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
[Note] Plugin 'FEEDBACK' is disabled.
[ERROR] Unknown/unsupported storage engine: InnoDB
[ERROR] Aborting'
systemd[1]: mariadb.service: control process exited, code=exited status=1
systemd[1]: Failed to start MariaDB 10.11.4 database server.
We then decided to try to updating one major version step at a time. Updating the cluster to version 10.7.8 succeeded. Attempt to update from 10.7.8 to 10.8.8 failed to a similar error message again.
I found out that there's an earlier bug MDEV-27437 about quite similar issue, but it seems that that issue should not affect our use case as we are using a later version of mariadb server and the suggested wsrep_sst_method=rsync. Attached our server.cnf.
Attachments
Issue Links
- relates to
-
MDEV-31536 Galera mariadb-backup to work as SST between major versions
- Open
-
MDEV-27437 Galera snapshot transfer fails to upgrade between some major versions
- Closed
-
MDEV-28483 Include the details about 10.4 to 10.5 and newer major versions upgrades for Galera clusters into the Knowledge Base
- Closed