[MDEV-9039] Can't upgrade MariaDB to to 10.1.8 version from 10.0.21 Created: 2015-10-30  Updated: 2015-11-16  Resolved: 2015-11-06

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - Sequence, XA
Affects Version/s: 10.1
Fix Version/s: 10.1.9

Type: Bug Priority: Major
Reporter: Mikhail Gavrilov Assignee: Sergei Golubchik
Resolution: Fixed Votes: 0
Labels: None

Attachments: File mariadb.log    
Issue Links:
Blocks
is blocked by MDEV-8794 Recovery failed! You must enable exac... Closed

 Description   

After removing old packages and install new.
Server couldn't run again.

[root@vmi25208 mysql]# systemctl status mariadb.service -l
 
● mariadb.service - MariaDB database server
 
   Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled; vendor preset: disabled)
 
  Drop-In: /etc/systemd/system/mariadb.service.d
 
           └─migrated-from-my.cnf-settings.conf
 
   Active: failed (Result: exit-code) since Пт 2015-10-30 03:27:11 YEKT; 3min 12s ago
 
  Process: 28986 ExecStart=/usr/sbin/mysqld $MYSQLD_OPTS (code=exited, status=1/FAILURE)
 
 Main PID: 28986 (code=exited, status=1/FAILURE)
 
   Status: "MariaDB server is down"
 
 
 
окт 30 03:27:09 vmi25208.contabo.net mysqld[28986]: 2015-10-30  3:27:09 139992865175296 [Note] InnoDB: Dumping buffer pool(s) not yet started
 
окт 30 03:27:09 vmi25208.contabo.net mysqld[28986]: 2015-10-30  3:27:09 139993609361600 [Note] Recovering after a crash using tc.log
 
окт 30 03:27:09 vmi25208.contabo.net mysqld[28986]: 2015-10-30  3:27:09 139993609361600 [ERROR] Recovery failed! You must enable exactly 2 storage engines that support two-phase commit protocol
 
окт 30 03:27:09 vmi25208.contabo.net mysqld[28986]: 2015-10-30  3:27:09 139993609361600 [ERROR] Crash recovery failed. Either correct the problem (if it's, for example, out of memory error) and restart, or delete tc log and start mysqld with --tc-heuristic-recover={commit|rollback}
 
окт 30 03:27:09 vmi25208.contabo.net mysqld[28986]: 2015-10-30  3:27:09 139993609361600 [ERROR] Can't init tc log
 
окт 30 03:27:09 vmi25208.contabo.net mysqld[28986]: 2015-10-30  3:27:09 139993609361600 [ERROR] Aborting
 
окт 30 03:27:11 vmi25208.contabo.net systemd[1]: mariadb.service: main process exited, code=exited, status=1/FAILURE
 
окт 30 03:27:11 vmi25208.contabo.net systemd[1]: Failed to start MariaDB database server.
 
окт 30 03:27:11 vmi25208.contabo.net systemd[1]: Unit mariadb.service entered failed state.
 
окт 30 03:27:11 vmi25208.contabo.net systemd[1]: mariadb.service failed.
 
 



 Comments   
Comment by Elena Stepanova [ 2015-10-30 ]

Hi Mikhail,

You've hit the bug MDEV-8794.
To start the server, it should be enough just to remove tc.log from your datadir.

Comment by Elena Stepanova [ 2015-10-30 ]

mikhail,

I would like to see which engines caused the problem in your upgrade. Could you please quote or attach the whole error log, and also output of SHOW ENGINES after you start the server?

Comment by Mikhail Gavrilov [ 2015-10-30 ]

Could you please quote or attach the whole error log

I see that new server not add anything to /var/log/mariadb/mariadb.log

151030  3:07:13 [Note] /usr/libexec/mysqld: ready for connections.
 
 
 
<- here must be new server logging
 
 
 
Version: '10.0.21-MariaDB'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MariaDB Server
 
151030 03:36:29 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql

and also output of SHOW ENGINES after you start the server?

How do it? New server not still not run ...
I can showing that the output of the previous server. 

Engine              Support  Comment                                                                     Transactions  XA      Savepoints  
------------------  -------  --------------------------------------------------------------------------  ------------  ------  ------------
 
MyISAM              YES      MyISAM storage engine                                                       NO            NO      NO          
CSV                 YES      CSV storage engine                                                          NO            NO      NO          
MRG_MyISAM          YES      Collection of identical MyISAM tables                                       NO            NO      NO          
BLACKHOLE           YES      /dev/null storage engine (anything you write to it disappears)              NO            NO      NO          
FEDERATED           YES      FederatedX pluggable storage engine                                         YES           NO      YES         
SEQUENCE            YES      Generated tables filled with sequential values                              YES           YES     YES         
PERFORMANCE_SCHEMA  YES      Performance Schema                                                          NO            NO      NO          
ARCHIVE             YES      Archive storage engine                                                      NO            NO      NO          
MEMORY              YES      Hash based, stored in memory, useful for temporary tables                   NO            NO      NO          
InnoDB              DEFAULT  Percona-XtraDB, Supports transactions, row-level locking, and foreign keys  YES           YES     YES         
Aria                YES      Crash-safe tables with MyISAM heritage                                      NO            NO      NO

Comment by Elena Stepanova [ 2015-10-30 ]

mikhail,
Thanks.

Did you see my first comment?
You've hit the bug MDEV-8794.
To start the server, it should be enough just to remove tc.log from your datadir.

Comment by Mikhail Gavrilov [ 2015-10-30 ]

To start the server, it should be enough just to remove tc.log from your datadir.

Very strange I make second attempt without removing tc.log and server start fine.
Gremlins?

# dnf remove mariadb*
 
Dependencies resolved.
 
===================================================================================================================================================================================================================
 
 Package                                               Arch                                          Version                                                 Repository                                       Size
 
===================================================================================================================================================================================================================
 
Removing:
 
 mariadb                                               x86_64                                        1:10.0.21-1.fc22                                        @updates                                         37 M
 
 mariadb-common                                        x86_64                                        1:10.0.21-1.fc22                                        @updates                                        328 k
 
 mariadb-config                                        x86_64                                        1:10.0.21-1.fc22                                        @updates                                        577  
 mariadb-errmsg                                        x86_64                                        1:10.0.21-1.fc22                                        @updates                                        1.9 M
 
 mariadb-libs                                          x86_64                                        1:10.0.21-1.fc22                                        @updates                                        3.4 M
 
 mariadb-server                                        x86_64                                        1:10.0.21-1.fc22                                        @updates                                         86 M
 
 perl-DBD-MySQL                                        x86_64                                        4.031-1.fc22                                            @fedora                                         347 k
 
 
 
Transaction Summary
 
===================================================================================================================================================================================================================
 
Remove  7 Packages
 
 
 
Installed size: 130 M
 
Is this ok [y/N]: y
 
Running transaction check
 
Transaction check succeeded.
 
Running transaction test
 
Transaction test succeeded.
 
Running transaction
 
  Erasing     : mariadb-server-1:10.0.21-1.fc22.x86_64                                                                                                                                                         1/7 
  Erasing     : mariadb-errmsg-1:10.0.21-1.fc22.x86_64                                                                                                                                                         2/7 
  Erasing     : mariadb-1:10.0.21-1.fc22.x86_64                                                                                                                                                                3/7 
  Erasing     : perl-DBD-MySQL-4.031-1.fc22.x86_64                                                                                                                                                             4/7 
  Erasing     : mariadb-libs-1:10.0.21-1.fc22.x86_64                                                                                                                                                           5/7 
  Erasing     : mariadb-common-1:10.0.21-1.fc22.x86_64                                                                                                                                                         6/7 
  Erasing     : mariadb-config-1:10.0.21-1.fc22.x86_64                                                                                                                                                         7/7 
  Verifying   : mariadb-1:10.0.21-1.fc22.x86_64                                                                                                                                                                1/7 
  Verifying   : mariadb-common-1:10.0.21-1.fc22.x86_64                                                                                                                                                         2/7 
  Verifying   : mariadb-config-1:10.0.21-1.fc22.x86_64                                                                                                                                                         3/7 
  Verifying   : mariadb-errmsg-1:10.0.21-1.fc22.x86_64                                                                                                                                                         4/7 
  Verifying   : mariadb-libs-1:10.0.21-1.fc22.x86_64                                                                                                                                                           5/7 
  Verifying   : mariadb-server-1:10.0.21-1.fc22.x86_64                                                                                                                                                         6/7 
  Verifying   : perl-DBD-MySQL-4.031-1.fc22.x86_64                                                                                                                                                             7/7 
 
 
Removed:
 
  mariadb.x86_64 1:10.0.21-1.fc22           mariadb-common.x86_64 1:10.0.21-1.fc22    mariadb-config.x86_64 1:10.0.21-1.fc22    mariadb-errmsg.x86_64 1:10.0.21-1.fc22    mariadb-libs.x86_64 1:10.0.21-1.fc22   
  mariadb-server.x86_64 1:10.0.21-1.fc22    perl-DBD-MySQL.x86_64 4.031-1.fc22       
 
 
Complete!
 
 
 
[root@vmi25208 yum.repos.d]# dnf install MariaDB-server
 
Last metadata expiration check performed 0:43:53 ago on Fri Oct 30 17:30:22 2015.
 
Dependencies resolved.
 
===================================================================================================================================================================================================================
 
 Package                                                Arch                                           Version                                               Repository                                       Size
 
===================================================================================================================================================================================================================
 
Installing:
 
 MariaDB-client                                         x86_64                                         10.1.8-1.fc21                                         mariadb                                          34 M
 
 MariaDB-common                                         x86_64                                         10.1.8-1.fc21                                         mariadb                                          47 k
 
 MariaDB-server                                         x86_64                                         10.1.8-1.fc21                                         mariadb                                          87 M
 
 jemalloc                                               x86_64                                         3.6.0-8.fc22                                          fedora                                          116 k
 
 
 
Transaction Summary
 
===================================================================================================================================================================================================================
 
Install  4 Packages
 
 
 
Total download size: 121 M
 
Installed size: 529 M
 
Is this ok [y/N]: y
 
Downloading Packages:
 
(1/4): MariaDB-10.1.8-fedora21-x86_64-common.rpm                                                                                                                                    50 kB/s |  47 kB     00:00    
(2/4): jemalloc-3.6.0-8.fc22.x86_64.rpm                                                                                                                                            279 kB/s | 116 kB     00:00    
(3/4): MariaDB-10.1.8-fedora21-x86_64-client.rpm                                                                                                                                   3.6 MB/s |  34 MB     00:09    
(4/4): MariaDB-10.1.8-fedora21-x86_64-server.rpm                                                                                                                                   2.9 MB/s |  87 MB     00:30    
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 
Total                                                                                                                                                                              3.8 MB/s | 121 MB     00:31     
Running transaction check
 
Transaction check succeeded.
 
Running transaction test
 
Transaction test succeeded.
 
Running transaction
 
  Installing  : MariaDB-common-10.1.8-1.fc21.x86_64                                                                                                                                                            1/4 
  Installing  : MariaDB-client-10.1.8-1.fc21.x86_64                                                                                                                                                            2/4 
  Installing  : jemalloc-3.6.0-8.fc22.x86_64                                                                                                                                                                   3/4 
  Installing  : MariaDB-server-10.1.8-1.fc21.x86_64                                                                                                                                                            4/4 
  Verifying   : MariaDB-server-10.1.8-1.fc21.x86_64                                                                                                                                                            1/4 
  Verifying   : MariaDB-client-10.1.8-1.fc21.x86_64                                                                                                                                                            2/4 
  Verifying   : MariaDB-common-10.1.8-1.fc21.x86_64                                                                                                                                                            3/4 
  Verifying   : jemalloc-3.6.0-8.fc22.x86_64                                                                                                                                                                   4/4 
 
 
Installed:
 
  MariaDB-client.x86_64 10.1.8-1.fc21                   MariaDB-common.x86_64 10.1.8-1.fc21                   MariaDB-server.x86_64 10.1.8-1.fc21                   jemalloc.x86_64 3.6.0-8.fc22                  
 
 
Complete!
 
[root@vmi25208 yum.repos.d]# systemctl start mariadb.service
 
 

Comment by Elena Stepanova [ 2015-10-31 ]

Well, it's a little surprising, but rather fortunate under the circumstances.

The problem (MDEV-8794) appears when three events coincide:

  • you had at least two XA-capable engines enabled (so that tc.log was created);
  • your server was not shut down gracefully;
  • you attempt to start 10.1 server with a different number of XA-capable engines.

It specifically affects upgrade to 10.1, because in 10.1 the SEQUENCE engine is enabled by default, so it changes the number of engines automatically.

There are two solutions when it happens: either remove tc.log, or get the server start and shut down gracefully. As soon as either of these happens, it's fine to proceed.
I suppose it's possible that at some point during re-installation server was able to start up and shut down properly, which solved the problem.

Generated at Thu Feb 08 07:31:40 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.