[MDEV-21292] MariaDB fails with timeout on boot Created: 2019-12-11  Updated: 2020-05-05  Resolved: 2020-05-05

Status: Closed
Project: MariaDB Server
Component/s: Scripts & Clients
Affects Version/s: 10.3.17
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: Matthias S Assignee: Axel Schwenke
Resolution: Not a Bug Votes: 0
Labels: need_feedback
Environment:

Raspian Buster (10) on Raspberry Pi


 Description   

I moved my data directory to another location. It works if this is on the same disk, just at a different location (i.e., moved /var/lib/mysql/ and edited the datadir entry in the server.cnf).

However, when the data directory is on a mounted drive it does not work well. It generally works, but when the system boots, MariaDB times out.

I changed the systemd service for mariadb to ensure that mounting took place: I added RequiresMountsFor=/mnt/foo. I can see by the timestamps that the service is started as soon as the disk is mounted. Starting MariaDB works occasionally, but most times mariadb times out:

Dec 11 10:41:59 rpi systemd[1]: Mounted /mnt/foo.
 
...
 
Dec 11 10:41:59 rpi systemd[1]: Starting MariaDB 10.3.17 database server...
 
Dec 11 10:41:59 rpi mysqld[962]: 2019-12-11 10:41:59 0 [Note] /usr/sbin/mysqld (mysqld 10.3.17-MariaDB-0+deb10u1) starting as process 962 ...
 
Dec 11 10:43:29 rpi systemd[1]: mariadb.service: Start operation timed out. Terminating.
 
Dec 11 10:43:32 rpi systemd[1]: mariadb.service: Failed with result 'timeout'.
 
Dec 11 10:43:32 rpi systemd[1]: Failed to start MariaDB 10.3.17 database server.

Starting it manually after works fine.

 Comments   
Comment by Axel Schwenke [ 2019-12-13 ]

Hello mattsch, if MariaDB comes up occasionally, this might simply be a case where the startup time is near the default timeout of 90 seconds. Sometimes it's faster and works, sometimes it's slower and fails.

Then it would be enough to increase the timeout. This is described in this Knowledge Base article.

On the other hand this might be a more serious problem. Do you happen to know how long MariaDB takes to start when it works? Please also check the MariaDB error log for anything unusual.

Comment by Matthias S [ 2019-12-14 ]

Hello axel, it takes around 2 seconds to start MariaDB. It's a fresh installation.

Setting the start timeout to infinite indeed helps in the sense that it eventually starts. However, it takes around 3 minutes.

Here's the part of the error log:

2019-12-13 22:20:03 0 [Note] InnoDB: Using Linux native AIO
 
2019-12-13 22:20:03 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
 
2019-12-13 22:20:03 0 [Note] InnoDB: Uses event mutexes
 
2019-12-13 22:20:03 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
 
2019-12-13 22:20:03 0 [Note] InnoDB: Number of pools: 1
 
2019-12-13 22:20:03 0 [Note] InnoDB: Using generic crc32 instructions
 
2019-12-13 22:20:03 0 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
 
2019-12-13 22:20:03 0 [Note] InnoDB: Completed initialization of buffer pool
 
2019-12-13 22:20:03 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
 
2019-12-13 22:22:45 0 [Note] InnoDB: 128 out of 128 rollback segments are active.
 
2019-12-13 22:23:16 0 [Note] InnoDB: Creating shared tablespace for temporary tables
 
2019-12-13 22:23:16 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
 
2019-12-13 22:23:16 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
 
2019-12-13 22:23:16 0 [Note] InnoDB: 10.3.17 started; log sequence number 1631022; transaction id 21
 
2019-12-13 22:23:16 0 [Note] InnoDB: Loading buffer pool(s) from /mnt/foo/mysql/ib_buffer_pool
 
2019-12-13 22:23:16 0 [Note] Plugin 'FEEDBACK' is disabled.
 
2019-12-13 22:23:16 0 [Note] InnoDB: Buffer pool(s) load completed at 191213 22:23:16
 
2019-12-13 22:23:16 0 [Note] Server socket created on IP: '127.0.0.1'.
 
2019-12-13 22:23:16 0 [Note] Reading of all Master_info entries succeeded
 
2019-12-13 22:23:16 0 [Note] Added new Master_info '' to hash table
 
2019-12-13 22:23:16 0 [Note] /usr/sbin/mysqld: ready for connections.
 
Version: '10.3.17-MariaDB-0+deb10u1'  socket: '/run/mysqld/mysqld.sock'  port: 3306  Raspbian 10

In the case that it works (datadir in /var/lib/mysql) I see the same in the error log just without the long pause in between.

Comment by Matthias S [ 2019-12-29 ]

Through a coincidence I found out that this is caused by the disk taking forever to read from after mounting. It is caused by an "UAS incompatibility" of the SATA-to-USB interface and the Raspberry Pi 4. Once I enabled the quirks mode for this drive MariaDB started without a problem after booting.

Please close this issue.

Generated at Thu Feb 08 09:06:02 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.