[MDEV-9335] service start fails with timeout on system using systemctl (tarball install) Created: 2015-12-28  Updated: 2016-01-22  Resolved: 2016-01-22

Status: Closed
Project: MariaDB Server
Component/s: Platform Debian
Affects Version/s: 10.1.10
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: Ryan RAJKOMAR Assignee: Unassigned
Resolution: Not a Bug Votes: 0
Labels: None
Environment:

Debian Jessie (8.2)



 Description   

Note : my file in /etc/init.d/ is called mariadb_10_1_10_main

When running sudo service mariadb_10_1_10_main start ,

sudo service mariadb_10_1_10_main status 

returns :

● mariadb_10_1_10_main.service - LSB: start and stop MySQL
   Loaded: loaded (/etc/init.d/mariadb_10_1_10_main)
   Active: activating (start) since lun. 2015-12-28 16:59:36 CET; 1min 34s ago
  Control: 29285 (mariadb_10_1_10)
   CGroup: /system.slice/mariadb_10_1_10_main.service
           ├─29285 /bin/sh /etc/init.d/mariadb_10_1_10_main start
           ├─29291 /bin/sh /opt/mariadb/10_1_10/main/bin/mysqld_safe --datadir=/opt/mariadb/10_1_10/main/data --p...
           ├─29528 /opt/mariadb/10_1_10/main/bin/mysqld --defaults-extra-file=/opt/mariadb/10_1_10/main/data/my.c...
           └─29839 sleep 1
 
déc. 28 16:59:36 gitlab mariadb_10_1_10_main[29285]: Starting MySQL

Once start has timeout-ed,

sudo service mariadb_10_1_10_main status 

returns :

● mariadb_10_1_10_main.service - LSB: start and stop MySQL
   Loaded: loaded (/etc/init.d/mariadb_10_1_10_main)
   Active: failed (Result: timeout) since lun. 2015-12-28 16:44:29 CET; 7min ago
  Process: 27521 ExecStart=/etc/init.d/mariadb_10_1_10_main start (code=killed, signal=TERM)
 
déc. 28 16:39:29 gitlab mariadb_10_1_10_main[27521]: Starting MySQL
déc. 28 16:44:29 gitlab systemd[1]: mariadb_10_1_10_main.service start operation timed out. Terminating.
déc. 28 16:44:29 gitlab systemd[1]: Failed to start LSB: start and stop MySQL.
déc. 28 16:44:29 gitlab systemd[1]: Unit mariadb_10_1_10_main.service entered failed state.
déc. 28 16:44:29 gitlab mariadb_10_1_10_main[27521]: ..............................................................
Hint: Some lines were ellipsized, use -l to show in full.

Although the server has correctly been started (I can conenct using the socket or tcp connection), it would ssem that the provided script in support-files (mysql.server) may not be up to date for systemd usage or that another version of the file designed for systemd should be provided as well



 Comments   
Comment by Ryan RAJKOMAR [ 2015-12-28 ]

FYI : this issue only happens on a system installed from scratch with Debian Jessie : system upgraded from Wheezy seem to keep using the "old way" and the issue mentionned in MDEV-8593 seems to be fixed in 10.1.10 (server starts fine even with a custom socket name)

Also, the issue did not occur with MySQL 5.6 on a similar system (Debian 8.2, with everything configured the same way and mysql installed also from tarball)

Comment by Elena Stepanova [ 2016-01-10 ]

Please describe step-by-step how you configure the service upon installation from the binary tarball.

Comment by Ryan RAJKOMAR [ 2016-01-12 ]

Here is the process I followed (intially provided by a MySQL DBA to me and I then adapted it to the mariadb install)

Requirements (these were installed prior to all the following depending on which version I was trying to install) :

Mariadb 10.0 : libaio1 libaio-dev
Mariadb 10.1 : deps Mariadb 10.0 + libjemalloc1 libjemalloc-dev

1

# Create a user "mariadb" and add it to a group "mariadb" : 
sudo groupadd mariadb && sudo useradd -g mariadb mariadb

2

# Get latest tarball from maridb website : 
sudo wget {url}
 
# untar it : 
sudo tar -xvzf ./{downloaded_file.tar.gz}

3

# a) Create if not exists an /opt/mariadb_{version} directory : 
sudo mkdir /opt/mariadb_{version}
 
# b) Rename unpacked folder in newly create one : 
sudo mv {downloaded_file} /opt/mariadb_{version}/{newname} && sudo rm {downloaded_file.tar.gz}

4

sudo chown root:root -R {newname} && sudo chown mariadb:mariadb -R {newname}/data

5

cd {newname}

6

sudo ./scripts/mysql_install_db --basedir={/path/to/newname} --user=mariadb

7

sudo cp ./support-files/my-medium.cnf ./data/my.cnf

8

sudo vi ./data/my.cnf
 
# > add client section :
 [client]
 #password       = your_password
 port            = {desiredport | ex :3307}
 socket          = {path/to/sock | ex : /tmp/mariadb3307.sock}
 
# > modify mysqld section 
 [mysqld]
 user  = {user defined in mysql_install_db, if not standard (=> not mysql)}
 port            = {desiredport | ex :3307}
 socket          = {path/to/sock | ex : /tmp/mariadb3307.sock}
 
# > activate innodb on the right folder :
 innodb_data_home_dir = {/path/to/newname}/data
 innodb_data_file_path = ibdata1:10M:autoextend
 innodb_log_group_home_dir = {/path/to/newname}/data
# {all other innodb_* lines were uncommented}
 
# > {these three lines were added in mysqld section }
# force utf8 :
 character-set-server=utf8
 skip-character-set-client-handshake

>> save and close

9

# customize startup script:
 
sudo cp ./support-files/mysql.server /etc/init.d/mariadb_{version}_{instancename}
sudo vi /etc/init.d/mariadb_{version}_{instancename}
 
# > modify lines 46 & 47:
 basedir={/path/to/instance/basedir >> /path/to/newname}
 datadir={/path/to/instance/basedir/data >> /path/to/newname/data}

>> save and close

10

# test : 
sudo /etc/init.d/mariadb_{version}_{instancename} status
[FAIL] MySQL is not running ... failed!

11

# Start instance : 
sudo /etc/init.d/mariadb_{version}_{instancename} start

12

# test status : 
sudo /etc/init.d/mariadb_{version}_{instancename} status
# >> should return something equivalent (this is the the wheezy response) 
[ ok ] MySQL running ({pid}).

13

# Setup root password
sudo ./bin/mysqladmin --socket={path_to_instance_socketfile} -u root password
# and at each prompt enter {rootpassword}

14

# Test root connection
sudo ./bin/mysql --socket={path_to_instance_socketfile} -uroot -p
# and at prompt enter {rootpassword}

15

# Security enhancements :
DROP DATABASE test;
DELETE FROM mysql.user WHERE user = '' OR password = ''; 

Comment by Elena Stepanova [ 2016-01-22 ]

rrajkomar,

Thank you for the full description.
There is a flaw in the procedure. You use a non-default port/socket and put your my.cnf file into a non-default location, but you don't point mysqladmin to this cnf file in wait_for_ready routine in mysql.server; so, it attempts to ping on a default socket, cannot, and eventually goes down with the timeout. Please try to fix it and check if it starts working as you expect.

Comment by Ryan RAJKOMAR [ 2016-01-22 ]

Hello,

Thank you for the time you took to checkout this issue but I'm afraid there might be also something else :

I tried to run the command "manually" by executing :

/opt/mariadb/10_1_10/main/bin/mysqladmin --socket=/tmp/mariadb_10_1_10_main.sock ping 

>> Error with user/pass (I disabled login without password)

sudo /opt/mariadb/10_1_10/main/bin/mysqladmin --socket=/tmp/mariadb_10_1_10_main.sock ping 

>> Error with user/pass (I disabled root login without password)

/opt/mariadb/10_1_10/main/bin/mysqladmin --socket=/tmp/mariadb_10_1_10_main.sock -uroot -p ping 

>> Worked (I typed password when prompted)

It would seem that we can't disable root login without password without the process breaking which is bad IMO.

That being said how can I make the process use the right socket (right my.cnf file) without changing too much (using a customizeable port/socket is a requirement in my case).
Please remember that I did not have this issue with MySQL 5.6 so there must be a way to configure this behaviour globally

Comment by Elena Stepanova [ 2016-01-22 ]

rrajkomar,

First, I don't really know what you mean by "disable root login without password". I'll take a wild guess that you mean setting an empty password for local root and will further write under this assumption.

Of course you can do everything with your logins and passwords as long as you do it right, MySQL and MariaDB are not different in this regard, as long as you do it right.

I already told you above that you need to point your mysqladmin at the exact cnf file, but you pointed it at the socket instead. It's not the same thing (and you didn't even use --no-defaults.
The most likely reason for the problem above is that you have a configuration file in one of default locations which already contains a password, so as long as you don't provide -p option, the client attempts to use the password from the config file.
Probably it's some debian.cnf file or alike.
Please read the manual about default configuration locations and ways to customize the usage of configuration files.

(Of course, the other reason could be that you didn't run FLUSH PRIVILEGES after updating mysql.user table, but for the sake of sanity I'll assume that you did, or that you used SET PASSWORD rather than a direct update).

The environment that you are trying to use is what they usually call 'advanced setup', and should only be used if one has solid understanding of MySQL configuration, otherwise it's just too error-prone. Basic problems like the one above can happen any unlucky day, no matter whether you are currently running MariaDB or MySQL. You should be able to solve them on the fly.

It seems that you need to involve the DBA who wrote instructions for you into your setup process, especially since apparently they didn't bother to actually try their own instruction even once, otherwise the problem with wait_for_ready would have become obvious.
If you don't have the DBA anymore, you might want to consider getting a support contract. Otherwise it's better to stick to the default configuration where you don't need to think about, well, almost anything

Comment by Ryan RAJKOMAR [ 2016-01-22 ]

As I explained before the process was given to me for a mysql server but since mariadb advertises everywhere that all config options working for mysql would work for mariadb I thought the script would work the same (and aside from this part it does).

After looking carefully for differences, it seems the issue come from the fact that mariadb waits for a pingback while mysql looks for the pid file being created.

After adding --defaults-file=

{path}

to themysqladmin command in wait_for_ready it did indeed work.

Generated at Thu Feb 08 07:33:53 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.