[MDEV-5540] MariaDB reports failure to start Created: 2014-01-18  Updated: 2015-07-20  Resolved: 2014-04-10

Status: Closed
Project: MariaDB Server
Component/s: None
Affects Version/s: 5.5.36, 10.0.6
Fix Version/s: 5.5.37, 10.0.11

Type: Bug Priority: Minor
Reporter: Philip orleans Assignee: Sergei Golubchik
Resolution: Duplicate Votes: 0
Labels: memory, timeout
Environment:

debian 7 64 bits

Issue Links:
Duplicate
is duplicated by MDEV-5068 import file from init script Closed
Relates
relates to MDEV-8509 make it possible to change MYSQLD_STA... Closed

 Description    

root@production3:/var/lib/mysql# service mysql start
 
[FAIL] Starting MariaDB database server: mysqld . . . . . . . . . . . . . . . . . . . failed!

but in fact if you wait 60 seconds, and try "mysql", the software is up.
The issue is using a large amount of memory and memlock=1
It seems that the OS takes time to lock the memory and the software reports a failure to start when in fact, the process has not finished yet.
The issue may be compounded by the fact that the machine is virtual, Vmware, and it may take longer to acquire a lock on memory.
I use large-pages=1.
I have no idea if the memory is actually locked or not

cat /proc/meminfo | grep -i pages
 
AnonPages:      11784856 kB
 
AnonHugePages:         0 kB
 
HugePages_Total:   27000
 
HugePages_Free:     1668
 
HugePages_Rsvd:        0
 
HugePages_Surp:        0
 
Hugepagesize:       2048 kB



 Comments   
Comment by Sergei Golubchik [ 2014-01-20 ]

I don't know what we can do here. I suppose one can always create a configuration where MariaDB will take a lot of time to start. I'm not sure it's possible in all cases to distinguish that from a hanged or locked up server (caused by some bug in MariaDB or even the Linux kernel).

Perhaps all you need to do is to increase MYSQLD_STARTUP_TIMEOUT, like in

root@production3:/var/lib/mysql# MYSQLD_STARTUP_TIMEOUT=60 service mysql start

Comment by Philip orleans [ 2014-01-20 ]

The solution did not work. I exported a new variable with 90, but it
did not affect the outcome.
What worked is this. In /etc/into.d/mysql, change this line

for i in $(seq 1 "${MYSQLD_STARTUP_TIMEOUT:-30}"); do
 
for
 
for i in $(seq 1 "${MYSQLD_STARTUP_TIMEOUT:-90}"); do

Maybe we should change it at least to 60?

Also, the innodb stat tables do not get created, on a brand new
install from the repository. It took me some time to figure out that
the solution was to create them manually,
The tables are called
`innodb_index_stats`
`innodb_table_stats`
and this one also is missing
`gtid_slave_pos`

but maybe we should fix this in the product.

Comment by Sergei Golubchik [ 2014-01-27 ]

about 30 vs. 60. vs 90 — I think an appropriate solution would be to source /etc/default/mysql and let MYSQLD_STARTUP_TIMEOUT to be set there.

Comment by Jan Ingvoldstad [ 2014-03-21 ]

Sourcing /etc/default/mysql would be very welcome, yes. See below for example code based on /etc/init.d/skeleton.

But IMHO, this should be solved differently, by permitting an upgrade of the package without enforcing a restart of the daemon at postinst.

As an example, one of my test cases uses a few minutes to start. This would be nice to do at a different time than package upgrade.

The work-around is to hold the mariadb-server packages and only upgrade these manually, rather than automatically.

NAME=mysql
 
 
 
# ...
 
 
 
# Read configuration variable file if it is present
 
[ -r /etc/default/$NAME ] && . /etc/default/$NAME
 
 
 
# Load the VERBOSE setting and other rcS variables
 
. /lib/init/vars.sh

Comment by Jan Ingvoldstad [ 2014-03-21 ]

On second thought: don't load /lib/init/vars.sh, since that will break service mysql [start|stop], at least as it stands now.

Comment by Jan Ingvoldstad [ 2014-03-21 ]

Here's a patch:

--- 10.0.9/etc/init.d/mysql     2014-03-21 11:21:13.848030546 +0100
 
+++ 10.0.10/etc/init.d/mysql    2014-03-21 11:51:11.284712299 +0100
 
@@ -17,8 +17,19 @@
 
 set -u
 
 ${DEBIAN_SCRIPT_DEBUG:+ set -v -x}
 
 
-test -x /usr/sbin/mysqld || exit 0
 
-
 
+DESC="MariaDB database server"
 
+NAME=mysql
 
+DAEMON=/usr/sbin/mysqld
 
+
 
+# Exit if the package is not installed
 
+[ -x "$DAEMON" ] || exit 0
 
+
 
+# Read configuration variable file if it is present
 
+[ -r /etc/default/$NAME ] && . /etc/default/$NAME
 
+
 
+# Define LSB log_* functions.
 
+# Depend on lsb-base (>= 3.2-14) to ensure that this file is present
 
+# and status_of_proc is working.
 
 . /lib/lsb/init-functions
 
 
 SELF=$(cd $(dirname $0); pwd -P)/$(basename $0)
 
@@ -26,7 +37,7 @@
 
 MYADMIN="/usr/bin/mysqladmin --defaults-file=/etc/mysql/debian.cnf"
 
 
 # priority can be overriden and "-s" adds output to stderr
 
-ERR_LOGGER="logger -p daemon.err -t /etc/init.d/mysql -i"
 
+ERR_LOGGER="logger -p daemon.err -t ${SELF} -i"
 
 
 # Safeguard (relative paths, core dumps..)
 
 cd /
 
@@ -41,7 +52,7 @@
 
 #
 
 # Usage: void mysqld_get_param option
 
 mysqld_get_param() {
 
-       /usr/sbin/mysqld --print-defaults \
 
+       ${DAEMON} --print-defaults \
 
                | tr " " "\n" \
 
                | grep -- "--$1" \
 
                | tail -n 1 \
 
@@ -97,7 +108,7 @@
 
   'start')
 
        sanity_checks;
 
        # Start daemon
 
-       log_daemon_msg "Starting MariaDB database server" "mysqld"
 
+       log_daemon_msg "Starting ${DESC}" "mysqld"
 
        if mysqld_status check_alive nowarn; then
 
           log_progress_msg "already running"
 
           log_end_msg 0
 
@@ -131,7 +142,7 @@
 
        # at least for cron, we can rely on it here, too. (although we have 
        # to specify it explicit as e.g. sudo environments points to the normal
 
        # users home and not /root)
 
-       log_daemon_msg "Stopping MariaDB database server" "mysqld"
 
+       log_daemon_msg "Stopping ${DESC}" "mysqld"
 
        if ! mysqld_status check_dead nowarn; then
 
          set +e
 
          shutdown_out=`$MYADMIN shutdown 2>&1`; r=$?
 
@@ -139,7 +150,7 @@
 
          if [ "$r" -ne 0 ]; then
 
            log_end_msg 1
 
            [ "$VERBOSE" != "no" ] && log_failure_msg "Error: $shutdown_out"
 
-           log_daemon_msg "Killing MariaDB database server by signal" "mysqld"
 
+           log_daemon_msg "Killing ${DESC} by signal" "mysqld"
 
            killall -15 mysqld
 
             server_down=
 
            for i in `seq 1 600`; do
 
@@ -165,7 +176,7 @@
 
        ;;
 
 
   'reload'|'force-reload')
 
-       log_daemon_msg "Reloading MariaDB database server" "mysqld"
 
+       log_daemon_msg "Reloading ${DESC}" "mysqld"
 
        $MYADMIN reload
 
        log_end_msg 0
 
        ;;

Comment by Sergei Golubchik [ 2014-04-10 ]

Fixed in MDEV-5068

Comment by Nils Meyer [ 2015-07-20 ]

The fix in MDEV-5068 doesn't apply to Debian.

Generated at Thu Feb 08 07:05:09 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.