Re: When installing 5.2 from debian packages, debian-sys-maint user is misconfigured

terminal output
LPexportBug674413_mariadb_bug_report.txt

Re: When installing 5.2 from debian packages, debian-sys-maint user is misconfigured
1. The debian-sys-maint account should be created during install, it is used internally by the postinst scripts. My guess is that something went wrong with this in the many failed install attempts. It is a huge problem that we do not have apt-get'able package repositories at the moment, packages need to be installed in a specific order, which is very hard to get right manually.

You may have to create that user manually (or remove --purge + reinstall in right order should also work, if you haven't yet put data in).

2. When the postinst script installs the initial database, it runs mysqld with special options that disable InnoDB (to reduce risk of failing to install due to problems with innodb options), this is nothing to worry about (agree it is a bit confusing to see these in the logs, and also the warnings about --log-xxx options).

3. The failure to shutdown is a consequence of the problem with the debian-sys-maint account.

Looking at the log about the problem with debian-sys-maint ... It says 'using password: NO'. This suggests something wrong with the file cat /etc/mysql/debian.cnf. Can you show the contents of this file, with any password replaced by XXX ? It should look something like this:

[client]
host = localhost
user = debian-sys-maint
password = XXX
socket = /var/run/mysqld/mysqld.sock
[mysql_upgrade]
user = debian-sys-maint
password = XXX
socket = /var/run/mysqld/mysqld.sock
basedir = /usr

If this looks ok, you can check that `mysqladmin --defaults-file=/etc/mysql/debian.cnf ping` works; if not then maybe the account inside mariadb is wrong/missing (but the Using password: NO suggests a problem with debian.cnf)

Once we understand better what the problem is, we can try see if the install procedure can be made more robust to not lose the debian-sys-maint account in case of install failure

(It is also possible that an existing /etc/mysql/debian.cnf file with wrong content could have caused this; seeing the content of the file should help determine this).

Re: [Bug 674413] Re: When installing 5.2 from debian packages, debian-sys-maint user is misconfigured
On Fri, Nov 12, 2010 at 17:43, Kristian Nielsen
<knielsen@knielsen-hq.org> wrote:
> 1. The debian-sys-maint account should be created during install, it is
> used internally by the postinst scripts. My guess is that something went
> wrong with this in the many failed install attempts. It is a huge
> problem that we do not have apt-get'able package repositories at the
> moment, packages need to be installed in a specific order, which is
> very hard to get right manually.
I was sitting next to Colin while doing the install and he warned me
about it actually. I went ahead and did it anyway, cos even though
dpkg sucks for dependencies like this, it shouldn't actually cause
mayhem like this
Also, i'll talk to Arjen some more to put the pressure on getting the
OurDelta repo's in shape.

> You may have to create that user manually (or remove --purge + reinstall
> in right order should also work, if you haven't yet put data in).
I'll create manually. This is a test VM for http://fossasia.org, so
i'm not fussed about messign with it

> 2. When the postinst script installs the initial database, it runs
> mysqld with special options that disable InnoDB (to reduce risk of
> failing to install due to problems with innodb options), this is nothing
> to worry about (agree it is a bit confusing to see these in the logs,
> and also the warnings about --log-xxx options).
Agreed.

> 3. The failure to shutdown is a consequence of the problem with the
> debian-sys-maint account.
Figured that.

> Looking at the log about the problem with debian-sys-maint ... It says
> 'using password: NO'. This suggests something wrong with the file cat
> /etc/mysql/debian.cnf. Can you show the contents of this file, with any
> password replaced by XXX ? It should look something like this:
db:~# cat /etc/mysql/debian.cnf

1. Automatically generated for Debian scripts. DO NOT TOUCH!
   [client]
   host = localhost
   user = debian-sys-maint
   password = yomamma
   socket = /var/run/mysqld/mysqld.sock
   [mysql_upgrade]
   host = localhost
   user = debian-sys-maint
   password = yomamma
   socket = /var/run/mysqld/mysqld.sock
   basedir = /usr

> If this looks ok, you can check that `mysqladmin -defaults
> file=/etc/mysql/debian.cnf ping` works; if not then maybe the account
> inside mariadb is wrong/missing (but the Using password: NO suggests a
> problem with debian.cnf)
The account is missing:
db:~# mysqladmin --defaults-file=/etc/mysql/debian.cnf ping
mysqladmin: connect to server at 'localhost' failed
error: 'Access denied for user 'debian-sys-maint'@'localhost' (using
password: NO)'

MariaDB [(none)]> select user, host, password from mysql.user;
-------------------------------------------------------------

-------------------------------------------------------------

-------------------------------------------------------------
3 rows in set (0.00 sec)

> (It is also possible that an existing /etc/mysql/debian.cnf file with
> wrong content could have caused this; seeing the content of the file
> should help determine this).
Nope, this was a brand new VM

–
Walter Heck

–
follow @walterheck on twitter to see what I'm up to!
–
Check out my new startup: Server Monitoring as a Service @ http://tribily.com
Follow @tribily on Twitter and/or 'Like' our Facebook page at
http://www.facebook.com/tribily

Re: [Bug 674413] Re: When installing 5.2 from debian packages, debian-sys-maint user is misconfigured
On Fri, Nov 12, 2010 at 17:43, Kristian Nielsen
<knielsen@knielsen-hq.org> wrote:
> 1. The debian-sys-maint account should be created during install, it is
> used internally by the postinst scripts. My guess is that something went
> wrong with this in the many failed install attempts. It is a huge
> problem that we do not have apt-get'able package repositories at the
> moment, packages need to be installed in a specific order, which is
> very hard to get right manually.
I was sitting next to Colin while doing the install and he warned me
about it actually. I went ahead and did it anyway, cos even though
dpkg sucks for dependencies like this, it shouldn't actually cause
mayhem like this
Also, i'll talk to Arjen some more to put the pressure on getting the
OurDelta repo's in shape.

> You may have to create that user manually (or remove --purge + reinstall
> in right order should also work, if you haven't yet put data in).
I'll create manually. This is a test VM for http://fossasia.org, so
i'm not fussed about messign with it

> 2. When the postinst script installs the initial database, it runs
> mysqld with special options that disable InnoDB (to reduce risk of
> failing to install due to problems with innodb options), this is nothing
> to worry about (agree it is a bit confusing to see these in the logs,
> and also the warnings about --log-xxx options).
Agreed.

> 3. The failure to shutdown is a consequence of the problem with the
> debian-sys-maint account.
Figured that.

> Looking at the log about the problem with debian-sys-maint ... It says
> 'using password: NO'. This suggests something wrong with the file cat
> /etc/mysql/debian.cnf. Can you show the contents of this file, with any
> password replaced by XXX ? It should look something like this:
db:~# cat /etc/mysql/debian.cnf

1. Automatically generated for Debian scripts. DO NOT TOUCH!
   [client]
   host = localhost
   user = debian-sys-maint
   password = yomamma
   socket = /var/run/mysqld/mysqld.sock
   [mysql_upgrade]
   host = localhost
   user = debian-sys-maint
   password = yomamma
   socket = /var/run/mysqld/mysqld.sock
   basedir = /usr

> If this looks ok, you can check that `mysqladmin -defaults
> file=/etc/mysql/debian.cnf ping` works; if not then maybe the account
> inside mariadb is wrong/missing (but the Using password: NO suggests a
> problem with debian.cnf)
The account is missing:
db:~# mysqladmin --defaults-file=/etc/mysql/debian.cnf ping
mysqladmin: connect to server at 'localhost' failed
error: 'Access denied for user 'debian-sys-maint'@'localhost' (using
password: NO)'

MariaDB [(none)]> select user, host, password from mysql.user;
-------------------------------------------------------------

-------------------------------------------------------------

-------------------------------------------------------------
3 rows in set (0.00 sec)

> (It is also possible that an existing /etc/mysql/debian.cnf file with
> wrong content could have caused this; seeing the content of the file
> should help determine this).
Nope, this was a brand new VM

–
Walter Heck

–
follow @walterheck on twitter to see what I'm up to!
–
Check out my new startup: Server Monitoring as a Service @ http://tribily.com
Follow @tribily on Twitter and/or 'Like' our Facebook page at
http://www.facebook.com/tribily

Re: [Bug 674413] Re: When installing 5.2 from debian packages, debian-sys-maint user is misconfigured
On Fri, Nov 12, 2010 at 17:43, Kristian Nielsen
<knielsen@knielsen-hq.org> wrote:
> 1. The debian-sys-maint account should be created during install, it is
> used internally by the postinst scripts. My guess is that something went
> wrong with this in the many failed install attempts. It is a huge
> problem that we do not have apt-get'able package repositories at the
> moment, packages need to be installed in a specific order, which is
> very hard to get right manually.
I was sitting next to Colin while doing the install and he warned me
about it actually. I went ahead and did it anyway, cos even though
dpkg sucks for dependencies like this, it shouldn't actually cause
mayhem like this
Also, i'll talk to Arjen some more to put the pressure on getting the
OurDelta repo's in shape.

> You may have to create that user manually (or remove --purge + reinstall
> in right order should also work, if you haven't yet put data in).
I'll create manually. This is a test VM for http://fossasia.org, so
i'm not fussed about messign with it

> 2. When the postinst script installs the initial database, it runs
> mysqld with special options that disable InnoDB (to reduce risk of
> failing to install due to problems with innodb options), this is nothing
> to worry about (agree it is a bit confusing to see these in the logs,
> and also the warnings about --log-xxx options).
Agreed.

> 3. The failure to shutdown is a consequence of the problem with the
> debian-sys-maint account.
Figured that.

> Looking at the log about the problem with debian-sys-maint ... It says
> 'using password: NO'. This suggests something wrong with the file cat
> /etc/mysql/debian.cnf. Can you show the contents of this file, with any
> password replaced by XXX ? It should look something like this:
db:~# cat /etc/mysql/debian.cnf

1. Automatically generated for Debian scripts. DO NOT TOUCH!
   [client]
   host = localhost
   user = debian-sys-maint
   password = yomamma
   socket = /var/run/mysqld/mysqld.sock
   [mysql_upgrade]
   host = localhost
   user = debian-sys-maint
   password = yomamma
   socket = /var/run/mysqld/mysqld.sock
   basedir = /usr

> If this looks ok, you can check that `mysqladmin -defaults
> file=/etc/mysql/debian.cnf ping` works; if not then maybe the account
> inside mariadb is wrong/missing (but the Using password: NO suggests a
> problem with debian.cnf)
The account is missing:
db:~# mysqladmin --defaults-file=/etc/mysql/debian.cnf ping
mysqladmin: connect to server at 'localhost' failed
error: 'Access denied for user 'debian-sys-maint'@'localhost' (using
password: NO)'

MariaDB [(none)]> select user, host, password from mysql.user;
-------------------------------------------------------------

-------------------------------------------------------------

-------------------------------------------------------------
3 rows in set (0.00 sec)

> (It is also possible that an existing /etc/mysql/debian.cnf file with
> wrong content could have caused this; seeing the content of the file
> should help determine this).
Nope, this was a brand new VM

–
Walter Heck

–
follow @walterheck on twitter to see what I'm up to!
–
Check out my new startup: Server Monitoring as a Service @ http://tribily.com
Follow @tribily on Twitter and/or 'Like' our Facebook page at
http://www.facebook.com/tribily

Re: When installing 5.2 from debian packages, debian-sys-maint user is misconfigured
Adding the debian-sys-maint user by hand fixed most of my problems:
==== Maria commands
version for the right syntax to use near '-sys-maint@localhost identified by 'j8KmNWOAiZuPbnfr'' at line 1
MariaDB [(none)]> grant all on . to 'debian-sys-maint'@localhost identified by 'j8KmNWOAiZuPbnfr';
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> exit
Bye
db:~# /etc/init.d/mysql stop
Stopping MariaDB database server: mysqld.
db:~# tail /var/log/syslog
Nov 12 22:07:30 db mysqld: 101112 22:07:30 [Warning] Access denied for user 'debian-sys-maint'@'localhost' (using password: NO)
Nov 12 22:10:01 db mysqld: 101112 22:10:01 [Note] /usr/sbin/mysqld: Normal shutdown
Nov 12 22:10:01 db mysqld:
Nov 12 22:10:01 db mysqld: 101112 22:10:01 [Note] Event Scheduler: Purging the queue. 0 events
Nov 12 22:10:03 db mysqld: 101112 22:10:03 InnoDB: Starting shutdown...
Nov 12 22:10:07 db mysqld: 101112 22:10:07 InnoDB: Shutdown completed; log sequence number 45356
Nov 12 22:10:07 db mysqld: 101112 22:10:07 [Note] PrimeBase XT Engine shutdown...
Nov 12 22:10:07 db mysqld: 101112 22:10:07 [Note] /usr/sbin/mysqld: Shutdown complete
Nov 12 22:10:07 db mysqld:
Nov 12 22:10:07 db mysqld_safe: mysqld from pid file /var/run/mysqld/mysqld.pid ended
db:~# /etc/init.d/mysql start
Starting MariaDB database server: mysqld . ..
Checking for corrupt, not cleanly closed and upgrade needing tables..
db:~# tail /var/log/syslog

==== Syslog output
Nov 12 22:10:51 db mysqld_safe: Starting mysqld daemon with databases from /var/lib/mysql
Nov 12 22:10:51 db mysqld: 101112 22:10:51 [Warning] options --log-slow-admin-statements, --log-queries-not-using-indexes and --log-slow-slave-statements have no effect if --log_slow_queries is not set
Nov 12 22:10:51 db mysqld: 101112 22:10:51 [Note] PrimeBase XT (PBXT) Engine 1.0.11-7 Pre-GA loaded...
Nov 12 22:10:51 db mysqld: 101112 22:10:51 [Note] Paul McCullagh, PrimeBase Technologies GmbH, http://www.primebase.org
Nov 12 22:10:52 db mysqld: InnoDB: The InnoDB memory heap is disabled
Nov 12 22:10:52 db mysqld: InnoDB: Mutexes and rw_locks use GCC atomic builtins
Nov 12 22:10:52 db mysqld: InnoDB: Compressed tables use zlib 1.2.3
Nov 12 22:10:52 db mysqld: 101112 22:10:52 InnoDB: highest supported file format is Barracuda.
Nov 12 22:10:52 db mysqld: 101112 22:10:52 Percona XtraDB (http://www.percona.com) 1.0.12-12.1 started; log sequence number 45356
Nov 12 22:10:52 db mysqld: 101112 22:10:52 [Note] Event Scheduler: Loaded 0 events
Nov 12 22:10:52 db mysqld: 101112 22:10:52 [Note] /usr/sbin/mysqld: ready for connections.
Nov 12 22:10:52 db mysqld: Version: '5.2.3-MariaDB-mariadb87-log' socket: '/var/run/mysqld/mysqld.sock' port: 3306 (MariaDB - http://mariadb.com/)
Nov 12 22:10:53 db /etc/mysql/debian-start[2871]: Upgrading MySQL tables if necessary.
Nov 12 22:10:54 db mysqld: 101112 22:10:54 [Warning] Statement may not be safe to log in statement format. Statement: ALTER TABLE general_log
Nov 12 22:10:54 db mysqld: MODIFY event_time TIMESTAMP NOT NULL,
Nov 12 22:10:54 db mysqld: MODIFY user_host MEDIUMTEXT NOT NULL,
Nov 12 22:10:54 db mysqld: MODIFY thread_id INTEGER NOT NULL,
Nov 12 22:10:54 db mysqld: MODIFY server_id INTEGER UNSIGNED NOT NULL,
Nov 12 22:10:54 db mysqld: MODIFY command_type VARCHAR(64) NOT NULL,
Nov 12 22:10:54 db mysqld: MODIFY argument MEDIUMTEXT NOT NULL
Nov 12 22:10:54 db mysqld: 101112 22:10:54 [Warning] Statement may not be safe to log in statement format. Statement: ALTER TABLE slow_log
Nov 12 22:10:54 db mysqld: MODIFY start_time TIMESTAMP NOT NULL,
Nov 12 22:10:54 db mysqld: MODIFY user_host MEDIUMTEXT NOT NULL,
Nov 12 22:10:54 db mysqld: MODIFY query_time TIME NOT NULL,
Nov 12 22:10:54 db mysqld: MODIFY lock_time TIME NOT NULL,
Nov 12 22:10:54 db mysqld: MODIFY rows_sent INTEGER NOT NULL,
Nov 12 22:10:54 db mysqld: MODIFY rows_examined INTEGER NOT NULL,
Nov 12 22:10:54 db mysqld: MODIFY db VARCHAR(512) NOT NULL,
Nov 12 22:10:54 db mysqld: MODIFY last_insert_id INTEGER NOT NULL,
Nov 12 22:10:54 db mysqld: MODIFY insert_id INTEGER NOT NULL,
Nov 12 22:10:54 db mysqld: MODIFY server_id INTEGER UNSIGNED NOT NULL,
Nov 12 22:10:54 db mysqld: MODIFY sql_text MEDIUMTEXT NOT NULL
Nov 12 22:10:54 db /etc/mysql/debian-start[2875]: /usr/bin/mysql_upgrade: the '--basedir' option is always ignored
Nov 12 22:10:54 db /etc/mysql/debian-start[2875]: Looking for 'mysql' as: /usr/bin/mysql
Nov 12 22:10:54 db /etc/mysql/debian-start[2875]: Looking for 'mysqlcheck' as: /usr/bin/mysqlcheck
Nov 12 22:10:54 db /etc/mysql/debian-start[2875]: Phase 1/3: Fixing table and database names
Nov 12 22:10:54 db /etc/mysql/debian-start[2875]: Processing databases
Nov 12 22:10:54 db /etc/mysql/debian-start[2875]: information_schema
Nov 12 22:10:54 db /etc/mysql/debian-start[2875]: mysql
Nov 12 22:10:54 db /etc/mysql/debian-start[2875]: Phase 2/3: Checking and upgrading tables
Nov 12 22:10:54 db /etc/mysql/debian-start[2875]: Processing databases
Nov 12 22:10:54 db /etc/mysql/debian-start[2875]: information_schema
Nov 12 22:10:54 db /etc/mysql/debian-start[2875]: mysql
Nov 12 22:10:54 db /etc/mysql/debian-start[2875]: mysql.columns_priv OK
Nov 12 22:10:54 db /etc/mysql/debian-start[2875]: mysql.db OK
Nov 12 22:10:54 db /etc/mysql/debian-start[2875]: mysql.event OK
Nov 12 22:10:54 db /etc/mysql/debian-start[2875]: mysql.func OK
Nov 12 22:10:54 db /etc/mysql/debian-start[2875]: mysql.help_category OK
Nov 12 22:10:54 db /etc/mysql/debian-start[2875]: mysql.help_keyword OK
Nov 12 22:10:54 db /etc/mysql/debian-start[2875]: mysql.help_relation OK
Nov 12 22:10:54 db /etc/mysql/debian-start[2875]: mysql.help_topic OK
Nov 12 22:10:54 db /etc/mysql/debian-start[2875]: mysql.host OK
Nov 12 22:10:54 db /etc/mysql/debian-start[2875]: mysql.ndb_binlog_index OK
Nov 12 22:10:54 db /etc/mysql/debian-start[2875]: mysql.plugin OK
Nov 12 22:10:54 db /etc/mysql/debian-start[2875]: mysql.proc OK
Nov 12 22:10:54 db /etc/mysql/debian-start[2875]: mysql.procs_priv OK
Nov 12 22:10:54 db /etc/mysql/debian-start[2875]: mysql.servers OK
Nov 12 22:10:54 db /etc/mysql/debian-start[2875]: mysql.tables_priv OK
Nov 12 22:10:54 db /etc/mysql/debian-start[2875]: mysql.time_zone OK
Nov 12 22:10:54 db /etc/mysql/debian-start[2875]: mysql.time_zone_leap_second OK
Nov 12 22:10:54 db /etc/mysql/debian-start[2875]: mysql.time_zone_name OK
Nov 12 22:10:54 db /etc/mysql/debian-start[2875]: mysql.time_zone_transition OK
Nov 12 22:10:54 db /etc/mysql/debian-start[2875]: mysql.time_zone_transition_type OK
Nov 12 22:10:54 db /etc/mysql/debian-start[2875]: mysql.user OK
Nov 12 22:10:54 db /etc/mysql/debian-start[2875]: Phase 3/3: Running 'mysql_fix_privilege_tables'...
Nov 12 22:10:54 db /etc/mysql/debian-start[2875]: OK
Nov 12 22:10:54 db /etc/mysql/debian-start[2890]: Checking for insecure root accounts.
Nov 12 22:10:54 db /etc/mysql/debian-start[2894]: Triggering myisam-recover for all MyISAM tables
db:~#

Re: When installing 5.2 from debian packages, debian-sys-maint user is misconfigured
Thanks for the detailed report, I was able to repeat it. I will see if I can find a way to make it more robust.

Re: When installing 5.2 from debian packages, debian-sys-maint user is misconfigured
Actually, the problem is not related to the many failed install attempts, it is much worse.
The script that is supposed to create the debian-sys-maint account has not been
updated to 5.2, which has a new column `auth_string` in the mysql.user table.
This means that any install of MariaDB 5.2 .debs will have the problem with missing
debian-sys-maint account!
I'll get a fix cooked up.

Walter, thanks a million for bringing this very serious problem to our attention, as well
as for the excellent bug report!

Re: When installing 5.2 from debian packages, debian-sys-maint user is misconfigured
I committed a fix for this in ourdelta-montyprogram-fixes (for failure to create debian-sys-maint account and default config causing warning about slow log options).

Re: When installing 5.2 from debian packages, debian-sys-maint user is misconfigured
Summary of effect on users:

This affects new installs of .debs for MariaDB 5.2. Upgrading from earlier version is not affected, nor is 5.1 or installing from other source than .deb.

The symptom is that /etc/init.d/mysql stop does not work, nor does reload, status, etc.

We will upload new .deb packages shortly. Installing the new package should fix the problem. However, as shutdown of the old server is broken, it may be necessary to manually stop the old server with something like this:

sudo kill -TERM $(pidof mysqld)

The new, fixed .deb packages have version string 5.2.3-mariadb90, while the old ones have version string 5.2.3-mariadb87

Re: When installing 5.2 from debian packages, debian-sys-maint user is misconfigured
Haven't verified but since we're at 5.2.4 now, I presume this is released?

Re: When installing 5.2 from debian packages, debian-sys-maint user is misconfigured
just marking this as "invalid" in OurDelta project, since MariaDB 5.2 is built within the Maria project (its build scripts are just based on OurDelta's, but naturally evolve)

Re: When installing 5.2 from debian packages, debian-sys-maint user is misconfigured
Yes, this is fixed in MariaDB 5.2.4.

Launchpad bug id: 674413