log.7z
[MDEV-17590] installation problem: mariadb service does not recognize that the server has been started Created: 2018-11-01 Updated: 2018-12-07 Resolved: 2018-12-07 |
|
| Status: | Closed |
| Project: | MariaDB Server |
| Component/s: | Configuration, Packaging, Platform Debian |
| Affects Version/s: | 10.1.26, 10.3.10 |
| Fix Version/s: | N/A |
| Type: | Bug | Priority: | Major |
| Reporter: | MadEmpire | Assignee: | Unassigned |
| Resolution: | Not a Bug | Votes: | 0 |
| Labels: | debian9, need_feedback, socket | ||
| Environment: |
debian 9, v Server |
||
| Attachments: |
log.7z
|
| Description |
|
We can not install mariadb or any other mysql version. We tried >20 times to install mariadb with a clear debian 9 img from our webhoster zap-hosting.com. It works fine with debian 8.
Our Linux version is: |
| Comments |
| Comment by Elena Stepanova [ 2018-11-01 ] | ||
|
It appears that you are installing a mix of Debian's MariaDB packages (10.1.26-0+deb9u1) and MariaDB's packages (1:10.3.10+maria~stretch). Unfortunately, it's an unsupported combination, here is a comment from official Debian maintenance. Could you please try to install either one or another and check if it works? | ||
| Comment by MadEmpire [ 2018-11-01 ] | ||
|
Hey we already tried this and treid it now again. There is no mariadb repo in the apt list: root@12505:~# apt install mariadb-server-10.1 mariadb-client-10.1
Nov 01 15:26:39 12505 systemd[1]: Starting MariaDB database server... | ||
| Comment by Elena Stepanova [ 2018-11-01 ] | ||
|
Thanks. And can you now provide the portion of syslog related to this unsuccessful attempt?
But the problem for 10.1 should be different, so we need to see what it is. | ||
| Comment by MadEmpire [ 2018-11-01 ] | ||
|
I uploaded the new log files, they should be attached. | ||
| Comment by Elena Stepanova [ 2018-11-01 ] | ||
|
Thanks. So indeed, 10.1 problem is different. 10.1.26 server starts just fine (as we can see in error.log), waits for 90 seconds and then gets shut down gracefully, at the same time when the service reports a timeout. It doesn't complain about any problems with socket creation. Unfortunately, I can't find in the last archive errors from mysqladmin or whatever other tool fails to connect to the server, but the timeout obviously occurs. Maybe 10.1 is just less verbose. When you said in the description "mysql socket not found and could not be started", did you mean that you checked for the existence of the socket manually, or is it the interpretation of mysqladmin complaining that it couldn't connect to the server via the socket? And If you did check for the existence of the socket, did you do it while the service was still "waiting" for the server to start? Because the socket would only exist during those 90 seconds while the service is waiting, as soon as the server shuts down cleanly, the socket file is removed. We did have one complaint from a user very long time ago about a socket file magically disappearing on their machine while the server was running, the reason for this has never been found. The suspicion was that the problem was environmental (if it was indeed happening at all), since nobody else ever reported it. It was something like CentOS, though. However, assuming that the socket is all right, the most likely reason of the problem is a mix up of authentication methods. Packages provided by Debian use unix_socket as a default authentication method for the local root, and thus their scripts assume that mysqladmin etc. will have a passwordless access to the server as long as they are run under root. On the other hand, MariaDB's packages use a special user/password configured in debian.conf for the tools, and also invite the user to create a password for the local root upon installation. When these two approaches collide, e.g. a database created by one is attempted to be used by another, or scripts/configs from them get mixed up, bad things happen. I saw that you did purge before installing, but still, given how strange debian packaging happens to be, I wouldn't rule out some garbage remaining there. One approach which should probably help to check it is adding skip-grant-tables to the server config file temporarily. Hopefully it will allow the service to start, and then you can connect to the server and inspect mysql.user table (important columns would be user, host, password, authentication_string) to see what's there. | ||
| Comment by MadEmpire [ 2018-11-01 ] | ||
|
The error message occured and then i checked the folder and there was no socket. I checked this directly after the installation and the error message. | ||
| Comment by MadEmpire [ 2018-11-01 ] | ||
|
if you want to test it yourself i can just tell you our root pw. I could also install a clean version of debian 9 from our web hoster. | ||
| Comment by Elena Stepanova [ 2018-11-01 ] | ||
|
Yes, sure, it might go faster if I have access to the box in question. I could take a look at your current (failing) installation, maybe there will be no need to re-install Debian. You can send me the password by email, or just add my ssh key to the authorized list. | ||
| Comment by Elena Stepanova [ 2018-11-02 ] | ||
|
I've spent some time digging into the machine in question. First of all, there is no problem with socket or server startup – server starts just fine, the socket gets created normally, the server is accessible via the socket until the service timeout (90 sec by default) occurs and the server gets shut down. Secondly, there was indeed a mix of different installations when I reached the machine, but it's unrelated to the main problem. Even after I cleaned up the mix and got a totally clean 10.1.26 (or 10.3.10) installation, the problem persisted. The problem happens upon service startup (not server, but service). MariaDB service has Type=notify. According to systemd documentation, it means that . It appears to be indeed the culprit. If I change the Type to simple, everything works file. Further, I noticed these errors in the system logs. They don't show up in system logs every time, I'm not sure how audit logging occurs, maybe there is some throttling. But they seem to be related.
Oddly, the machine doesn't have apparmor installed, but there are some pieces which might indicate an inconsistent state, e.g. /sys/module/apparmor/parameters/mode shows enforce, etc. I tried to re-install/uninstall apparmor, but it didn't help. Possibly it requires some changes in system startup configuration, or at least system restart. Finally, I tried the exact same installation (same MariaDB version from the same repo) on our Stretch VM, and it works fine (We don't have VServer though). The difference is that we don't have those pieces of apparmor suggesting that it could have been enabled at some point, so again, it indirectly confirms the theory. Also, since the year-old 10.1.26 packages built by Debian and new 10.3.10 packages built by MariaDB hit the exact same problem, and according to you, it only started happening recently, it's extremely unlikely that the problem is on the server packaging side. Far more likely is that the environment changed somehow. I suggest that you try to figure out
We'll be waiting for news from you, as it might be useful for other community users. | ||
| Comment by MadEmpire [ 2018-11-05 ] | ||
|
First of all, thank you for your great help | ||
| Comment by Elena Stepanova [ 2018-12-07 ] | ||
|
I will close it for now. Please comment if you have further evidence that it's a MariaDB bug. |
We reported the problems to our Webhoster and zap hosting will create a new virtual server for us. I will give the task to find the problem to our Webhoster. If i have news about this i will post them here and again thank you for your help.