[MDEV-25177] Better indication of refusing to start because of ProtectHome Created: 2021-03-17  Updated: 2023-09-19  Resolved: 2023-09-04

Status: Closed
Project: MariaDB Server
Component/s: Server
Affects Version/s: 10.5.9
Fix Version/s: 10.4.32, 10.5.23, 10.6.16, 10.10.7, 10.11.6, 11.0.4, 11.1.3, 11.2.2

Type: Bug Priority: Major
Reporter: Premek Assignee: Daniel Black
Resolution: Fixed Votes: 0
Labels: None
Environment:

debian

Issue Links:
Relates
relates to MDEV-26317 Distributed mariadb.service Systed se... Closed
relates to MDEV-12243 upgrade to systemd can failed with di... Open

 Description   

I just spent unreasonable amount of time until I learnt about existence of the `ProtectHome` setting. My use case is I needed to change the directory to home because I don't have space anywhere else on my dev environment (which I think could be common)

I believe when mariadb refuses to start because ProtectHome=true it is not logged/displayed, not obvious to someone who doesn't know about this setting.

Please ignore/close the issue if this is not true and I just didn't see it

 Comments   
Comment by Tuukka Pasanen [ 2021-08-23 ]

This seems to be mostly same than MDEV-26317 so solutions should work with both

Comment by Tuukka Pasanen [ 2021-09-12 ]

There is now Pull request against this bug

Comment by Otto Kekäläinen [ 2021-09-14 ]

i dived into this a bit deeper. If you move your /var/lib/mysql under /home (as the bug reporter here indicates they did, though exact paths are not in the bug report), MariaDB fill quite clearly stop starting and there is an error message immediately visible:

$ ll /var/lib/mysql
 
lrwxrwxrwx 1 root root 32 syys  13 20:56 /var/lib/mysql -> /home/otto/mariadb/var-lib-mysql/
 
 
 
$ sudo systemctl start mariadb
 
Job for mariadb.service failed because the control process exited with error code.
 
See "systemctl status mariadb.service" and "journalctl -xe" for details.
 
 
 
$ journalctl -u mariadb -n 10
 
-- Logs begin at Mon 2018-06-11 23:33:19 PDT, end at Mon 2021-09-13 20:57:28 PDT. --
 
syys 13 20:54:22 XPS-13-9370 mariadbd[784]: 2021-09-13 20:54:22 0 [Note] /usr/sbin/mariadbd: Shutdown complete
 
syys 13 20:54:22 XPS-13-9370 systemd[1]: Stopped MariaDB 10.5.12 database server.
 
syys 13 20:56:54 XPS-13-9370 systemd[1]: Starting MariaDB 10.5.12 database server...
 
syys 13 20:56:54 XPS-13-9370 mariadbd[2821]: 2021-09-13 20:56:54 0 [Note] /usr/sbin/mariadbd (mysqld 10.5.12-MariaDB-1:10.5.12+maria~bionic) sta
 
syys 13 20:56:54 XPS-13-9370 mariadbd[2821]: 2021-09-13 20:56:54 0 [Warning] Can't create test file /var/lib/mysql/XPS-13-9370.lower-test
 
syys 13 20:56:54 XPS-13-9370 mariadbd[2821]: [92B blob data]
 
syys 13 20:56:54 XPS-13-9370 mariadbd[2821]: 2021-09-13 20:56:54 0 [ERROR] Aborting
 
syys 13 20:56:54 XPS-13-9370 systemd[1]: mariadb.service: Main process exited, code=exited, status=1/FAILURE
 
syys 13 20:56:54 XPS-13-9370 systemd[1]: mariadb.service: Failed with result 'exit-code'.
 
syys 13 20:56:54 XPS-13-9370 systemd[1]: Failed to start MariaDB 10.5.12 database server.

Indeed, it does not say why it can't write into the datadir /var/lib/mysql.

..and it seems that the root cause isn't ProtectedHome, as setting it to `no` does not fix the issue:

$ grep Protect /lib/systemd/system/mariadb.service
 
ProtectSystem=no
 
ProtectHome=no
 
 
 
$ sudo systemctl daemon-reload
 
 
 
$ sudo systemctl restart mariadb
 
Job for mariadb.service failed because the control process exited with error code.
 
See "systemctl status mariadb.service" and "journalctl -xe" for details.
 
 
 
$ journalctl -u mariadb -n 10
 
-- Logs begin at Mon 2018-06-11 23:33:19 PDT, end at Mon 2021-09-13 21:05:01 PDT. --
 
syys 13 21:04:50 XPS-13-9370 systemd[1]: mariadb.service: Failed with result 'exit-code'.
 
syys 13 21:04:50 XPS-13-9370 systemd[1]: Failed to start MariaDB 10.5.12 database server.
 
syys 13 21:04:57 XPS-13-9370 systemd[1]: Starting MariaDB 10.5.12 database server...
 
syys 13 21:04:58 XPS-13-9370 mariadbd[5113]: 2021-09-13 21:04:58 0 [Note] /usr/sbin/mariadbd (mysqld 10.5.12-MariaDB-1:10.5.12+maria~bionic) sta
 
syys 13 21:04:58 XPS-13-9370 mariadbd[5113]: 2021-09-13 21:04:58 0 [Warning] Can't create test file /var/lib/mysql/XPS-13-9370.lower-test
 
syys 13 21:04:58 XPS-13-9370 mariadbd[5113]: [92B blob data]
 
syys 13 21:04:58 XPS-13-9370 mariadbd[5113]: 2021-09-13 21:04:58 0 [ERROR] Aborting
 
syys 13 21:04:58 XPS-13-9370 systemd[1]: mariadb.service: Main process exited, code=exited, status=1/FAILURE
 
syys 13 21:04:58 XPS-13-9370 systemd[1]: mariadb.service: Failed with result 'exit-code'.
 
syys 13 21:04:58 XPS-13-9370 systemd[1]: Failed to start MariaDB 10.5.12 database server.

Comment by Tuukka Pasanen [ 2021-09-14 ]

I expect chmod and chown are correct? I love how systemd just puts [92B blob data] when it expects it's binary data. Would it be more verbose on start if chmod/chown is incorrect way to handle this...

Comment by Otto Kekäläinen [ 2021-09-14 ]

Yes, chmod and chown are correct (actually unchanged as I used 'cp -a' to
copy).

Your can do the same test yourself and see if you get different results.

Indeed, systemd is not very helpful here. Maybe file an issue upstream at
systemd requesting better warnings about things that systemd itself does?

Comment by Premek [ 2021-09-14 ]

Hi, thanks for looking into this. Sorry I don't remember exactly what my error message was, but I configured it using

datadir                 = /home/mysql

not a symlink

Comment by Tuukka Pasanen [ 2021-12-03 ]

premek Ok then Pull Request shell script should give enough information. Can you give feedback is it working? You can find script from PR.

Comment by Daniel Black [ 2022-01-31 ]

the environment systemd provides in ExecStartPre isn't as hardened as ExecStart so some of the more practical tests need re-examining.

Comment by Premek [ 2022-01-31 ]

Hi, thank you for looking into my issue, but I'm currently not able to test the fix, sorry

Comment by Daniel Black [ 2022-02-15 ]

In the testing of the PR:

  • examination of paths is faulty as Otto gave an example - could be a symlink. I don't know if bind-mounts are similar
  • ExecStartPre scripts aren't the same protected environment as ExecStart and the special execute modes don't include a protected mode.

As datadir is the likely variable to hit this restriction, an errno=13 + systemd could result in a more verbose warning hint in the server in test_if_case_insensitive.

secure_file_priv seems the other variable likely to be changed to a home/system location can could get the same error checks.

premek (or anyone watching) can you think of any other variables require special testing/error handing?

Comment by Daniel Black [ 2023-08-30 ]

Acceptable? https://github.com/MariaDB/server/pull/2743

Generated at Thu Feb 08 09:35:44 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.