[MDEV-9165] Debian postinstall chown or the datadir takes too long on servers with many files on the datadir Created: 2015-11-23 Updated: 2018-01-15 Resolved: 2016-11-26 |
|
| Status: | Closed |
| Project: | MariaDB Server |
| Component/s: | Platform Debian |
| Affects Version/s: | 5.5.46, 10.0.22, 10.1.8 |
| Fix Version/s: | 10.2.3 |
| Type: | Bug | Priority: | Minor |
| Reporter: | Jean Weisbuch | Assignee: | Otto Kekäläinen |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Environment: |
Debian |
||
| Description |
|
I recently experienced a slow MariaDB upgrade/migration from MySQL 5.5 to MariaDB 5.5, after the server has been stopped it took maybe one or two minutes before starting the new one. A process doing a "chown mysql -R /var/lib/mysql" was runing for quite a long time. Replacing the line on "mariadb-server-*.postinst" :
With something like (there is probably a better alternative) :
It seems to be marginally slower (less than 1/100th of a second on my tests) on small directories but faster on directories with many files on it (about 20-40% faster on a directory with only 25k files on a software raid1 with 2* 7200k HDDs). |
| Comments |
| Comment by Jean Weisbuch [ 2015-11-24 ] |
|
I did some better testings on the same system with 850,000 unique files on a directory (3.3G of space) located on an ext4 partition and all files are already owned by "mysql" (or the target user). It takes between 1min30s and 1min40s to run the classic "chown -R" while the "find" variant only takes around 1.8 seconds, which is significantly faster. Having only one file owned by a different owner (for example "debian-10.0.flag" owned by root) on the folder wont make an significant difference. The main drawback of the "find" alternative, is that it will be slower if an important number of files in the folder are owned by a different user than "mysql" (or the target one). Using "find" and "xargs" would require to add a dependency to the "findutils" package (both files are in it) but its possible to stick to the old behavior in case the package is not present on the system. |
| Comment by Otto Kekäläinen [ 2016-11-25 ] |
|
jb-boin Is find /var/lib/mysql/ ! -uid $(id -u mysql) -print0 |xargs -0 -r chown mysql now your final suggestion or do you have improved versions of it? |
| Comment by Otto Kekäläinen [ 2016-11-25 ] |
|
How about find $mysql_datadir ! -uid $(id -u mysql) -exec chown mysql:mysql {} \; ? |
| Comment by Otto Kekäläinen [ 2016-11-26 ] |
|
Fixed in https://github.com/MariaDB/server/commit/3a6e781ea39b67fabf1a528e6a071215a65f50b0 for 10.2 |
| Comment by Jean Weisbuch [ 2017-01-23 ] |
|
Sorry for the delay but just to answer to you last question, the use of "find ... -exec" would call chown on each file wile piping to xargs would call chown with multiple arguments and thus doing less forking and being faster. |
| Comment by Otto Kekäläinen [ 2017-01-24 ] |
|
Fixed for 10.1 in Debian in https://github.com/ottok/mariadb-10.1/commit/7d52f516e41f4af9cb5ec2eb97f6f35240914302 |
| Comment by Daniel Black [ 2018-01-15 ] |
|
For find, -uid $(id -u mysql) is equivalent to -user mysql. xargs has a --max-procs=max-procs which could be useful for parallelism of the changes. |