[MDBF-350] buildbot ppc64le-ubuntu-2110 /home/buildbot/ppc64le-ubuntu-2110/build/mysql-test/var/22/mysqld.1': Operation not permitted Created: 2022-02-25 Updated: 2022-11-04 Resolved: 2022-11-04 |
|
| Status: | Closed |
| Project: | MariaDB Foundation Development |
| Component/s: | None |
| Affects Version/s: | N/A |
| Fix Version/s: | N/A |
| Type: | Bug | Priority: | Major |
| Reporter: | Daniel Black | Assignee: | Faustin Lammler |
| Resolution: | Incomplete | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | 0d | ||
| Time Spent: | 0.25d | ||
| Original Estimate: | 0d | ||
| Issue Links: |
|
||||||||
| Description |
|
https://buildbot.mariadb.org/#/builders/337/builds/899/steps/6/logs/stdio
Fails from the very first test. Is the /dev/shm mount incorrect? |
| Comments |
| Comment by Vlad Bogolin [ 2022-02-28 ] | |
|
What is particular to the Power builders is that they use
This happened in order to solve some out of space errors that were related to this. Can the shm_size be too large? Other than that, the builder is updated and should use the latests quay image. | |
| Comment by Daniel Black [ 2022-02-28 ] | |
|
shm_size can only be too large if it consumes memory off other things that need it. 20G looks like a good figure for the tests. It would need a lot of parallel tests to run out. | |
| Comment by Vlad Bogolin [ 2022-02-28 ] | |
|
Then I guess it must be a permission issue or something along those lines. However the issue seems weird and since things seem to be fine on one worker it might be something related to host machine configurations. | |
| Comment by Faustin Lammler [ 2022-03-01 ] | |
|
Not sure about the issue here but /dev/shm is automatically filtered from Zabbix low level discovery so it may be a good idea to start collecting metrics about it. I will try to add it we may see some interesting stuff. | |
| Comment by Faustin Lammler [ 2022-03-29 ] | |
|
Adding `shm` to monitoring is not a good Idea as it will generate too much false positive. There is also something else different on that particular builder (fi-bbw-p9rhel7), the /var/lib/docker is a symlink to /srv/buildbot/docker and /srv/buildbot is a symlink to /home/buildbot. If I recall correctly, we did this because of space issue, but this and We could maybe test/confirm this by re-running a problematic build/test without those symlinks. vladbogo we can schedule such a build when you are back. | |
| Comment by Faustin Lammler [ 2022-04-07 ] | |
|
Clean users ID and fix rights in /home/buildbot. As discussed with Vlad, the symlink stuff should be tested too. | |
| Comment by Daniel Black [ 2022-04-08 ] | |
|
Or is there sufficient IO to actually test on a filesystem (MDBF-321)? Appears to be p9-rhel7-bbw1-docker-ubuntu-2110 and p9-db-bbw1-docker-ubuntu-2110. | |
| Comment by Faustin Lammler [ 2022-09-19 ] | |
|
vladbogo do we have this problem still? |