[MDEV-31529] MariaDB docker pipeline is failing Created: 2023-06-22 Updated: 2023-06-27 Resolved: 2023-06-27 |
|
| Status: | Closed |
| Project: | MariaDB Server |
| Component/s: | Docker |
| Affects Version/s: | None |
| Fix Version/s: | 11.0.2, 10.5.21, 10.6.14, 10.9.7, 10.10.5, 10.11.4 |
| Type: | Bug | Priority: | Major |
| Reporter: | Ehsan Kiani Far | Assignee: | Daniel Black |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Environment: |
s390x |
||
| Attachments: |
|
| Description |
|
I observed that the docker image build CI for s390x has been timing out since March 2023 therefore some images are missing from the docker registry. I was able to build the image on my s390x machine with no problem. Looks like it is an issue with the build pipeline. Could anyone from the MariaDB community take a look at this issue. I can provide any help if required. Thanks. |
| Comments |
| Comment by Daniel Black [ 2023-06-22 ] |
|
Thanks for the bug report. It appears to be something in the Docker CI (infosiftr was acquired by Docker Inc). I've forwarded the bug to them to understand: https://github.com/docker-library/official-images/issues/14923 |
| Comment by Daniel Black [ 2023-06-23 ] |
|
I'm a bit worried that 10.5, 10.6 succeed and the others failed. This could distro kernel related, 10.9+ are jammy based and would be using iouring, 10.5/10.6 wouldn't). So this could be a s390x kernel uring problem. I removed the initialization in https://github.com/MariaDB/mariadb-docker/pull/518 that should make it quicker, and avoid the build time exercising of database functions. If there is really a problem in s390x initialization of a database in 10.9+ this needs to be found. I'll look at pushing a release early next week that passes hopefully and there should at least be a release there. Can you test quay.io/mariadb-foundation/mariadb-devel:$version images on real hardware as they haven't had trouble building at all. As a test clone https://github.com/MariaDB/mariadb-docker and: .test/run quay.io/mariadb-foundation/mariadb-devel:10.9 Will start/stop exercise the functionality. Pushing a hammerdb/sysbench oltp (RW) at the container would be useful too. |
| Comment by Daniel Black [ 2023-06-27 ] |
|
The s390x released on Docker Hub on the official releases have been updated so are available in all 10.5+ containers like they should have been. There is still most likely an iouring problem with at least some kernels on s390x for mariadb containers 10.9+. To confirm, after exercising a stalling test case (like previous), disable the liburing with --innodb_use_native_aio=0 and see the same behaviour exists. |
| Comment by Ehsan Kiani Far [ 2023-06-27 ] |
|
Thanks @danblack for your help and support. I am deeply thankful. |