Details
-
Bug
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Fixed
-
10.11.13
-
Can result in hang or crash
-
-
Q3/2025 Maintenance
Description
Hi,
We in kolla-ansible openstack project are testing mariadb backup procedure, but we see this in our CI
Here is output of backup :
"stderr_lines": [ |
"2025-05-23 18:45:58.517 INFO Loading config file at /var/lib/kolla/config_files/config.json", |
"2025-05-23 18:45:58.518 INFO Validating config file", |
"2025-05-23 18:45:58.518 INFO Kolla config strategy set to: COPY_ALWAYS", |
"2025-05-23 18:45:58.519 INFO Copying /etc/mysql/my.cnf to /etc/kolla/defaults/etc/mysql/my.cnf", |
"2025-05-23 18:45:58.519 INFO Copying permissions from /etc/mysql/my.cnf onto /etc/kolla/defaults/etc/mysql/my.cnf", |
"2025-05-23 18:45:58.519 INFO Copying service configuration files", |
"2025-05-23 18:45:58.520 INFO Deleting /etc/mysql/my.cnf", |
"2025-05-23 18:45:58.520 INFO Copying /var/lib/kolla/config_files/my.cnf to /etc/mysql/my.cnf", |
"2025-05-23 18:45:58.521 INFO Setting permission for /etc/mysql/my.cnf", |
"2025-05-23 18:45:58.521 INFO Writing out command to execute", |
"2025-05-23 18:45:58.522 INFO Setting permission for /var/log/kolla/mariadb", |
"2025-05-23 18:45:58.522 INFO Setting permission for /backup", |
"[00] 2025-05-23 18:45:58 Connecting to MariaDB server host: 192.0.2.2, user: backup_shard_0, password: set, port: 3306, socket: not set", |
"[00] 2025-05-23 18:45:58 Using server version 10.11.13-MariaDB-deb12-log", |
"mariabackup based on MariaDB server 10.11.13-MariaDB debian-linux-gnu (x86_64)", |
"[00] 2025-05-23 18:45:58 incremental backup from 0 is enabled.", |
"[00] 2025-05-23 18:45:58 uses posix_fadvise().", |
"[00] 2025-05-23 18:45:58 cd to /var/lib/mysql/", |
"[00] 2025-05-23 18:45:58 open files limit requested 0, set to 1048576", |
"[00] 2025-05-23 18:45:58 mariabackup: using the following InnoDB configuration:", |
"[00] 2025-05-23 18:45:58 innodb_data_home_dir = ", |
"[00] 2025-05-23 18:45:58 innodb_data_file_path = ibdata1:12M:autoextend", |
"[00] 2025-05-23 18:45:58 innodb_log_group_home_dir = ./", |
"[00] 2025-05-23 18:45:58 InnoDB: Using liburing", |
"2025-05-23 18:45:58 0 [Note] InnoDB: Number of transaction pools: 1", |
"mariabackup: io_uring_queue_init() failed with EPERM: sysctl kernel.io_uring_disabled has the value 2, or 1 and the user of the process is not a member of sysctl kernel.io_uring_group. (see man 2 io_uring_setup).", |
"2025-05-23 18:45:58 0 [Warning] InnoDB: liburing disabled: falling back to innodb_use_native_aio=OFF", |
"2025-05-23 18:45:58 0 [Note] InnoDB: Memory-mapped log (block size=512 bytes)", |
"250523 18:45:58 [ERROR] mariabackup got signal 11 ;", |
"Sorry, we probably made a mistake, and this is a bug.", |
"", |
"Your assistance in bug reporting will enable us to fix this for the next release.", |
"To report this bug, see https://mariadb.com/kb/en/reporting-bugs about how to report", |
"a bug on https://jira.mariadb.org/.", |
"", |
"Please include the information from the server start above, to the end of the", |
"information below.", |
"", |
"Server version: 10.11.13-MariaDB-deb12 source revision: 8fb09426b98583916ccfd4f8c49741adc115bac3", |
"", |
"The information page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mariadbd/", |
"contains instructions to obtain a better version of the backtrace below.", |
"Following these instructions will help MariaDB developers provide a fix quicker.", |
"", |
"Attempting backtrace. Include this in the bug report.", |
"(note: Retrieving this information may fail)", |
"", |
"Thread pointer: 0x0", |
"stack_bottom = 0x0 thread_stack 0x49000", |
"Printing to addr2line failed", |
"mariabackup(my_print_stacktrace+0x2e)[0x55d3302aa3ae]", |
"mariabackup(handle_fatal_signal+0x229)[0x55d32fdcd6d9]", |
"/lib/x86_64-linux-gnu/libc.so.6(+0x3c050)[0x7f0230668050]", |
"mariabackup(server_mysql_fetch_row+0x14)[0x55d32fa19474]", |
"mariabackup(+0x76ca87)[0x55d32f9eba87]", |
"mariabackup(+0x75f37a)[0x55d32f9de37a]", |
"mariabackup(main+0x163)[0x55d32f983053]", |
"/lib/x86_64-linux-gnu/libc.so.6(+0x2724a)[0x7f023065324a]", |
"/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x85)[0x7f0230653305]", |
"mariabackup(_start+0x21)[0x55d32f9c8161]", |
"Writing a core file...", |
"Working directory at /var/lib/mysql", |
"Resource Limits (excludes unlimited resources):", |
"Limit Soft Limit Hard Limit Units ", |
"Max stack size 8388608 unlimited bytes ", |
"Max open files 1048576 1048576 files ", |
"Max locked memory 8388608 8388608 bytes ", |
"Max pending signals 30827 30827 signals ", |
"Max msgqueue size 819200 819200 bytes ", |
"Max nice priority 0 0 ", |
"Max realtime priority 0 0 ", |
"Core pattern: core", |
"", |
"Kernel version: Linux version 6.1.0-35-amd64 (debian-kernel@lists.debian.org) (gcc-12 (Debian 12.2.0-14+deb12u1) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40) #1 SMP PREEMPT_DYNAMIC Debian 6.1.137-1 (2025-05-07)", |
"", |
"/usr/local/bin/kolla_mariadb_backup_replica.sh: line 36: 44 Segmentation fault (core dumped) mariabackup --defaults-file=\"${REPLICA_MY_CNF}\" --backup --stream=mbstream --incremental-history-name=\"${LAST_FULL_DATE}\" --history=\"${LAST_FULL_DATE}\"", |
" 45 Done | gzip > \"${BACKUP_DIR}/incremental-$(date +%H)-mysqlbackup-${LAST_FULL_DATE}.qp.mbc.mbs.gz\"" |
],
|
"stdout": "Taking an incremental backup\n", |
"stdout_lines": [ |
"Taking an incremental backup" |
]
|
}
|
mariabackup: io_uring_queue_init() failed with EPERM: sysctl kernel.io_uring_disabled has the value 2, or 1 and the user of the process is not a member of sysctl kernel.io_uring_group. (see man 2 io_uring_setup)........... |
.
|
.
|
Sysctl :
kernel.apparmor_restrict_unprivileged_io_uring = 0 |
kernel.io_uring_disabled = 0 |
kernel.io_uring_group = -1 |
|
Server and backup process both running in container, I've also tried to run backup procedure as privileged container, or with seccomp unconfined , nothing helped.
Mariadb-server and mariadb-backup is 10.11.13, if I make a APT pin and disable installation of 10.11.13 here https://review.opendev.org/c/openstack/kolla/+/950820 tests are working again
https://review.opendev.org/c/openstack/kolla-ansible/+/950794 / https://6dee196209eb9fb193b3-af1c41ea995a25ff0cf4e07cd0399165.ssl.cf5.rackcdn.com/openstack/8eee012a8cf345f09b2d505071186d11/primary/logs/ansible/test-mariadb .
where it can be seen 10.11.11 is working.
Thanks for fix
Attachments
Issue Links
- causes
-
MDEV-37406 mariadb-backup fails on 10.11.14
-
- Confirmed
-
- is caused by
-
MDEV-30000 make mariadb-backup to force an innodb checkpoint
-
- Closed
-
- is duplicated by
-
MDEV-36967 No backup_history saved, mariabackup seg faults
-
- Closed
-
- relates to
-
MDEV-37454 mariabackup fails to populate mariadb_backup_history table
-
- Open
-