Details
-
Bug
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Not a Bug
-
11.7.2
-
None
-
Docker container running in Windows 11 24H2
-
Not for Release Notes
Description
I am running MariaDB 11.7.2 in Docker container in Windows 11 24H2 and I am getting often errors like
"Error on rename of './simbiatr_simbiat/uc__users' to './simbiatr_simbiat/#sql-backup-10-15be' (errno: 194 "Tablespace is missing for a table")"
|
They happen with various ALTER operations (inconsistent) and after OPTIMIZE (consistent), at least. With OPTIMIZE the replication steps for me are:
- Run OPTIMIZE on a table
- Try running it again
With ALTER it seem to be happening after adding/removing columns and sometimes changing indexes, but I can't figure out a clear pattern, unfortunately. But there the "double rename" from MDEV-24189 is also replicable for me, even though it has a different error. Thus I linked it and a couple of other old tickets, that seem to be the same issue.
Regardless of replication scenario, once I get the error, the only way to solve it is to specifically STOP the container and then start again. If I do restart (the one from Docker Desktop UI) - it does not go away in like 95% of the cases (it did seem to work a couple of times, though). The issue is not replicable on my PROD environment running Ubuntu.
I also found this old Docker issue, and it looks like there was an issue with how it was handling files in similar conditions. It may be recurrence of the same issue on 24H2 or newer version of Docker, but can't say for sure, since I got the need to do a lot of ALTERs within last month, and I had 24H2 for at least 2 months now, and I think there was at least 2 Docker update during that time.
Below are my Dockerfile, part of Docker-compose and my.cnf. Let me know if I can provide anything else to help fix this bug.
Dockerfile:
#Version needs to be updated in RUN as well
|
FROM mariadb:11.7.2-noble
|
#Copying files for initial restore. Only really needed in case no physical backup (from mariadb-backup or alternatives) is available
|
COPY --chown=root ./build/DDL/*.sql /docker-entrypoint-initdb.d/
|
 |
# Need cron, LZ4 compression plugin for MariaDB, and 7zip for backups compression
|
RUN apt-get -y update&&apt-get -y --no-install-recommends upgrade&&apt-get -y --no-install-recommends install cron mariadb-plugin-provider-lz4=1:11.7.2+maria~ubu2404 7zip&&apt-get -y autoremove&&apt-get clean
|
Docker-compose:
mariadb:
|
container_name: mariadb
|
build:
|
context: .
|
dockerfile: config/mysql/Dockerfile
|
restart: unless-stopped
|
environment:
|
WEB_SERVER_TEST: ${WEB_SERVER_TEST}
|
MARIADB_AUTO_UPGRADE: true
|
#Password for backups created by CRON job
|
MARIADB_BACKUP_PASSWORD: ${MARIADB_BACKUP_PASSWORD}
|
#Limit root access from localhost only
|
MARIADB_ROOT_HOST: localhost
|
MARIADB_RANDOM_ROOT_PASSWORD: true
|
MARIADB_MYSQL_LOCALHOST_USER: true
|
networks:
|
- webserver
|
ports:
|
- "3306:3306"
|
volumes:
|
- mariadb_socket:/run/mysqld/:rw
|
- ${MARIADB_DATA_DIR}:/var/lib/mysql:rw
|
- ./config/mysql:/etc/mysql/conf.d:ro
|
- ./logs:/usr/local/logs:rw
|
- ${MARIADB_BACKUP_DIR}:/usr/local/backups:rw
|
- ./build/DDL:/usr/local/DDL:ro
|
security_opt:
|
- no-new-privileges=true
|
healthcheck:
|
test: [ "CMD", "healthcheck.sh", "--defaults-file=/etc/mysql/conf.d/my.cnf", "--connect", "--innodb_initialized", "--innodb_buffer_pool_loaded", "--mariadbupgrade" ]
|
start_period: 120s
|
interval: 10s
|
timeout: 5s
|
retries: 3
|
command: >
|
/bin/sh -c "printenv | grep -v "no_proxy" > /etc/environment &&
|
#We need to ensure that buffer pool file exists
|
touch /var/lib/mysql/ib_buffer_pool &&
|
chown mysql:mysql /var/lib/mysql/ib_buffer_pool &&
|
#We need to register cron tasks and start cron service before the endpoint
|
cp /etc/mysql/conf.d/cron/mariadb.cron /etc/cron.d/mariadb.cron &&
|
chmod 0644 /etc/cron.d/mariadb.cron &&
|
crontab /etc/cron.d/mariadb.cron &&
|
cron &&
|
docker-entrypoint.sh mariadbd"
|
my.cnf:
# The MariaDB configuration file
|
#
|
# The MariaDB/MySQL tools read configuration files in the following order:
|
# 0. "/etc/mysql/my.cnf" symlinks to this file, reason why all the rest is read.
|
# 1. "/etc/mysql/mariadb.cnf" (this file) to set global defaults,
|
# 2. "/etc/mysql/conf.d/*.cnf" to set global options.
|
# 3. "/etc/mysql/mariadb.conf.d/*.cnf" to set MariaDB-only options.
|
# 4. "~/.my.cnf" to set user-specific options.
|
#
|
# If the same option is defined multiple times, the last one will apply.
|
#
|
# One can use all long options that the program supports.
|
# Run program with --help to get a list of available options and with
|
# --print-defaults to see which it would actually understand and use.
|
#
|
# If you are new to MariaDB, check out https://mariadb.com/kb/en/basic-mariadb-articles/
|
 |
#
|
# This group is read both by the client and the server
|
# use it for options that affect everything
|
#
|
 |
[client-server]
|
# These are commonly set, remove the # and set as required.
|
# basedir = .....
|
# datadir = .....
|
# port = .....
|
# server_id = .....
|
# Port or socket location where to connect
|
# port = 19913
|
socket = /run/mysqld/mysqld.sock
|
 |
# Import all .cnf files from configuration directory
|
#!include /etc/mysql/conf.d/security.cnf
|
#!includedir /etc/mysql/mariadb.conf.d/
|
 |
[client]
|
default-character-set=utf8mb4
|
 |
[mysqld]
|
# Remove leading # to turn on a very important data integrity option: logging
|
# changes to the binary log between backups.
|
# log_bin
|
default_time_zone='+00:00'
|
 |
bind-address = *
|
 |
require_secure_transport=on
|
ssl=0
|
skip_ssl
|
ssl_ca=
|
ssl_cert=
|
ssl_key=
|
 |
plugin_maturity=stable
|
plugin_load_add=auth_ed25519
|
plugin_load_add=provider_lz4
|
provider_lz4=FORCE_PLUS_PERMANENT
|
 |
log_error=/usr/local/logs/mariadb.log
|
log_warnings=2
|
 |
character_set_filesystem=utf8mb4
|
 |
character-set-client-handshake = false #force encoding to uft8
|
character_set_server=utf8mb4
|
collation-server=utf8mb4_0900_as_cs
|
default_password_lifetime=0
|
 |
local-infile=0
|
innodb_force_recovery = 0
|
 |
# Remove leading # and set to the amount of RAM for the most important data
|
# cache in MySQL. Start at 70% of total RAM for dedicated server, else 10%.
|
# innodb_buffer_pool_size = 128M
|
#For large inserts
|
#innodb_buffer_pool_size = 20G
|
#For regular operation
|
innodb_buffer_pool_size = 8G
|
innodb_max_dirty_pages_pct_lwm=1
|
innodb_buffer_pool_load_at_startup=1
|
innodb_buffer_pool_dump_at_shutdown=1
|
innodb_fast_shutdown = 0
|
 |
innodb_log_file_size = 256M
|
innodb_log_buffer_size = 256MB
|
#Required for page compression
|
innodb_default_row_format='dynamic'
|
innodb_compression_default=ON
|
innodb_compression_algorithm='lz4'
|
innodb_io_capacity=10000
|
innodb_io_capacity_max=20000
|
innodb_read_io_threads=16
|
innodb_write_io_threads=16
|
 |
innodb_undo_log_truncate=ON
|
 |
# Copied from Fozzy
|
log_queries_not_using_indexes = ON
|
max_allowed_packet = 33554432
|
max_connect_errors = 10
|
max_connections = 500
|
max_user_connections = 50
|
query_cache_size = 64MB
|
query_cache_type = ON
|
query_cache_strip_comments = ON
|
query_cache_limit = 4MB
|
sync_binlog = 1
|
 |
# FULLTEXT settings
|
ft_min_word_len = 1
|
innodb_ft_min_token_size = 1
|
innodb_ft_num_word_optimize = 10000
|
innodb_ft_sort_pll_degree = 4
|
 |
# Additional optimizations from https://releem.com/blog/tpost/9kdjxj8ve1-mysql-8-performance-benchmark
|
sort_buffer_size = 262144
|
join_buffer_size = 8M
|
table_open_cache = 4096
|
table_definition_cache = 2432
|
 |
# Temp tables sizes
|
max_heap_table_size = 512M
|
tmp_table_size = 512M
|
tmp_disk_table_size = 1G
|
 |
#Optimizations based on https://dba.stackexchange.com/questions/212884/innodb-after-48-hours-of-optimizing-10mb-sec-write-speed
|
innodb_lru_scan_depth=512
|
innodb_flushing_avg_loops=15
|
max_seeks_for_key=32
|
max_write_lock_count=16
|
read_rnd_buffer_size=256K
|
 |
#Be more aggressive with sleeping connections
|
interactive_timeout=600
|
wait_timeout=600
|
 |
#Monitoring stuff
|
performance_schema=ON
|
log_output=TABLE
|
slow_query_log=ON
|
#slow_query_log_file=/usr/local/logs/mariadb-slow.log
|
long_query_time=2
|
log_queries_not_using_indexes=OFF
|
log_slow_max_warnings=5
|
log_slow_min_examined_row_limit=10000
|
log_slow_rate_limit=10
|
general_log=OFF
|
Attachments
Issue Links
- relates to
-
MDEV-24189 Rename Table twice raise error "Tablespace is missing for a table"
-
- Closed
-
-
MDEV-32207 altering table returns errno: 168 "Unknown (generic) error from engine"
-
- Closed
-
-
MDEV-33284 ERROR 1025 (HY000): Error on rename of './test/t1' to './test/t2' (errno: 150 "Foreign key constraint is incorrectly formed") on RENAME
-
- Open
-
- links to