[MDEV-31099] InnoDB: Attempted to open a previously opened tablespace Created: 2023-04-20  Updated: 2024-02-07

Status: Stalled
Project: MariaDB Server
Component/s: Backup
Affects Version/s: 10.6.13, 10.5.25
Fix Version/s: 10.5, 10.6

Type: Bug Priority: Major
Reporter: Matthias Leich Assignee: Vladislav Lesin
Resolution: Unresolved Votes: 1
Labels: rr-profile


 Description   

origin/bb-10.6-MDEV-26782 3d589c462b7e17145e055ce9066bbf27856ca215 2023-04-20T09:14:17+03:00
 
Scenario:
1. Start the server on location A and generate some initial data. 
2. Two sessions run some DDL/DML mix concurent.
3. During 2. is ongoing run in a loop
     mariabackup --backup ... (store data on location B)
     mariabackup --prepare ... on location B
     Start a server on the data in location B
     Check the data in that server
 
In one of these rounds fails mariabackup --backup with
# 2023-04-20T12:12:04 [2149587] | [rr 2228638 2730][00] 2023-04-20 12:12:03 mariabackup: Generating a list of tablespaces
# 2023-04-20T12:12:04 [2149587] | [rr 2228638 2866]2023-04-20 12:12:03 0 [ERROR] InnoDB: Attempted to open a previously opened tablespace. Previous tablespace: ./test/t4.ibd uses space ID: 31. Cannot open filepath: ./cool_down/t4.ibd which uses the same space ID.
# 2023-04-20T12:12:04 [2149587] | [rr 2228638 2870][00] FATAL ERROR: 2023-04-20 12:12:03 Failed to validate first page of the file cool_down/t4, error 40
 
pluto:/data/results/1681983232/TBR-1279$ _RR_TRACE_DIR=./1_clone/rr/ rr replay --mark-stdio
 
 # git clone https://github.com/mleich1/rqg --branch <pick the right branch> RQG
#
# GIT_SHOW: HEAD -> master, origin/master, origin/HEAD 499b2f32d99115987374e521c44f6369aded4f6a 2023-04-19T17:38:04+02:00
# rqg.pl  : Version 4.2.1 (2022-12)
#
# $RQG_HOME/rqg.pl \
# --grammar=conf/mariadb/table_stress_innodb_nocopy1.yy \
# --gendata=conf/mariadb/table_stress.zz \
# --gendata_sql=conf/mariadb/table_stress.sql \
# --reporters=Mariabackup_linux \
# --mysqld=--loose-innodb-log-file-size=150M \
# --mysqld=--loose-innodb_lock_schedule_algorithm=fcfs \
# --mysqld=--loose-idle_write_transaction_timeout=0 \
# --mysqld=--loose-idle_transaction_timeout=0 \
# --mysqld=--loose-idle_readonly_transaction_timeout=0 \
# --mysqld=--connect_timeout=60 \
# --mysqld=--interactive_timeout=28800 \
# --mysqld=--slave_net_timeout=60 \
# --mysqld=--net_read_timeout=30 \
# --mysqld=--net_write_timeout=60 \
# --mysqld=--loose-table_lock_wait_timeout=50 \
# --mysqld=--wait_timeout=28800 \
# --mysqld=--lock-wait-timeout=86400 \
# --mysqld=--innodb-lock-wait-timeout=50 \
# --no_mask \
# --queries=10000000 \
# --seed=random \
# --reporters=Backtrace \
# --reporters=ErrorLog \
# --reporters=Deadlock \
# --validators=None \
# --mysqld=--log_output=none \
# --mysqld=--log_bin_trust_function_creators=1 \
# --mysqld=--loose-debug_assert_on_not_freed_memory=0 \
# --engine=InnoDB \
# --restart_timeout=240 \
# --mysqld=--plugin-load-add=file_key_management.so \
# --mysqld=--loose-file-key-management-filename=$RQG_HOME/conf/mariadb/encryption_keys.txt \
# --mysqld=--plugin-load-add=provider_lzo.so \
# --mysqld=--plugin-load-add=provider_bzip2.so \
# --mysqld=--plugin-load-add=provider_lzma.so \
# --mysqld=--plugin-load-add=provider_snappy.so \
# --mysqld=--plugin-load-add=provider_lz4.so \
# --mysqld=--loose-innodb_compression_level=1 \
# --duration=300 \
# --mysqld=--loose-innodb_fatal_semaphore_wait_threshold=300 \
# --mysqld=--loose-innodb_read_only_compressed=OFF \
# --mysqld=--innodb_stats_persistent=off \
# --mysqld=--innodb_adaptive_hash_index=on \
# --mysqld=--log-bin \
# --mysqld=--sync-binlog=1 \
# --mysqld=--loose-innodb_evict_tables_on_commit_debug=on \
# --mysqld=--loose-max-statement-time=30 \
# --threads=2 \
# --mysqld=--innodb-use-native-aio=0 \
# --mysqld=--loose-gdb \
# --mysqld=--loose-debug-gdb \
# --rr=Extended \
# --rr_options=--chaos --wait \
# --mysqld=--innodb_undo_log_truncate=OFF \
# --mysqld=--innodb_rollback_on_timeout=OFF \
# --mysqld=--innodb_page_size=16K \
# --mysqld=--innodb-buffer-pool-size=8M \
# <local settings>
   



 Comments   
Comment by Matthias Leich [ 2023-05-30 ]

origin/bb-10.6-MDEV-31264 7340eb0921c6f71a963a56cca915a08f6c1ac660 2023-05-26T17:34:41+05:30
mariabackup --backup fails with

  1. 2023-05-30T05:09:17 [3655187] | 2023-05-30 5:09:16 0 [Note] InnoDB: Opened 16 undo tablespaces
  2. 2023-05-30T05:09:17 [3655187] | [00] 2023-05-30 05:09:16 mariabackup: Generating a list of tablespaces
  3. 2023-05-30T05:09:17 [3655187] | 2023-05-30 5:09:16 0 [ERROR] InnoDB: Operating system error number 2 in a file operation.
  4. 2023-05-30T05:09:17 [3655187] | 2023-05-30 5:09:16 0 [ERROR] InnoDB: The error means the system cannot find the path specified.
  5. 2023-05-30T05:09:17 [3655187] | 2023-05-30 5:09:16 0 [ERROR] InnoDB: File ./test/t4.ibd: 'open' returned OS error 71.
  6. 2023-05-30T05:09:17 [3655187] | 2023-05-30 5:09:16 0 [ERROR] InnoDB: Operating system error number 2 in a file operation.
  7. 2023-05-30T05:09:17 [3655187] | 2023-05-30 5:09:16 0 [ERROR] InnoDB: The error means the system cannot find the path specified.
  8. 2023-05-30T05:09:17 [3655187] | 2023-05-30 5:09:16 0 [Warning] InnoDB: Cannot open './test/t4.ibd'.
  9. 2023-05-30T05:09:17 [3655187] | [00] FATAL ERROR: 2023-05-30 05:09:16 Failed to validate first page of the file test/t4, error 62
    But without rr trace or core. (missing stuff might be problem with RQG)
    sdp:/data1/results/1685444106/TBR-1279/
Comment by Vladislav Lesin [ 2023-12-18 ]

It looks similar to MDEV-31256.

Generated at Thu Feb 08 10:21:16 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.