Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-29909

SST fails when table is defined with DATA DIRECTORY='/path/to' and datafile is larger than datadir space

    XMLWordPrintable

Details

    • Bug
    • Status: Stalled (View Workflow)
    • Critical
    • Resolution: Unresolved
    • 10.6
    • 10.6
    • Galera SST
    • None

    Description

      I first tested using mariabackup to restore a non-clustered database with table defined with DATA DIRECTORY and datafile larger than space in datadir. That succeeded OK. So, mariabackup is able to restore into the directory outside of datadir.

      Next, I tried with a two node galera cluser 10.6.9-5-MariaDB-enterprise-log using mariabackup as wsrep_sst_method. This fails. In this case, datadir=/var/lib/mysql and alternate for large table is /var/lib/mariadb.

      root@gg2:~$ df -h /var/lib/mysql/
      Filesystem      Size  Used Avail Use% Mounted on
      /dev/loop0      974M  447M  460M  50% /var/lib/mysql
      root@gg2:~$ df -h /var/lib/mariadb/
      Filesystem      Size  Used Avail Use% Mounted on
      /dev/sda1       4.9G  3.8G  903M  81% /
      root@gg2:~$ find /var/lib/mariadb/ -type f -exec ls -lh {} \;
      -rw-rw---- 1 mysql mysql 40M Oct 28 10:50 /var/lib/mariadb/testr/fast_rows.ibd
      -rw-rw---- 1 mysql mysql 732M Oct 28 11:00 /var/lib/mariadb/testr/bigtable.ibd
      root@gg2:~$ systemctl stop mariadb
      root@gg2:~$ rm -fr /var/lib/mysql/*
      root@gg2:~$ rm -fr /var/lib/mariadb/*
      root@gg2:~$ echo "=========================" >> /var/log/mysql/mariadb.err
      root@gg2:~$ systemctl start mariadb
      Job for mariadb.service failed because a fatal signal was delivered to the control process.
      See "systemctl status mariadb.service" and "journalctl -xe" for details.
      root@gg2:~$ tail -300 /var/log/mysql/mariadb.err | grep -i error
      2022-10-28 11:03:11 0 [ERROR] WSREP: ./gcs/src/gcs_group.cpp:gcs_group_handle_join_msg():1207: Will never receive state. Need to abort.
      2022-10-28 11:03:11 0 [ERROR] WSREP: Process completed with error: wsrep_sst_mariabackup --role 'joiner' --address '192.168.68.192' --datadir '/var/lib/mysql/' --parent 1942 --progress 0 --binlog '/var/log/mysql/mariadb-bin' --mysqld-args --wsrep_start_position=00000000-0000-0000-0000-000000000000:-1: 2 (No such file or directory)
      2022-10-28 11:03:11 0 [ERROR] WSREP: Failed to read uuid:seqno and wsrep_gtid_domain_id from joiner script.
      2022-10-28 11:03:11 1 [ERROR] WSREP: Application received wrong state:
      2022-10-28 11:03:11 1 [ERROR] WSREP: Application state transfer failed. This is unrecoverable condition, restart required.
      root@gg2:~$ rm -fr /var/lib/mysql/*
      root@gg2:~$ rm -fr /var/lib/mariadb/*
      root@gg2:~$ touch /tmp/t.txt
      root@gg2:~$ echo "-----------------------------------" >> /var/log/mysql/mariadb.err
      root@gg2:~$ systemctl start mariadb
      Job for mariadb.service failed because the control process exited with error code.
      See "systemctl status mariadb.service" and "journalctl -xe" for details.
      root@gg2:~$ grep -A500 "\-\-\-\-\-\-\-\-\-\-\-" /var/log/mysql/mariadb.err |grep -i ERROR
      2022-10-28 11:08:18 0 [ERROR] WSREP: Requested size 268436760 for '/var/lib/mysql//galera.cache' exceeds available storage space 154144768: 28 (No space left on device)
      2022-10-28 11:08:18 0 [ERROR] WSREP: Failed to create a new provider '/usr/lib/libgalera_smm.so' with options 'socket.ssl=false; cert.optimistic_pa=no; gcache.recover=yes; gcache.size=256M; gcs.fc_factor=0.8; gcs.fc_limit=96; gcs.fc_master_slave=yes;': Failed to initialize wsrep provider
      2022-10-28 11:08:18 0 [ERROR] WSREP: Failed to load provider
      2022-10-28 11:08:18 0 [ERROR] Plugin 'wsrep' init function returned error.
      2022-10-28 11:08:18 0 [ERROR] Could not open mysql.plugin table: "Table 'mysql.plugin' doesn't exist". Some plugins may be not loaded
      2022-10-28 11:08:18 0 [ERROR] Failed to initialize plugins.
      2022-10-28 11:08:18 0 [ERROR] Aborting
      2022-10-28 11:08:20 0 [ERROR] WSREP: Requested size 268436760 for '/var/lib/mysql//galera.cache' exceeds available storage space 154120192: 28 (No space left on device)
      2022-10-28 11:08:20 0 [ERROR] WSREP: Failed to create a new provider '/usr/lib/libgalera_smm.so' with options 'socket.ssl=false; cert.optimistic_pa=no; gcache.recover=yes; gcache.size=256M; gcs.fc_factor=0.8; gcs.fc_limit=96; gcs.fc_master_slave=yes;': Failed to initialize wsrep provider
      2022-10-28 11:08:20 0 [ERROR] WSREP: Failed to load provider
      2022-10-28 11:08:20 0 [ERROR] Plugin 'wsrep' init function returned error.
      2022-10-28 11:08:20 0 [ERROR] Could not open mysql.plugin table: "Table 'mysql.plugin' doesn't exist". Some plugins may be not loaded
      2022-10-28 11:08:20 0 [ERROR] Failed to initialize plugins.
      2022-10-28 11:08:20 0 [ERROR] Aborting
      root@gg2:~$
      

      Attachments

        Issue Links

          Activity

            People

              sysprg Julius Goryavsky
              edward Edward Stoever
              Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.