Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-17218

Assertion `rw_lock_own(dict_operation_lock, RW_LOCK_S) || node->vcol_info.is_used()' failed in row0purge.cc:885: void row_purge_upd_exist_or_extern_func after crash recovery with virtual columns

    XMLWordPrintable

Details

    Description

      Note: might be a duplicate of (or have the same root cause as) MDEV-17215, but the assertion is in a different place, so filing it separately just in case.

      10.3 90b292ce311

      mysqld: /data/src/10.3/storage/innobase/row/row0purge.cc:885: void row_purge_upd_exist_or_extern_func(const que_thr_t*, purge_node_t*, trx_undo_rec_t*): Assertion `rw_lock_own(dict_operation_lock, RW_LOCK_S) || node->vcol_info.is_used()' failed.
      180922 21:58:55 [ERROR] mysqld got signal 6 ;
       
      #7  0x00007fa47ca83ee2 in __assert_fail () from /lib/x86_64-linux-gnu/libc.so.6
      #8  0x0000563023d65eed in row_purge_upd_exist_or_extern_func (thr=0x563027230a80, node=0x563027230f28, undo_rec=0x7fa418014078 "\001\206\034\001\030") at /data/src/10.3/storage/innobase/row/row0purge.cc:884
      #9  0x0000563023d66d14 in row_purge_record_func (node=0x563027230f28, undo_rec=0x7fa418014078 "\001\206\034\001\030", thr=0x563027230a80, updated_extern=false) at /data/src/10.3/storage/innobase/row/row0purge.cc:1201
      #10 0x0000563023d66eb6 in row_purge (node=0x563027230f28, undo_rec=0x7fa418014078 "\001\206\034\001\030", thr=0x563027230a80) at /data/src/10.3/storage/innobase/row/row0purge.cc:1245
      #11 0x0000563023d67264 in row_purge_step (thr=0x563027230a80) at /data/src/10.3/storage/innobase/row/row0purge.cc:1331
      #12 0x0000563023ceae2a in que_thr_step (thr=0x563027230a80) at /data/src/10.3/storage/innobase/que/que0que.cc:1046
      #13 0x0000563023ceb05d in que_run_threads_low (thr=0x563027230a80) at /data/src/10.3/storage/innobase/que/que0que.cc:1108
      #14 0x0000563023ceb24e in que_run_threads (thr=0x563027230a80) at /data/src/10.3/storage/innobase/que/que0que.cc:1148
      #15 0x0000563023db253c in srv_task_execute () at /data/src/10.3/storage/innobase/srv/srv0srv.cc:2452
      #16 0x0000563023db26e1 in srv_worker_thread (arg=0x0) at /data/src/10.3/storage/innobase/srv/srv0srv.cc:2500
      #17 0x00007fa47e75a494 in start_thread (arg=0x7fa4397fa700) at pthread_create.c:333
      #18 0x00007fa47cb4093f in clone () from /lib/x86_64-linux-gnu/libc.so.6
      

      To reproduce:

      • download the datadir from ftp://ftp.askmonty.org/public/data-mdev17218.tar.gz
      • remove a directory named data if exists in the current folder;
      • untar data-mdev17218.tar.gz, it will create data folder with some contents in it;
      • start 10.3 server on the created datadir data with the following options (modify the path to keys.txt if necessary):

        --file-key-management --file-key-management-filename=`pwd`/mysql-test/std_data/keys.txt --plugin-load-add=file_key_management.so --innodb-encrypt-tables --innodb-encrypt-log --innodb-encryption-threads=4 --aria-encrypt-tables=1 --encrypt-tmp-disk-tables=1 --encrypt-binlog --innodb-page-size=8K --innodb-compression-algorithm=none --loose-innodb_log_compressed_pages=on
        

      The server starts all right, but in a few seconds it aborts with the assertion above.

      The provided datadir was created on the same version/build of MariaDB server, by creating tables with virtual columns, performing some DML on them and at some point killing the server with SIGKILL. The datadir was stored after that, before any attempt to restart the server. Full general log and error log from the beginning up to SIGKILL can be found in logs-mdev17218.tar.gz.

      All threads' stack trace is also attached.

      It's not easy to reproduce the initial datadir creation, because usually MDEV-17215 happens first. On the same reason, I can't say for sure which of the server options above are relevant and which are not, it's just the same options with which this particular datadir was created.
      10.2 and 10.3 are known to be affected, I have no information about other versions.

      The test below can reproduce the problem from scratch (might need to re-run several times if it hits MDEV-17215 instead):

      github.com/MariaDB/randgen --branch elenst-mdev14046 1348571d03a

      perl ./runall-trials.pl --trials=20 --grammar=conf/mariadb/oltp-transactional.yy --grammar2=conf/mariadb/oltp_and_ddl.yy --gendata=conf/mariadb/innodb_upgrade.zz --gendata-advanced --vcols --mysqld=--server-id=111 --scenario=CrashUpgrade --duration=90 --basedir=/data/bld/10.3 --mysqld=--file-key-management --mysqld=--file-key-management-filename=/data/bld/keys.txt --mysqld=--plugin-load-add=file_key_management.so --mysqld=--innodb-encrypt-tables --mysqld=--innodb-encrypt-log --mysqld=--innodb-encryption-threads=4 --mysqld=--aria-encrypt-tables=1 --mysqld=--encrypt-tmp-disk-tables=1 --mysqld=--encrypt-binlog --mysqld=--innodb-page-size=8K --mysqld=--innodb-compression-algorithm=none --mysqld=--loose-innodb-file-format=Barracuda --mysqld=--loose-innodb-file-per-table=1 --gendata=conf/mariadb/innodb_upgrade.zz --no-mask --seed=1537618555 --threads=4 --queries=100M --mysqld=--loose-max-statement-time=20 --mysqld=--loose-lock-wait-timeout=20 --mysqld=--loose-innodb-lock-wait-timeout=10 --mysqld=--loose-innodb_log_compressed_pages=on --mtr-build-thread=300 --vardir1=/dev/shm/vardir
      

      Attachments

        1. logs-mdev17218.tar.gz
          3.29 MB
          Elena Stepanova
        2. threads
          36 kB
          Elena Stepanova

        Issue Links

          Activity

            People

              thiru Thirunarayanan Balathandayuthapani
              elenst Elena Stepanova
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.