Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-15855

Assertion `mysql_table' failed in innobase_get_computed_value upon DDL/DML on a table with virtual columns

Details

    Description

      Run the test case with --mysqld=--lock-wait-timeout=2 --repeat=50 --mem.

      --source include/have_innodb.inc
       
      CREATE TABLE t1 (y YEAR, vy YEAR AS (y) VIRTUAL, pk INT, PRIMARY KEY(pk), UNIQUE(vy)) ENGINE=InnoDB;
       
      INSERT IGNORE INTO t1 (pk,y) VALUES (1,2022);
      REPLACE INTO t1 (pk,y) SELECT pk,y FROM t1;
      ALTER TABLE t1 ENGINE=InnoDB;
      REPLACE INTO t1 (pk,y) SELECT pk,y FROM t1;
      TRUNCATE TABLE t1;
       
      # Cleanup
      DROP TABLE t1;
      

      10.3 18e8d420b796

      mysqld: /data/src/10.3/storage/innobase/handler/ha_innodb.cc:21079: dfield_t* innobase_get_computed_value(const dtuple_t*, const dict_v_col_t*, const dict_index_t*, mem_heap_t**, mem_heap_t*, const dict_field_t*, THD*, TABLE*, const dict_table_t*, upd_t*, dict_foreign_t*): Assertion `mysql_table' failed.
      180412 21:12:28 [ERROR] mysqld got signal 6 ;
       
      #7  0x00007ff408cd1ee2 in __assert_fail () from /lib/x86_64-linux-gnu/libc.so.6
      #8  0x0000559e7ce7dbef in innobase_get_computed_value (row=0x7ff3cc015dd0, col=0x7ff3ac075198, index=0x7ff3ac1738f8, local_heap=0x7ff3ecff7990, heap=0x7ff3cc015d00, ifield=0x0, thd=0x7ff3cc000b00, mysql_table=0x0, old_table=0x0, parent_update=0x0, foreign=0x0) at /data/src/10.3/storage/innobase/handler/ha_innodb.cc:21079
      #9  0x0000559e7d0281bd in row_vers_build_clust_v_col (row=0x7ff3cc015dd0, clust_index=0x7ff3ac1738f8, index=0x7ff3ac174658, heap=0x7ff3cc015d00) at /data/src/10.3/storage/innobase/row/row0vers.cc:473
      #10 0x0000559e7d0291dc in row_vers_old_has_index_entry (also_curr=1, rec=0x7ff3fe76c07e "\200", mtr=0x7ff3ecff7ac0, index=0x7ff3ac174658, ientry=0x7ff3cc0144e8, roll_ptr=3940649694921243, trx_id=0) at /data/src/10.3/storage/innobase/row/row0vers.cc:921
      #11 0x0000559e7cfdda78 in row_purge_poss_sec (node=0x559e7f8df308, index=0x7ff3ac174658, entry=0x7ff3cc0144e8) at /data/src/10.3/storage/innobase/row/row0purge.cc:265
      #12 0x0000559e7cfde2a5 in row_purge_remove_sec_if_poss_leaf (node=0x559e7f8df308, index=0x7ff3ac174658, entry=0x7ff3cc0144e8) at /data/src/10.3/storage/innobase/row/row0purge.cc:477
      #13 0x0000559e7cfde75e in row_purge_remove_sec_if_poss (node=0x559e7f8df308, index=0x7ff3ac174658, entry=0x7ff3cc0144e8) at /data/src/10.3/storage/innobase/row/row0purge.cc:584
      #14 0x0000559e7cfde95b in row_purge_del_mark (node=0x559e7f8df308) at /data/src/10.3/storage/innobase/row/row0purge.cc:654
      #15 0x0000559e7cfdfdc7 in row_purge_record_func (node=0x559e7f8df308, undo_rec=0x559e7f8df8b8 "", thr=0x559e7f8df138, updated_extern=false) at /data/src/10.3/storage/innobase/row/row0purge.cc:1043
      #16 0x0000559e7cfe00e4 in row_purge (node=0x559e7f8df308, undo_rec=0x559e7f8df8b8 "", thr=0x559e7f8df138) at /data/src/10.3/storage/innobase/row/row0purge.cc:1108
      #17 0x0000559e7cfe03e5 in row_purge_step (thr=0x559e7f8df138) at /data/src/10.3/storage/innobase/row/row0purge.cc:1187
      #18 0x0000559e7cf646cc in que_thr_step (thr=0x559e7f8df138) at /data/src/10.3/storage/innobase/que/que0que.cc:1051
      #19 0x0000559e7cf648ff in que_run_threads_low (thr=0x559e7f8df138) at /data/src/10.3/storage/innobase/que/que0que.cc:1113
      #20 0x0000559e7cf64af0 in que_run_threads (thr=0x559e7f8df138) at /data/src/10.3/storage/innobase/que/que0que.cc:1153
      #21 0x0000559e7d031916 in srv_task_execute () at /data/src/10.3/storage/innobase/srv/srv0srv.cc:2485
      #22 0x0000559e7d031abb in srv_worker_thread (arg=0x0) at /data/src/10.3/storage/innobase/srv/srv0srv.cc:2533
      #23 0x00007ff40a9a8494 in start_thread (arg=0x7ff3ecff9700) at pthread_create.c:333
      #24 0x00007ff408d8e93f in clone () from /lib/x86_64-linux-gnu/libc.so.6
      

      Couldn't reproduce on 10.2.
      See comments for the test case which affects non-debug build.

      Attachments

        Issue Links

          Activity

            Problem is that there is a deadlock between purge and truncate operation.
            Truncate operation holds MDL LOCK on the table (t1) and tries to acquire InnoDB dict_operation_lock.
            Purge holds InnoDB dict_operation_lock and tries to acquire MDL LOCK on the table (t1) to calculate
            the virtual column value. It leads to hang of server.

            Test case scenario has lock_wait_timeout as 2 seconds. So it leads to crash else it could lead to hang.

            thiru Thirunarayanan Balathandayuthapani added a comment - Problem is that there is a deadlock between purge and truncate operation. Truncate operation holds MDL LOCK on the table (t1) and tries to acquire InnoDB dict_operation_lock. Purge holds InnoDB dict_operation_lock and tries to acquire MDL LOCK on the table (t1) to calculate the virtual column value. It leads to hang of server. Test case scenario has lock_wait_timeout as 2 seconds. So it leads to crash else it could lead to hang.

            Results of RQG testing on bb-10.3-thiru-GA commit af9e7e9f35bd7352e6c17920b9c353111cad6d19:
            1. Compile error
            storage/innobase/dict/dict0dict.cc:1033:13: error: invalid conversion from ‘char’ to ‘char*’ [-fpermissive]
            for is_part = '\0'; with mit gcc (Ubuntu 7.2.0-8ubuntu3.2) 7.2.0.
            The compile worked well with gcc (Ubuntu 6.3.0-12ubuntu2) 6.3.0 20170406.
            Marko's proposal to use is_part = 0; worked well on gcc (Ubuntu 7.2.0-8ubuntu3.2) 7.2.0.

            2. The binaries were compiled with debug and without ASAN.

            3. Apply the replay test for the ASAN failures (https://jira.mariadb.org/browse/MDEV-16136) causes hitting asserts at roughly all code lines which showed up with ASAN failures in MDEV-16136.
            Content of test: Heavy concurrent

            • CREATE/DROP SCHEMA , CHECK TABLE
              Btw: It looks like this DDL is not strict required. But it raises the likelihood to replay trouble serious.
            • CREATE TABLE ... AS SELECT ... /DROP TABLE
            • ALTER TABLE ... ENGINE ...

            4. Mix of other RQG tests with concurrent DDL+DML (In sum 100 test runs)

            • Several times: RQG means to have detected an in minimum partially hanging server. mysql.err entries like
              --Thread 139821989754624 has waited at dict0boot.ic line 36 for 923.00 seconds the semaphore:
              Mutex at 0x5630128a9ba0, Mutex DICT_SYS created dict0dict.cc:1259, lock var 2
              Last time reserved in file storage/innobase/dict/dict0stats_bg.cc line 328
            • SEGV of a CREATE SEQUENCE seq4 MAXVALUE 3249 INCREMENT BY 3 NOCYCLE
            • SELECT * from `view_table1_innodb_int` WHERE ...subquery with t5 --> failed: 1032 Can't find record in 't5'
              I assume this is rather some non InnoDB Problem.
            • INSERT INTO `table100_innodb_int_autoinc` ... VALUES ...--> failed: 1032 Can't find record in 'table100_innodb_int_autoinc'
            • mysqld: storage/innobase/handler/handler0alter.cc:7482: bool rollback_inplace_alter_table(Alter_inplace_info*,
              const TABLE*, row_prebuilt_t*): Assertion `!clust_index->online_log' failed.
            • There were ~ 10 times running servers left over from already finished tests.
              Either the already known shutdown problems or internal failure of RQG.
            mleich Matthias Leich added a comment - Results of RQG testing on bb-10.3-thiru-GA commit af9e7e9f35bd7352e6c17920b9c353111cad6d19: 1. Compile error storage/innobase/dict/dict0dict.cc:1033:13: error: invalid conversion from ‘char’ to ‘char*’ [-fpermissive] for is_part = '\0'; with mit gcc (Ubuntu 7.2.0-8ubuntu3.2) 7.2.0. The compile worked well with gcc (Ubuntu 6.3.0-12ubuntu2) 6.3.0 20170406. Marko's proposal to use is_part = 0; worked well on gcc (Ubuntu 7.2.0-8ubuntu3.2) 7.2.0. 2. The binaries were compiled with debug and without ASAN. 3. Apply the replay test for the ASAN failures ( https://jira.mariadb.org/browse/MDEV-16136 ) causes hitting asserts at roughly all code lines which showed up with ASAN failures in MDEV-16136 . Content of test: Heavy concurrent CREATE/DROP SCHEMA , CHECK TABLE Btw: It looks like this DDL is not strict required. But it raises the likelihood to replay trouble serious. CREATE TABLE ... AS SELECT ... /DROP TABLE ALTER TABLE ... ENGINE ... 4. Mix of other RQG tests with concurrent DDL+DML (In sum 100 test runs) Several times: RQG means to have detected an in minimum partially hanging server. mysql.err entries like --Thread 139821989754624 has waited at dict0boot.ic line 36 for 923.00 seconds the semaphore: Mutex at 0x5630128a9ba0, Mutex DICT_SYS created dict0dict.cc:1259, lock var 2 Last time reserved in file storage/innobase/dict/dict0stats_bg.cc line 328 SEGV of a CREATE SEQUENCE seq4 MAXVALUE 3249 INCREMENT BY 3 NOCYCLE SELECT * from `view_table1_innodb_int` WHERE ...subquery with t5 --> failed: 1032 Can't find record in 't5' I assume this is rather some non InnoDB Problem. INSERT INTO `table100_innodb_int_autoinc` ... VALUES ...--> failed: 1032 Can't find record in 'table100_innodb_int_autoinc' mysqld: storage/innobase/handler/handler0alter.cc:7482: bool rollback_inplace_alter_table(Alter_inplace_info*, const TABLE*, row_prebuilt_t*): Assertion `!clust_index->online_log' failed. There were ~ 10 times running servers left over from already finished tests. Either the already known shutdown problems or internal failure of RQG.

            I pushed a cleanup and rebase of the code to 10.3-MDEV-15855. mleich, sorry, the proper fix for the compilation error would have been

            *is_part = '\0';
            

            This could have caused trouble when testing partitioned tables.

            I think that the final goal should be that all access to InnoDB tables is protected by MDL. We should introduce some "THD holds MDL on this table name" assertions to back that goal.

            Furthermore, I think that MDL acquisition or dict_sys->mutex ownership (dict_locked=true) when opening tables should be mutually exclusive. It is never acceptable to hold any InnoDB mutexes or rw-locks while waiting for MDL. This is the biggest issue that I was able to find in my review.

            I think that more work is needed to remove table lookup by index_id and possibly to clean up the DDL operations. Especially CREATE TABLE should perhaps not add the table to the dict_sys cache until it is all done.

            Some of this is admittedly "feature creep"; I think that we need the purge thread changes of this in 10.2 already, once the code has matured and it has been thoroughly tested in 10.4.

            marko Marko Mäkelä added a comment - I pushed a cleanup and rebase of the code to 10.3-MDEV-15855 . mleich , sorry, the proper fix for the compilation error would have been *is_part = '\0' ; This could have caused trouble when testing partitioned tables. I think that the final goal should be that all access to InnoDB tables is protected by MDL. We should introduce some "THD holds MDL on this table name" assertions to back that goal. Furthermore, I think that MDL acquisition or dict_sys->mutex  ownership ( dict_locked=true ) when opening tables should be mutually exclusive. It is never acceptable to hold any InnoDB mutexes or rw-locks while waiting for MDL. This is the biggest issue that I was able to find in my review. I think that more work is needed to remove table lookup by index_id and possibly to clean up the DDL operations. Especially CREATE TABLE should perhaps not add the table to the dict_sys cache until it is all done. Some of this is admittedly "feature creep"; I think that we need the purge thread changes of this in 10.2 already, once the code has matured and it has been thoroughly tested in 10.4.

            I'm removing the note about non-debug builds being unaffected, because the test case below causes different failures on all of current 10.3 debug/release and bb-10.3-release (tentative 10.3.8 release).
            To be run with --mysqld=--lock-wait-timeout=5 --mem --repeat=100

            --source include/have_innodb.inc
             
            CREATE TABLE t1 ( 
                pk INT NOT NULL,
                a INT,
                b CHAR(8) NULL,
                v CHAR(8) AS (b) VIRTUAL,
                PRIMARY KEY(pk),
                UNIQUE KEY(pk),
                INDEX(a,v)
            ) ENGINE=InnoDB;
            INSERT INTO t1 (pk,a,b) VALUES (1,10,'foo');
             
            REPLACE INTO t1 (pk,a,b) SELECT pk,a,b FROM t1;
            ALTER TABLE t1 RENAME TO t2;
             
            # Cleanup
            DROP TABLE t2;
            

            10.3 debug 46857a860c

            mysqld: /data/src/10.3/storage/innobase/handler/ha_innodb.cc:20767: dfield_t* innobase_get_computed_value(const dtuple_t*, const dict_v_col_t*, const dict_index_t*, mem_heap_t**, mem_heap_t*, const dict_field_t*, THD*, TABLE*, const dict_table_t*, upd_t*, dict_foreign_t*): Assertion `mysql_table' failed.
            180701 20:44:24 [ERROR] mysqld got signal 6 ;
             
            #7  0x00007f16a1635ee2 in __assert_fail () from /lib/x86_64-linux-gnu/libc.so.6
            #8  0x00005573b0c2375d in innobase_get_computed_value (row=0x7f1664015bd8, col=0x7f16480a3c08, index=0x7f16480904d8, local_heap=0x7f16867fa990, heap=0x7f1664015b00, ifield=0x0, thd=0x7f1664000b00, mysql_table=0x0, old_table=0x0, parent_update=0x0, foreign=0x0) at /data/src/10.3/storage/innobase/handler/ha_innodb.cc:20767
            #9  0x00005573b0dccd34 in row_vers_build_clust_v_col (row=0x7f1664015bd8, clust_index=0x7f16480904d8, index=0x7f164804f888, heap=0x7f1664015b00) at /data/src/10.3/storage/innobase/row/row0vers.cc:450
            #10 0x00005573b0dcdd47 in row_vers_old_has_index_entry (also_curr=1, rec=0x7f16970fc07e "\200", mtr=0x7f16867faac0, index=0x7f164804f888, ientry=0x7f16640156f8, roll_ptr=3659174718407633, trx_id=0) at /data/src/10.3/storage/innobase/row/row0vers.cc:896
            #11 0x00005573b0d832be in row_purge_poss_sec (node=0x5573b41aaf38, index=0x7f164804f888, entry=0x7f16640156f8) at /data/src/10.3/storage/innobase/row/row0purge.cc:265
            #12 0x00005573b0d83ae1 in row_purge_remove_sec_if_poss_leaf (node=0x5573b41aaf38, index=0x7f164804f888, entry=0x7f16640156f8) at /data/src/10.3/storage/innobase/row/row0purge.cc:477
            #13 0x00005573b0d83f9a in row_purge_remove_sec_if_poss (node=0x5573b41aaf38, index=0x7f164804f888, entry=0x7f16640156f8) at /data/src/10.3/storage/innobase/row/row0purge.cc:584
            #14 0x00005573b0d84195 in row_purge_del_mark (node=0x5573b41aaf38) at /data/src/10.3/storage/innobase/row/row0purge.cc:654
            #15 0x00005573b0d855f5 in row_purge_record_func (node=0x5573b41aaf38, undo_rec=0x5573b41ab4e8 "", thr=0x5573b41aad68, updated_extern=false) at /data/src/10.3/storage/innobase/row/row0purge.cc:1042
            #16 0x00005573b0d85912 in row_purge (node=0x5573b41aaf38, undo_rec=0x5573b41ab4e8 "", thr=0x5573b41aad68) at /data/src/10.3/storage/innobase/row/row0purge.cc:1107
            #17 0x00005573b0d85c13 in row_purge_step (thr=0x5573b41aad68) at /data/src/10.3/storage/innobase/row/row0purge.cc:1186
            #18 0x00005573b0d0a62e in que_thr_step (thr=0x5573b41aad68) at /data/src/10.3/storage/innobase/que/que0que.cc:1046
            #19 0x00005573b0d0a861 in que_run_threads_low (thr=0x5573b41aad68) at /data/src/10.3/storage/innobase/que/que0que.cc:1108
            #20 0x00005573b0d0aa52 in que_run_threads (thr=0x5573b41aad68) at /data/src/10.3/storage/innobase/que/que0que.cc:1148
            #21 0x00005573b0dd63b7 in srv_task_execute () at /data/src/10.3/storage/innobase/srv/srv0srv.cc:2437
            #22 0x00005573b0dd655c in srv_worker_thread (arg=0x0) at /data/src/10.3/storage/innobase/srv/srv0srv.cc:2485
            #23 0x00007f16a330c494 in start_thread (arg=0x7f16867fc700) at pthread_create.c:333
            #24 0x00007f16a16f293f in clone () from /lib/x86_64-linux-gnu/libc.so.6
            

            10.3 release 46857a860c

            #2  <signal handler called>
            #3  innobase_get_computed_value (row=row@entry=0x7f57d800b970, col=col@entry=0x7f57bc05a5a0, index=index@entry=0x7f57bc0528e0, local_heap=local_heap@entry=0x7f57dfffd838, heap=heap@entry=0x7f57d800b8c0, ifield=ifield@entry=0x0, thd=0x7f57d80009a8, mysql_table=0x0, old_table=0x0, parent_update=0x0, foreign=0x0) at /data/src/10.3/storage/innobase/handler/ha_innodb.cc:20769
            #4  0x000056192282d1a6 in row_vers_build_clust_v_col (row=row@entry=0x7f57d800b970, clust_index=clust_index@entry=0x7f57bc0528e0, index=index@entry=0x7f57bc053dd0, heap=0x7f57d800b8c0) at /data/src/10.3/storage/innobase/row/row0vers.cc:450
            #5  0x000056192282fb73 in row_vers_old_has_index_entry (also_curr=also_curr@entry=1, rec=0x7f5808dc807e <error: Cannot access memory at address 0x7f5808dc807e>, mtr=mtr@entry=0x7f57dfffd980, index=index@entry=0x7f57bc053dd0, ientry=ientry@entry=0x7f57d800b4b0, roll_ptr=2533274815628030, trx_id=0) at /data/src/10.3/storage/innobase/row/row0vers.cc:896
            #6  0x000056192280a0e0 in row_purge_poss_sec (node=node@entry=0x5619243fe660, index=index@entry=0x7f57bc053dd0, entry=entry@entry=0x7f57d800b4b0) at /data/src/10.3/storage/innobase/row/row0purge.cc:265
            #7  0x000056192280a466 in row_purge_remove_sec_if_poss_leaf (node=node@entry=0x5619243fe660, index=index@entry=0x7f57bc053dd0, entry=entry@entry=0x7f57d800b4b0) at /data/src/10.3/storage/innobase/row/row0purge.cc:477
            #8  0x000056192280b38f in row_purge_remove_sec_if_poss (entry=0x7f57d800b4b0, index=0x7f57bc053dd0, node=0x5619243fe660) at /data/src/10.3/storage/innobase/row/row0purge.cc:584
            #9  row_purge_del_mark (node=0x5619243fe660) at /data/src/10.3/storage/innobase/row/row0purge.cc:654
            #10 row_purge_record_func (node=0x5619243fe660, undo_rec=0x5619243febd0 "", updated_extern=<optimized out>) at /data/src/10.3/storage/innobase/row/row0purge.cc:1042
            #11 0x000056192280c71e in row_purge (thr=<optimized out>, undo_rec=<optimized out>, node=<optimized out>) at /data/src/10.3/storage/innobase/row/row0purge.cc:1107
            #12 row_purge_step (thr=0x7f57d80009a8, thr@entry=0x5619243fe498) at /data/src/10.3/storage/innobase/row/row0purge.cc:1186
            #13 0x00005619227cd7da in que_thr_step (thr=0x5619243fe498) at /data/src/10.3/storage/innobase/que/que0que.cc:1046
            #14 que_run_threads_low (thr=0x5619243fe498) at /data/src/10.3/storage/innobase/que/que0que.cc:1108
            #15 que_run_threads (thr=thr@entry=0x5619243fe498) at /data/src/10.3/storage/innobase/que/que0que.cc:1148
            #16 0x0000561922837c2f in srv_task_execute () at /data/src/10.3/storage/innobase/srv/srv0srv.cc:2437
            #17 srv_worker_thread (arg=<optimized out>) at /data/src/10.3/storage/innobase/srv/srv0srv.cc:2485
            #18 0x00007f5811088494 in start_thread (arg=0x7f57dffff700) at pthread_create.c:333
            #19 0x00007f580f46e93f in clone () from /lib/x86_64-linux-gnu/libc.so.6
            

            bb-10.3-release debug fc6fe26dc

            #3  <signal handler called>
            #4  0x00005621626c68f8 in innobase_allocate_row_for_vcol (thd=0x7f2e1c001d50, index=0x7f2ddc04fca8, heap=0x7f2e20ff7970, table=0x7f2e20ff7980, record=0x7f2e20ff7988, storage=0x7f2e20ff7978) at /data/src/bb-10.3-release/storage/innobase/handler/ha_innodb.cc:20822
            #5  0x0000562162870708 in row_vers_build_clust_v_col (row=0x7f2e1c00eb08, clust_index=0x7f2ddc099238, index=0x7f2ddc04fca8, heap=0x7f2e1c00ea30) at /data/src/bb-10.3-release/storage/innobase/row/row0vers.cc:448
            #6  0x00005621628717a8 in row_vers_old_has_index_entry (also_curr=1, rec=0x7f2e3549807e "\200", mtr=0x7f2e20ff7ac0, index=0x7f2ddc04fca8, ientry=0x7f2e1c00e548, roll_ptr=3659174718407629, trx_id=0) at /data/src/bb-10.3-release/storage/innobase/row/row0vers.cc:912
            #7  0x0000562162826c7c in row_purge_poss_sec (node=0x5621647b9f58, index=0x7f2ddc04fca8, entry=0x7f2e1c00e548) at /data/src/bb-10.3-release/storage/innobase/row/row0purge.cc:265
            #8  0x000056216282749f in row_purge_remove_sec_if_poss_leaf (node=0x5621647b9f58, index=0x7f2ddc04fca8, entry=0x7f2e1c00e548) at /data/src/bb-10.3-release/storage/innobase/row/row0purge.cc:477
            #9  0x0000562162827958 in row_purge_remove_sec_if_poss (node=0x5621647b9f58, index=0x7f2ddc04fca8, entry=0x7f2e1c00e548) at /data/src/bb-10.3-release/storage/innobase/row/row0purge.cc:584
            #10 0x0000562162827b53 in row_purge_del_mark (node=0x5621647b9f58) at /data/src/bb-10.3-release/storage/innobase/row/row0purge.cc:654
            #11 0x0000562162828fb3 in row_purge_record_func (node=0x5621647b9f58, undo_rec=0x5621647ba508 "", thr=0x5621647b9d88, updated_extern=false) at /data/src/bb-10.3-release/storage/innobase/row/row0purge.cc:1042
            #12 0x00005621628292d0 in row_purge (node=0x5621647b9f58, undo_rec=0x5621647ba508 "", thr=0x5621647b9d88) at /data/src/bb-10.3-release/storage/innobase/row/row0purge.cc:1107
            #13 0x00005621628295d1 in row_purge_step (thr=0x5621647b9d88) at /data/src/bb-10.3-release/storage/innobase/row/row0purge.cc:1186
            #14 0x00005621627ade78 in que_thr_step (thr=0x5621647b9d88) at /data/src/bb-10.3-release/storage/innobase/que/que0que.cc:1046
            #15 0x00005621627ae0ab in que_run_threads_low (thr=0x5621647b9d88) at /data/src/bb-10.3-release/storage/innobase/que/que0que.cc:1108
            #16 0x00005621627ae29c in que_run_threads (thr=0x5621647b9d88) at /data/src/bb-10.3-release/storage/innobase/que/que0que.cc:1148
            #17 0x0000562162879e17 in srv_task_execute () at /data/src/bb-10.3-release/storage/innobase/srv/srv0srv.cc:2437
            #18 0x0000562162879fbc in srv_worker_thread (arg=0x0) at /data/src/bb-10.3-release/storage/innobase/srv/srv0srv.cc:2485
            #19 0x00007f2e3d636494 in start_thread (arg=0x7f2e20ff9700) at pthread_create.c:333
            #20 0x00007f2e3ba1c93f in clone () from /lib/x86_64-linux-gnu/libc.so.6
            

            bb-10.3-release ASAN fc6fe26dc

            ASAN:SIGSEGV
            =================================================================
            ==8869==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x565002951472 sp 0x7efc40d803c0 bp 0x7efc40d80410 T19)
                #0 0x565002951471 in innobase_allocate_row_for_vcol(THD*, dict_index_t*, mem_block_info_t**, TABLE**, unsigned char**, VCOL_STORAGE**) /data/src/bb-10.3-release/storage/innobase/handler/ha_innodb.cc:20822
                #1 0x565002c72bd3 in row_vers_build_clust_v_col /data/src/bb-10.3-release/storage/innobase/row/row0vers.cc:448
                #2 0x565002c74b84 in row_vers_old_has_index_entry(unsigned long, unsigned char const*, mtr_t*, dict_index_t*, dtuple_t const*, unsigned long, unsigned long) /data/src/bb-10.3-release/storage/innobase/row/row0vers.cc:912
                #3 0x565002bebf59 in row_purge_poss_sec(purge_node_t*, dict_index_t*, dtuple_t const*) /data/src/bb-10.3-release/storage/innobase/row/row0purge.cc:265
                #4 0x565002becae1 in row_purge_remove_sec_if_poss_leaf /data/src/bb-10.3-release/storage/innobase/row/row0purge.cc:477
                #5 0x565002bed0c5 in row_purge_remove_sec_if_poss /data/src/bb-10.3-release/storage/innobase/row/row0purge.cc:584
                #6 0x565002bed4b6 in row_purge_del_mark /data/src/bb-10.3-release/storage/innobase/row/row0purge.cc:654
                #7 0x565002befd12 in row_purge_record_func /data/src/bb-10.3-release/storage/innobase/row/row0purge.cc:1042
                #8 0x565002bf0341 in row_purge /data/src/bb-10.3-release/storage/innobase/row/row0purge.cc:1107
                #9 0x565002bf0a0f in row_purge_step(que_thr_t*) /data/src/bb-10.3-release/storage/innobase/row/row0purge.cc:1186
                #10 0x565002b09369 in que_thr_step /data/src/bb-10.3-release/storage/innobase/que/que0que.cc:1046
                #11 0x565002b0978c in que_run_threads_low /data/src/bb-10.3-release/storage/innobase/que/que0que.cc:1108
                #12 0x565002b09ae3 in que_run_threads(que_thr_t*) /data/src/bb-10.3-release/storage/innobase/que/que0que.cc:1148
                #13 0x565002c82f4c in srv_task_execute /data/src/bb-10.3-release/storage/innobase/srv/srv0srv.cc:2437
                #14 0x565002c83155 in srv_worker_thread /data/src/bb-10.3-release/storage/innobase/srv/srv0srv.cc:2485
                #15 0x7efc572ae493 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x7493)
                #16 0x7efc5569493e in __clone (/lib/x86_64-linux-gnu/libc.so.6+0xe893e)
             
            AddressSanitizer can not provide additional info.
            SUMMARY: AddressSanitizer: SEGV /data/src/bb-10.3-release/storage/innobase/handler/ha_innodb.cc:20822 innobase_allocate_row_for_vcol(THD*, dict_index_t*, mem_block_info_t**, TABLE**, unsigned char**, VCOL_STORAGE**)
            Thread T19 created by T0 here:
                #0 0x7efc574e7bba in pthread_create (/usr/lib/x86_64-linux-gnu/libasan.so.1+0x23bba)
                #1 0x565002ab0994 in os_thread_create_func(void* (*)(void*), void*, unsigned long*) /data/src/bb-10.3-release/storage/innobase/os/os0thread.cc:137
                #2 0x565002c9263b in srv_start(bool) /data/src/bb-10.3-release/storage/innobase/srv/srv0start.cc:2460
                #3 0x56500290804c in innodb_init /data/src/bb-10.3-release/storage/innobase/handler/ha_innodb.cc:4271
                #4 0x565002150a29 in ha_initialize_handlerton(st_plugin_int*) /data/src/bb-10.3-release/sql/handler.cc:522
                #5 0x565001a8f8b0 in plugin_initialize /data/src/bb-10.3-release/sql/sql_plugin.cc:1432
                #6 0x565001a911d0 in plugin_init(int*, char**, int) /data/src/bb-10.3-release/sql/sql_plugin.cc:1715
                #7 0x5650017cad56 in init_server_components /data/src/bb-10.3-release/sql/mysqld.cc:5377
                #8 0x5650017ccee2 in mysqld_main(int, char**) /data/src/bb-10.3-release/sql/mysqld.cc:5982
                #9 0x5650017b646f in main /data/src/bb-10.3-release/sql/main.cc:25
                #10 0x7efc555cc2b0 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x202b0)
             
            ==8869==ABORTING
            

            elenst Elena Stepanova added a comment - I'm removing the note about non-debug builds being unaffected, because the test case below causes different failures on all of current 10.3 debug/release and bb-10.3-release (tentative 10.3.8 release). To be run with --mysqld=--lock-wait-timeout=5 --mem --repeat=100 --source include/have_innodb.inc   CREATE TABLE t1 ( pk INT NOT NULL , a INT , b CHAR (8) NULL , v CHAR (8) AS (b) VIRTUAL, PRIMARY KEY (pk), UNIQUE KEY (pk), INDEX (a,v) ) ENGINE=InnoDB; INSERT INTO t1 (pk,a,b) VALUES (1,10, 'foo' );   REPLACE INTO t1 (pk,a,b) SELECT pk,a,b FROM t1; ALTER TABLE t1 RENAME TO t2;   # Cleanup DROP TABLE t2; 10.3 debug 46857a860c mysqld: /data/src/10.3/storage/innobase/handler/ha_innodb.cc:20767: dfield_t* innobase_get_computed_value(const dtuple_t*, const dict_v_col_t*, const dict_index_t*, mem_heap_t**, mem_heap_t*, const dict_field_t*, THD*, TABLE*, const dict_table_t*, upd_t*, dict_foreign_t*): Assertion `mysql_table' failed. 180701 20:44:24 [ERROR] mysqld got signal 6 ;   #7 0x00007f16a1635ee2 in __assert_fail () from /lib/x86_64-linux-gnu/libc.so.6 #8 0x00005573b0c2375d in innobase_get_computed_value (row=0x7f1664015bd8, col=0x7f16480a3c08, index=0x7f16480904d8, local_heap=0x7f16867fa990, heap=0x7f1664015b00, ifield=0x0, thd=0x7f1664000b00, mysql_table=0x0, old_table=0x0, parent_update=0x0, foreign=0x0) at /data/src/10.3/storage/innobase/handler/ha_innodb.cc:20767 #9 0x00005573b0dccd34 in row_vers_build_clust_v_col (row=0x7f1664015bd8, clust_index=0x7f16480904d8, index=0x7f164804f888, heap=0x7f1664015b00) at /data/src/10.3/storage/innobase/row/row0vers.cc:450 #10 0x00005573b0dcdd47 in row_vers_old_has_index_entry (also_curr=1, rec=0x7f16970fc07e "\200", mtr=0x7f16867faac0, index=0x7f164804f888, ientry=0x7f16640156f8, roll_ptr=3659174718407633, trx_id=0) at /data/src/10.3/storage/innobase/row/row0vers.cc:896 #11 0x00005573b0d832be in row_purge_poss_sec (node=0x5573b41aaf38, index=0x7f164804f888, entry=0x7f16640156f8) at /data/src/10.3/storage/innobase/row/row0purge.cc:265 #12 0x00005573b0d83ae1 in row_purge_remove_sec_if_poss_leaf (node=0x5573b41aaf38, index=0x7f164804f888, entry=0x7f16640156f8) at /data/src/10.3/storage/innobase/row/row0purge.cc:477 #13 0x00005573b0d83f9a in row_purge_remove_sec_if_poss (node=0x5573b41aaf38, index=0x7f164804f888, entry=0x7f16640156f8) at /data/src/10.3/storage/innobase/row/row0purge.cc:584 #14 0x00005573b0d84195 in row_purge_del_mark (node=0x5573b41aaf38) at /data/src/10.3/storage/innobase/row/row0purge.cc:654 #15 0x00005573b0d855f5 in row_purge_record_func (node=0x5573b41aaf38, undo_rec=0x5573b41ab4e8 "", thr=0x5573b41aad68, updated_extern=false) at /data/src/10.3/storage/innobase/row/row0purge.cc:1042 #16 0x00005573b0d85912 in row_purge (node=0x5573b41aaf38, undo_rec=0x5573b41ab4e8 "", thr=0x5573b41aad68) at /data/src/10.3/storage/innobase/row/row0purge.cc:1107 #17 0x00005573b0d85c13 in row_purge_step (thr=0x5573b41aad68) at /data/src/10.3/storage/innobase/row/row0purge.cc:1186 #18 0x00005573b0d0a62e in que_thr_step (thr=0x5573b41aad68) at /data/src/10.3/storage/innobase/que/que0que.cc:1046 #19 0x00005573b0d0a861 in que_run_threads_low (thr=0x5573b41aad68) at /data/src/10.3/storage/innobase/que/que0que.cc:1108 #20 0x00005573b0d0aa52 in que_run_threads (thr=0x5573b41aad68) at /data/src/10.3/storage/innobase/que/que0que.cc:1148 #21 0x00005573b0dd63b7 in srv_task_execute () at /data/src/10.3/storage/innobase/srv/srv0srv.cc:2437 #22 0x00005573b0dd655c in srv_worker_thread (arg=0x0) at /data/src/10.3/storage/innobase/srv/srv0srv.cc:2485 #23 0x00007f16a330c494 in start_thread (arg=0x7f16867fc700) at pthread_create.c:333 #24 0x00007f16a16f293f in clone () from /lib/x86_64-linux-gnu/libc.so.6 10.3 release 46857a860c #2 <signal handler called> #3 innobase_get_computed_value (row=row@entry=0x7f57d800b970, col=col@entry=0x7f57bc05a5a0, index=index@entry=0x7f57bc0528e0, local_heap=local_heap@entry=0x7f57dfffd838, heap=heap@entry=0x7f57d800b8c0, ifield=ifield@entry=0x0, thd=0x7f57d80009a8, mysql_table=0x0, old_table=0x0, parent_update=0x0, foreign=0x0) at /data/src/10.3/storage/innobase/handler/ha_innodb.cc:20769 #4 0x000056192282d1a6 in row_vers_build_clust_v_col (row=row@entry=0x7f57d800b970, clust_index=clust_index@entry=0x7f57bc0528e0, index=index@entry=0x7f57bc053dd0, heap=0x7f57d800b8c0) at /data/src/10.3/storage/innobase/row/row0vers.cc:450 #5 0x000056192282fb73 in row_vers_old_has_index_entry (also_curr=also_curr@entry=1, rec=0x7f5808dc807e <error: Cannot access memory at address 0x7f5808dc807e>, mtr=mtr@entry=0x7f57dfffd980, index=index@entry=0x7f57bc053dd0, ientry=ientry@entry=0x7f57d800b4b0, roll_ptr=2533274815628030, trx_id=0) at /data/src/10.3/storage/innobase/row/row0vers.cc:896 #6 0x000056192280a0e0 in row_purge_poss_sec (node=node@entry=0x5619243fe660, index=index@entry=0x7f57bc053dd0, entry=entry@entry=0x7f57d800b4b0) at /data/src/10.3/storage/innobase/row/row0purge.cc:265 #7 0x000056192280a466 in row_purge_remove_sec_if_poss_leaf (node=node@entry=0x5619243fe660, index=index@entry=0x7f57bc053dd0, entry=entry@entry=0x7f57d800b4b0) at /data/src/10.3/storage/innobase/row/row0purge.cc:477 #8 0x000056192280b38f in row_purge_remove_sec_if_poss (entry=0x7f57d800b4b0, index=0x7f57bc053dd0, node=0x5619243fe660) at /data/src/10.3/storage/innobase/row/row0purge.cc:584 #9 row_purge_del_mark (node=0x5619243fe660) at /data/src/10.3/storage/innobase/row/row0purge.cc:654 #10 row_purge_record_func (node=0x5619243fe660, undo_rec=0x5619243febd0 "", updated_extern=<optimized out>) at /data/src/10.3/storage/innobase/row/row0purge.cc:1042 #11 0x000056192280c71e in row_purge (thr=<optimized out>, undo_rec=<optimized out>, node=<optimized out>) at /data/src/10.3/storage/innobase/row/row0purge.cc:1107 #12 row_purge_step (thr=0x7f57d80009a8, thr@entry=0x5619243fe498) at /data/src/10.3/storage/innobase/row/row0purge.cc:1186 #13 0x00005619227cd7da in que_thr_step (thr=0x5619243fe498) at /data/src/10.3/storage/innobase/que/que0que.cc:1046 #14 que_run_threads_low (thr=0x5619243fe498) at /data/src/10.3/storage/innobase/que/que0que.cc:1108 #15 que_run_threads (thr=thr@entry=0x5619243fe498) at /data/src/10.3/storage/innobase/que/que0que.cc:1148 #16 0x0000561922837c2f in srv_task_execute () at /data/src/10.3/storage/innobase/srv/srv0srv.cc:2437 #17 srv_worker_thread (arg=<optimized out>) at /data/src/10.3/storage/innobase/srv/srv0srv.cc:2485 #18 0x00007f5811088494 in start_thread (arg=0x7f57dffff700) at pthread_create.c:333 #19 0x00007f580f46e93f in clone () from /lib/x86_64-linux-gnu/libc.so.6 bb-10.3-release debug fc6fe26dc #3 <signal handler called> #4 0x00005621626c68f8 in innobase_allocate_row_for_vcol (thd=0x7f2e1c001d50, index=0x7f2ddc04fca8, heap=0x7f2e20ff7970, table=0x7f2e20ff7980, record=0x7f2e20ff7988, storage=0x7f2e20ff7978) at /data/src/bb-10.3-release/storage/innobase/handler/ha_innodb.cc:20822 #5 0x0000562162870708 in row_vers_build_clust_v_col (row=0x7f2e1c00eb08, clust_index=0x7f2ddc099238, index=0x7f2ddc04fca8, heap=0x7f2e1c00ea30) at /data/src/bb-10.3-release/storage/innobase/row/row0vers.cc:448 #6 0x00005621628717a8 in row_vers_old_has_index_entry (also_curr=1, rec=0x7f2e3549807e "\200", mtr=0x7f2e20ff7ac0, index=0x7f2ddc04fca8, ientry=0x7f2e1c00e548, roll_ptr=3659174718407629, trx_id=0) at /data/src/bb-10.3-release/storage/innobase/row/row0vers.cc:912 #7 0x0000562162826c7c in row_purge_poss_sec (node=0x5621647b9f58, index=0x7f2ddc04fca8, entry=0x7f2e1c00e548) at /data/src/bb-10.3-release/storage/innobase/row/row0purge.cc:265 #8 0x000056216282749f in row_purge_remove_sec_if_poss_leaf (node=0x5621647b9f58, index=0x7f2ddc04fca8, entry=0x7f2e1c00e548) at /data/src/bb-10.3-release/storage/innobase/row/row0purge.cc:477 #9 0x0000562162827958 in row_purge_remove_sec_if_poss (node=0x5621647b9f58, index=0x7f2ddc04fca8, entry=0x7f2e1c00e548) at /data/src/bb-10.3-release/storage/innobase/row/row0purge.cc:584 #10 0x0000562162827b53 in row_purge_del_mark (node=0x5621647b9f58) at /data/src/bb-10.3-release/storage/innobase/row/row0purge.cc:654 #11 0x0000562162828fb3 in row_purge_record_func (node=0x5621647b9f58, undo_rec=0x5621647ba508 "", thr=0x5621647b9d88, updated_extern=false) at /data/src/bb-10.3-release/storage/innobase/row/row0purge.cc:1042 #12 0x00005621628292d0 in row_purge (node=0x5621647b9f58, undo_rec=0x5621647ba508 "", thr=0x5621647b9d88) at /data/src/bb-10.3-release/storage/innobase/row/row0purge.cc:1107 #13 0x00005621628295d1 in row_purge_step (thr=0x5621647b9d88) at /data/src/bb-10.3-release/storage/innobase/row/row0purge.cc:1186 #14 0x00005621627ade78 in que_thr_step (thr=0x5621647b9d88) at /data/src/bb-10.3-release/storage/innobase/que/que0que.cc:1046 #15 0x00005621627ae0ab in que_run_threads_low (thr=0x5621647b9d88) at /data/src/bb-10.3-release/storage/innobase/que/que0que.cc:1108 #16 0x00005621627ae29c in que_run_threads (thr=0x5621647b9d88) at /data/src/bb-10.3-release/storage/innobase/que/que0que.cc:1148 #17 0x0000562162879e17 in srv_task_execute () at /data/src/bb-10.3-release/storage/innobase/srv/srv0srv.cc:2437 #18 0x0000562162879fbc in srv_worker_thread (arg=0x0) at /data/src/bb-10.3-release/storage/innobase/srv/srv0srv.cc:2485 #19 0x00007f2e3d636494 in start_thread (arg=0x7f2e20ff9700) at pthread_create.c:333 #20 0x00007f2e3ba1c93f in clone () from /lib/x86_64-linux-gnu/libc.so.6 bb-10.3-release ASAN fc6fe26dc ASAN:SIGSEGV ================================================================= ==8869==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x565002951472 sp 0x7efc40d803c0 bp 0x7efc40d80410 T19) #0 0x565002951471 in innobase_allocate_row_for_vcol(THD*, dict_index_t*, mem_block_info_t**, TABLE**, unsigned char**, VCOL_STORAGE**) /data/src/bb-10.3-release/storage/innobase/handler/ha_innodb.cc:20822 #1 0x565002c72bd3 in row_vers_build_clust_v_col /data/src/bb-10.3-release/storage/innobase/row/row0vers.cc:448 #2 0x565002c74b84 in row_vers_old_has_index_entry(unsigned long, unsigned char const*, mtr_t*, dict_index_t*, dtuple_t const*, unsigned long, unsigned long) /data/src/bb-10.3-release/storage/innobase/row/row0vers.cc:912 #3 0x565002bebf59 in row_purge_poss_sec(purge_node_t*, dict_index_t*, dtuple_t const*) /data/src/bb-10.3-release/storage/innobase/row/row0purge.cc:265 #4 0x565002becae1 in row_purge_remove_sec_if_poss_leaf /data/src/bb-10.3-release/storage/innobase/row/row0purge.cc:477 #5 0x565002bed0c5 in row_purge_remove_sec_if_poss /data/src/bb-10.3-release/storage/innobase/row/row0purge.cc:584 #6 0x565002bed4b6 in row_purge_del_mark /data/src/bb-10.3-release/storage/innobase/row/row0purge.cc:654 #7 0x565002befd12 in row_purge_record_func /data/src/bb-10.3-release/storage/innobase/row/row0purge.cc:1042 #8 0x565002bf0341 in row_purge /data/src/bb-10.3-release/storage/innobase/row/row0purge.cc:1107 #9 0x565002bf0a0f in row_purge_step(que_thr_t*) /data/src/bb-10.3-release/storage/innobase/row/row0purge.cc:1186 #10 0x565002b09369 in que_thr_step /data/src/bb-10.3-release/storage/innobase/que/que0que.cc:1046 #11 0x565002b0978c in que_run_threads_low /data/src/bb-10.3-release/storage/innobase/que/que0que.cc:1108 #12 0x565002b09ae3 in que_run_threads(que_thr_t*) /data/src/bb-10.3-release/storage/innobase/que/que0que.cc:1148 #13 0x565002c82f4c in srv_task_execute /data/src/bb-10.3-release/storage/innobase/srv/srv0srv.cc:2437 #14 0x565002c83155 in srv_worker_thread /data/src/bb-10.3-release/storage/innobase/srv/srv0srv.cc:2485 #15 0x7efc572ae493 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x7493) #16 0x7efc5569493e in __clone (/lib/x86_64-linux-gnu/libc.so.6+0xe893e)   AddressSanitizer can not provide additional info. SUMMARY: AddressSanitizer: SEGV /data/src/bb-10.3-release/storage/innobase/handler/ha_innodb.cc:20822 innobase_allocate_row_for_vcol(THD*, dict_index_t*, mem_block_info_t**, TABLE**, unsigned char**, VCOL_STORAGE**) Thread T19 created by T0 here: #0 0x7efc574e7bba in pthread_create (/usr/lib/x86_64-linux-gnu/libasan.so.1+0x23bba) #1 0x565002ab0994 in os_thread_create_func(void* (*)(void*), void*, unsigned long*) /data/src/bb-10.3-release/storage/innobase/os/os0thread.cc:137 #2 0x565002c9263b in srv_start(bool) /data/src/bb-10.3-release/storage/innobase/srv/srv0start.cc:2460 #3 0x56500290804c in innodb_init /data/src/bb-10.3-release/storage/innobase/handler/ha_innodb.cc:4271 #4 0x565002150a29 in ha_initialize_handlerton(st_plugin_int*) /data/src/bb-10.3-release/sql/handler.cc:522 #5 0x565001a8f8b0 in plugin_initialize /data/src/bb-10.3-release/sql/sql_plugin.cc:1432 #6 0x565001a911d0 in plugin_init(int*, char**, int) /data/src/bb-10.3-release/sql/sql_plugin.cc:1715 #7 0x5650017cad56 in init_server_components /data/src/bb-10.3-release/sql/mysqld.cc:5377 #8 0x5650017ccee2 in mysqld_main(int, char**) /data/src/bb-10.3-release/sql/mysqld.cc:5982 #9 0x5650017b646f in main /data/src/bb-10.3-release/sql/main.cc:25 #10 0x7efc555cc2b0 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x202b0)   ==8869==ABORTING

            I ended up changing the patch in my review, so it should be considered a joint work between me and thiru. Local ASAN testing found two memory leaks, which were fixed. Also the memory allocation was simplified. There was one hang in buildbot during gcol.innodb_virtual_basic, but I was not able to repeat it (instead, repeatedly got 3 different BLOB-related crashes when reading a record). Without having core dumps from buildbot, I cannot tell if the hang could be a duplicate of MDEV-14637.

            Once the tests have passed on bb-10.2-marko and bb-10.3-marko, I will push to 10.2 and 10.3.

            marko Marko Mäkelä added a comment - I ended up changing the patch in my review, so it should be considered a joint work between me and thiru . Local ASAN testing found two memory leaks, which were fixed. Also the memory allocation was simplified. There was one hang in buildbot during gcol.innodb_virtual_basic, but I was not able to repeat it (instead, repeatedly got 3 different BLOB-related crashes when reading a record). Without having core dumps from buildbot, I cannot tell if the hang could be a duplicate of MDEV-14637 . Once the tests have passed on bb-10.2-marko and bb-10.3-marko, I will push to 10.2 and 10.3.

            People

              marko Marko Mäkelä
              elenst Elena Stepanova
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.