Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-24202

Assertion `dest_size <= sizeof(ulint)' failed in mach0data.ic:691

    XMLWordPrintable

Details

    Description

      Workflow:
      1. Start the server
      2. One session generates some initial data
      3. One session runs some DDL/DML mix.
      4. During 3. is ongoing the DB server process gets killed intentionally
      5. Restart attempt
      6. Various SQL which tries to find defects
      Within 6.  I harvest
      mysqld: /Server/bb-10.6-MDEV-23497/storage/innobase/include/mach0data.ic:691: void mach_write_to_n_little_endian(byte*, ulint, ulint): Assertion `dest_size <= sizeof(ulint)' failed.
      (rr) bt
      #0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
      #1  0x00005aff78c8a859 in __GI_abort () at abort.c:79
      #2  0x00005aff78c8a729 in __assert_fail_base (fmt=0x5aff78e20588 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=0x55ea195f4980 "dest_size <= sizeof(ulint)", file=0x55ea195f1220 "/Server/bb-10.6-MDEV-23497/storage/innobase/include/mach0data.ic", 
          line=691, function=<optimized out>) at assert.c:92
      #3  0x00005aff78c9bf36 in __GI___assert_fail (assertion=assertion@entry=0x55ea195f4980 "dest_size <= sizeof(ulint)", file=file@entry=0x55ea195f1220 "/Server/bb-10.6-MDEV-23497/storage/innobase/include/mach0data.ic", line=line@entry=691, 
          function=function@entry=0x55ea195f49c0 "void mach_write_to_n_little_endian(byte*, ulint, ulint)") at assert.c:101
      #4  0x000055ea18404318 in mach_write_to_n_little_endian (n=32769, dest_size=18446744073709551612, dest=0x61a000319ebd "") at /Server/bb-10.6-MDEV-23497/storage/innobase/include/mach0data.ic:691
      #5  row_mysql_store_blob_ref (dest=dest@entry=0x61a000319ebd "", col_len=4, data=data@entry=0x6310045248a0, len=len@entry=32769) at /Server/bb-10.6-MDEV-23497/storage/innobase/row/row0mysql.cc:251
      #6  0x000055ea184a833d in row_sel_field_store_in_mysql_format_func (dest=0x61a000319ebd "", templ=templ@entry=0x618000046528, index=index@entry=0x6170000203a0, field_no=field_no@entry=7, data=0x6310045248a0 '3' <repeats 200 times>..., len=32769)
          at /Server/bb-10.6-MDEV-23497/storage/innobase/row/row0sel.cc:2746
      #7  0x000055ea184b0f64 in row_sel_store_mysql_field (mysql_rec=mysql_rec@entry=0x61a000319eb8 "\377\002", prebuilt=prebuilt@entry=0x6210001041a0, rec=rec@entry=0x240c7a5900ad "\200", index=index@entry=0x6170000203a0, 
          offsets=offsets@entry=0x73dc10ff3f40, field_no=<optimized out>, templ=<optimized out>) at /Server/bb-10.6-MDEV-23497/storage/innobase/row/row0sel.cc:2908
      #8  0x000055ea184b2b79 in row_sel_store_mysql_rec (mysql_rec=mysql_rec@entry=0x61a000319eb8 "\377\002", prebuilt=prebuilt@entry=0x6210001041a0, rec=<optimized out>, vrow=<optimized out>, rec_clust=false, index=0x6170000203a0, offsets=<optimized out>)
          at /Server/bb-10.6-MDEV-23497/storage/innobase/row/row0sel.cc:3120
      #9  0x000055ea184c9d6b in row_search_mvcc (buf=buf@entry=0x61a000319eb8 "\377\002", mode=<optimized out>, prebuilt=<optimized out>, match_mode=<optimized out>, direction=direction@entry=0)
          at /Server/bb-10.6-MDEV-23497/storage/innobase/row/row0sel.cc:5489
      #10 0x000055ea17fe8f3c in ha_innobase::index_read (this=this@entry=0x61d0001af4b8, buf=buf@entry=0x61a000319eb8 "\377\002", key_ptr=key_ptr@entry=0x0, key_len=key_len@entry=0, find_flag=find_flag@entry=HA_READ_AFTER_KEY)
          at /Server/bb-10.6-MDEV-23497/storage/innobase/handler/ha_innodb.cc:8537
      #11 0x000055ea17fe96de in ha_innobase::index_first (this=this@entry=0x61d0001af4b8, buf=buf@entry=0x61a000319eb8 "\377\002") at /Server/bb-10.6-MDEV-23497/storage/innobase/handler/ha_innodb.cc:8899
      #12 0x000055ea17fe98a2 in ha_innobase::rnd_next (this=0x61d0001af4b8, buf=0x61a000319eb8 "\377\002") at /Server/bb-10.6-MDEV-23497/storage/innobase/handler/ha_innodb.cc:8992
      #13 0x000055ea1734ffcd in handler::ha_rnd_next (this=0x61d0001af4b8, buf=0x61a000319eb8 "\377\002") at /Server/bb-10.6-MDEV-23497/sql/handler.cc:3066
      #14 0x000055ea17970a01 in rr_sequential (info=0x62b0000c8f68) at /Server/bb-10.6-MDEV-23497/sql/records.h:82
      #15 0x000055ea16bbf429 in READ_RECORD::read_record (this=0x62b0000c8f68) at /Server/bb-10.6-MDEV-23497/sql/records.h:81
      #16 join_init_read_record (tab=0x62b0000c8ea0) at /Server/bb-10.6-MDEV-23497/sql/sql_select.cc:21562
      #17 0x000055ea16b6bfbb in sub_select (join=0x62b0000c6610, join_tab=0x62b0000c8ea0, end_of_records=<optimized out>) at /Server/bb-10.6-MDEV-23497/sql/sql_select.cc:20584
      #18 0x000055ea16bfe79e in do_select (join=join@entry=0x62b0000c6610, procedure=<optimized out>) at /Server/bb-10.6-MDEV-23497/sql/sql_select.cc:20134
      #19 0x000055ea16c2ba22 in JOIN::exec_inner (this=this@entry=0x62b0000c6610) at /Server/bb-10.6-MDEV-23497/sql/sql_select.cc:4469
      #20 0x000055ea16c2be85 in JOIN::exec (this=this@entry=0x62b0000c6610) at /Server/bb-10.6-MDEV-23497/sql/sql_select.cc:4250
      #21 0x000055ea16c2661e in mysql_select (thd=thd@entry=0x62b0000bd218, tables=<optimized out>, fields=..., conds=<optimized out>, og_num=<optimized out>, order=<optimized out>, group=<optimized out>, having=<optimized out>, proc_param=<optimized out>, 
          select_options=<optimized out>, result=<optimized out>, unit=<optimized out>, select_lex=<optimized out>) at /Server/bb-10.6-MDEV-23497/sql/sql_select.cc:4665
      #22 0x000055ea16c27099 in handle_select (thd=thd@entry=0x62b0000bd218, lex=lex@entry=0x62b0000c12e0, result=result@entry=0x62b0000c65e0, setup_tables_done_option=setup_tables_done_option@entry=0) at /Server/bb-10.6-MDEV-23497/sql/sql_select.cc:417
      #23 0x000055ea16a5c18e in execute_sqlcom_select (thd=thd@entry=0x62b0000bd218, all_tables=<optimized out>) at /Server/bb-10.6-MDEV-23497/sql/sql_parse.cc:6118
      #24 0x000055ea16a86a89 in mysql_execute_command (thd=thd@entry=0x62b0000bd218) at /Server/bb-10.6-MDEV-23497/sql/sql_parse.cc:3820
      #25 0x000055ea16a42963 in mysql_parse (thd=thd@entry=0x62b0000bd218, rawbuf=<optimized out>, length=<optimized out>, parser_state=parser_state@entry=0x73dc10ff6ed0) at /Server/bb-10.6-MDEV-23497/sql/sql_parse.cc:7883
      #26 0x000055ea16a73fe5 in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x62b0000bd218, packet=packet@entry=0x6290000eb219 "", packet_length=packet_length@entry=141) at /Server/bb-10.6-MDEV-23497/sql/sql_class.h:1251
      #27 0x000055ea16a7e92a in do_command (thd=0x62b0000bd218) at /Server/bb-10.6-MDEV-23497/sql/sql_parse.cc:1348
      #28 0x000055ea16f18292 in do_handle_one_connection (connect=<optimized out>, connect@entry=0x6080004b5438, put_in_cache=put_in_cache@entry=true) at /Server/bb-10.6-MDEV-23497/sql/sql_connect.cc:1410
      #29 0x000055ea16f194d2 in handle_one_connection (arg=arg@entry=0x6080004b5438) at /Server/bb-10.6-MDEV-23497/sql/sql_connect.cc:1312
      #30 0x000055ea17da5cba in pfs_spawn_thread (arg=0x61500000f818) at /Server/bb-10.6-MDEV-23497/storage/perfschema/pfs.cc:2201
      #31 0x000038ad68fb0609 in start_thread (arg=<optimized out>) at pthread_create.c:477
      #32 0x00005aff78d87293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
      (rr)
       
      for 
      SELECT * FROM `test`.`t7` FORCE INDEX (`Marvão_uidx1`) WHERE `col_int` >= -9223372036854775808 OR `col_int` IS NULL OR `col_int` IS NOT NULL
       
      origin/bb-10.6-MDEV-23497 5b2b9e8b8a36259c4aeabb5ef1fadd52d7ae5b26 2020-11-10
      complied with debug, asan and Og
       
      sdp:/RQG/storage/1605036797/tmp/dev/shm/vardir/1605036797/121/1/rr
      _RR_TRACE_DIR="." rr replay --mark-stdio
      For some first analysis of the rr trace please have a look at https://jira.mariadb.org/browse/MDEV-24156
       
      I am aware that the crash recovery is sometimes unable to heal inconsistencies
      between the InnoDB and the Server Data Dictionary because of technical reasons.
      So for me the main question is:
      Is hitting the assert
      - a bug within server+InnoDB which could be fixed now
      or
      - some DD inconsistency which we cannot handle in the moment
      ?
       
      RQG
      ====
      git clone https://github.com/mleich1/rqg --branch experimental RQG
       
      perl rqg.pl \
      --grammar=conf/mariadb/table_stress_innodb_nocopy1.yy \
      --gendata=conf/mariadb/table_stress.zz \
      --gendata_sql=conf/mariadb/table_stress.sql \
      --reporters=CrashRecovery1 \
      --mysqld=--innodb_use_native_aio=1 \
      --mysqld=--innodb_lock_schedule_algorithm=fcfs \
      --mysqld=--loose-idle_write_transaction_timeout=0 \
      --mysqld=--loose-idle_transaction_timeout=0 \
      --mysqld=--loose-idle_readonly_transaction_timeout=0 \
      --mysqld=--connect_timeout=60 \
      --mysqld=--interactive_timeout=28800 \
      --mysqld=--slave_net_timeout=60 \
      --mysqld=--net_read_timeout=30 \
      --mysqld=--net_write_timeout=60 \
      --mysqld=--loose-table_lock_wait_timeout=50 \
      --mysqld=--wait_timeout=28800 \
      --mysqld=--lock-wait-timeout=86400 \
      --mysqld=--innodb-lock-wait-timeout=50 \
      --no-mask \
      --queries=10000000 \
      --seed=random \
      --reporters=Backtrace \
      --reporters=ErrorLog \
      --reporters=Deadlock1 \
      --validators=None \
      --mysqld=--log_output=none \
      --mysqld=--log-bin \
      --mysqld=--log_bin_trust_function_creators=1 \
      --mysqld=--loose-debug_assert_on_not_freed_memory=0 \
      --engine=InnoDB \
      --restart_timeout=120 \
      --redefine=conf/mariadb/redefine_innodb_read_only_compressed.yy \
      --mysqld=--plugin-load-add=file_key_management.so \
      --mysqld=--loose-file-key-management-filename=/home/mleich/RQG/conf/mariadb/encryption_keys.txt \
      --duration=300 \
      --mysqld=--loose-innodb_fatal_semaphore_wait_threshold=300 \
      --mysqld=--loose-innodb-sync-debug \
      --mysqld=--innodb_stats_persistent=on \
      --mysqld=--loose-max-statement-time=30 \
      --threads=1 \
      --mysqld=--innodb_page_size=64K \
      --mysqld=--innodb-buffer-pool-size=256M \
      --duration=300 \
      --no_mask \
      --workdir=<local settings> \
      --vardir=<local settings> \
      --mtr-build-thread=<local settings> \
      --basedir1=<local settings> \
      --script_debug=_nix_ \
      --rr=Extended \
      --rr_options=--chaos
       
      
      

      Attachments

        Activity

          People

            mleich Matthias Leich
            mleich Matthias Leich
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.