[MDEV-29336] mysqld: storage/innobase/include/sux_lock.h:404: void sux_lock<ssux>::u_x_upgrade() [with ssux = ssux_lock_impl<true>]: Assertion `have_u_not_x()' failed. Created: 2022-08-19  Updated: 2022-08-30  Resolved: 2022-08-23

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - InnoDB
Affects Version/s: 10.6
Fix Version/s: 10.6.10, 10.7.6, 10.8.5, 10.9.3, 10.10.2

Type: Bug Priority: Critical
Reporter: Matthias Leich Assignee: Marko Mäkelä
Resolution: Fixed Votes: 0
Labels: rr-profile

Attachments: File MDEV-29336.cc    
Issue Links:
Blocks
blocks MDEV-27983 InnoDB hangs on multiple concurrent r... Closed
Problem/Incident
is caused by MDEV-24142 rw_lock_t has unnecessarily complex w... Closed

 Description    

origin/bb-10.6-MDEV-27700 eed7f33027a7debae2cbff4e33f921a6e1b65e65 2022-08-08T22:13:40+05:30
 
but per Thiru MDEV-27700 only helped to reveal that bug.
 
sdp:/data/results/1660579751/TBR-1581/dev/shm/rqg/1660579751/177/1/rr$ _RR_TRACE_DIR="." rr replay --mark-stdio
 
...
 
[rr 3712636 43306]mysqld: /data/Server/bb-10.6-MDEV-27700/storage/innobase/include/sux_lock.h:404: void sux_lock<ssux>::u_x_upgrade() [with ssux = ssux_lock_impl<true>]: Assertion `have_u_not_x()' failed.
 
...
 
Thread 3 received signal SIGABRT, Aborted.
 
[Switching to Thread 3712636.3714634]
 
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
 
50      ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
 
(rr) bt
 
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
 
#1  0x00007fad21ae0859 in __GI_abort () at abort.c:79
 
#2  0x00007fad21ae0729 in __assert_fail_base (fmt=0x7fad21c76588 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=0x563aaa9e8d80 "have_u_not_x()",     file=0x563aaa9e7ea0 "/data/Server/bb-10.6-MDEV-27700/storage/innobase/include/sux_lock.h", line=404, function=<optimized out>) at assert.c:92
 
#3  0x00007fad21af1f36 in __GI___assert_fail (assertion=0x563aaa9e8d80 "have_u_not_x()", file=0x563aaa9e7ea0 "/data/Server/bb-10.6-MDEV-27700/storage/innobase/include/sux_lock.h", line=404,     function=0x563aaa9e8dc0 "void sux_lock<ssux>::u_x_upgrade() [with ssux = ssux_lock_impl<true>]") at assert.c:101
 
#4  0x0000563aa96dc5ed in sux_lock<ssux_lock_impl<true> >::u_x_upgrade (this=0x7fad157eb568) at /data/Server/bb-10.6-MDEV-27700/storage/innobase/include/sux_lock.h:404
 
#5  0x0000563aa96d0239 in mtr_t::defer_drop_ahi (block=0x7fad157eb550, fix_type=MTR_MEMO_PAGE_SX_FIX) at /data/Server/bb-10.6-MDEV-27700/storage/innobase/mtr/mtr0mtr.cc:1291
 
#6  0x0000563aa9a54239 in buf_page_get_low (page_id=..., zip_size=0, rw_latch=4, guess=0x0, mode=10, mtr=0x7facfdc44380, err=0x7facfdc44220, allow_ibuf_merge=false)    at /data/Server/bb-10.6-MDEV-27700/storage/innobase/buf/buf0buf.cc:2823
 
#7  0x0000563aa9a5503b in buf_page_get_gen (page_id=..., zip_size=0, rw_latch=4, guess=0x0, mode=10, mtr=0x7facfdc44380, err=0x7facfdc44220, allow_ibuf_merge=false)    at /data/Server/bb-10.6-MDEV-27700/storage/innobase/buf/buf0buf.cc:2960
 
#8  0x0000563aa99a4337 in btr_block_get (index=..., page=3, mode=4, merge=false, mtr=0x7facfdc44380, err=0x7facfdc44220) at /data/Server/bb-10.6-MDEV-27700/storage/innobase/btr/btr0btr.cc:228
 
#9  0x0000563aa99a48d2 in btr_root_block_get (index=0x6160022fa808, mode=RW_SX_LATCH, mtr=0x7facfdc44380, err=0x7facfdc44220) at /data/Server/bb-10.6-MDEV-27700/storage/innobase/btr/btr0btr.cc:269
 
#10 0x0000563aa99a5e2e in btr_page_alloc_low (index=0x6160022fa808, hint_page_no=4, file_direction=113 'q', level=0, mtr=0x7facfdc44380, init_mtr=0x7facfdc44380, err=0x7facfdc44220)    at /data/Server/bb-10.6-MDEV-27700/storage/innobase/btr/btr0btr.cc:516
 
#11 0x0000563aa99a5fab in btr_page_alloc (index=0x6160022fa808, hint_page_no=4, file_direction=113 'q', level=0, mtr=0x7facfdc44380, init_mtr=0x7facfdc44380, err=0x7facfdc44220)    at /data/Server/bb-10.6-MDEV-27700/storage/innobase/btr/btr0btr.cc:548
 
#12 0x0000563aa9a15c37 in btr_store_big_rec_extern_fields (pcur=0x6110000b1200, offsets=0x61d000ac6118, big_rec_vec=0x6200002fb108, btr_mtr=0x7facfdc44dc0, op=BTR_STORE_UPDATE)    at /data/Server/bb-10.6-MDEV-27700/storage/innobase/btr/btr0cur.cc:7045
 
#13 0x0000563aa98cbd15 in row_upd_clust_rec (flags=0, node=0x621000342678, index=0x6160022fa808, offsets=0x61d000ac6118, offsets_heap=0x7facfdc44ac0, thr=0x621000342ad8, mtr=0x7facfdc44dc0)    at /data/Server/bb-10.6-MDEV-27700/storage/innobase/row/row0upd.cc:2464
 
#14 0x0000563aa98cd23e in row_upd_clust_step (node=0x621000342678, thr=0x621000342ad8) at /data/Server/bb-10.6-MDEV-27700/storage/innobase/row/row0upd.cc:2690
 
#15 0x0000563aa98cdc67 in row_upd (node=0x621000342678, thr=0x621000342ad8) at /data/Server/bb-10.6-MDEV-27700/storage/innobase/row/row0upd.cc:2791
 
#16 0x0000563aa98ce769 in row_upd_step (thr=0x621000342ad8) at /data/Server/bb-10.6-MDEV-27700/storage/innobase/row/row0upd.cc:2933
 
#17 0x0000563aa9830637 in row_update_for_mysql (prebuilt=0x621000341d88) at /data/Server/bb-10.6-MDEV-27700/storage/innobase/row/row0mysql.cc:1695
 
#18 0x0000563aa94a484c in ha_innobase::update_row (this=0x61d0008b74b8, old_row=0x61a0002816f8 "\366\377\067\017", new_row=0x61a0002814b8 "\326\377\067\017")    at /data/Server/bb-10.6-MDEV-27700/storage/innobase/handler/ha_innodb.cc:8675
 
#19 0x0000563aa8bc3f31 in handler::ha_update_row (this=0x61d0008b74b8, old_data=0x61a0002816f8 "\366\377\067\017", new_data=0x61a0002814b8 "\326\377\067\017") at /data/Server/bb-10.6-MDEV-27700/sql/handler.cc:7635
 
#20 0x0000563aa86a0fed in mysql_update (thd=0x62b00020d218, table_list=0x62b0002143f8, fields=..., values=..., conds=0x0, order_num=0, order=0x0, limit=18446744073709551606, ignore=false, found_return=0x7facfdc461f0, updated_return=0x7facfdc46210) at /data/Server/bb-10.6-MDEV-27700/sql/sql_update.cc:1087
 
#21 0x0000563aa83bfd6f in mysql_execute_command (thd=0x62b00020d218, is_called_from_prepared_stmt=false) at /data/Server/bb-10.6-MDEV-27700/sql/sql_parse.cc:4408
 
#22 0x0000563aa83d8144 in mysql_parse (thd=0x62b00020d218, rawbuf=0x62b000214238 "UPDATE t1 SET col_text = REPEAT(SUBSTR(CAST( 1247 AS CHAR),1,1), @fill_amount)  /* E_R Thread21 QNO 19 CON_ID 35 */", length=115, parser_state=0x7facfdc46a20) at /data/Server/bb-10.6-MDEV-27700/sql/sql_parse.cc:8030
 
#23 0x0000563aa83b0552 in dispatch_command (command=COM_QUERY, thd=0x62b00020d218, packet=0x629000e0b219 " UPDATE t1 SET col_text = REPEAT(SUBSTR(CAST( 1247 AS CHAR),1,1), @fill_amount)  /* E_R Thread21 QNO 19 CON_ID 35 */ ", packet_length=117, blocking=true) at /data/Server/bb-10.6-MDEV-27700/sql/sql_parse.cc:1896
 
#24 0x0000563aa83ad999 in do_command (thd=0x62b00020d218, blocking=true) at /data/Server/bb-10.6-MDEV-27700/sql/sql_parse.cc:1409
 
#25 0x0000563aa87b1587 in do_handle_one_connection (connect=0x608000003938, put_in_cache=true) at /data/Server/bb-10.6-MDEV-27700/sql/sql_connect.cc:1418
 
#26 0x0000563aa87b0e13 in handle_one_connection (arg=0x608000003938) at /data/Server/bb-10.6-MDEV-27700/sql/sql_connect.cc:1312
 
#27 0x00007fad22008609 in start_thread (arg=<optimized out>) at pthread_create.c:477
 
#28 0x00007fad21bdd293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
 
(rr)



 Comments   
Comment by Matthias Leich [ 2022-08-19 ]

Alternative rr trace
pluto:/data/results/1660917950/TBR-1581$ _RR_TRACE_DIR=./1/rr/ rr replay

Comment by Marko Mäkelä [ 2022-08-22 ]

The trace pluto:/data/results/1660917950/TBR-1581/1/rr/latest-trace is for bb-10.6-MDEV-27700 eed7f33027a7debae2cbff4e33f921a6e1b65e65 and not my fix. That said, I think that similar to how I adjusted btr_page_alloc_low(), the function btr_page_free() must be adjusted as well.

Comment by Matthias Leich [ 2022-08-22 ] 

origin/bb-10.6-MDEV-29336 6251ae4ea219a5b14f978b79efd8c8584a655fc5 2022-08-22T16:24:10+03:00
 
which contains fixes for MDEV-27700, MDEV-29336 and MDEV-29250 performed well in RQG testing.

Comment by Marko Mäkelä [ 2022-08-23 ]

As far as I can tell, this problem is likely to only affect BLOB operations with small tables (consisting of a single index page) when the adaptive hash index is enabled and the table metadata is being frequently rebuilt as a result of some non-rebuilding ALTER TABLE operation.

Generated at Thu Feb 08 10:07:44 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.