InnoDB indexes are inconsistent after dropping partition

2023-10-26 16:32:48 467 [ERROR] Found index <index> whose column info does not match that of MariaDB.

2023-10-26 16:32:48 467 [ERROR] InnoDB indexes are inconsistent with what defined in .frm for table ./<db>/<table>#P#P20230901

2023-10-26 15:38:46 64307678 [Warning] InnoDB: Using a partial-field key prefix in search, index `<index>` of table `<db>`.`<table>` /* Partition `P20231101` */. Last data field length 8 bytes, key ptr now exceeds key end by 3 bytes. Key value in the MySQL format:

 len 7; hex b50399b1740000; asc     t  ;

2023-10-26 15:38:46 64307678 [ERROR] InnoDB: Clustered record for sec rec not found index `<index>` of table `<db>`.`<table>` /* Partition `P20231101` */

InnoDB: sec index record PHYSICAL RECORD: n_fields 3; compact format; info bits 0

 0: len 2; hex 03b5; asc   ;;

 1: len 8; hex 87c174b199007005; asc   t   p ;;

 2: len 6; hex af05f8af05e2; asc       ;;

InnoDB: clust index record PHYSICAL RECORD: n_fields 10; compact format; info bits 0

 0: len 6; hex 007005af066c; asc  p   l;;

 1: len 6; hex 000000000000; asc       ;;

 2: len 7; hex 80000000000000; asc        ;;

 3: len 2; hex 03b5; asc   ;;

 4: len 4; hex 000146ef; asc   F ;;

 5: len 8; hex 87e074b199000000; asc   t     ;;

 6: len 4; hex 00000000; asc     ;;

 7: len 4; hex 00000000; asc     ;;

 8: len 4; hex 15000000; asc     ;;

 9: len 2; hex 0000; asc   ;;

TRANSACTION 421589657076240, ACTIVE 0 sec starting index read

mysql tables in use 8, locked 0

0 lock struct(s), heap size 1128, 0 row lock(s)

MySQL thread id 64307678, OS thread handle 140095744042752, query id 791359259 <hostname> 192.168.255.80 statsadm Sending data

SELECT

 <tableb>.hostname  AS hostname,

 MAX( <table>.numCurrentSessions ) AS max

FROM

 <table>,

 <tableb>,

 <tablec>

WHERE

 <tablec>.name = @site AND

 <table>.time BETWEEN @date AND @date + INTERVAL 1 DAY AND

 <table>.siteid = <tablec>.id AND

 <tableb>.id= <table>.serverid

GROUP BY hostname

InnoDB: Submit a detailed bug report to https://jira.mariadb.org/

2023-10-26 15:40:30 64307291 [ERROR] [FATAL] InnoDB: Cannot insert tuple TUPLE (info_bits=0, 3 fields): {[2]  (0x02E2),[5]  t  (0x1DDE74B199),[6] p    (0x007005BDDCCA)}into index `<index>` of table `<db>`.`<table>` /* Partition `P20231101` */. Max size: 19

231026 15:40:30 [ERROR] mysqld got signal 6 ;

This could be because you hit a bug. It is also possible that this binary

or one of the libraries it was linked against is corrupt, improperly built,

or misconfigured. This error can also be caused by malfunctioning hardware.

To report this bug, see https://mariadb.com/kb/en/reporting-bugs

We will try our best to scrape up some info that will hopefully help

diagnose the problem, but since we have already crashed,

something is definitely wrong and this may fail.

Server version: 10.5.16-MariaDB-log

key_buffer_size=134217728

read_buffer_size=131072

max_used_connections=56

max_threads=153

thread_count=30

It is possible that mysqld could use up to

key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 467873 K  bytes of memory

Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x7f6a9c39c5a8

Attempting backtrace. You can use the following information to find out

where mysqld died. If you see no messages after this, something went

terribly wrong...

stack_bottom = 0x7f6a94b44bd8 thread_stack 0x49000

Can't start addr2line

/usr/libexec/mysqld(my_print_stacktrace+0x41)[0x55a8ee32add1]

/usr/libexec/mysqld(handle_fatal_signal+0x4a5)[0x55a8eddc0355]

/lib64/libpthread.so.0(+0x12cf0)[0x7f6f0959fcf0]

/lib64/libc.so.6(gsignal+0x10f)[0x7f6f088fbacf]

/lib64/libc.so.6(abort+0x127)[0x7f6f088ceea5]

/usr/libexec/mysqld(+0x644be7)[0x55a8edaa8be7]

/usr/libexec/mysqld(+0x6485f6)[0x55a8edaac5f6]

/usr/libexec/mysqld(+0xce28fd)[0x55a8ee1468fd]

/usr/libexec/mysqld(+0xce78de)[0x55a8ee14b8de]

/usr/libexec/mysqld(+0xce7fe4)[0x55a8ee14bfe4]

/usr/libexec/mysqld(+0xcf8bd0)[0x55a8ee15cbd0]

/usr/libexec/mysqld(+0xc411ed)[0x55a8ee0a51ed]

/usr/libexec/mysqld(_ZN7handler12ha_write_rowEPKh+0x177)[0x55a8eddcdee7]

/usr/libexec/mysqld(+0xb812f0)[0x55a8edfe52f0]

/usr/libexec/mysqld(_ZN7handler12ha_write_rowEPKh+0x177)[0x55a8eddcdee7]

/usr/libexec/mysqld(_Z12write_recordP3THDP5TABLEP12st_copy_infoP13select_result+0x1dd)[0x55a8edb8d34d]

/usr/libexec/mysqld(_Z10mysql_loadP3THDPK12sql_exchangeP10TABLE_LISTR4ListI4ItemES9_S9_15enum_duplicatesbb+0x17de)[0x55a8edbba71e]

/usr/libexec/mysqld(_Z21mysql_execute_commandP3THD+0x185f)[0x55a8edbcba0e]

/usr/libexec/mysqld(_Z11mysql_parseP3THDPcjP12Parser_statebb+0x238)[0x55a8edbbc956]

/usr/libexec/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcjbb+0x177d)[0x55a8edbc842c]

/usr/libexec/mysqld(_Z10do_commandP3THD+0x122)[0x55a8edbc9702]

/usr/libexec/mysqld(_Z24do_handle_one_connectionP7CONNECTb+0x4e0)[0x55a8edcbaa00]

/usr/libexec/mysqld(handle_one_connection+0x5d)[0x55a8edcbadad]

/usr/libexec/mysqld(+0xb882d8)[0x55a8edfec2d8]

/lib64/libpthread.so.0(+0x81ca)[0x7f6f095951ca]

/lib64/libc.so.6(clone+0x43)[0x7f6f088e6e73]

Trying to get some variables.

Some pointers may be invalid and cause the dump to abort.

Query (0x7f6a9c04a370): LOAD DATA LOCAL INFILE '/data/tmp/makeStats_12/<table>.bcp' INTO TABLE <table> (time,serverid,siteid,cpuUsed,memoryUsed,processes,numCurrentSessions)

Connection ID (thread ID): 64307291

Status: NOT_KILLED

Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=on,condition_pushdown_from_having=on,not_null_range_scan=off

The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains

information that should help you find out what is causing the crash.

Writing a core file...

Working directory at /data/db/data

Resource Limits:

Limit                     Soft Limit           Hard Limit           Units

Max cpu time              unlimited            unlimited            seconds

Max file size             unlimited            unlimited            bytes

Max data size             unlimited            unlimited            bytes

Max stack size            8388608              unlimited            bytes

Max core file size        0                    unlimited            bytes

Max resident set          unlimited            unlimited            bytes

Max processes             94970                94970                processes

Max open files            160000               160000               files

Max locked memory         65536                65536                bytes

Max address space         unlimited            unlimited            bytes

Max file locks            unlimited            unlimited            locks

Max pending signals       94970                94970                signals

Max msgqueue size         819200               819200               bytes

Max nice priority         0                    0

Max realtime priority     0                    0

Max realtime timeout      unlimited            unlimited            us

Core pattern: |/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h %e