[MDEV-29675] "free(): invalid pointer" with TABLE_TYPE=ODBC and the MariaDB ODBC Driver Created: 2022-09-30  Updated: 2022-11-30  Resolved: 2022-11-30

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - Connect
Affects Version/s: 10.9.3
Fix Version/s: 10.9.4

Type: Bug Priority: Major
Reporter: markus makela Assignee: Andrew Hutchings
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
Relates
relates to MDEV-29734 Add SSL to TABLE_TYPE=MYSQL Open

 Description   

I was testing how the CONNECT engine works with Postgres and ended up using the wrong driver while connecting. The connection appeared to hang (as expected) but then the whole server crashed.
Stacktrace:

2022-09-30 10:18:50 24 [Note] CONNECT: Version 1.07.0002 March 22, 2021
free(): invalid pointer
220930 13:04:09 [ERROR] mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed, 
something is definitely wrong and this may fail.
Server version: 10.9.3-MariaDB-1:10.9.3+maria~ubu2204-log
key_buffer_size=134217728
read_buffer_size=131072
max_used_connections=5
max_threads=10002
thread_count=5
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 22159044 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
Thread pointer: 0x7f6668000c68
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x7f66945a8c78 thread_stack 0x49000
Printing to addr2line failed
mysqld(my_print_stacktrace+0x32)[0x55a2e5062342]
mysqld(handle_fatal_signal+0x478)[0x55a2e4b338e8]
/lib/x86_64-linux-gnu/libc.so.6(+0x42520)[0x7f66b7073520]
/lib/x86_64-linux-gnu/libc.so.6(pthread_kill+0x12c)[0x7f66b70c7a7c]
/lib/x86_64-linux-gnu/libc.so.6(raise+0x16)[0x7f66b7073476]
/lib/x86_64-linux-gnu/libc.so.6(abort+0xd3)[0x7f66b70597f3]
/lib/x86_64-linux-gnu/libc.so.6(+0x896f6)[0x7f66b70ba6f6]
/lib/x86_64-linux-gnu/libc.so.6(+0xa0d7c)[0x7f66b70d1d7c]
/lib/x86_64-linux-gnu/libc.so.6(+0xa2ac4)[0x7f66b70d3ac4]
/lib/x86_64-linux-gnu/libc.so.6(free+0x73)[0x7f66b70d64d3]
mysqld(+0xa1e04a)[0x55a2e4b0604a]
mysqld(mysql_real_connect+0x3a8)[0x55a2e4b0aa68]
/usr/lib/x86_64-linux-gnu/odbc/libmaodbc.so(+0x18033)[0x7f668dfc6033]
/usr/lib/x86_64-linux-gnu/odbc/libmaodbc.so(+0x1f789)[0x7f668dfcd789]
/usr/lib/x86_64-linux-gnu/odbc/libmaodbc.so(SQLDriverConnect+0xc4)[0x7f668dfbb964]
/lib/x86_64-linux-gnu/libodbc.so.2(SQLDriverConnect+0x9c0)[0x7f669415c370]
/usr/lib/mysql/plugin/ha_connect.so(_ZN7ODBConn13DriverConnectEj+0x50)[0x7f66944e8060]
/usr/lib/mysql/plugin/ha_connect.so(_ZN7ODBConn4OpenEPKcP10odbc_parmsj+0x63)[0x7f66944e8133]
/usr/lib/mysql/plugin/ha_connect.so(_Z11ODBCColumnsP7_globalPKcS2_S2_S2_ibP10odbc_parms+0x257)[0x7f66944e8497]
/usr/lib/mysql/plugin/ha_connect.so(+0xa3f12)[0x7f6694436f12]
mysqld(+0x8acdae)[0x55a2e4994dae]
mysqld(_Z26mysql_create_table_no_lockP3THDP16st_ddl_log_stateS2_PK25st_mysql_const_lex_stringS5_P22Table_specification_stP10Alter_infoPbiP10TABLE_LIST+0x148)[0x55a2e4995458]
mysqld(_ZN25Sql_cmd_create_table_like7executeEP3THD+0x120a)[0x55a2e499fb8a]
mysqld(_Z21mysql_execute_commandP3THDb+0x4be6)[0x55a2e48e3406]
mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x1e7)[0x55a2e48e4697]
mysqld(_Z16dispatch_command19enum_server_commandP3THDPcjb+0x14d5)[0x55a2e48e6e35]
mysqld(_Z10do_commandP3THDb+0x138)[0x55a2e48e8b38]
mysqld(_Z24do_handle_one_connectionP7CONNECTb+0x3bf)[0x55a2e4a0741f]
mysqld(handle_one_connection+0x5d)[0x55a2e4a0776d]
mysqld(+0xc76de6)[0x55a2e4d5ede6]
/lib/x86_64-linux-gnu/libc.so.6(+0x94b43)[0x7f66b70c5b43]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x44)[0x7f66b7156bb4]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (0x7f66680109c0): create Or replace table pg_acc ENGINE=CONNECT TABLE_TYPE=ODBC CONNECTION='Driver=MariaDB Unicode;UID=maxuser;PWD=maxpwd;SERVER=127.0.0.1;PORT=5432' TABNAME='pgbench_accounts'
Connection ID (thread ID): 27
Status: KILL_QUERY
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=on,condition_pushdown_from_having=on,not_null_range_scan=off
The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains
information that should help you find out what is causing the crash.
Writing a core file...
Working directory at /var/lib/mysql
Resource Limits:
Limit                     Soft Limit           Hard Limit           Units     
Max cpu time              unlimited            unlimited            seconds   
Max file size             unlimited            unlimited            bytes     
Max data size             unlimited            unlimited            bytes     
Max stack size            8388608              unlimited            bytes     
Max core file size        unlimited            unlimited            bytes     
Max resident set          unlimited            unlimited            bytes     
Max processes             unlimited            unlimited            processes 
Max open files            60000                60000                files     
Max locked memory         8388608              8388608              bytes     
Max address space         unlimited            unlimited            bytes     
Max file locks            unlimited            unlimited            locks     
Max pending signals       127103               127103               signals   
Max msgqueue size         819200               819200               bytes     
Max nice priority         0                    0                    
Max realtime priority     0                    0                    
Max realtime timeout      unlimited            unlimited            us        
Core pattern: |/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h
Kernel version: Linux version 5.19.11-200.fc36.x86_64 (mockbuild@bkernel02.iad2.fedoraproject.org) (gcc (GCC) 12.2.1 20220819 (Red Hat 12.2.1-2), GNU ld version 2.37-36.fc36) #1 SMP PREEMPT_DYNAMIC Fri Sep 23 15:07:44 UTC 2022
Fatal signal 11 while backtracing



 Comments   
Comment by Andrew Hutchings [ 2022-09-30 ]

Note to self: symbol clash between external client and internal mysqld for `mysql_real_connect`.

Comment by Andrew Hutchings [ 2022-09-30 ]

More notes: needs an ODBC setup with libmariadb, odbc-mariadb package in Ubuntu. Possible fix might be linking odbc with `-Bsymbolic`, which is the linker equivalent of `RTLD_DEEPBIND`.

Comment by Andrew Hutchings [ 2022-11-02 ]

markus makela I think this might have been fixed as part of MDEV-29490. Are you able to check?

Comment by markus makela [ 2022-11-03 ]

I don't think I have a build environment for the server available right now. Once the 10.9.4 release is out, I can use the docker images to test it.

Comment by markus makela [ 2022-11-30 ]

Yup, seems to be fixed by MDEV-29490.

Comment by Andrew Hutchings [ 2022-11-30 ]

This was fixed by another commit which remapped the internal client symbols so they nolonger conflict.

Generated at Thu Feb 08 10:10:25 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.