[MCOL-954] mysqld crashing on slave replication UMs Created: 2017-10-04  Updated: 2020-08-25  Resolved: 2017-12-04

Status: Closed
Project: MariaDB ColumnStore
Component/s: MariaDB Server
Affects Version/s: 1.0.11
Fix Version/s: 1.0.12, 1.1.3

Type: Bug Priority: Major
Reporter: David Hill (Inactive) Assignee: Daniel Lee (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Environment:

amazon EC2 root install 3um / 5pm system

Sprint: 2017-23, 2017-24

 Description   

Customer reproting that mysqld continually crashed on UM2/UM3, which are the slave replication nodes. Don't know if this is related or not.

Thread pointer: 0x7f89e8001550
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x7f8a3c10f810 thread_stack 0x80000
/usr/local/mariadb/columnstore/mysql//bin/mysqld(my_print_stacktrace+0x38)[0x7f8a79ace835]
/usr/local/mariadb/columnstore/mysql//bin/mysqld(handle_fatal_signal+0x38f)[0x7f8a7947a7fd]
/lib64/libpthread.so.0(+0xf370)[0x7f8a78942370]
/lib64/libc.so.6(gsignal+0x37)[0x7f8a7689f1d7]
/lib64/libc.so.6(abort+0x148)[0x7f8a768a08c8]
mysys/stacktrace.c:267(my_print_stacktrace)[0x7f8a771a39d5]
/lib64/libstdc++.so.6(+0x5e946)[0x7f8a771a1946]
/lib64/libstdc++.so.6(+0x5e973)[0x7f8a771a1973]
/lib64/libstdc++.so.6(+0x5eb93)[0x7f8a771a1b93]
/usr/local/mariadb/columnstore/mysql/lib/plugin/libcalmysql.so(_Z29ha_calpont_impl_external_lockP3THDP5TABLEi+0x2c3)[0x7f8a735f2314]
/usr/local/mariadb/columnstore/mysql/lib/plugin/libcalmysql.so(_ZN10ha_calpont13external_lockEP3THDi+0x83)[0x7f8a735dd08d]
sql/handler.cc:5867(handler::ha_external_lock(THD*, int))[0x7f8a79488832]
sql/lock.cc:364(lock_external(THD*, TABLE**, unsigned int))[0x7f8a79568fbb]
sql/lock.cc:309(mysql_lock_tables(THD*, st_mysql_lock*, unsigned int))[0x7f8a79568ce9]
sql/lock.cc:275(mysql_lock_tables(THD*, TABLE**, unsigned int, unsigned int))[0x7f8a79568bd7]
sql/sql_base.cc:5442(lock_tables(THD*, TABLE_LIST*, unsigned int, unsigned int))[0x7f8a791ef570]
sql/sql_base.cc:5271(open_and_lock_tables(THD*, DDL_options_st const&, TABLE_LIST*, bool, unsigned int, Prelocking_strategy*))[0x7f8a791ef059]
/usr/local/mariadb/columnstore/mysql//bin/mysqld(_Z20open_and_lock_tablesP3THDP10TABLE_LISTbj+0x55)[0x7f8a791e2bb7]
/usr/local/mariadb/columnstore/mysql//bin/mysqld(_Z17mysql_create_viewP3THDP10TABLE_LIST21enum_view_create_mode+0x251)[0x7f8a79335c45]
/usr/local/mariadb/columnstore/mysql//bin/mysqld(_Z21mysql_execute_commandP3THD+0x843b)[0x7f8a79257c35]
/usr/local/mariadb/columnstore/mysql//bin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x2b9)[0x7f8a7925bd60]
/usr/local/mariadb/columnstore/mysql//bin/mysqld(_ZN15Query_log_event14do_apply_eventEP14rpl_group_infoPKcj+0xe43)[0x7f8a795731f3]
sql/sql_base.h:534(open_and_lock_tables(THD*, TABLE_LIST*, bool, unsigned int))[0x7f8a79572349]
sql/log_event.h:1343(Log_event::apply_event(rpl_group_info*))[0x7f8a791bc6d3]
sql/slave.cc:3480(apply_event_and_update_pos_apply(Log_event*, THD*, rpl_group_info*, int))[0x7f8a791b2652]
sql/slave.cc:3602(apply_event_and_update_pos(Log_event*, THD*, rpl_group_info*))[0x7f8a791b29a3]
sql/slave.cc:3886(exec_relay_log_event(THD*, Relay_log_info*, rpl_group_info*))[0x7f8a791b308e]
sql/slave.cc:4982(handle_slave_sql)[0x7f8a791b6119]
/lib64/libpthread.so.0(+0x7dc5)[0x7f8a7893adc5]
/lib64/libc.so.6(clone+0x6d)[0x7f8a7696176d]

 Comments   
Comment by Andrew Hutchings (Inactive) [ 2017-11-21 ]

vtable_state was in a random state which caused an assert style crash when locking a table during a slave view create. The patch initializes vtable_state correctly.

For QA: You need a 2UM (or more) setup with schema replication turned on and then do:

create table toview (a int, b int) engine=columnstore;
 
create view tview as select * from toview;

Before the patch it will crash the slave when it is replicated. After the patch it should create the view correctly on the slave.

Comment by Daniel Lee (Inactive) [ 2017-12-04 ]

Builds verified: Github source

1.0.12-1

[root@localhost ~]# cat mariadb-columnstore-1.0.12-1-centos7.x86_64.bin.tar.txt
/root/columnstore/mariadb-columnstore-server
commit 25e9d054cd3d05683fade1b974e1730316d256ed
Merge: 89b2ea1 7c52a83
Author: David.Hall <david.hall@mariadb.com>
Date: Tue Nov 21 10:49:11 2017 -0600

Merge pull request #79 from mariadb-corporation/MCOL-954-1.0

MCOL-954 Init vtable state

/root/columnstore/mariadb-columnstore-server/mariadb-columnstore-engine
commit b112e826a2793228f5f3c1312fec5291fc1d8bf5
Merge: 7c2640f b657938
Author: David.Hall <david.hall@mariadb.com>
Date: Fri Dec 1 16:17:28 2017 -0600

Merge pull request #338 from mariadb-corporation/MCOL-1068

MCOL-1068 Improve compression_ratio() procedure

1.1.3-1

/root/columnstore/mariadb-columnstore-server
commit 632e265687674fb66bd1d704bc18032b00dd6b17
Merge: 5e9fe52 200f5be
Author: david hill <david.hill@mariadb.com>
Date: Tue Nov 21 15:22:06 2017 -0600

Merge branch 'develop-1.1' of https://github.com/mariadb-corporation/mariadb-columnstore-server into develop-1.1

/root/columnstore/mariadb-columnstore-server/mariadb-columnstore-engine
commit 4d8026618cfb5377c9a200170848092ce5660f10
Author: david hill <david.hill@mariadb.com>
Date: Wed Nov 29 09:36:24 2017 -0600

change how the os_detect is run on remote nodes

Retested test scenario in the ticket, as well as more complicated scenarios. Also tested DML and queries.

Generated at Thu Feb 08 02:25:03 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.