[MCOL-3999] FAILOVER: create table after failover process completed Created: 2020-05-12  Updated: 2020-05-15  Resolved: 2020-05-15

Status: Closed
Project: MariaDB ColumnStore
Component/s: DDLProc
Affects Version/s: 1.4.4
Fix Version/s: 1.4.4

Type: Bug Priority: Critical
Reporter: Daniel Lee (Inactive) Assignee: Daniel Lee (Inactive)
Resolution: Fixed Votes: 0
Labels: None

Attachments: File columnstoreSupportReport.vagrantTestStack.tar.gz    
Issue Links:
Problem/Incident
is caused by MCOL-3842 1.4.2 centos 7 with gluster setup - p... Closed
Sprint: 2020-7

 Description   

Build tested: 1.4.4-1 (Jenkins 20200508)

I started using the columnstoreSupport tool to collect log files, as well as other stack related informations. columnstoreSupport output has been attached.

With a newly installed 3PM combo stack with glusterfs, QA sanity tests (DDL, DML, LDI, cpimport, queries) passed. I took PM3 (cold standby node) out of service. Failover process completed. Create table command returned the following error:

MariaDB [mytest]> create table t1 (c1 int) engine=columnstore;
ERROR 1815 (HY000): Internal error: CAL0009: (6)Create table failed due to CAL0001: we_readthread caught exception boost unique_lock has no mutex: Operation not permitted
MariaDB [mytest]>

mcsadmin> getprocessstatus
getprocessstatus Tue May 12 19:22:59 2020

MariaDB ColumnStore Process statuses

Process Module Status Last Status Change Process ID
------------------ ------ --------------- ------------------------ ----------
ProcessMonitor pm1 ACTIVE Tue May 12 19:11:12 2020 4015
ProcessManager pm1 ACTIVE Tue May 12 19:11:18 2020 4236
DBRMControllerNode pm1 ACTIVE Tue May 12 19:12:51 2020 6090
ServerMonitor pm1 ACTIVE Tue May 12 19:12:54 2020 6133
DBRMWorkerNode pm1 ACTIVE Tue May 12 19:12:55 2020 6157
PrimProc pm1 ACTIVE Tue May 12 19:12:59 2020 6253
ExeMgr pm1 ACTIVE Tue May 12 19:18:54 2020 17686
WriteEngineServer pm1 ACTIVE Tue May 12 19:13:21 2020 8035
DDLProc pm1 ACTIVE Tue May 12 19:19:04 2020 17831
DMLProc pm1 ACTIVE Tue May 12 19:19:28 2020 18207
mysqld pm1 ACTIVE Tue May 12 19:13:57 2020 10438

ProcessMonitor pm2 ACTIVE Tue May 12 19:12:37 2020 3523
ProcessManager pm2 HOT_STANDBY Tue May 12 19:12:46 2020 3710
DBRMControllerNode pm2 COLD_STANDBY Tue May 12 19:19:25 2020
ServerMonitor pm2 ACTIVE Tue May 12 19:12:59 2020 3912
DBRMWorkerNode pm2 ACTIVE Tue May 12 19:13:00 2020 3926
PrimProc pm2 ACTIVE Tue May 12 19:13:04 2020 3955
ExeMgr pm2 ACTIVE Tue May 12 19:18:58 2020 5997
WriteEngineServer pm2 ACTIVE Tue May 12 19:13:22 2020 4487
DDLProc pm2 COLD_STANDBY Tue May 12 19:19:25 2020
DMLProc pm2 COLD_STANDBY Tue May 12 19:19:25 2020
mysqld pm2 ACTIVE Tue May 12 19:19:25 2020 4865

ProcessMonitor pm3 AUTO_OFFLINE Tue May 12 19:18:48 2020
ProcessManager pm3 AUTO_OFFLINE Tue May 12 19:18:48 2020
DBRMControllerNode pm3 AUTO_OFFLINE Tue May 12 19:18:48 2020
ServerMonitor pm3 AUTO_OFFLINE Tue May 12 19:18:49 2020
DBRMWorkerNode pm3 AUTO_OFFLINE Tue May 12 19:18:49 2020
PrimProc pm3 AUTO_OFFLINE Tue May 12 19:18:49 2020
ExeMgr pm3 AUTO_OFFLINE Tue May 12 19:18:49 2020
WriteEngineServer pm3 AUTO_OFFLINE Tue May 12 19:18:49 2020
DDLProc pm3 AUTO_OFFLINE Tue May 12 19:18:49 2020
DMLProc pm3 AUTO_OFFLINE Tue May 12 19:18:49 2020
mysqld pm3 AUTO_OFFLINE Tue May 12 19:18:49 2020

 Comments   
Comment by Daniel Lee (Inactive) [ 2020-05-12 ]

After putting PM3 back online, 'create table' worked fine.

Comment by Patrick LeBlanc (Inactive) [ 2020-05-14 ]

merged it into dev-1.4; won't be available to test until i trigger a build of course.

Comment by Daniel Lee (Inactive) [ 2020-05-15 ]

Build verified: 1.4.4-1 (Jenkins 20200515)

Repeat test case and more.

Generated at Thu Feb 08 02:47:01 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.