[MCOL-4954] Add support for new DBRM socket bytestream format. Created: 2021-12-27  Updated: 2022-03-30  Resolved: 2022-02-04

Status: Closed
Project: MariaDB ColumnStore
Component/s: cmapi
Affects Version/s: cmapi-1.5, cmapi-1.6
Fix Version/s: cmapi-1.6.2

Type: Task Priority: Blocker
Reporter: Alan Mologorsky Assignee: Daniel Lee (Inactive)
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
Duplicate
is duplicated by MCOL-4947 Releases 6.1.2 and higher are much sl... Closed
is duplicated by MCOL-4972 Undefined behaviour while DBRM commun... Closed
PartOf
includes MCOL-4919 DBRM issue in tests from develop6 Closed
is part of MCOL-4850 Refactor CMAPI to meet PEP-8 styleguide Stalled
Problem/Incident
is caused by MCOL-4810 Redundant copying and wasting memory ... Closed
is caused by MCOL-4973 Update MariaDB Columnstore to 6.2.2, ... Closed
Relates
relates to MCOL-4988 Table lock remained after DML failure... Closed
Sprint: 2021-16, 2021-17

 Description   

DBRM socket bytestream protocol has been changed since MCS 6.2.1 release.
Add support for both new and old protocols.
In new version add 4 extra bytes for long strings counting purpose to the header.

 Comments   
Comment by Alan Mologorsky [ 2022-02-01 ]

For testing.
Steps for the new test case:
1. add at least one node to the cluster 

curl -k -s -X PUT https://s2pm1:8640/cmapi/0.4.0/cluster/node \
 
  --header 'Content-Type:application/json' \
 
  --header "x-api-key:$MCSAPIKEY" \
 
  --data '{"timeout":120, "node": "s2pm1"}' \
 
  | jq

2. set cluster mode to readonly

curl -k -s -X PUT https://s2pm1:8640/cmapi/0.4.0/cluster/mode-set \
 
  --header 'Content-Type:application/json' \
 
  --header "x-api-key:$MCSAPIKEY" \
 
  --data '{"mode": "readonly"}' \
 
  | jq

3. set cluster mode to readwrite

curl -k -s -X PUT https://s2pm1:8640/cmapi/0.4.0/cluster/mode-set \
 
  --header 'Content-Type:application/json' \
 
  --header "x-api-key:$MCSAPIKEY" \
 
  --data '{"mode": "readwrite"}' \
 
  | jq

Comment by Daniel Lee (Inactive) [ 2022-02-02 ]

Build tested: 6.2.3-1 (#3752), CMAPI 1.6.2 (#600)
Test on a 3-node cluster using Centos 8

Reproduced the readonly and readwrite issues on 6.2.2

Tested the follow commands (aliases)

mcsModule
mcsShutdown
mcsStart
mcsReadOnly
mcsReadWrite

After setting the cluster in readonly mode, I tried an insert statement. A DBRM read-only error returned as expected, but it left a table lock in place. After setting the cluster back to readwrite mode, subsequent DML statements also failed because of the table lock.

When the cluster is in readonly, a failed DML state should clear the table lock. I don't know if the reported issues is within CMAPI or the DMLPro. Please investigate. If it is a latter case, I will create a new ticket for it.

[centos8:root~]# $MCSCLIENT mytest -vvv -e "insert into quicktest values (12, 'readwrite mode')"
 
--------------
 
insert into quicktest values (12, 'readwrite mode')
 
--------------
 
 
 
Query OK, 1 row affected (0.236 sec)
 
 
 
Bye
 
[centos8:root~]# mcsReadOnly
 
{
 
  "timestamp": "2022-02-02 20:26:38.826101",
 
  "cluster-mode": "readonly"
 
}
 
[centos8:root~]# $MCSCLIENT mytest -vvv -e "insert into quicktest values (12, 'readwrite mode')"
 
--------------
 
insert into quicktest values (12, 'readwrite mode')
 
--------------
 
 
 
ERROR 1815 (HY000) at line 1: Internal error: CAL0001: Insert Failed:   a BRM Set hwm error. [BRM error status: DBRM is in READ-ONLY mode]  
Bye
 
[centos8:root~]# viewtablelock
 
 There is 1 table lock
 
 
 
  Table             LockID  Process  PID   Session  Txn  CreationTime              State    DBRoots  
  mytest.quicktest  7       DMLProc  7782  86       15   Wed Feb  2 20:26:48 2022  LOADING  1,2,3    
[centos8:root~]# mcsReadWrite
 
{
 
  "timestamp": "2022-02-02 20:29:41.277906",
 
  "cluster-mode": "readwrite"
 
}
 
[centos8:root~]# $MCSCLIENT mytest -vvv -e "insert into quicktest values (12, 'readwrite mode')"
 
--------------
 
insert into quicktest values (12, 'readwrite mode')
 
--------------
 
 
 
ERROR 1815 (HY000) at line 1: Internal error: CAL0001: Insert Failed:  MCS-2009: Unable to perform the insert operation because DMLProc with PID 7782 is currently holding the table lock for session 86.

Comment by Daniel Lee (Inactive) [ 2022-02-04 ]

A ticket MCOL-4988 has been opened to track the reported table lock issue. Closing this ticket

Comment by Alan Mologorsky [ 2022-02-10 ]

This feature fixes undefined behaviour in internal communication between CMAPI and Engine. Such behaviour described in MCOL-4973

Generated at Thu Feb 08 02:54:16 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.