[MCOL-3532] DBRM RO Mode Created: 2019-09-30  Updated: 2021-01-15

Status: Closed
Project: MariaDB ColumnStore
Component/s: None
Affects Version/s: 1.0.14
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Abhinav santi Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

AWS EC2, C3.8XL



 Description   

um1 keeps going down after system restart with BRM errors and DBRM RO with timeouts.
Unfortunately there is not much info in the err.log to debug the issue

Below is the log from PM1 err.log

L0000: DBRM Controller: Network error reading from node 1. Reading response to command 6, length 13. 0 length response, possible time-out. Will see if retry is possible.
Sep 30 18:54:39 xxx-pm1 controllernode[37474]: 39.228624 |0|0|0| C 29 CAL0000: InetStreamSocket::readToMagic(): I/O error1: rc-1; poll signal interrupt ( POLLHUP POLLERR )
Sep 30 18:59:38 xxx-pm1 controllernode[37474]: 38.316568 |0|0|0| C 29 CAL0000: A node is unresponsive for cmd = 6, no reconfigure in at least 300 seconds. Setting read-only mode.
Sep 30 18:59:38 xxx-pm1 writeengine[14124]: 38.606614 |0|0|0| E 19 CAL0085: ClearTableLock: Ending bulk rollback for table fact_hash (OID-3646); lock-517; initiated by cpimport. (rollback failed; Error rolling back column extents from extent map for 3647; dbRoot-1; partition-3; segment-1; hwm-3149; BRM error performing bulk rollback of column extents. [BRM error status: DBRM is in READ-ONLY mode]).
Sep 30 18:59:38 xxx-pm1 cpimport.bin[22478]: 38.884391 |0|0|0| E 34 CAL0087: BulkLoad Error: closeDctnryStore: error closing store file for OID-5428; file-/home/mariadb-user/mariadb/columnstore/data1/000.dir/000.dir/021.dir/052.dir/000.dir/FILE001.cdf; a BRM Set hwm error. [BRM error status: DBRM is in READ-ONLY mode]
Sep 30 18:59:38 xxx-pm1 cpimport.bin[22478]: 38.884632 |0|0|0| E 34 CAL0087: BulkLoad Error: finishParsing: close dictionary file error with column flightnumber_iata; a BRM Set hwm error.
Sep 30 18:59:38 xxx-pm1 cpimport.bin[22478]: 38.884718 |0|0|0| E 34 CAL0087: BulkLoad Error: setParseComplete completion error; Failed to load table: pantry.dim_tails; a BRM Set hwm error.
Sep 30 18:59:38 xxx-pm1 cpimport.bin[22478]: 38.884783 |0|0|0| E 34 CAL0087: BulkLoad Error: Bulkload Parse (thread 0) Failed for Table pantry.dim_tails during parse completion. Terminating this job.
Sep 30 18:59:38 xxx-pm1 writeengineserver[14124]: 38.897238 |0|0|0| E 32 CAL0000: 4636 : cpimport exit on failure (signal -1)
Sep 30 18:59:38 xxx-pm1 writeengine[14124]: 38.967832 |0|0|0| E 19 CAL0085: ClearTableLock: Ending bulk rollback for table dim_tails (OID-5400); lock-516; initiated by cpimport. (rollback failed; Bulk rollback for table dim_tails (OID-5400) not performed; BRM is in read-only state.).



 Comments   
Comment by Abhinav santi [ 2019-09-30 ]

Uploaded columnstore support information to ftp location at ftp.mariadb.com
filename: new-columnstoreSupportReport.prod-cs-pm1-09-30-2019.tar.gz

Generated at Thu Feb 08 02:43:25 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.