[MCOL-5623] S3 Cluster Left Read-Only | BRM lock state & DBRMSnapshotInterval Created: 2023-12-05  Updated: 2024-01-24  Resolved: 2024-01-24

Status: Closed
Project: MariaDB ColumnStore
Component/s: N/A
Affects Version/s: 23.02.3, 23.02.4
Fix Version/s: 23.02.8

Type: Bug Priority: Blocker
Reporter: Allen Herrera Assignee: Denis Khalikov
Resolution: Fixed Votes: 0
Labels: None
Environment:

Customer: 48x1024 3 node Cohesity On-prem S3 cluster
Reproduction: Rocky 8 16x64 EC2 S3 single Node

Issue Links:
PartOf
includes MCOL-5631 Optimize BRM load operation when Stor... In Progress
Sprint: 2023-11, 2023-12

 Description   

Currently an S3 cluster customer cant load data for more than ~10 days as a BRM lock state likely coincidently occurring during cpimports cascades into rollbacks and leaving the deployment in Read-Only mode until manual intervention. This is unacceptable and prevents the customer from running their production workload.

Error

Dec  5 04:06:27 ip-172-31-27-120 Calpont[125899]: 27.238073 |0|0|0| W 00 CAL0094: Attempting to fix the BRM lock state. Diagnostic values: r=1 rwt=0 w=0 wwt=0.

See developer comments for reproduction
Generated at Thu Feb 08 02:59:14 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.