[MCOL-948] Columnstore engine(1.0.11): Transaction is NOT integrated with TRIGGER in an INSERT statement ! Created: 2017-09-29  Updated: 2021-01-17  Resolved: 2021-01-17

Status: Closed
Project: MariaDB ColumnStore
Component/s: MariaDB Server, writeengine
Affects Version/s: 1.0.11
Fix Version/s: N/A

Type: New Feature Priority: Major
Reporter: Gavin Chen Assignee: Todd Stoffel (Inactive)
Resolution: Won't Do Votes: 0
Labels: Transaction, integrity
Environment:

CentOS 7


Epic Link: ColumnStore Compatibility Improvements

 Description   

Current culumnstore can't support constraints FOREIGN KEY,UNIQUE,PRIMARY KEY, but these constraints is very important for avoid logic error from applications, I try use TRIGGER to avoid the limitations, but I found NOT an integrated transaction in an INSERT statement:
CREATE TABLE head
(id int PRIMARY KEY,
sVal varchar(50)
) ENGINE=INNODB;
INSERT INTO head(id,sVal) values(1,'Val1'),(2,'Val3'),(3,'Val3');

CREATE TABLE detail_inno
(
id int AUTO_INCREMENT PRIMARY KEY,
fk int NOT NULL,
sData varchar(30)
)ENGINE=INNODB;
CREATE TABLE detail_columnstore
(
id int,
fk int NOT NULL,
sData varchar(30)
)ENGINE=COLUMNSTORE COMMENT = 'autoincrement=id';;

DELIMITER //
CREATE TRIGGER fktest_inno BEFORE INSERT ON detail_inno FOR EACH ROW
BEGIN
IF NOT EXISTS(SELECT * FROM head WHERE id=NEW.fk) THEN
SIGNAL sqlstate '45001' set message_text = "Force Key fk NOT exists in table head ! ";
END IF;
END//
CREATE TRIGGER fktest_columnstore BEFORE INSERT ON detail_columnstore FOR EACH ROW
BEGIN
IF NOT EXISTS(SELECT * FROM head WHERE id=NEW.fk) THEN
SIGNAL sqlstate '45001' set message_text = "Force Key fk NOT exists in table head ! ";
END IF;
END//
DELIMITER ;

INSERT INTO detail_inno(fk,sData) VALUES(1,'data1'),(2,'data2'),(4,'data4');--GOOD: NOT any records inserted at here.
INSERT INTO detail_columnstore(fk,sData) VALUES(1,'data1'),(2,'data2'),(4,'data4');-- BAD: id 1,2 is inserted at here, id 4 not inserted.



 Comments   
Comment by David Thompson (Inactive) [ 2017-09-29 ]

Correct, most analytics databases don't always support more complex referential integrity due to larger data scales since normally the data is not system of record. Even if we fix the trigger support for dml inserts, it would be a very high amount of work to enforce this with bulk load since that is working at the lower level writeengine level rather than mysqld level.

If you need to do some validation and cleanup of your source data, you would be better loading and validating to an innodb table and perform any clean up there before doing an insert select to the columnstore table (this is essentially the ELT paradigm).

Comment by Gavin Chen [ 2017-09-30 ]

Hi,David:
about conflict between bulk load and integrity, I think should let user to decide choise which one and balance them, such as in INNODB or SQL server, we are setup constraints FOREIGN KEY,UNIQUE in develop or try-run stage, once into release stage or found performance bottleneck, we will disable these constraints.

Generated at Thu Feb 08 02:25:01 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.