[MCOL-1183] Incorrectly formatted file can cause cpimport to crash and leave behind locks - Jira

Details

Type: Bug
Status: Closed (View Workflow)
Priority: Major
Resolution: Cannot Reproduce
Affects Version/s: 1.1.2
Fix Version/s: 1.1.7
Component/s: cpimport
Labels:
None

Sprint:
2018-05, 2018-06, 2018-07, 2018-08, 2018-09, 2018-10, 2018-11, 2018-12, 2018-13, 2018-14, 2018-15, 2018-16, 2018-17, 2018-18, 2018-19, 2018-20, 2018-21

Description

A user tried to load some incorrectly formatted tab-delimited files using cpimport.

gdb says that the crash was the following:

Program terminated with signal 11, Segmentation fault.

#0  0x00007f0e40940cf4 in WriteEngine::WEFileReadThread::getNextRow (this=this@entry=0x7fff11bb0018, ifs=...,

    pBuf=pBuf@entry=0x7fff11bb03a4 "...redacted..."..., MaxLen=MaxLen@entry=1048575)

    at /home/builder/mariadb-columnstore-server/mariadb-columnstore-engine/writeengine/splitter/we_filereadthread.cpp:495

After cpimport crashed, ColumnStore also did not clear the table locks originally taken by cpimport, so those had to be cleared with cleartablelock.

Core dumps and log files will be provided privately.

Attachments

Activity

Ascending order - Click to sort in descending order

Geoff Montee (Inactive) created issue - 2018-01-26 21:36

Andrew Hutchings (Inactive) added a comment - 2018-01-29 16:32

Many thanks for the gdb output. Whilst I haven't been able to mock up a way of reproducing this yet it does look like the buffer is blown when scanning for the end enclosing character once one is found.

The user is outputting data using:

mysql --defaults-extra-file=<.cnf file> -h <server> --compress --quick --skip-column-names --default-character-set=utf8 -Ns -D $schema --execute="select * from $table" > $table.txt

Then importing with:

cpimport -E '\"' -e 1 -n1 -s '\t' $schema $table $file

The user has been told to use -B on the export instead.

My guess is we need to create a table that will spit out at least 1MB of data once a quote is in the output so that there is enough data to blow the buffer.

Andrew Hutchings (Inactive) added a comment - 2018-01-29 16:32 Many thanks for the gdb output. Whilst I haven't been able to mock up a way of reproducing this yet it does look like the buffer is blown when scanning for the end enclosing character once one is found. The user is outputting data using: mysql --defaults-extra-file=<.cnf file> -h <server> --compress --quick --skip-column-names --default-character-set=utf8 -Ns -D $schema --execute="select * from $table" > $table.txt Then importing with: cpimport -E '\"' -e 1 -n1 -s '\t' $schema $table $file The user has been told to use -B on the export instead. My guess is we need to create a table that will spit out at least 1MB of data once a quote is in the output so that there is enough data to blow the buffer.

Andrew Hutchings (Inactive) made changes - 2018-02-16 13:39

Field	Original Value	New Value
Fix Version/s		1.1.4 [ 22918 ]

Andrew Hutchings (Inactive) made changes - 2018-02-16 13:39

Assignee

Andrew Hutchings [ linuxjedi ]

Andrew Hutchings (Inactive) made changes - 2018-02-24 12:06

Sprint

2018-05 [ 232 ]

Andrew Hutchings (Inactive) made changes - 2018-03-12 12:40

Sprint

2018-05 [ 232 ]

2018-05, 2018-06 [ 232, 234 ]

Andrew Hutchings (Inactive) made changes - 2018-03-26 07:17

Sprint

2018-05, 2018-06 [ 232, 234 ]

2018-05, 2018-06, 2018-07 [ 232, 234, 238 ]

Andrew Hutchings (Inactive) made changes - 2018-04-09 14:35

Sprint

2018-05, 2018-06, 2018-07 [ 232, 234, 238 ]

2018-05, 2018-06, 2018-07, 2018-08 [ 232, 234, 238, 242 ]

Andrew Hutchings (Inactive) made changes - 2018-04-16 08:47

Fix Version/s		1.1.5 [ 23013 ]
Fix Version/s	1.1.4 [ 22918 ]

Andrew Hutchings (Inactive) made changes - 2018-04-23 19:40

Sprint

2018-05, 2018-06, 2018-07, 2018-08 [ 232, 234, 238, 242 ]

2018-05, 2018-06, 2018-07, 2018-08, 2018-09 [ 232, 234, 238, 242, 244 ]

Andrew Hutchings (Inactive) made changes - 2018-05-05 05:40

Sprint

2018-05, 2018-06, 2018-07, 2018-08, 2018-09 [ 232, 234, 238, 242, 244 ]

2018-05, 2018-06, 2018-07, 2018-08, 2018-09, 2018-10 [ 232, 234, 238, 242, 244, 248 ]

Andrew Hutchings (Inactive) made changes - 2018-05-21 12:19

Sprint

2018-05, 2018-06, 2018-07, 2018-08, 2018-09, 2018-10 [ 232, 234, 238, 242, 244, 248 ]

2018-05, 2018-06, 2018-07, 2018-08, 2018-09, 2018-10, 2018-11 [ 232, 234, 238, 242, 244, 248, 252 ]

Andrew Hutchings (Inactive) made changes - 2018-05-21 18:17

Fix Version/s		1.1.6 [ 23121 ]
Fix Version/s	1.1.5 [ 23013 ]

Andrew Hutchings (Inactive) made changes - 2018-06-04 09:49

Sprint

2018-05, 2018-06, 2018-07, 2018-08, 2018-09, 2018-10, 2018-11 [ 232, 234, 238, 242, 244, 248, 252 ]

2018-05, 2018-06, 2018-07, 2018-08, 2018-09, 2018-10, 2018-11, 2018-12 [ 232, 234, 238, 242, 244, 248, 252, 256 ]

Andrew Hutchings (Inactive) made changes - 2018-06-18 07:38

Sprint

2018-05, 2018-06, 2018-07, 2018-08, 2018-09, 2018-10, 2018-11, 2018-12 [ 232, 234, 238, 242, 244, 248, 252, 256 ]

2018-05, 2018-06, 2018-07, 2018-08, 2018-09, 2018-10, 2018-11, 2018-12, 2018-13 [ 232, 234, 238, 242, 244, 248, 252, 256, 260 ]

Andrew Hutchings (Inactive) made changes - 2018-07-02 08:45

Sprint

2018-05, 2018-06, 2018-07, 2018-08, 2018-09, 2018-10, 2018-11, 2018-12, 2018-13 [ 232, 234, 238, 242, 244, 248, 252, 256, 260 ]

2018-05, 2018-06, 2018-07, 2018-08, 2018-09, 2018-10, 2018-11, 2018-12, 2018-13, 2018-14 [ 232, 234, 238, 242, 244, 248, 252, 256, 260, 263 ]

Andrew Hutchings (Inactive) made changes - 2018-07-20 13:07

Sprint

2018-05, 2018-06, 2018-07, 2018-08, 2018-09, 2018-10, 2018-11, 2018-12, 2018-13, 2018-14 [ 232, 234, 238, 242, 244, 248, 252, 256, 260, 263 ]

2018-05, 2018-06, 2018-07, 2018-08, 2018-09, 2018-10, 2018-11, 2018-12, 2018-13, 2018-14, 2018-15 [ 232, 234, 238, 242, 244, 248, 252, 256, 260, 263, 267 ]

Andrew Hutchings (Inactive) made changes - 2018-07-31 06:58

Fix Version/s		1.1.7 [ 23136 ]
Fix Version/s	1.1.6 [ 23121 ]

Andrew Hutchings (Inactive) made changes - 2018-08-06 15:01

Sprint

2018-05, 2018-06, 2018-07, 2018-08, 2018-09, 2018-10, 2018-11, 2018-12, 2018-13, 2018-14, 2018-15 [ 232, 234, 238, 242, 244, 248, 252, 256, 260, 263, 267 ]

2018-05, 2018-06, 2018-07, 2018-08, 2018-09, 2018-10, 2018-11, 2018-12, 2018-13, 2018-14, 2018-15, 2018-16 [ 232, 234, 238, 242, 244, 248, 252, 256, 260, 263, 267, 269 ]

Andrew Hutchings (Inactive) made changes - 2018-09-03 12:38

Sprint

2018-05, 2018-06, 2018-07, 2018-08, 2018-09, 2018-10, 2018-11, 2018-12, 2018-13, 2018-14, 2018-15, 2018-16 [ 232, 234, 238, 242, 244, 248, 252, 256, 260, 263, 267, 269 ]

2018-05, 2018-06, 2018-07, 2018-08, 2018-09, 2018-10, 2018-11, 2018-12, 2018-13, 2018-14, 2018-15, 2018-16, 2018-17 [ 232, 234, 238, 242, 244, 248, 252, 256, 260, 263, 267, 269, 276 ]

Andrew Hutchings (Inactive) made changes - 2018-09-24 10:15

Sprint

2018-05, 2018-06, 2018-07, 2018-08, 2018-09, 2018-10, 2018-11, 2018-12, 2018-13, 2018-14, 2018-15, 2018-16, 2018-17 [ 232, 234, 238, 242, 244, 248, 252, 256, 260, 263, 267, 269, 276 ]

2018-05, 2018-06, 2018-07, 2018-08, 2018-09, 2018-10, 2018-11, 2018-12, 2018-13, 2018-14, 2018-15, 2018-16, 2018-17, 2018-18 [ 232, 234, 238, 242, 244, 248, 252, 256, 260, 263, 267, 269, 276, 278 ]

David Thompson (Inactive) made changes - 2018-09-26 15:26

Assignee

Andrew Hutchings [ linuxjedi ]

Jens Röwekamp [ jens.rowekamp ]

Andrew Hutchings (Inactive) made changes - 2018-10-16 09:57

Sprint

2018-05, 2018-06, 2018-07, 2018-08, 2018-09, 2018-10, 2018-11, 2018-12, 2018-13, 2018-14, 2018-15, 2018-16, 2018-17, 2018-18 [ 232, 234, 238, 242, 244, 248, 252, 256, 260, 263, 267, 269, 276, 278 ]

2018-05, 2018-06, 2018-07, 2018-08, 2018-09, 2018-10, 2018-11, 2018-12, 2018-13, 2018-14, 2018-15, 2018-16, 2018-17, 2018-18, 2018-19 [ 232, 234, 238, 242, 244, 248, 252, 256, 260, 263, 267, 269, 276, 278, 283 ]

Andrew Hutchings (Inactive) made changes - 2018-11-05 09:38

Sprint

2018-05, 2018-06, 2018-07, 2018-08, 2018-09, 2018-10, 2018-11, 2018-12, 2018-13, 2018-14, 2018-15, 2018-16, 2018-17, 2018-18, 2018-19, 2018-20 [ 232, 234, 238, 242, 244, 248, 252, 256, 260, 263, 267, 269, 276, 278, 283, 288 ]

Andrew Hutchings (Inactive) made changes - 2018-12-10 15:04

Sprint

2018-05, 2018-06, 2018-07, 2018-08, 2018-09, 2018-10, 2018-11, 2018-12, 2018-13, 2018-14, 2018-15, 2018-16, 2018-17, 2018-18, 2018-19, 2018-20, 2018-21 [ 232, 234, 238, 242, 244, 248, 252, 256, 260, 263, 267, 269, 276, 278, 283, 288, 295 ]

Jens Röwekamp (Inactive) made changes - 2019-01-07 09:51

Resolution		Cannot Reproduce [ 5 ]
Status	Open [ 1 ]	Closed [ 6 ]

Jira Automation (IT) made changes - 2024-07-06 16:03

Zendesk Related Tickets

188888

People

Assignee:: Jens Röwekamp (Inactive)

Reporter:: Geoff Montee (Inactive)

Votes:: 0 Vote for this issue

Watchers:: 5 Start watching this issue

Dates

Created:: 2018-01-26 21:36

Updated:: 2024-07-08 02:45

Resolved:: 2019-01-07 09:51

Git Integration

Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.

MariaDB ColumnStore

Details

Description

Attachments

Activity

People

Dates

Git Integration