[MCOL-1245] getting A fatal error in bulkinsert mariadb columnstore java api Created: 2018-03-06  Updated: 2023-10-26  Resolved: 2018-03-23

Status: Closed
Project: MariaDB ColumnStore
Component/s: None
Affects Version/s: 1.1.3
Fix Version/s: 1.1.4

Type: Bug Priority: Major
Reporter: Paresh B Assignee: Daniel Lee (Inactive)
Resolution: Fixed Votes: 0
Labels: Mariadb-Columnstore
Environment:

OS:Red Hat Enterprise Linux Server release 7.4 (Maipo)

Attachments: Zip Archive MCOL-1245.zip    
Issue Links:
Duplicate
is duplicated by MCOL-1295 Python bulk load API encounters error... Closed
Sprint: 2018-06

 Description   

I am trying to use MariaDB ColumnStore Bulk Write SDK in Java to bulk insert 10 rows in a table with 50 columns(data type int) but it never finished had to kill the process in 10 -15 tried couple times but same results and now I am getting following error.

this is the code sample from their doc https://mariadb.com/kb/en/library/columnstore-bulk-write-sdk/#rhel-centos-7

import com.mariadb.columnstore.api.*;

public class MCSAPITest {

public static void main(String[] args) {
ColumnStoreDriver d = new ColumnStoreDriver();
ColumnStoreBulkInsert b = d.createBulkInsert("test", "t1", (short)0, 0);
try {
int rows = 10;
int columns = 50;
for (int i = 0; i < rows; ++i) {
for (int c = 0; c < columns; ++c)

{ b.setColumn(c, 0); }

System.out.println("Writing row: " + i);
b.writeRow();
}
System.out.println("Commiting Now");
b.commit();
}
catch (ColumnStoreException e)

{ b.rollback(); e.printStackTrace(); }

}
}

A fatal error has been detected by the Java Runtime Environment:
#

  1. SIGSEGV (0xb) at pc=0x00007f73b32a49ea, pid=17005, tid=0x00007f73e20f2700
    #
  2. JRE version: Java(TM) SE Runtime Environment (8.0_121-b13) (build 1.8.0_121-b13)
  3. Java VM: Java HotSpot(TM) 64-Bit Server VM (25.121-b13 mixed mode linux-amd64 compressed oops)
  4. Problematic frame:
  5. C [libmcsapi.so.1+0xc9ea] mcsapi::ColumnStoreSystemCatalogColumn::ColumnStoreSystemCatalogColumn(mcsapi::ColumnStoreSystemCatalogColumn const&)+0xaa
    #
  6. Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
    #
  7. If you would like to submit a bug report, please visit:
  8. http://bugreport.java.com/bugreport/crash.jsp
  9. The crash happened outside the Java Virtual Machine in native code.
  10. See problematic frame for where to report the bug.

 Comments   
Comment by Andrew Hutchings (Inactive) [ 2018-03-07 ]

Hi,

In your ticket description it states you are running 1.0.13. Is this the case? The connectors are not compatible with this version, they require 1.1.x (1.1.3 recommended).

Comment by Paresh B [ 2018-03-07 ]

Sorry that was typo. This is 1.1.3. This is the rpm used mariadb-columnstore-api-1.1.3-1-x86_64-centos7.rpm

Comment by Andrew Hutchings (Inactive) [ 2018-03-07 ]

And the MariaDB server is definitely 1.1.3 too?

Comment by Paresh B [ 2018-03-07 ]

Yes
Name : mariadb-columnstore-platform
Version : 1.1.3
Release : 1
Architecture: x86_64
Install Date: Thu 01 Mar 2018 12:38:24 PM EST

Comment by Jens Röwekamp (Inactive) [ 2018-03-07 ]

Compiled and tested above code on Ubuntu 16.04 with CS 1.1.3-1 and Bulk API 1.1.3, and CentOS 7 with CS 1.1.3-1 and 1.1.3-2, and Bulk API 1.1.3.

Compilation, execution and ingestion successful without any errors.

Without more information I won't be able to reproduce the error and help.

Could you provide me with the java file you tried to execute, a core dump of ColumnStore during execution, information about the java compiler you used, and a core dump of the java error produced?

Comment by David Thompson (Inactive) [ 2018-03-09 ]

i'd also double check some basics, if you are running the sample on a non columnstore node server, make sure that you have copied the right ColumnStore.xml locally to /usr/local/mariadb/columnstore/etc and that server can reach the columnstore pm nodes.

Comment by Paresh B [ 2018-03-09 ]

Yes I have copied ColumnStore.xml locally to /usr/local/mariadb/columnstore/etc and server can rach pm nodes, In fact it works sometime, here are some test results. I have enabled to core dump at maria but there is no core dump created. I do have java core dump though, but its pretty large how do I upload it here?

sr # Rows Columns Total duration for bulk insert (Sec)
1 20 50 257
3 20 50 30
2 20000 50 122
4 200000 50 53
5 1-100 500 not inserted yet

Comment by Jens Röwekamp (Inactive) [ 2018-03-09 ]

Could you send the files via ftp to ftp://ftp.mariadb.com/uploads ? The directory is write only so you won't be able to see the files once uploaded.

Comment by Paresh B [ 2018-03-09 ]

uploaded.

Comment by Paresh B [ 2018-03-09 ]

sometimes I am getting this error if this helps.
Error during mcsapi write operations: com.mariadb.columnstore.api.ColumnStoreException: Compressed data corruption

Comment by Jens Röwekamp (Inactive) [ 2018-03-16 ]

Short status update:

Can reproduce the error on a setup with a remote CS server and the Java code executed from a different machine. In this case CS was running on CentOS 7 and the Bulk Write API was running on Debian 9.

Can't reproduce on a single node installation where CS and the Bulk Write API are executed on the same machine.

Increased the number of rows to 50 and number of columns to 500.

Will investigate further.

Comment by Jens Röwekamp (Inactive) [ 2018-03-16 ]

Is a bug in the C++ API. Transferred the example to Python and C++ (attached) MCOL-1245.zip and they produce reproducible a "segmentation fault".

Will move ticket to LinuxJedi, as he has more C++ expertise than I have.

Comment by Andrew Hutchings (Inactive) [ 2018-03-19 ]

The decompression buffer has a double free problem when it has to be used more than once. The key to triggering it is a remote connection with lots of columns as it causes the HWM and system catalog packets to be large enough to both trigger compression.

Comment by Andrew Hutchings (Inactive) [ 2018-03-19 ]

Patch fixes the double-free problem.

For QA:
1. You need a 1UM/PM server
2. You need another machine with network access to the ColumnStore server for the API.
3. Decompress MCOL-1245.zip and compile it (for the C++ test, Python and Java available). This will be something like:

g++ MCSAPITest.cxx -o MCSAPITest -std=c++11 `pkg-config libmcsapi --cflags --libs`

4. mcsmysql test < t1.sql
5. Copy the ColumnStore server's Columnstore.xml file to Columnstore_CentOS.xml in the directory as the source files for the test
6. Edit that file and change all instances of '127.0.0.1' to the IP of ColumnStore server
7. Run the executable generated

Before the fix this will crash towards the end.

Comment by Daniel Lee (Inactive) [ 2018-03-23 ]

Build verified: 1.1.4-1 source

/root/columnstore/mariadb-columnstore-server
commit 3b5242143b394423dbdf96d888410a3c33f9ff97
Merge: b7d93b7 23900e6
Author: benthompson15 <ben.thompson@mariadb.com>
Date: Wed Mar 7 10:39:40 2018 -0600

Merge pull request #104 from mariadb-corporation/davidhilldallas-patch-3

update version

/root/columnstore/mariadb-columnstore-server/mariadb-columnstore-engine
commit 4e167285c6665c9aa0e204eac2de0add61709be4
Merge: 819c69c b800fc7
Author: benthompson15 <ben.thompson@mariadb.com>
Date: Wed Mar 14 13:25:21 2018 -0500

Merge pull request #427 from mariadb-corporation/MCOL-1225

Mcol 1225 - fix mysql/lib library path

[root@localhost mariadb-columnstore-api]# git show
commit 87f624263dd78d261c35395392a676597081168f
Merge: 595c74d 2e2a0f8
Author: sneJ p <sneJ-@users.noreply.github.com>
Date: Mon Mar 19 10:34:09 2018 -0700

Merge pull request #60 from mariadb-corporation/MCOL-1245

MCOL-1245 Fix decompression double-free

Platform: centos 7
I tried to reproduce the issue using the python script on the 1.1.3-1 API. The test did not crash for me. Instead, it hung when committing after bulk insert. Python was utilizing 100% CPU and the commit never finished. Same behavior observed for api 1.1.3-1 running against Columnstore 1.1.3-1 and 1.1.4-1.

API 1.1.4-1 worked on both Columnstore 1.1.3-1 and 1.1.4-1.

Comment by Christoffer [ 2018-06-12 ]

Although the ticket is in closed state and clearly states the fix is included in 1.1.4, I would just like to confirm that it indeed works, tested through Python.

Thank you for the fix

Generated at Thu Feb 08 02:27:18 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.