Jens Röwekamp (Inactive) added a comment - 2018-03-30 17:36 LinuxJedi could you have a look at the core dumps [1] that were produced by NodeJS 8 for "test_million_row_debug.js" and maybe figure out if it is a Swig or Bulk Write SDK bug? The first one (2125) quit with a segmentation fault after calling setColumn(): Core was generated by `node test_million_row_debug.js'. Program terminated with signal SIGSEGV, Segmentation fault. #0 mcsapi::ColumnStoreSystemCatalogTable::getColumn (this=0x29557e0, columnNumber=columnNumber@entry=1) at /home/jens/mariadb-columnstore-api/src/mcsapi_types.cpp:595 595 return *mImpl->columns[columnNumber]; [Current thread is 1 (Thread 0x7fb13c83ab80 (LWP 2125))] (gdb) bt #0 mcsapi::ColumnStoreSystemCatalogTable::getColumn (this=0x29557e0, columnNumber=columnNumber@entry=1) at /home/jens/mariadb-columnstore-api/src/mcsapi_types.cpp:595 #1 0x00007fb138f6cbe8 in mcsapi::ColumnStoreBulkInsert::setColumn (this=0x2990d10, columnNumber=1, value=970922, status=0x0) at /home/jens/mariadb-columnstore-api/src/mcsapi_bulk.cpp:122 #2 0x00007fb1391b3b5b in mcsapi::ColumnStoreBulkInsert::setColumn (this=0x2990d10, columnNumber=1, value=970922, status=0x0) at /usr/local/include/libmcsapi/mcsapi_bulk.h:50 #3 0x00007fb1391ab84f in _wrap_ColumnStoreBulkInsert_setColumn_int32 (args=...) at ../nodemcsapi_wrap.cxx:6703 #4 0x0000171738c0e0e7 in ?? () #5 0x00007fff0aa005d0 in ?? () #6 0x00007fff0aa00618 in ?? () #7 0x0000000000000002 in ?? () #8 0x0000171738c0dfc1 in ?? () #9 0x00007fff0aa00590 in ?? () #10 0x0000000000000006 in ?? () #11 0x00007fff0aa006d8 in ?? () #12 0x0000171738c0e926 in ?? () #13 0x00000fce08b8cb81 in ?? () #14 0x00000000028666a0 in ?? () #15 0x00002a1027f822d1 in ?? () #16 0x00002a1027f822d1 in ?? () #17 0x00002a1027f822d1 in ?? () #18 0x00000b33660616b9 in ?? () #19 0x00000fce08b8caf1 in ?? () #20 0x00002a1027f822d1 in ?? () #21 0x000ed0aa00000000 in ?? () #22 0x0000000100000000 in ?? () #23 0x00000fce08b8cb81 in ?? () #24 0x00000fce08b8cb81 in ?? () #25 0x0000719600000000 in ?? () #26 0x00000fce08b8c9e9 in ?? () #27 0x00000fce08b8cac1 in ?? () #28 0x0000000000007196 in ?? () #29 0x00000fce08b8caf1 in ?? () #30 0x0000000000000000 in ?? () The second one (2383) exited with a ColumnStoreUsageError that not all columns of this row have been filled, even though the log says that setColumn() was executed before writeRow(). Core was generated by `node test_million_row_debug.js'. Program terminated with signal SIGABRT, Aborted. #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51 51 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory. [Current thread is 1 (Thread 0x7f895bbd4b80 (LWP 2383))] (gdb) bt #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51 #1 0x00007f895a79642a in __GI_abort () at abort.c:89 #2 0x00007f895b2ca0ad in __gnu_cxx::__verbose_terminate_handler() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6 #3 0x00007f895b2c8066 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6 #4 0x00007f895b2c80b1 in std::terminate() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6 #5 0x00007f895b2c82c9 in __cxa_throw () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6 #6 0x00007f8958304a05 in mcsapi::ColumnStoreBulkInsert::writeRow (this=<optimized out>) at /home/jens/mariadb-columnstore-api/src/mcsapi_bulk.cpp:249 #7 0x00007f8958548ed0 in _wrap_ColumnStoreBulkInsert_writeRow (args=...) at ../nodemcsapi_wrap.cxx:7826 #8 0x0000137f0850dfc7 in ?? () #9 0x00007ffcba5691b0 in ?? () #10 0x00007ffcba5691e8 in ?? () #11 0x0000000000000000 in ?? () When I used custom debug statements within the bulk write sdk's setColumn() statements (cf. [2]), the bug somehow disappears. Therefore, I currently don't know how to proceed further with my analysis. You need to compile a special Swig version from [3] to produce the NodeJS 8 wrapper and reproduce the error. [1] https://drive.google.com/drive/folders/1yc7n_nOfm9iCALxRA6jHpMdnOIzpiq4i?usp=sharing [2] https://github.com/mariadb-corporation/mariadb-columnstore-api/tree/MCOL-1239_debug [3] https://github.com/meetnow/swig/tree/js-v8-52

Andrew Hutchings (Inactive) added a comment - 2018-03-30 18:19 Both of them give me the feeling that something is freeing ColumnStore's objects (or making copies of them) that shouldn't. If it is possible to run using valgrind that might give us more information. You could also: 1. set breakpoints in the destructors for the objects 2. add mcsdebug calls in the constructors/destructors.

Jens Röwekamp (Inactive) added a comment - 2018-04-09 18:39 LinuxJedi bulls-eye: ColumnStoreDriver gets destructed and causes the issue. Thanks. Seems to be the NodeJS equivalent of MCOL-1091 . MCOL-1091 's proposed temporary hotfix to call ColumnStoreDriver.getVersion() at the end of the ingestion process works. call writeRow 30341 call setColumn 0 30342 call setColumn 1 30342 call writeRow 30342 call setColumn 0 30343 call setColumn 1 30343 call writeRow 30343 call setColumn 0 30344 call setColumn 1 30344 call writeRow 30344 call setColumn 0 30345 call setColumn 1 30345 [mcsapi][11:30:35.716150] src/util_debug.cpp:48 Class 0x42513e0 ColumnStoreDriver destruct call [mcsapi][11:30:35.716180] src/util_debug.cpp:48 Class 0x4252290 ColumnStoreDriverImpl initialize call [mcsapi][11:30:35.716245] src/util_debug.cpp:48 Class 0x4252290 ColumnStoreDriverImpl initialize called [mcsapi][11:30:35.716250] src/util_debug.cpp:48 Class 0x42513e0 ColumnStoreDriver destruct called call writeRow 30345 terminate called after throwing an instance of 'mcsapi::ColumnStoreUsageError' what(): Not all the columns for this row have been filled Aborted

Jens Röwekamp (Inactive) added a comment - 2018-04-09 23:03 Unfortunately we can't apply the same fix as we did in MCOL-1091 . Swig doesn't offer an option to manually set references in the NodeJS module code as done in Java. Therefore, I currently see four options: 1. fix Swig's memory management for NodeJS 2. use the above mentioned temporary fix and wait until Swig's community fixes the bugs for us 3. write a wrapper ourselves without Swig 4. don't support NodeJS Regarding option 1: There are three open issues referring to garbage collection in Swig's GitHub repository, namely https://github.com/swig/swig/issues/317 https://github.com/swig/swig/issues/329 https://github.com/swig/swig/pull/343 which will make it a more complicated operation. This will probably take month. Deep knowledge of C++, NodeJS and Swig will be required. Regarding option 2: Issues 317, 329 and 343 were all filed in 2015 and haven't been updated since. The chances are good that we have to wait a couple of more years until a fix will be provided. Until a decision is made on how to proceed further, this ticket will be stalled.

Jens Röwekamp (Inactive) added a comment - 2018-04-16 19:26 - edited Contrary to my initial statement, Swigs NodeJS 6 implementation is also affected by the premature garbage collection bug experienced in NodeJS 8. If ColumnStoreDriver is used without try catch statements, in NodeJS 6 it is also premature garbage collected. A fix would be to change the garbage collection behaviour of ColumnStoreDriver in Swig's nodemcsapi_wrap.cxx file from SWIG_POINTER_OWN to 0. Then we need to take care of the collection on the C++ layer. I further tested the memory loss of a 1Million row test with Valgrind (mem-test full) with temporarily keeping ColumnStoreDriver in memory by calling .getVersion() at the end of the script and with above mentioned temporary solution, without freeing ColumnStoreDriver manually. The observed memory loss is quite high: language definitely lost indirectly lost possibly lost still reachable suppressed C++ 0 bytes in 0 blocks 0 bytes in 0 blocks 0 bytes in 0 blocks 1,170 bytes in 22 blocks 0 bytes in 0 blocks Java 3,789 bytes in 25 blocks 4,392 bytes in 15 blocks 3,360,522 bytes in 140 blocks 32,124,631 bytes in 204,942 blocks 0 bytes in 0 blocks Python 8 bytes in 1 blocks 3,402 bytes in 12 blocks 22,400,368 bytes in 199,995 blocks 554,204 bytes in 340 blocks 0 bytes in 0 blocks Python 3 0 bytes in 0 blocks 0 bytes in 0 blocks 8,424 bytes in 15 blocks 23,053,420 bytes in 200,652 blocks 0 bytes in 0 blocks Node 8 – version call 704,384 bytes in 22,005 blocks 8 bytes in 1 blocks 22,602,041 bytes in 203,138 blocks 99,908 bytes in 1,216 blocks 0 bytes in 0 blocks Node 8 – c memory handling 704,264 bytes in 22,002 blocks 181,669 bytes in 2,777 blocks 22,420,164 bytes in 200,358 blocks 100,084 bytes in 1,218 blocks 0 bytes in 0 blocks Node 6 – version call 255,008 bytes in 7,968 blocks 8 bytes in 1 blocks 22,580,933 bytes in 202,768 blocks 20,752 bytes in 84 blocks 0 bytes in 0 blocks Node 6 – c memory handling 254,728 bytes in 7,960 blocks 5,596 bytes in 57 blocks 22,573,369 bytes in 202,688 blocks 22,688 bytes in 106 blocks 0 bytes in 0 blocks

Todd Stoffel (Inactive) added a comment - 2021-01-17 07:14 Deprecated