[MCOL-4130] ExeMgr segfaults when its connection gets a SIGPIPE Created: 2020-06-30  Updated: 2023-10-25  Resolved: 2023-10-25

Status: Closed
Project: MariaDB ColumnStore
Component/s: PrimProc
Affects Version/s: None
Fix Version/s: Icebox

Type: Task Priority: Minor
Reporter: Daniel Black Assignee: Unassigned
Resolution: Won't Fix Votes: 0
Labels: None

Issue Links:
PartOf
is part of MCOL-4134 Clean and fix remaining columnstore c... Closed

 Description   

(gdb) r
Starting program: /usr/bin/ExeMgr 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/powerpc64le-linux-gnu/libthread_db.so.1".
[New Thread 0x7ffff37ff160 (LWP 25650)]
[New Thread 0x7ffff2fef160 (LWP 25651)]
[New Thread 0x7ffff24ff160 (LWP 25652)]
[New Thread 0x7ffff1cef160 (LWP 25654)]
[New Thread 0x7ffff148f160 (LWP 25656)]
Starting ExeMgr: st = 50, qs = 20, mx = 95, cf = /etc/columnstore/Columnstore.xml
 
Thread 1 "ExeMgr" received signal SIGPIPE, Broken pipe.
0x00007ffff7f28464 in __libc_write (fd=<optimised out>, buf=0x100115b20, nbytes=13) at ../sysdeps/unix/sysv/linux/write.c:27
27	../sysdeps/unix/sysv/linux/write.c: No such file or directory.
(gdb) bt
#0  0x00007ffff7f28464 in __libc_write (fd=<optimised out>, buf=0x100115b20, nbytes=13) at ../sysdeps/unix/sysv/linux/write.c:27
#1  0x00007ffff6aaeb80 in messageqcpp::InetStreamSocket::written (this=<optimised out>, fd=<optimised out>, ptr=<optimised out>, nbytes=13)
    at ./storage/columnstore/columnstore/utils/messageqcpp/inetstreamsocket.cpp:1060
#2  0x00007ffff6aaf864 in messageqcpp::InetStreamSocket::do_write (this=<optimised out>, msg=<optimised out>, 
    whichMagic=<error reading variable: value has been optimised out>, stats=0x0) at ./storage/columnstore/columnstore/utils/messageqcpp/inetstreamsocket.cpp:704
#3  0x00007ffff6aafadc in messageqcpp::InetStreamSocket::write (this=<optimised out>, msg=<optimised out>, stats=<optimised out>)
    at ./storage/columnstore/columnstore/utils/messageqcpp/inetstreamsocket.cpp:719
#4  0x00007ffff6ab925c in messageqcpp::CompressedInetStreamSocket::write (this=0x10013fbb0, msg=
      @0x7fffffffdc00: {<messageqcpp::Serializeable> = {_vptr.Serializeable = 0x7ffff6aeef70 <vtable for messageqcpp::ByteStream+16>}, static BlockSize = 4096, static ISSOverhead = 8, fBuf = 0x100115b20 "7\301\373\024\005", fCurInPtr = 0x100115b2d "\177", fCurOutPtr = 0x100115b28 "7@", fMaxLen = 8192}, stats=0x0)
    at ./storage/columnstore/columnstore/utils/messageqcpp/compressed_iss.cpp:127
#5  0x00007ffff6a971fc in messageqcpp::IOSocket::write (stats=0x0, msg=
      @0x7fffffffdc00: {<messageqcpp::Serializeable> = {_vptr.Serializeable = 0x7ffff6aeef70 <vtable for messageqcpp::ByteStream+16>}, static BlockSize = 4096, static ISSOverhead = 8, fBuf = 0x100115b20 "7\301\373\024\005", fCurInPtr = 0x100115b2d "\177", fCurOutPtr = 0x100115b28 "7@", fMaxLen = 8192}, this=0x10013ebe8)
    at ./storage/columnstore/columnstore/utils/messageqcpp/iosocket.h:250
#6  messageqcpp::MessageQueueClient::write (this=0x10013ebb0, msg=
      @0x7fffffffdc00: {<messageqcpp::Serializeable> = {_vptr.Serializeable = 0x7ffff6aeef70 <vtable for messageqcpp::ByteStream+16>}, static BlockSize = 4096, static ISSOverhead = 8, fBuf = 0x100115b20 "7\301\373\024\005", fCurInPtr = 0x100115b2d "\177", fCurOutPtr = 0x100115b28 "7@", fMaxLen = 8192}, timeout=<optimised out>, 
    stats=0x0) at ./storage/columnstore/columnstore/utils/messageqcpp/messagequeue.cpp:333
#7  0x00007ffff6d80750 in BRM::DBRM::send_recv (this=0x100121220, in=
      @0x7fffffffdc00: {<messageqcpp::Serializeable> = {_vptr.Serializeable = 0x7ffff6aeef70 <vtable for messageqcpp::ByteStream+16>}, static BlockSize = 4096, static ISSOverhead = 8, fBuf = 0x100115b20 "7\301\373\024\005", fCurInPtr = 0x100115b2d "\177", fCurOutPtr = 0x100115b28 "7@", fMaxLen = 8192}, out=
      @0x7fffffffdc28: {<messageqcpp::Serializeable> = {_vptr.Serializeable = 0x7ffff6aeef70 <vtable for messageqcpp::ByteStream+16>}, static BlockSize = 4096, static ISSOverhead = 8, fBuf = 0x100117b30 "\030\025\342\365\377\177", fCurInPtr = 0x100117b38 "\030\025\342\365\377\177", fCurOutPtr = 0x100117b38 "\030\025\342\365\377\177", fMaxLen = 8192}) at ./storage/columnstore/columnstore/versioning/BRM/dbrm.cpp:877
#8  0x00007ffff6d88c34 in BRM::DBRM::setSystemState (this=0x100121220, stateFlags=<optimised out>) at ./storage/columnstore/columnstore/versioning/BRM/dbrm.cpp:3644
#9  0x00007ffff6d89234 in BRM::DBRM::setSystemQueryReady (this=<optimised out>, bReady=<optimised out>)
    at ./storage/columnstore/columnstore/versioning/BRM/dbrm.cpp:3497
#10 0x0000000100014a4c in main (argc=<optimised out>, argv=<optimised out>) at ./storage/columnstore/columnstore/exemgr/main.cpp:1655
(gdb) c
Continuing.
 
Thread 1 "ExeMgr" received signal SIGSEGV, Segmentation fault.
__memcpy_power7 () at ../sysdeps/powerpc/powerpc64/power7/memcpy.S:378
378	../sysdeps/powerpc/powerpc64/power7/memcpy.S: No such file or directory.
(gdb) bt
#0  __memcpy_power7 () at ../sysdeps/powerpc/powerpc64/power7/memcpy.S:378
#1  0x00007ffff6aab204 in memcpy (__len=<optimised out>, __src=<optimised out>, __dest=<optimised out>)
    at /usr/include/powerpc64le-linux-gnu/bits/string_fortified.h:34
#2  messageqcpp::ByteStream::doCopy (this=0x7fffffffdc28, rhs=
      @0x7fffffffd860: {<messageqcpp::Serializeable> = {_vptr.Serializeable = 0x7fffffffd9a0}, static BlockSize = 4096, static ISSOverhead = 8, fBuf = 0x7fff22002222 "", fCurInPtr = 0x7ffff5cb085c <_int_malloc+4748> "y\033o|P\373\202A", fCurOutPtr = 0x7ffff6af7a00 "", fMaxLen = 4294958120})
    at ./storage/columnstore/columnstore/utils/messageqcpp/bytestream.cpp:58
#3  0x00007ffff6aab40c in messageqcpp::ByteStream::operator= (this=<optimised out>, rhs=<optimised out>)
    at ./storage/columnstore/columnstore/utils/messageqcpp/bytestream.cpp:83
#4  0x00007ffff6d80784 in messageqcpp::ByteStream::operator= (rhs=@0x7fffffffdb28: {px = 0x7fffffffd860, pn = {pi_ = 0x7fffffffd9d0}}, this=<optimised out>)
    at ./storage/columnstore/columnstore/utils/messageqcpp/bytestream.h:548
#5  BRM::DBRM::send_recv (this=0x100121220, in=
      @0x7fffffffdc00: {<messageqcpp::Serializeable> = {_vptr.Serializeable = 0x7ffff6aeef70 <vtable for messageqcpp::ByteStream+16>}, static BlockSize = 4096, static ISSOverhead = 8, fBuf = 0x100115b20 "7\301\373\024\005", fCurInPtr = 0x100115b2d "\177", fCurOutPtr = 0x100115b28 "7@", fMaxLen = 8192}, out=
      @0x7fffffffdc28: {<messageqcpp::Serializeable> = {_vptr.Serializeable = 0x7ffff6aeef70 <vtable for messageqcpp::ByteStream+16>}, static BlockSize = 4096, static ISSOverhead = 8, fBuf = 0x7ffee4060010 "", fCurInPtr = 0x100117b38 "\030\025\342\365\377\177", fCurOutPtr = 0x100117b38 "\030\025\342\365\377\177", fMaxLen = 4294562012}) at ./storage/columnstore/columnstore/versioning/BRM/dbrm.cpp:878
#6  0x00007ffff6d88c34 in BRM::DBRM::setSystemState (this=0x100121220, stateFlags=<optimised out>) at ./storage/columnstore/columnstore/versioning/BRM/dbrm.cpp:3644
#7  0x00007ffff6d89234 in BRM::DBRM::setSystemQueryReady (this=<optimised out>, bReady=<optimised out>)
    at ./storage/columnstore/columnstore/versioning/BRM/dbrm.cpp:3497
#8  0x0000000100014a4c in main (argc=<optimised out>, argv=<optimised out>) at ./storage/columnstore/columnstore/exemgr/main.cpp:1655



 Comments   
Comment by Roman [ 2020-07-13 ]

danblack Could you give more details on how to reproduce this w/o artificially sending a SIGPIPE to ExeMgr.

Comment by Daniel Black [ 2020-07-17 ]

I didn't artificially send the SIGPIPE. I suspect the process it was connected with died. I was hoping the stacktrace made it obvious to you what it was connected to (because I didn't attempt to identify it).

I was mainly just looking for a cleaner recovery of errors.

Comment by Roman [ 2020-07-20 ]

We'll handle the case according with the priority.

Generated at Thu Feb 08 02:48:02 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.