[MXS-335] Crash in readwritesplit Created: 2015-08-26  Updated: 2015-09-17  Resolved: 2015-08-31

Status: Closed
Project: MariaDB MaxScale
Component/s: mariadbbackend, readwritesplit
Affects Version/s: 1.2.0
Fix Version/s: 1.2.1

Type: Bug Priority: Blocker
Reporter: markus makela Assignee: Johan Wikman
Resolution: Fixed Votes: 0
Labels: None
Environment:

Binaries:
http://maxscale-jenkins.mariadb.com/repo/1.2.0-patch/centos6.5_x86_64/
Commit: 337249291b20176484f434bd7f635d7126619066


 Description    

2015-08-25 23:22:59   Fatal: MaxScale received fatal signal 11. Attempting backtrace.
 
2015-08-25 23:22:59     /usr/bin/maxscale() [0x5238ab] 
2015-08-25 23:22:59     /lib64/libpthread.so.0() [0x3fd140f710] 
2015-08-25 23:22:59     /usr/lib64/maxscale/libreadwritesplit.so(+0x862e) [0x7f9fa454b62e] 
2015-08-25 23:22:59     /usr/lib64/maxscale/libMySQLBackend.so(+0x5077) [0x7f9f90337077] 
2015-08-25 23:22:59     /usr/bin/maxscale() [0x539129] 
2015-08-25 23:22:59     /usr/bin/maxscale(poll_waitevents+0x634) [0x5389e8] 
2015-08-25 23:22:59     /lib64/libpthread.so.0() [0x3fd14079d1] 
2015-08-25 23:22:59     /lib64/libc.so.6(clone+0x6d) [0x3fd10e8b6d]

which corresponds to:

/home/ec2-user/workspace/server/core/gateway.c:358
 
/lib64/libpthread.so.0() [0x3fd140f710]
 
/home/ec2-user/workspace/server/modules/routing/readwritesplit/readwritesplit.c:2754
 
/home/ec2-user/workspace/server/modules/protocol/mysql_backend.c:568
 
/home/ec2-user/workspace/server/core/poll.c:878
 
/home/ec2-user/workspace/server/core/poll.c:610
 
/lib64/libpthread.so.0() [0x3fd14079d1]
 
/lib64/libc.so.6(clone+0x6d) [0x3fd10e8b6d]

Crash happens when the GWBUF is accessed on line 2754 of readwritesplit.c.

 Comments   
Comment by Dipti Joshi (Inactive) [ 2015-08-27 ]

johan.wikman This is a new blocker issue.

Comment by Johan Wikman [ 2015-08-27 ]

Assuming the crash occurred because writebuf was NULL at line 2754 in function clientReply in readwritesplit.c, it must have been NULL when passed in to clientReply. It may be set to NULL in clientReply, but then the crash line is not executed.

In the caller - gw_read_backend_event in mysql_backend.c - the buffer is called read_buf and it can be NULL, if process_response_data on line 533 returns NULL without dcb being marked as complete.

If the processing of a response is aborted in mysql_backend.c@process_responsedata due to a packet being incomplete, it must be ensured that it also appears that way after the return from the function, so that the processing is also aborted in mysql_backend.c@gw_read_backend_event

Comment by Johan Wikman [ 2015-08-27 ]

A fix has now been made and is being regression tested.

Comment by Johan Wikman [ 2015-08-31 ]

Crash not repeated, but likely cause found by code review. Fix made, and change regression tested.

Comment by Guillaume Lefranc [ 2015-08-31 ]

Crash repeated in production, core dump files coming up.

Generated at Thu Feb 08 03:58:31 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.