[MXS-414] Maxscale crashed every day! Created: 2015-10-17 Updated: 2015-11-24 Resolved: 2015-11-24 |
|
| Status: | Closed |
| Project: | MariaDB MaxScale |
| Component/s: | Core |
| Affects Version/s: | 1.2.0, 1.2.1 |
| Fix Version/s: | 1.3.0 |
| Type: | Bug | Priority: | Blocker |
| Reporter: | cai sunny | Assignee: | Johan Wikman |
| Resolution: | Fixed | Votes: | 1 |
| Labels: | None | ||
| Environment: |
Linux 2.6.32-573.3.1.el6.x86_64 #1 SMP Thu Aug 13 22:55:16 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux |
||
| Attachments: |
|
||||||||||||||||
| Issue Links: |
|
||||||||||||||||
| Description |
|
Maxscale crashed every day! The error message like below: |
| Comments |
| Comment by cai sunny [ 2015-10-17 ] | ||||||||||||||||
|
Please help check how to find the root cause. | ||||||||||||||||
| Comment by markus makela [ 2015-10-19 ] | ||||||||||||||||
|
Is there any specific types of queries you were doing through MaxScale when this happened? Did this crash happen at the same time when the master server went down? This seems like | ||||||||||||||||
| Comment by cai sunny [ 2015-10-19 ] | ||||||||||||||||
|
It happened on different time. | ||||||||||||||||
| Comment by markus makela [ 2015-10-23 ] | ||||||||||||||||
|
Enabling the trace log with maxadmin enable log trace should show which queries are being routed. | ||||||||||||||||
| Comment by Michaël de groot [ 2015-11-04 ] | ||||||||||||||||
|
I experienced this as well. Please find the (full, anonimized) error log and (partial, anonimized) trace log attached. Please keep in mind the full and original logs are sent to you in our internal channel as well. | ||||||||||||||||
| Comment by Johan Wikman [ 2015-11-05 ] | ||||||||||||||||
|
Interpreted stacktrace: /home/ec2-user/workspace/server/core/gateway.c:362 | ||||||||||||||||
| Comment by Johan Wikman [ 2015-11-05 ] | ||||||||||||||||
|
The relevant code is:
The crash occurs on line 1704. Since it was checked on line 1695 that master_dcb and master_dcb->session are non-NULL, there are basically two options:
The former would require that not only has the session object pointed to by master_dcb->session been freed, without master_dcb->session having been set to NULL, but also that the memory would have been reclaimed by the operating system. That seems unlikely. So, the likely reason is that master_dcb->session has been set to NULL by another thread after the check was made on line 1695, but before master_dcb->session->data is accessed on line 1704. This is the kind of issue that the fix of | ||||||||||||||||
| Comment by Johan Wikman [ 2015-11-24 ] | ||||||||||||||||
|
I will close this now since the expection is that |