[MDEV-18248] Multiple crashes on Galera 5-nodes after simple ALTER-statement Created: 2019-01-15 Updated: 2019-03-14 Resolved: 2019-03-14 |
|
| Status: | Closed |
| Project: | MariaDB Server |
| Component/s: | Data Definition - Alter Table, Galera, Storage Engine - InnoDB |
| Affects Version/s: | 10.2.14 |
| Fix Version/s: | N/A |
| Type: | Bug | Priority: | Major |
| Reporter: | Martijn Keijzer | Assignee: | Unassigned |
| Resolution: | Incomplete | Votes: | 1 |
| Labels: | crash, galera, innodb, need_feedback | ||
| Environment: |
RedHat 6.10 |
||
| Description |
|
While update a simple table with an simple alter statement the 5-node multimaster Galera stops working an the table gets corrupted. This has happened several times nog with different tables and simple alterstatements (without triggers). The errorlog shows this: 2019-01-15 10:47:19 140487941920512 [Note] WSREP: Member 1.0 (elavsu6211.company.local) synced with group. schema: databasename InnoDB: Failing assertion: table->get_ref_count() == 0 190115 12:24:40 [ERROR] mysqld got signal 6 ; To report this bug, see https://mariadb.com/kb/en/reporting-bugs We will try our best to scrape up some info that will hopefully help It is possible that mysqld could use up to Thread pointer: 0x7fbe2d906c18 Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query (0x7fbe2d9141f0): is an invalid pointer Connection ID (thread ID): 8227134 Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_push$ The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains Query: ALTER TABLE `subject_bpztools_aagenda` ADD `id_subject_cat` int(11) NULL DEFAULT '0' AFTER `id_subject`, ADD INDEX `id_subject_cat` (`id_subject_cat`) |
| Comments |
| Comment by Elena Stepanova [ 2019-01-15 ] |
|
We have an open issue (general, not Galera-specific) with this assertion failure: martijnk, is it possible that your instances could be running two parallel ALTER TABLE at the same time? (Asking because that's what our test case in |
| Comment by Martijn Keijzer [ 2019-01-15 ] |
|
We have only a very critical production-environment and an extrabackup, for me it is not possible to do other tests. The same as in An ALTER works for me now whe I stop mysql on 4 of the 5 nodes and do the changes at the first node and then connect the 4 nodes after the alters... I a previous (other) setup of my databasesystem no problems did occur (Linux Redhat 6, 3 node cluster) an d I did several alters in a week on the cluster. Other dan in |
| Comment by Elena Stepanova [ 2019-02-14 ] |
|
Sorry for not being clear, I didn't suggest you to run the tests, I was asking if it's possible that your instance was running two parallel ALTER TABLE around the time of failure. It would indicate possible relation to the above-mentioned bug For not being able to re-create or drop table, it's an unfortunate known result of DDL not being completely crash-safe. When your instance happens to crash at a certain stage of performing DDL, it can indeed leave the table in a half-dead state. Maybe marko can suggest how to get rid of the remains. |