Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-17334

Crash on prepared SELECT statement

    XMLWordPrintable

Details

    Description

      A three nodes Galera cluster behind MaxScale routing gets all three nodes crashed on a prepared statement (SELECT).
      The application is in Java and uses mysql-connector-java-5.1.45.jar.

      First node crashes at `11:57:51` and the other two at `11:57:58`

      What I still don't understand is why Node 1 and 3 crash at the same time, I'd expect a bit of delay between the two.
      To be noted that MaxScale (afaik) pre-sends prepared statements to all nodes.

      Server version: 10.3.9-MariaDB-1:10.3.9+maria~xenial-log

      _THE SELECT spotted to be causing the crash_

      SELECT
      	t.id,
      	t.org_id,
      	t.int_id,
      	t.ext_id,
      	t.ext_name,
      	t.type,
      	t.flags,
      	t.aut,
      	t.ins
      FROM
      	myTableName t
      WHERE
      	t.org_id = 9
      AND
      	type = ?
      AND
      	int_id = ?
      

      MAXSCALE (first routes the query to Node2, then it tries Node1 and Node3 apparently)

      _NODE 2 CRASH_

      2018-09-28 11:57:53   error  : Monitor was unable to connect to server [192.168.12.-92-]:3306 : "Can't connect to MySQL server on '192.168.12.-92-' (115)"
      2018-09-28 11:57:53   notice : Server changed state: devdb2[192.168.12.92:3306]: slave_down. [Slave, Synced, Running] -> [Down]
      
      

      _NODE 1 & 3 CRASH_

      2018-09-28 11:57:59   error  : Monitor was unable to connect to server [192.168.12.-91-]:3306 : "Lost connection to MySQL server at 'handshake: reading initial communication packet', system error: 107"
      2018-09-28 11:57:59   error  : Monitor was unable to connect to server [192.168.12.-93-]:3306 : "Can't connect to MySQL server on '192.168.12.-93-' (115)"
      2018-09-28 11:57:59   error  : [galeramon] There are no cluster members
      2018-09-28 11:57:59   notice : Server changed state: devdb1[192.168.12.-91-:3306]: master_down. [Master, Synced, Running] -> [Down]
      2018-09-28 11:57:59   notice : Server changed state: devdb3[192.168.12.-93-:3306]: slave_down. [Slave, Synced, Running] -> [Down]
      

      _ONE STACK TRACE (from Node 2):_

      Sep 28 11:57:51 galera2 mysqld[21934]: terribly wrong...
      Sep 28 11:57:51 galera2 mysqld[21934]: stack_bottom = 0x7f7b0ab46cf8 thread_stack 0x49000
      Sep 28 11:57:51 galera2 mysqld[21934]: /usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x561af1385ace]
      Sep 28 11:57:51 galera2 mysqld[21934]: /usr/sbin/mysqld(handle_fatal_signal+0x347)[0x561af0e2b8f7]
      Sep 28 11:57:51 galera2 mysqld[21934]: /lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7f7c628d5390]
      Sep 28 11:57:51 galera2 mysqld[21934]: /usr/sbin/mysqld(append_query_string(charset_info_st const*, String*, char const*, unsigned long, bool)+0x5d)[0x561af0f0d6fd]
      Sep 28 11:57:51 galera2 mysqld[21934]: /usr/sbin/mysqld(Item_param::value_query_val_str(THD*, String*) const+0xac)[0x561af0e4a8bc]
      Sep 28 11:57:51 galera2 mysqld[21934]: /usr/sbin/mysqld(Item_param::append_for_log(THD*, String*)+0x63)[0x561af0e4ac43]
      Sep 28 11:57:51 galera2 mysqld[21934]: /usr/sbin/mysqld(+0x58c7d1)[0x561af0c647d1]
      Sep 28 11:57:51 galera2 mysqld[21934]: /usr/sbin/mysqld(Prepared_statement::set_parameters(String*, unsigned char*, unsigned char*)+0x6d)[0x561af0c67e9d]
      Sep 28 11:57:51 galera2 mysqld[21934]: /usr/sbin/mysqld(Prepared_statement::execute_loop(String*, bool, unsigned char*, unsigned char*)+0x36)[0x561af0c6a376]
      Sep 28 11:57:51 galera2 mysqld[21934]: /usr/sbin/mysqld(+0x592f48)[0x561af0c6af48]
      Sep 28 11:57:51 galera2 mysqld[21934]: /usr/sbin/mysqld(mysqld_stmt_execute(THD*, char*, unsigned int)+0x25)[0x561af0c6b085]
      Sep 28 11:57:51 galera2 mysqld[21934]: /usr/sbin/mysqld(dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool)+0x14a0)[0x561af0c581a0]
      Sep 28 11:57:51 galera2 mysqld[21934]: /usr/sbin/mysqld(do_command(THD*)+0x13c)[0x561af0c5944c]
      Sep 28 11:57:51 galera2 mysqld[21934]: /usr/sbin/mysqld(do_handle_one_connection(CONNECT*)+0x252)[0x561af0d29fd2]
      Sep 28 11:57:51 galera2 mysqld[21934]: /usr/sbin/mysqld(handle_one_connection+0x3d)[0x561af0d2a14d]
      Sep 28 11:57:51 galera2 mysqld[21934]: /lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba)[0x7f7c628cb6ba]
      Sep 28 11:57:51 galera2 mysqld[21934]: /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7f7c61f7641d]
      

      _THE THREE STACK TRACES ORDERED BY POSITION:_

      Sep 28 11:57:58 galera3 mysqld[]: /usr/sbin/mysqld(+0x58c7d1)[0x555e956dc7d1]
      Sep 28 11:57:58 galera1 mysqld[]: /usr/sbin/mysqld(+0x58c7d1)[0x55aa91c7b7d1]
      Sep 28 11:57:51 galera2 mysqld[]: /usr/sbin/mysqld(+0x58c7d1)[0x561af0c647d1]
      Sep 28 11:57:51 galera2 mysqld[]: /usr/sbin/mysqld(append_query_string(charset_info_st const*, String*, char const*, unsigned long, bool)+0x5d)[0
      Sep 28 11:57:58 galera1 mysqld[]: /usr/sbin/mysqld(append_query_string(charset_info_st const*, String*, char const*, unsigned long, bool)+0x5d)[0
      Sep 28 11:57:58 galera3 mysqld[]: /usr/sbin/mysqld(append_query_string(charset_info_st const*, String*, char const*, unsigned long, bool)+0x5d)[0x
      Sep 28 11:57:58 galera3 mysqld[]: /usr/sbin/mysqld(Item_param::append_for_log(THD*, String*)+0x63)[0x555e958c2c43]
      Sep 28 11:57:58 galera1 mysqld[]: /usr/sbin/mysqld(Item_param::append_for_log(THD*, String*)+0x63)[0x55aa91e61c43]
      Sep 28 11:57:51 galera2 mysqld[]: /usr/sbin/mysqld(Item_param::append_for_log(THD*, String*)+0x63)[0x561af0e4ac43]
      Sep 28 11:57:58 galera3 mysqld[]: /usr/sbin/mysqld(Item_param::value_query_val_str(THD*, String*) const+0xac)[0x555e958c28bc]
      Sep 28 11:57:58 galera1 mysqld[]: /usr/sbin/mysqld(Item_param::value_query_val_str(THD*, String*) const+0xac)[0x55aa91e618bc]
      Sep 28 11:57:51 galera2 mysqld[]: /usr/sbin/mysqld(Item_param::value_query_val_str(THD*, String*) const+0xac)[0x561af0e4a8bc]
      Sep 28 11:57:58 galera3 mysqld[]: /usr/sbin/mysqld(Prepared_statement::execute_loop(String*, bool, unsigned char*, unsigned char*)+0x36)[0x555e956
      Sep 28 11:57:58 galera1 mysqld[]: /usr/sbin/mysqld(Prepared_statement::execute_loop(String*, bool, unsigned char*, unsigned char*)+0x36)[0x55aa91
      Sep 28 11:57:58 galera3 mysqld[]: /usr/sbin/mysqld(Prepared_statement::set_parameters(String*, unsigned char*, unsigned char*)+0x6d)[0x555e956dfe9
      Sep 28 11:57:58 galera1 mysqld[]: /usr/sbin/mysqld(Prepared_statement::set_parameters(String*, unsigned char*, unsigned char*)+0x6d)[0x55aa91c7ee
      Sep 28 11:57:51 galera2 mysqld[]: /usr/sbin/mysqld(Prepared_statement::set_parameters(String*, unsigned char*, unsigned char*)+0x6d)[0x561af0c67e
      
      

      Attachments

        Issue Links

          Activity

            People

              suresh.ramagiri@mariadb.com suresh ramagiri
              claudio.nanni Claudio Nanni
              Votes:
              2 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.