Details

    Description

      A three nodes Galera cluster behind MaxScale routing gets all three nodes crashed on a prepared statement (SELECT).
      The application is in Java and uses mysql-connector-java-5.1.45.jar.

      First node crashes at `11:57:51` and the other two at `11:57:58`

      What I still don't understand is why Node 1 and 3 crash at the same time, I'd expect a bit of delay between the two.
      To be noted that MaxScale (afaik) pre-sends prepared statements to all nodes.

      Server version: 10.3.9-MariaDB-1:10.3.9+maria~xenial-log

      _THE SELECT spotted to be causing the crash_

      SELECT
      	t.id,
      	t.org_id,
      	t.int_id,
      	t.ext_id,
      	t.ext_name,
      	t.type,
      	t.flags,
      	t.aut,
      	t.ins
      FROM
      	myTableName t
      WHERE
      	t.org_id = 9
      AND
      	type = ?
      AND
      	int_id = ?
      

      MAXSCALE (first routes the query to Node2, then it tries Node1 and Node3 apparently)

      _NODE 2 CRASH_

      2018-09-28 11:57:53   error  : Monitor was unable to connect to server [192.168.12.-92-]:3306 : "Can't connect to MySQL server on '192.168.12.-92-' (115)"
      2018-09-28 11:57:53   notice : Server changed state: devdb2[192.168.12.92:3306]: slave_down. [Slave, Synced, Running] -> [Down]
      
      

      _NODE 1 & 3 CRASH_

      2018-09-28 11:57:59   error  : Monitor was unable to connect to server [192.168.12.-91-]:3306 : "Lost connection to MySQL server at 'handshake: reading initial communication packet', system error: 107"
      2018-09-28 11:57:59   error  : Monitor was unable to connect to server [192.168.12.-93-]:3306 : "Can't connect to MySQL server on '192.168.12.-93-' (115)"
      2018-09-28 11:57:59   error  : [galeramon] There are no cluster members
      2018-09-28 11:57:59   notice : Server changed state: devdb1[192.168.12.-91-:3306]: master_down. [Master, Synced, Running] -> [Down]
      2018-09-28 11:57:59   notice : Server changed state: devdb3[192.168.12.-93-:3306]: slave_down. [Slave, Synced, Running] -> [Down]
      

      _ONE STACK TRACE (from Node 2):_

      Sep 28 11:57:51 galera2 mysqld[21934]: terribly wrong...
      Sep 28 11:57:51 galera2 mysqld[21934]: stack_bottom = 0x7f7b0ab46cf8 thread_stack 0x49000
      Sep 28 11:57:51 galera2 mysqld[21934]: /usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x561af1385ace]
      Sep 28 11:57:51 galera2 mysqld[21934]: /usr/sbin/mysqld(handle_fatal_signal+0x347)[0x561af0e2b8f7]
      Sep 28 11:57:51 galera2 mysqld[21934]: /lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7f7c628d5390]
      Sep 28 11:57:51 galera2 mysqld[21934]: /usr/sbin/mysqld(append_query_string(charset_info_st const*, String*, char const*, unsigned long, bool)+0x5d)[0x561af0f0d6fd]
      Sep 28 11:57:51 galera2 mysqld[21934]: /usr/sbin/mysqld(Item_param::value_query_val_str(THD*, String*) const+0xac)[0x561af0e4a8bc]
      Sep 28 11:57:51 galera2 mysqld[21934]: /usr/sbin/mysqld(Item_param::append_for_log(THD*, String*)+0x63)[0x561af0e4ac43]
      Sep 28 11:57:51 galera2 mysqld[21934]: /usr/sbin/mysqld(+0x58c7d1)[0x561af0c647d1]
      Sep 28 11:57:51 galera2 mysqld[21934]: /usr/sbin/mysqld(Prepared_statement::set_parameters(String*, unsigned char*, unsigned char*)+0x6d)[0x561af0c67e9d]
      Sep 28 11:57:51 galera2 mysqld[21934]: /usr/sbin/mysqld(Prepared_statement::execute_loop(String*, bool, unsigned char*, unsigned char*)+0x36)[0x561af0c6a376]
      Sep 28 11:57:51 galera2 mysqld[21934]: /usr/sbin/mysqld(+0x592f48)[0x561af0c6af48]
      Sep 28 11:57:51 galera2 mysqld[21934]: /usr/sbin/mysqld(mysqld_stmt_execute(THD*, char*, unsigned int)+0x25)[0x561af0c6b085]
      Sep 28 11:57:51 galera2 mysqld[21934]: /usr/sbin/mysqld(dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool)+0x14a0)[0x561af0c581a0]
      Sep 28 11:57:51 galera2 mysqld[21934]: /usr/sbin/mysqld(do_command(THD*)+0x13c)[0x561af0c5944c]
      Sep 28 11:57:51 galera2 mysqld[21934]: /usr/sbin/mysqld(do_handle_one_connection(CONNECT*)+0x252)[0x561af0d29fd2]
      Sep 28 11:57:51 galera2 mysqld[21934]: /usr/sbin/mysqld(handle_one_connection+0x3d)[0x561af0d2a14d]
      Sep 28 11:57:51 galera2 mysqld[21934]: /lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba)[0x7f7c628cb6ba]
      Sep 28 11:57:51 galera2 mysqld[21934]: /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7f7c61f7641d]
      

      _THE THREE STACK TRACES ORDERED BY POSITION:_

      Sep 28 11:57:58 galera3 mysqld[]: /usr/sbin/mysqld(+0x58c7d1)[0x555e956dc7d1]
      Sep 28 11:57:58 galera1 mysqld[]: /usr/sbin/mysqld(+0x58c7d1)[0x55aa91c7b7d1]
      Sep 28 11:57:51 galera2 mysqld[]: /usr/sbin/mysqld(+0x58c7d1)[0x561af0c647d1]
      Sep 28 11:57:51 galera2 mysqld[]: /usr/sbin/mysqld(append_query_string(charset_info_st const*, String*, char const*, unsigned long, bool)+0x5d)[0
      Sep 28 11:57:58 galera1 mysqld[]: /usr/sbin/mysqld(append_query_string(charset_info_st const*, String*, char const*, unsigned long, bool)+0x5d)[0
      Sep 28 11:57:58 galera3 mysqld[]: /usr/sbin/mysqld(append_query_string(charset_info_st const*, String*, char const*, unsigned long, bool)+0x5d)[0x
      Sep 28 11:57:58 galera3 mysqld[]: /usr/sbin/mysqld(Item_param::append_for_log(THD*, String*)+0x63)[0x555e958c2c43]
      Sep 28 11:57:58 galera1 mysqld[]: /usr/sbin/mysqld(Item_param::append_for_log(THD*, String*)+0x63)[0x55aa91e61c43]
      Sep 28 11:57:51 galera2 mysqld[]: /usr/sbin/mysqld(Item_param::append_for_log(THD*, String*)+0x63)[0x561af0e4ac43]
      Sep 28 11:57:58 galera3 mysqld[]: /usr/sbin/mysqld(Item_param::value_query_val_str(THD*, String*) const+0xac)[0x555e958c28bc]
      Sep 28 11:57:58 galera1 mysqld[]: /usr/sbin/mysqld(Item_param::value_query_val_str(THD*, String*) const+0xac)[0x55aa91e618bc]
      Sep 28 11:57:51 galera2 mysqld[]: /usr/sbin/mysqld(Item_param::value_query_val_str(THD*, String*) const+0xac)[0x561af0e4a8bc]
      Sep 28 11:57:58 galera3 mysqld[]: /usr/sbin/mysqld(Prepared_statement::execute_loop(String*, bool, unsigned char*, unsigned char*)+0x36)[0x555e956
      Sep 28 11:57:58 galera1 mysqld[]: /usr/sbin/mysqld(Prepared_statement::execute_loop(String*, bool, unsigned char*, unsigned char*)+0x36)[0x55aa91
      Sep 28 11:57:58 galera3 mysqld[]: /usr/sbin/mysqld(Prepared_statement::set_parameters(String*, unsigned char*, unsigned char*)+0x6d)[0x555e956dfe9
      Sep 28 11:57:58 galera1 mysqld[]: /usr/sbin/mysqld(Prepared_statement::set_parameters(String*, unsigned char*, unsigned char*)+0x6d)[0x55aa91c7ee
      Sep 28 11:57:51 galera2 mysqld[]: /usr/sbin/mysqld(Prepared_statement::set_parameters(String*, unsigned char*, unsigned char*)+0x6d)[0x561af0c67e
      
      

      Attachments

        Issue Links

          Activity

            markus makela markus makela added a comment - - edited

            Managed to get a crash on 10.4.12 and 10.3.21:

            200203 12:29:37 [ERROR] mysqld got signal 11 ;
            This could be because you hit a bug. It is also possible that this binary
            or one of the libraries it was linked against is corrupt, improperly built,
            or misconfigured. This error can also be caused by malfunctioning hardware.
            To report this bug, see https://mariadb.com/kb/en/reporting-bugs
            We will try our best to scrape up some info that will hopefully help
            diagnose the problem, but since we have already crashed, 
            something is definitely wrong and this may fail.
            Server version: 10.4.12-MariaDB-1:10.4.12+maria~bionic-log
            key_buffer_size=134217728
            read_buffer_size=2097152
            max_used_connections=8
            max_threads=10002
            thread_count=15
            It is possible that mysqld could use up to 
            key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 61830494 K  bytes of memory
            Hope that's ok; if not, decrease some variables in the equation.
            Thread pointer: 0x7f709c000c08
            Attempting backtrace. You can use the following information to find out
            where mysqld died. If you see no messages after this, something went
            terribly wrong...
            stack_bottom = 0x7f7110694dd8 thread_stack 0x49000
            mysqld(my_print_stacktrace+0x2e)[0x55fe6f106c3e]
            mysqld(handle_fatal_signal+0x515)[0x55fe6eb7acf5]
            /lib/x86_64-linux-gnu/libpthread.so.0(+0x12890)[0x7f712d94d890]
            mysqld(_Z19append_query_stringPK15charset_info_stP6StringPKcmb+0x61)[0x55fe6ec6f901]
            mysqld(_ZNK10Item_param19value_query_val_strEP3THDP6String+0xb6)[0x55fe6eb9bfc6]
            mysqld(_ZThn144_N10Item_param14append_for_logEP3THDP6String+0x7b)[0x55fe6eb9c4db]
            mysqld(+0x677377)[0x55fe6e97d377]
            mysqld(_ZN18Prepared_statement14set_parametersEP6StringPhS2_+0xe1)[0x55fe6e980de1]
            mysqld(_ZN18Prepared_statement12execute_loopEP6StringbPhS2_+0x35)[0x55fe6e982cc5]
            mysqld(+0x67da72)[0x55fe6e983a72]
            mysqld(_Z19mysqld_stmt_executeP3THDPcj+0x25)[0x55fe6e983b15]
            mysqld(_Z16dispatch_command19enum_server_commandP3THDPcjbb+0xe35)[0x55fe6e96fda5]
            mysqld(_Z10do_commandP3THD+0x148)[0x55fe6e971be8]
            mysqld(_Z24do_handle_one_connectionP7CONNECT+0x25e)[0x55fe6ea4ddae]
            mysqld(handle_one_connection+0x3d)[0x55fe6ea4de6d]
            /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db)[0x7f712d9426db]
            /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7f712c36488f]
            Trying to get some variables.
            Some pointers may be invalid and cause the dump to abort.
            Query (0x0): 
            Connection ID (thread ID): 17
            Status: NOT_KILLED
            Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=on,condition_pushdown_from_having=on
            The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
            information that should help you find out what is causing the crash.
            Writing a core file...
            Working directory at /var/lib/mysql
            Resource Limits:
            Limit                     Soft Limit           Hard Limit           Units     
            Max cpu time              unlimited            unlimited            seconds   
            Max file size             unlimited            unlimited            bytes     
            Max data size             unlimited            unlimited            bytes     
            Max stack size            8388608              unlimited            bytes     
            Max core file size        unlimited            unlimited            bytes     
            Max resident set          unlimited            unlimited            bytes     
            Max processes             1048576              1048576              processes 
            Max open files            1048576              1048576              files     
            Max locked memory         65536                65536                bytes     
            Max address space         unlimited            unlimited            bytes     
            Max file locks            unlimited            unlimited            locks     
            Max pending signals       127226               127226               signals   
            Max msgqueue size         819200               819200               bytes     
            Max nice priority         0                    0                    
            Max realtime priority     0                    0                    
            Max realtime timeout      unlimited            unlimited            us        
            Core pattern: |/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h %e
            
            

            Here's the test program I used (maxscale listens on port 4006 with readwritesplit):

            #include <stdio.h>
            #include <iostream>
            #include <stdlib.h>
            #include <mysql.h>
            #include <string.h>
            #include <unistd.h>
             
            using namespace std;
             
            int main(int argc, char** argv)
            {
                MYSQL *mysql;
                MYSQL_RES* result;
                MYSQL_ROW row;
                MYSQL_FIELD* fields;
             
                if ((mysql = mysql_init(NULL)) == NULL)
                {
                    cout << "Failed initialize client library." << endl;
                }
             
                cout << "Connect" << endl;
             
                if (!mysql_real_connect(mysql, "127.0.0.1", "maxuser", "maxpwd", NULL, 4006, NULL, 0))
                {
                    cout << "Failed to connect to database: " << mysql_error(mysql) << endl;
                    mysql_close(mysql);
                    return 1;
                }
             
                cout << "Create table" << endl;
             
                mysql_query(mysql, "DROP TABLE IF EXISTS test.t1");
                mysql_query(mysql, "CREATE  TABLE test.t1(id BIGINT)");
                mysql_query(mysql, "INSERT INTO test.t1 VALUES (1), (2), (3)");
             
                MYSQL_STMT *stmt = mysql_stmt_init(mysql);
                MYSQL_BIND param;
             
                int value = 1;
                my_bool isnull = false;
                const char* data = "SELECT * FROM test.t1 WHERE id = ?";
             
                memset(&param, 0, sizeof(MYSQL_BIND));
                param.buffer = &value;
                param.buffer_type = MYSQL_TYPE_LONG;
                param.is_null = &isnull;
                param.is_unsigned = false;
             
                cout << "Prepare" << endl;
             
                if (mysql_stmt_prepare(stmt, data, strlen(data)) ||
                    mysql_stmt_bind_param(stmt, &param))
                {
                    cout << "Prepared statement preparation failure for '" << data << "': " << mysql_stmt_error(stmt) << endl;
                }
             
                sleep(1);
                cout << "Execute" << endl;
                        
                if (mysql_stmt_execute(stmt))
                {
                    cout << "Prepared statement execution failure for '" << data << "': " << mysql_stmt_error(stmt) << endl;
                }
             
                value = 0;
                mysql_stmt_bind_result(stmt, &param);
                mysql_stmt_store_result(stmt);
                mysql_stmt_fetch(stmt);
             
                cout << value << endl;
             
                mysql_query(mysql, "BEGIN");
             
                // This is where it crashes
                if (mysql_stmt_execute(stmt))
                {
                    cout << "Prepared statement execution failure for '" << data << "': " << mysql_stmt_error(stmt) << endl;
                }
             
                value = 0;
                mysql_stmt_bind_result(stmt, &param);
                mysql_stmt_store_result(stmt);
                mysql_stmt_fetch(stmt);
             
                cout << value << endl;
             
                mysql_query(mysql, "COMMIT");
             
                cout << "Close" << endl;
                mysql_stmt_close(stmt);
             
                cout << "Disconnect" << endl;
             
                mysql_close(mysql);
             
                return 0;
            }
            

            I think the fact that the first time a COM_STMT_EXECUTE is executed inside the transaction the parameter types aren't sent.

            markus makela markus makela added a comment - - edited Managed to get a crash on 10.4.12 and 10.3.21: 200203 12:29:37 [ERROR] mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. To report this bug, see https://mariadb.com/kb/en/reporting-bugs We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. Server version: 10.4.12-MariaDB-1:10.4.12+maria~bionic-log key_buffer_size=134217728 read_buffer_size=2097152 max_used_connections=8 max_threads=10002 thread_count=15 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 61830494 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x7f709c000c08 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x7f7110694dd8 thread_stack 0x49000 mysqld(my_print_stacktrace+0x2e)[0x55fe6f106c3e] mysqld(handle_fatal_signal+0x515)[0x55fe6eb7acf5] /lib/x86_64-linux-gnu/libpthread.so.0(+0x12890)[0x7f712d94d890] mysqld(_Z19append_query_stringPK15charset_info_stP6StringPKcmb+0x61)[0x55fe6ec6f901] mysqld(_ZNK10Item_param19value_query_val_strEP3THDP6String+0xb6)[0x55fe6eb9bfc6] mysqld(_ZThn144_N10Item_param14append_for_logEP3THDP6String+0x7b)[0x55fe6eb9c4db] mysqld(+0x677377)[0x55fe6e97d377] mysqld(_ZN18Prepared_statement14set_parametersEP6StringPhS2_+0xe1)[0x55fe6e980de1] mysqld(_ZN18Prepared_statement12execute_loopEP6StringbPhS2_+0x35)[0x55fe6e982cc5] mysqld(+0x67da72)[0x55fe6e983a72] mysqld(_Z19mysqld_stmt_executeP3THDPcj+0x25)[0x55fe6e983b15] mysqld(_Z16dispatch_command19enum_server_commandP3THDPcjbb+0xe35)[0x55fe6e96fda5] mysqld(_Z10do_commandP3THD+0x148)[0x55fe6e971be8] mysqld(_Z24do_handle_one_connectionP7CONNECT+0x25e)[0x55fe6ea4ddae] mysqld(handle_one_connection+0x3d)[0x55fe6ea4de6d] /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db)[0x7f712d9426db] /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7f712c36488f] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query (0x0): Connection ID (thread ID): 17 Status: NOT_KILLED Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=on,condition_pushdown_from_having=on The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. Writing a core file... Working directory at /var/lib/mysql Resource Limits: Limit Soft Limit Hard Limit Units Max cpu time unlimited unlimited seconds Max file size unlimited unlimited bytes Max data size unlimited unlimited bytes Max stack size 8388608 unlimited bytes Max core file size unlimited unlimited bytes Max resident set unlimited unlimited bytes Max processes 1048576 1048576 processes Max open files 1048576 1048576 files Max locked memory 65536 65536 bytes Max address space unlimited unlimited bytes Max file locks unlimited unlimited locks Max pending signals 127226 127226 signals Max msgqueue size 819200 819200 bytes Max nice priority 0 0 Max realtime priority 0 0 Max realtime timeout unlimited unlimited us Core pattern: |/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h %e Here's the test program I used (maxscale listens on port 4006 with readwritesplit): #include <stdio.h> #include <iostream> #include <stdlib.h> #include <mysql.h> #include <string.h> #include <unistd.h>   using namespace std;   int main( int argc, char ** argv) { MYSQL *mysql; MYSQL_RES* result; MYSQL_ROW row; MYSQL_FIELD* fields;   if ((mysql = mysql_init(NULL)) == NULL) { cout << "Failed initialize client library." << endl; }   cout << "Connect" << endl;   if (!mysql_real_connect(mysql, "127.0.0.1" , "maxuser" , "maxpwd" , NULL, 4006, NULL, 0)) { cout << "Failed to connect to database: " << mysql_error(mysql) << endl; mysql_close(mysql); return 1; }   cout << "Create table" << endl;   mysql_query(mysql, "DROP TABLE IF EXISTS test.t1" ); mysql_query(mysql, "CREATE TABLE test.t1(id BIGINT)" ); mysql_query(mysql, "INSERT INTO test.t1 VALUES (1), (2), (3)" );   MYSQL_STMT *stmt = mysql_stmt_init(mysql); MYSQL_BIND param;   int value = 1; my_bool isnull = false ; const char * data = "SELECT * FROM test.t1 WHERE id = ?" ;   memset (&param, 0, sizeof (MYSQL_BIND)); param.buffer = &value; param.buffer_type = MYSQL_TYPE_LONG; param.is_null = &isnull; param.is_unsigned = false ;   cout << "Prepare" << endl;   if (mysql_stmt_prepare(stmt, data, strlen (data)) || mysql_stmt_bind_param(stmt, &param)) { cout << "Prepared statement preparation failure for '" << data << "': " << mysql_stmt_error(stmt) << endl; }   sleep(1); cout << "Execute" << endl; if (mysql_stmt_execute(stmt)) { cout << "Prepared statement execution failure for '" << data << "': " << mysql_stmt_error(stmt) << endl; }   value = 0; mysql_stmt_bind_result(stmt, &param); mysql_stmt_store_result(stmt); mysql_stmt_fetch(stmt);   cout << value << endl;   mysql_query(mysql, "BEGIN" );   // This is where it crashes if (mysql_stmt_execute(stmt)) { cout << "Prepared statement execution failure for '" << data << "': " << mysql_stmt_error(stmt) << endl; }   value = 0; mysql_stmt_bind_result(stmt, &param); mysql_stmt_store_result(stmt); mysql_stmt_fetch(stmt);   cout << value << endl;   mysql_query(mysql, "COMMIT" );   cout << "Close" << endl; mysql_stmt_close(stmt);   cout << "Disconnect" << endl;   mysql_close(mysql);   return 0; } I think the fact that the first time a COM_STMT_EXECUTE is executed inside the transaction the parameter types aren't sent.
            shulga Dmitry Shulga added a comment -

            I didn't manage to reproduce this bug report (during my experiments mariadb server was run with the --general-log option turned on).

            Looking at the source code of the function append_query_string() also didn't find any suspicious place that could be theoretically a reason for server crashing.

            So, to continue working on this task a test case that reproduces the bug report is required.

            shulga Dmitry Shulga added a comment - I didn't manage to reproduce this bug report (during my experiments mariadb server was run with the --general-log option turned on). Looking at the source code of the function append_query_string() also didn't find any suspicious place that could be theoretically a reason for server crashing. So, to continue working on this task a test case that reproduces the bug report is required.
            alice Alice Sherepa added a comment -

            This is likely the same bug as MDEV-19838 (fixed in 10.3.26, 10.4.16). Please check if the problem still exists in the recent versions.

            alice Alice Sherepa added a comment - This is likely the same bug as MDEV-19838 (fixed in 10.3.26, 10.4.16). Please check if the problem still exists in the recent versions.

            >This is likely the same bug as MDEV-19838 (fixed in 10.3.26, 10.4.16). Please check if the problem still exists in the recent versions.

            Occurance happened 10.6.17-12

            241021 15:04:33 [ERROR] mysqld got signal 11 ;
            Sorry, we probably made a mistake, and this is a bug.
             
            Your assistance in bug reporting will enable us to fix this for the next release.
            To report this bug, see https://mariadb.com/kb/en/reporting-bugs
             
            We will try our best to scrape up some info that will hopefully help
            diagnose the problem, but since we have already crashed, 
            something is definitely wrong and this may fail.
             
            Server version: 10.6.17-12-MariaDB-enterprise-log source revision: 249b7e2a0440ae1eb3ba14c607bdb61393bd464e
            key_buffer_size=134217728
            read_buffer_size=2097152
            max_used_connections=750
            max_threads=5002
            thread_count=750
            It is possible that mysqld could use up to 
            key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 30995573 K  bytes of memory
            Hope that's ok; if not, decrease some variables in the equation.
             
            Thread pointer: 0x7f9304000c58
            Attempting backtrace. You can use the following information to find out
            where mysqld died. If you see no messages after this, something went
            terribly wrong...
            stack_bottom = 0x7f94c8ad2d18 thread_stack 0x49000
            /usr/sbin/mariadbd(my_print_stacktrace+0x32)[0x55757bd80052]
            /usr/sbin/mariadbd(handle_fatal_signal+0x475)[0x55757b82b135]
            /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f9d075fb420]
            /usr/sbin/mariadbd(_Z19append_query_stringPK15charset_info_stP6StringPKcmb+0x65)[0x55757b95c425]
            /usr/sbin/mariadbd(_ZNK10Item_param19value_query_val_strEP3THDP6String+0x114)[0x55757b851bc4]
            /usr/sbin/mariadbd(_ZThn120_N10Item_param14append_for_logEP3THDP6String+0x7d)[0x55757b85215d]
            /usr/sbin/mariadbd(+0x821312)[0x55757b5e5312]
            /usr/sbin/mariadbd(_ZN18Prepared_statement14set_parametersEP6StringPhS2_+0x6f)[0x55757b5e95df]
            /usr/sbin/mariadbd(_ZN18Prepared_statement12execute_loopEP6StringbPhS2_+0x3a)[0x55757b5ea84a]
            /usr/sbin/mariadbd(+0x827b25)[0x55757b5ebb25]
            /usr/sbin/mariadbd(_Z19mysqld_stmt_executeP3THDPcj+0x30)[0x55757b5ebd10]
            /usr/sbin/mariadbd(_Z16dispatch_command19enum_server_commandP3THDPcjb+0x1e9f)[0x55757b5c466f]
            /usr/sbin/mariadbd(_Z10do_commandP3THDb+0x138)[0x55757b5c60b8]
            /usr/sbin/mariadbd(_Z24do_handle_one_connectionP7CONNECTb+0x3b7)[0x55757b6efa07]
            /usr/sbin/mariadbd(handle_one_connection+0x5d)[0x55757b6efd5d]
            /usr/sbin/mariadbd(+0xcb9bd6)[0x55757ba7dbd6]
            /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609)[0x7f9d075ef609]
            /lib/x86_64-linux-gnu/libc.so.6(clone+0x43)[0x7f9d071db353]
             
            Trying to get some variables.
            Some pointers may be invalid and cause the dump to abort.
            Query (0x0): (null)
            Connection ID (thread ID): 2997
            Status: NOT_KILLED
             
            Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=on,condition_pushdown_from_having=on,not_null_range_scan=off,hash_join_cardinality=off,cset_narrowing=off,sargable_casefold=off
             
            The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mariadbd/ contains
            information that should help you find out what is causing the crash.
             
            We think the query pointer is invalid, but we will try to print it anyway. 
            Query: 
             
            Writing a core file...
            Working directory at /data/mysql
            Resource Limits:
            Limit                     Soft Limit           Hard Limit           Units     
            Max cpu time              unlimited            unlimited            seconds   
            Max file size             unlimited            unlimited            bytes     
            Max data size             unlimited            unlimited            bytes     
            Max stack size            8388608              unlimited            bytes     
            Max core file size        0                    unlimited            bytes     
            Max resident set          unlimited            unlimited            bytes     
            Max processes             255411               255411               processes 
            Max open files            32768                32768                files     
            Max locked memory         65536                65536                bytes     
            Max address space         unlimited            unlimited            bytes     
            Max file locks            unlimited            unlimited            locks     
            Max pending signals       255411               255411               signals   
            Max msgqueue size         819200               819200               bytes     
            Max nice priority         0                    0                    
            Max realtime priority     0                    0                    
            Max realtime timeout      unlimited            unlimited            us        
            Core pattern: |/usr/share/apport/apport -p%p -s%s -c%c -d%d -P%P -u%u -g%g -- %E
             
            Kernel version: Linux version 5.4.0-132-generic (buildd@lcy02-amd64-059) (gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.1)) #148-Ubuntu SMP Mon Oct 17 16:02:06 UTC 2022
            

            Richard Richard Stracke added a comment - >This is likely the same bug as MDEV-19838 (fixed in 10.3.26, 10.4.16). Please check if the problem still exists in the recent versions. Occurance happened 10.6.17-12 241021 15 : 04 : 33 [ERROR] mysqld got signal 11 ; Sorry, we probably made a mistake, and this is a bug.   Your assistance in bug reporting will enable us to fix this for the next release. To report this bug, see https: //mariadb.com/kb/en/reporting-bugs   We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail.   Server version: 10.6 . 17 - 12 -MariaDB-enterprise-log source revision: 249b7e2a0440ae1eb3ba14c607bdb61393bd464e key_buffer_size= 134217728 read_buffer_size= 2097152 max_used_connections= 750 max_threads= 5002 thread_count= 750 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 30995573 K bytes of memory Hope that's ok; if not, decrease some variables in the equation.   Thread pointer: 0x7f9304000c58 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this , something went terribly wrong... stack_bottom = 0x7f94c8ad2d18 thread_stack 0x49000 /usr/sbin/mariadbd(my_print_stacktrace+ 0x32 )[ 0x55757bd80052 ] /usr/sbin/mariadbd(handle_fatal_signal+ 0x475 )[ 0x55757b82b135 ] /lib/x86_64-linux-gnu/libpthread.so. 0 (+ 0x14420 )[ 0x7f9d075fb420 ] /usr/sbin/mariadbd(_Z19append_query_stringPK15charset_info_stP6StringPKcmb+ 0x65 )[ 0x55757b95c425 ] /usr/sbin/mariadbd(_ZNK10Item_param19value_query_val_strEP3THDP6String+ 0x114 )[ 0x55757b851bc4 ] /usr/sbin/mariadbd(_ZThn120_N10Item_param14append_for_logEP3THDP6String+ 0x7d )[ 0x55757b85215d ] /usr/sbin/mariadbd(+ 0x821312 )[ 0x55757b5e5312 ] /usr/sbin/mariadbd(_ZN18Prepared_statement14set_parametersEP6StringPhS2_+ 0x6f )[ 0x55757b5e95df ] /usr/sbin/mariadbd(_ZN18Prepared_statement12execute_loopEP6StringbPhS2_+ 0x3a )[ 0x55757b5ea84a ] /usr/sbin/mariadbd(+ 0x827b25 )[ 0x55757b5ebb25 ] /usr/sbin/mariadbd(_Z19mysqld_stmt_executeP3THDPcj+ 0x30 )[ 0x55757b5ebd10 ] /usr/sbin/mariadbd(_Z16dispatch_command19enum_server_commandP3THDPcjb+ 0x1e9f )[ 0x55757b5c466f ] /usr/sbin/mariadbd(_Z10do_commandP3THDb+ 0x138 )[ 0x55757b5c60b8 ] /usr/sbin/mariadbd(_Z24do_handle_one_connectionP7CONNECTb+ 0x3b7 )[ 0x55757b6efa07 ] /usr/sbin/mariadbd(handle_one_connection+ 0x5d )[ 0x55757b6efd5d ] /usr/sbin/mariadbd(+ 0xcb9bd6 )[ 0x55757ba7dbd6 ] /lib/x86_64-linux-gnu/libpthread.so. 0 (+ 0x8609 )[ 0x7f9d075ef609 ] /lib/x86_64-linux-gnu/libc.so. 6 (clone+ 0x43 )[ 0x7f9d071db353 ]   Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ( 0x0 ): ( null ) Connection ID (thread ID): 2997 Status: NOT_KILLED   Optimizer switch : index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=on,condition_pushdown_from_having=on,not_null_range_scan=off,hash_join_cardinality=off,cset_narrowing=off,sargable_casefold=off   The manual page at https: //mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mariadbd/ contains information that should help you find out what is causing the crash.   We think the query pointer is invalid, but we will try to print it anyway. Query:   Writing a core file... Working directory at /data/mysql Resource Limits: Limit Soft Limit Hard Limit Units Max cpu time unlimited unlimited seconds Max file size unlimited unlimited bytes Max data size unlimited unlimited bytes Max stack size 8388608 unlimited bytes Max core file size 0 unlimited bytes Max resident set unlimited unlimited bytes Max processes 255411 255411 processes Max open files 32768 32768 files Max locked memory 65536 65536 bytes Max address space unlimited unlimited bytes Max file locks unlimited unlimited locks Max pending signals 255411 255411 signals Max msgqueue size 819200 819200 bytes Max nice priority 0 0 Max realtime priority 0 0 Max realtime timeout unlimited unlimited us Core pattern: |/usr/share/apport/apport -p%p -s%s -c%c -d%d -P%P -u%u -g%g -- %E   Kernel version: Linux version 5.4 . 0 - 132 -generic (buildd @lcy02 -amd64- 059 ) (gcc version 9.4 . 0 (Ubuntu 9.4 . 0 -1ubuntu1~ 20.04 . 1 )) # 148 -Ubuntu SMP Mon Oct 17 16 : 02 : 06 UTC 2022

            People

              alice Alice Sherepa
              claudio.nanni Claudio Nanni
              Votes:
              2 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.