Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-15186

Server Crash on Subquery / Union / Temp Table in Nested Stored Proc

    Details

      Description

      I upgraded from 10.2.7 to 10.2.12 the other night and one nightly stored procedure I run started crashing MariaDB.

      Basically, I think this is related to temporary table creation (create temp table / subquery / union) in a nested or double nested stored procedure. I'm assuming it's some sort of memory issue, but I have not run into anything like this in previous versions. Maybe there is some ini setting that could easily fix this, but there were no issues in 10.2.7 with the same ini file.

      This is happening on both my dev box and my production box.

      Here is one error dump:

      mysqld got exception 0xc0000005 ;
      This could be because you hit a bug. It is also possible that this binary
      or one of the libraries it was linked against is corrupt, improperly built,
      or misconfigured. This error can also be caused by malfunctioning hardware.
      To report this bug, see https://mariadb.com/kb/en/reporting-bugs
      We will try our best to scrape up some info that will hopefully help
      diagnose the problem, but since we have already crashed,
      something is definitely wrong and this may fail.
      Server version: 10.2.12-MariaDB
      key_buffer_size=134217728
      read_buffer_size=131072
      max_used_connections=339
      max_threads=65537
      thread_count=330
      It is possible that mysqld could use up to
      key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 324254 K bytes of memory
      Hope that's ok; if not, decrease some variables in the equation.
      Thread pointer: 0xcf46788108
      Attempting backtrace. You can use the following information to find out
      where mysqld died. If you see no messages after this, something went
      terribly wrong...
      mysqld.exe!Explain_query::get_select()[sql_explain.cc:86]
      mysqld.exe!Explain_query::add_node()[sql_explain.cc:122]
      mysqld.exe!JOIN::save_explain_data_intern()[sql_select.cc:24890]
      mysqld.exe!JOIN::save_explain_data()[sql_select.cc:3313]
      mysqld.exe!JOIN::optimize()[sql_select.cc:1110]
      mysqld.exe!mysql_derived_optimize()[sql_derived.cc:906]
      mysqld.exe!mysql_handle_single_derived()[sql_derived.cc:197]
      mysqld.exe!TABLE_LIST::handle_derived()[table.cc:7801]
      mysqld.exe!st_select_lex::handle_derived()[sql_lex.cc:3907]
      mysqld.exe!TABLE_LIST::handle_derived()[table.cc:7798]
      mysqld.exe!st_select_lex::handle_derived()[sql_lex.cc:3907]
      mysqld.exe!JOIN::optimize_inner()[sql_select.cc:1361]
      mysqld.exe!JOIN::optimize()[sql_select.cc:1101]
      mysqld.exe!mysql_select()[sql_select.cc:3711]
      mysqld.exe!handle_select()[sql_select.cc:361]
      mysqld.exe!mysql_execute_command()[sql_parse.cc:3933]
      mysqld.exe!sp_instr_stmt::exec_core()[sp_head.cc:3266]
      mysqld.exe!sp_lex_keeper::reset_lex_and_exec_core()[sp_head.cc:3027]
      mysqld.exe!sp_instr_stmt::execute()[sp_head.cc:3180]
      mysqld.exe!sp_head::execute()[sp_head.cc:1336]
      mysqld.exe!sp_head::execute_procedure()[sp_head.cc:2135]
      mysqld.exe!do_execute_sp()[sql_parse.cc:2888]
      mysqld.exe!mysql_execute_command()[sql_parse.cc:5809]
      mysqld.exe!sp_instr_stmt::exec_core()[sp_head.cc:3266]
      mysqld.exe!sp_lex_keeper::reset_lex_and_exec_core()[sp_head.cc:3027]
      mysqld.exe!sp_instr_stmt::execute()[sp_head.cc:3180]
      mysqld.exe!sp_head::execute()[sp_head.cc:1336]
      mysqld.exe!sp_head::execute_procedure()[sp_head.cc:2135]
      mysqld.exe!do_execute_sp()[sql_parse.cc:2888]
      mysqld.exe!mysql_execute_command()[sql_parse.cc:5809]
      mysqld.exe!mysql_parse()[sql_parse.cc:7905]
      mysqld.exe!dispatch_command()[sql_parse.cc:1807]
      mysqld.exe!do_command()[sql_parse.cc:1359]
      mysqld.exe!threadpool_process_request()[threadpool_common.cc:366]
      mysqld.exe!tp_callback()[threadpool_common.cc:192]
      ntdll.dll!RtlFreeUnicodeString()
      ntdll.dll!RtlFreeUnicodeString()
      KERNEL32.DLL!BaseThreadInitThunk()
      ntdll.dll!RtlUserThreadStart()
      Trying to get some variables.
      Some pointers may be invalid and cause the dump to abort.
      Query (0xcf4af06450): create temporary table mlb_split_weights as
      select *
      , if(season <= if(@days>=14, @proj_year, @proj_year-1), power(@lg_decay, @proj_year-season),0) as weight
      , if(season <= if(@days>=14, @proj_year, @proj_year-1), power(@lg_K_decay, @proj_year-season),0) as Kweight
      , if(season <= if(@days>=14, @proj_year, @proj_year-1), power(@lg_BB_decay, @proj_year-season),0) as BBweight
      , if(season <= if(@days>=14, @proj_year, @proj_year-1), power(@lg_HR_decay, @proj_year-season),0) as HRweight
      from mlb_pitching_league_role_splits where season >= @proj_year-5
      Connection ID (thread ID): 5424
      Status: NOT_KILLED
      Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loos [truncated, 689 bytes total]
      The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
      information that should help you find out what is causing the crash.

      -----------------------------------------------------

      Here's another one:

      Timestamp, Thread, Type, Details
      2018-02-02 16:27:22, , [ERROR], mysqld got exception 0xc0000005 ;
      , , , This could be because you hit a bug. It is also possible that this binary
      , , , or one of the libraries it was linked against is corrupt, improperly built,
      , , , or misconfigured. This error can also be caused by malfunctioning hardware.
      , , , To report this bug, see https://mariadb.com/kb/en/reporting-bugs
      , , , We will try our best to scrape up some info that will hopefully help
      , , , diagnose the problem, but since we have already crashed,
      , , , something is definitely wrong and this may fail.
      , , , Server version: 10.2.12-MariaDB
      , , , key_buffer_size=134217728
      , , , read_buffer_size=131072
      , , , max_used_connections=321
      , , , max_threads=65537
      , , , thread_count=319
      , , , It is possible that mysqld could use up to
      , , , key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 324254 K bytes of memory
      , , , Hope that's ok; if not, decrease some variables in the equation.
      , , , Thread pointer: 0x4f42b670e8
      , , , Attempting backtrace. You can use the following information to find out
      , , , where mysqld died. If you see no messages after this, something went
      , , , terribly wrong...
      , , , mysqld.exe!memset()[memset.asm:81]
      , , , mysqld.exe!set_dynamic()[array.c:204]
      , , , mysqld.exe!Explain_query::add_node()[sql_explain.cc:122]
      , , , mysqld.exe!JOIN::save_explain_data_intern()[sql_select.cc:24890]
      , , , mysqld.exe!JOIN::save_explain_data()[sql_select.cc:3313]
      , , , mysqld.exe!JOIN::optimize()[sql_select.cc:1110]
      , , , mysqld.exe!mysql_derived_optimize()[sql_derived.cc:906]
      , , , mysqld.exe!mysql_handle_single_derived()[sql_derived.cc:197]
      , , , mysqld.exe!TABLE_LIST::handle_derived()[table.cc:7801]
      , , , mysqld.exe!st_select_lex::handle_derived()[sql_lex.cc:3907]
      , , , mysqld.exe!TABLE_LIST::handle_derived()[table.cc:7798]
      , , , mysqld.exe!st_select_lex::handle_derived()[sql_lex.cc:3907]
      , , , mysqld.exe!JOIN::optimize_inner()[sql_select.cc:1361]
      , , , mysqld.exe!JOIN::optimize()[sql_select.cc:1101]
      , , , mysqld.exe!st_select_lex_unit::optimize()[sql_union.cc:919]
      , , , mysqld.exe!st_select_lex_unit::exec()[sql_union.cc:953]
      , , , mysqld.exe!mysql_union()[sql_union.cc:41]
      , , , mysqld.exe!handle_select()[sql_select.cc:351]
      , , , mysqld.exe!mysql_execute_command()[sql_parse.cc:3933]
      , , , mysqld.exe!sp_instr_stmt::exec_core()[sp_head.cc:3266]
      , , , mysqld.exe!sp_lex_keeper::reset_lex_and_exec_core()[sp_head.cc:3027]
      , , , mysqld.exe!sp_instr_stmt::execute()[sp_head.cc:3180]
      , , , mysqld.exe!sp_head::execute()[sp_head.cc:1336]
      , , , mysqld.exe!sp_head::execute_procedure()[sp_head.cc:2135]
      , , , mysqld.exe!do_execute_sp()[sql_parse.cc:2888]
      , , , mysqld.exe!mysql_execute_command()[sql_parse.cc:5809]
      , , , mysqld.exe!sp_instr_stmt::exec_core()[sp_head.cc:3266]
      , , , mysqld.exe!sp_lex_keeper::reset_lex_and_exec_core()[sp_head.cc:3027]
      , , , mysqld.exe!sp_instr_stmt::execute()[sp_head.cc:3180]
      , , , mysqld.exe!sp_head::execute()[sp_head.cc:1336]
      , , , mysqld.exe!sp_head::execute_procedure()[sp_head.cc:2135]
      , , , mysqld.exe!do_execute_sp()[sql_parse.cc:2888]
      , , , mysqld.exe!mysql_execute_command()[sql_parse.cc:5809]
      , , , mysqld.exe!sp_instr_stmt::exec_core()[sp_head.cc:3266]
      , , , mysqld.exe!sp_lex_keeper::reset_lex_and_exec_core()[sp_head.cc:3027]
      , , , mysqld.exe!sp_instr_stmt::execute()[sp_head.cc:3180]
      , , , mysqld.exe!sp_head::execute()[sp_head.cc:1336]
      , , , mysqld.exe!sp_head::execute_procedure()[sp_head.cc:2135]
      , , , mysqld.exe!do_execute_sp()[sql_parse.cc:2888]
      , , , mysqld.exe!mysql_execute_command()[sql_parse.cc:5809]
      , , , mysqld.exe!sp_instr_stmt::exec_core()[sp_head.cc:3266]
      , , , mysqld.exe!sp_lex_keeper::reset_lex_and_exec_core()[sp_head.cc:3027]
      , , , mysqld.exe!sp_instr_stmt::execute()[sp_head.cc:3180]
      , , , mysqld.exe!sp_head::execute()[sp_head.cc:1336]
      , , , mysqld.exe!sp_head::execute_procedure()[sp_head.cc:2135]
      , , , mysqld.exe!do_execute_sp()[sql_parse.cc:2888]
      , , , mysqld.exe!mysql_execute_command()[sql_parse.cc:5809]
      , , , mysqld.exe!mysql_parse()[sql_parse.cc:7905]
      , , , mysqld.exe!dispatch_command()[sql_parse.cc:1807]
      , , , mysqld.exe!do_command()[sql_parse.cc:1359]
      , , , mysqld.exe!threadpool_process_request()[threadpool_common.cc:366]
      , , , mysqld.exe!tp_callback()[threadpool_common.cc:192]
      , , , ntdll.dll!RtlFreeUnicodeString()
      , , , ntdll.dll!RtlFreeUnicodeString()
      , , , KERNEL32.DLL!BaseThreadInitThunk()
      , , , ntdll.dll!RtlUserThreadStart()
      , , , Trying to get some variables.
      , , , Some pointers may be invalid and cause the dump to abort.
      , , , Query (0x4f46a806b0): create table projected_bb_wOBAcon_unreg as
      , , , select NAME_CONST('projyear',2018) as season, coalesce(s1.batter,s2.batter) as batter, s1.batted_balls as s1batted_balls, s1.pred_wOBAcon as s1wOBAcon
      , , , , s1.batted_balls/(25+s1.batted_balls) as s1weight
      , , , ,s2.batted_balls as s2batted_balls
      , , , , coalesce(25*s2.batted_balls/((25+s1.batted_balls)*(25+s2.batted_balls)),0) as s2weight
      , , , , s2.pred_wOBAcon as s2wOBAcon
      , , , ,(s1.pred_wOBAcon*s1.batted_balls/(25+s1.batted_balls)+ coalesce(s2.pred_wOBAcon*25*s2.batted_balls/((25+s1.batted_balls)(25+s2.batted_balls)),0))/(s1.batted_balls/(25+s1.batted_balls) + coalesce(25*s2.batted_balls/((25+s1.batted_balls)(25+s2.batted_ball [truncated, 279 bytes total]
      , , , ,s1.batted_balls/(25+s1.batted_balls) + coalesce(25*s2.batted_balls/((25+s1.batted_balls)*(25+s2.batted_balls)),0) as bb_reliability
      , , , from
      , , , (select * from gd_savant_pred_wobacon where game_year = NAME_CONST('projyear',2018)) s1
      , , , left join
      , , , (select * from gd_savant_pred_wobacon where game_year = NAME_CONST('projyear',2018)-1) s2
      , , , on s1.batter=s2.batter
      , , , union
      , , , select NAME_CONST('projyear',2018) as season, s2.batter, s1.batted_balls s1batted_balls, s1.pred_wOBAcon as s1wOBAcon
      , , , , coalesce(s1.batted_balls/(25+s1.batted_balls),0) as s1weight
      , , , ,s2.batted_balls as s2batted_balls
      , , , , coalesce(25*s2.batted_balls/((25+coalesce(s1.batted_balls,0))*(25+s2.batted_balls)),0) as s2weight
      , , , , s2.pred_wOBAcon as s2wOBAcon
      , , , ,(coalesce(s1.pred_wOBAcon*s1.batted_balls/(25+s1.batted_balls),0)+ coalesce(s2.pred_wOBAcon*25*s2.batted_balls/((25+coalesce(s1.batted_balls,0))*(25+s2.batted_balls)),0))/(coalesce(s1.batted_balls/(25+s1.batted_balls),0) + coalesce(25*s2.batted_balls/((25 [truncated, 326 bytes total]
      , , , ,coalesce(s1.batted_balls/(25+s1.batted_balls),0) + coalesce(25*s2.batted_balls/((25+coalesce(s1.batted_balls,0))*(25+s2.batted_balls)),0) as bb_reliability
      , , , from
      , , , (select * from gd_savant_pred_wobacon where game_year = NAME_CONST('projyear',2018)) s1
      , , , right join
      , , , (select * from gd_savant_pred_wobacon where game_year = NAME_CONST('projyear',2018)-1) s2
      , , , on s1.batter=s2.batter
      , , , Connection ID (thread ID): 222
      , , , Status: NOT_KILLED
      , , , Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loos [truncated, 689 bytes total]
      , , , The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
      , , , information that should help you find out what is causing the crash.

      ---------------------------------------------------

      Here is my .ini file:

      [mysqld]

      datadir=C:/Program Files/MariaDB 10.0/data
      port=3306

      character-set-server=latin1
      default_storage_engine=innodb

      sql_mode="NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION"

      innodb_file_per_table
      innodb_buffer_pool_size=80G
      innodb_buffer_pool_instances=8
      innodb_log_file_size=4G
      innodb_flush_log_at_trx_commit=2
      innodb_flush_neighbors=0
      innodb_thread_concurrency=32
      innodb_sort_buffer_size=8M
      innodb_io_capacity=2000
      innodb_io_capacity_max=4000

      read_rnd_buffer_size = 16M
      sort_buffer_size = 8M
      join_buffer_size = 8M

      aria_pagecache_buffer_size=2G

      max_connections=10000

      skip-name-resolve

      tmp_table_size=256M
      max_heap_table_size=256M

      max_allowed_packet=16M

      table_open_cache = 2000
      table-definition-cache = 2000
      open_files_limit = 14000
      innodb_open_files = 6000

      transaction-isolation=READ-COMMITTED
      binlog_format=ROW

      [client]

      port=3306

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                dkappelman David Appelman
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: