Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-13226

Server crashes when tmpdir runs out of space

Details

    Description

      A user running 10.2.7 experienced the following crash:

      2017-06-28 13:37:14 139942218483456 [Warning] mysqld: Disk is full writing '/mariadb/tmp/#sql_c8b_5.MAD' (Errcode: 28 "No space left on device"). Waiting for someone to free space... (Expect up to 60 secs delay for server to continue after freeing disk space)
      2017-06-28 13:37:14 139942218483456 [Warning] mysqld: Retry in 60 secs. Message reprinted in 600 secs
      170628 13:38:40 [ERROR] mysqld got signal 11 ;
      This could be because you hit a bug. It is also possible that this binary
      or one of the libraries it was linked against is corrupt, improperly built,
      or misconfigured. This error can also be caused by malfunctioning hardware.
       
      To report this bug, see https://mariadb.com/kb/en/reporting-bugs
       
      We will try our best to scrape up some info that will hopefully help
      diagnose the problem, but since we have already crashed,
      something is definitely wrong and this may fail.
       
      Server version: 10.2.7-MariaDB-log
      key_buffer_size=67108864
      read_buffer_size=131072
      max_used_connections=206
      max_threads=1002
      thread_count=71
      It is possible that mysqld could use up to
      key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 2267268 K  bytes of memory
      Hope that's ok; if not, decrease some variables in the equation.
       
      Thread pointer: 0x7f44400e0268
      Attempting backtrace. You can use the following information to find out
      where mysqld died. If you see no messages after this, something went
      terribly wrong...
      stack_bottom = 0x7f46d5c2ed70 thread_stack 0x40000
      *** buffer overflow detected ***: /usr/sbin/mysqld terminated
      ======= Backtrace: =========
      /lib64/libc.so.6(__fortify_fail+0x37)[0x7f5deb77c047]
      /lib64/libc.so.6(+0x10d200)[0x7f5deb77a200]
      /lib64/libc.so.6(+0x10efb7)[0x7f5deb77bfb7]
      /usr/sbin/mysqld(my_addr_resolve+0x48)[0x7f5dee201198]
      /usr/sbin/mysqld(my_print_stacktrace+0x1c2)[0x7f5dee1eac32]
      /usr/sbin/mysqld(handle_fatal_signal+0x30d)[0x7f5dedc31a8d]
      /lib64/libpthread.so.0(+0xf370)[0x7f5ded118370]
      /usr/sbin/mysqld(+0xba0524)[0x7f5dee0e7524]
      /usr/sbin/mysqld(_ZN7AGGR_OP8end_sendEv+0x4a)[0x7f5dedacc8ea]
      /usr/sbin/mysqld(_Z24sub_select_postjoin_aggrP4JOINP13st_join_tableb+0x35)[0x7f5dedaccbc5]
      /usr/sbin/mysqld(_ZN4JOIN10exec_innerEv+0x8d3)[0x7f5dedacd553]
      /usr/sbin/mysqld(_ZN4JOIN4execEv+0x33)[0x7f5dedacd8a3]
      /usr/sbin/mysqld(_ZN18st_select_lex_unit4execEv+0x2ac)[0x7f5dedb0a82c]
      /usr/sbin/mysqld(_Z11mysql_unionP3THDP3LEXP13select_resultP18st_select_lex_unitm+0x2e)[0x7f5dedb0cb4e]
      /usr/sbin/mysqld(_Z13handle_selectP3THDP3LEXP13select_resultm+0x55)[0x7f5dedace345]
      /usr/sbin/mysqld(+0x528376)[0x7f5deda6f376]
      /usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x60e4)[0x7f5deda7c0a4]
      /usr/sbin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_statebb+0x2de)[0x7f5deda7f46e]
      /usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcjbb+0x2008)[0x7f5deda822e8]
      /usr/sbin/mysqld(_Z10do_commandP3THD+0x161)[0x7f5deda82e91]
      /usr/sbin/mysqld(_Z24do_handle_one_connectionP7CONNECT+0x1aa)[0x7f5dedb4605a]
      /usr/sbin/mysqld(handle_one_connection+0x3d)[0x7f5dedb4617d]
      /lib64/libpthread.so.0(+0x7dc5)[0x7f5ded110dc5]
      /lib64/libc.so.6(clone+0x6d)[0x7f5deb76473d]
      

      Attachments

        Activity

          #3  <signal handler called>
          #4  0x000055c58d355316 in ha_maria::extra (this=0x7febc0148948, operation=HA_EXTRA_NO_CACHE) at /home/elenst/git/10.2-14601/storage/maria/ha_maria.cc:2563
          #5  0x000055c58cba923c in AGGR_OP::end_send (this=0x7febc014e1b8) at /home/elenst/git/10.2-14601/sql/sql_select.cc:26338
          #6  0x000055c58cb94616 in sub_select_postjoin_aggr (join=0x7febc0134498, join_tab=0x7febc0108a48, end_of_records=true) at /home/elenst/git/10.2-14601/sql/sql_select.cc:18203
          #7  0x000055c58cb9493a in sub_select (join=0x7febc0134498, join_tab=0x7febc0108698, end_of_records=true) at /home/elenst/git/10.2-14601/sql/sql_select.cc:18439
          #8  0x000055c58cb9493a in sub_select (join=0x7febc0134498, join_tab=0x7febc01082e8, end_of_records=true) at /home/elenst/git/10.2-14601/sql/sql_select.cc:18439
          #9  0x000055c58cb9493a in sub_select (join=0x7febc0134498, join_tab=0x7febc0107f38, end_of_records=true) at /home/elenst/git/10.2-14601/sql/sql_select.cc:18439
          #10 0x000055c58cb9493a in sub_select (join=0x7febc0134498, join_tab=0x7febc0107b88, end_of_records=true) at /home/elenst/git/10.2-14601/sql/sql_select.cc:18439
          #11 0x000055c58cb94152 in do_select (join=0x7febc0134498, procedure=0x0) at /home/elenst/git/10.2-14601/sql/sql_select.cc:18034
          #12 0x000055c58cb6f085 in JOIN::exec_inner (this=0x7febc0134498) at /home/elenst/git/10.2-14601/sql/sql_select.cc:3473
          #13 0x000055c58cb6e556 in JOIN::exec (this=0x7febc0134498) at /home/elenst/git/10.2-14601/sql/sql_select.cc:3274
          #14 0x000055c58cc0aec5 in st_select_lex_unit::exec (this=0x7febc00045e0) at /home/elenst/git/10.2-14601/sql/sql_union.cc:990
          #15 0x000055c58cc07a54 in mysql_union (thd=0x7febc0000b00, lex=0x7febc0004518, result=0x7febc0130940, unit=0x7febc00045e0, setup_tables_done_option=0) at /home/elenst/git/10.2-14601/sql/sql_union.cc:41
          #16 0x000055c58cb64033 in handle_select (thd=0x7febc0000b00, lex=0x7febc0004518, result=0x7febc0130940, setup_tables_done_option=0) at /home/elenst/git/10.2-14601/sql/sql_select.cc:351
          #17 0x000055c58cb2ffc8 in execute_sqlcom_select (thd=0x7febc0000b00, all_tables=0x7febc013e188) at /home/elenst/git/10.2-14601/sql/sql_parse.cc:6442
          #18 0x000055c58cb25d18 in mysql_execute_command (thd=0x7febc0000b00) at /home/elenst/git/10.2-14601/sql/sql_parse.cc:3457
          #19 0x000055c58cb33951 in mysql_parse (thd=0x7febc0000b00, rawbuf=0x7febc0010f38 "<query has been redacted>"..., length=1881, parser_state=0x7fec2c18a250, is_com_multi=false, is_next_command=false) at /home/elenst/git/10.2-14601/sql/sql_parse.cc:7879
          #20 0x000055c58cb21725 in dispatch_command (command=COM_QUERY, thd=0x7febc0000b00, packet=0x7febc00086c1 "<query has been redacted>"..., packet_length=1881, is_com_multi=false, is_next_command=false) at /home/elenst/git/10.2-14601/sql/sql_parse.cc:1817
          #21 0x000055c58cb2005f in do_command (thd=0x7febc0000b00) at /home/elenst/git/10.2-14601/sql/sql_parse.cc:1362
          #22 0x000055c58cc6b402 in do_handle_one_connection (connect=0x55c591217920) at /home/elenst/git/10.2-14601/sql/sql_connect.cc:1354
          #23 0x000055c58cc6b182 in handle_one_connection (arg=0x55c591217920) at /home/elenst/git/10.2-14601/sql/sql_connect.cc:1260
          #24 0x00007fec2fda46ba in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
          #25 0x00007fec2f23982d in clone () from /lib/x86_64-linux-gnu/libc.so.6
          

          elenst Elena Stepanova added a comment - #3 <signal handler called> #4 0x000055c58d355316 in ha_maria::extra (this=0x7febc0148948, operation=HA_EXTRA_NO_CACHE) at /home/elenst/git/10.2-14601/storage/maria/ha_maria.cc:2563 #5 0x000055c58cba923c in AGGR_OP::end_send (this=0x7febc014e1b8) at /home/elenst/git/10.2-14601/sql/sql_select.cc:26338 #6 0x000055c58cb94616 in sub_select_postjoin_aggr (join=0x7febc0134498, join_tab=0x7febc0108a48, end_of_records=true) at /home/elenst/git/10.2-14601/sql/sql_select.cc:18203 #7 0x000055c58cb9493a in sub_select (join=0x7febc0134498, join_tab=0x7febc0108698, end_of_records=true) at /home/elenst/git/10.2-14601/sql/sql_select.cc:18439 #8 0x000055c58cb9493a in sub_select (join=0x7febc0134498, join_tab=0x7febc01082e8, end_of_records=true) at /home/elenst/git/10.2-14601/sql/sql_select.cc:18439 #9 0x000055c58cb9493a in sub_select (join=0x7febc0134498, join_tab=0x7febc0107f38, end_of_records=true) at /home/elenst/git/10.2-14601/sql/sql_select.cc:18439 #10 0x000055c58cb9493a in sub_select (join=0x7febc0134498, join_tab=0x7febc0107b88, end_of_records=true) at /home/elenst/git/10.2-14601/sql/sql_select.cc:18439 #11 0x000055c58cb94152 in do_select (join=0x7febc0134498, procedure=0x0) at /home/elenst/git/10.2-14601/sql/sql_select.cc:18034 #12 0x000055c58cb6f085 in JOIN::exec_inner (this=0x7febc0134498) at /home/elenst/git/10.2-14601/sql/sql_select.cc:3473 #13 0x000055c58cb6e556 in JOIN::exec (this=0x7febc0134498) at /home/elenst/git/10.2-14601/sql/sql_select.cc:3274 #14 0x000055c58cc0aec5 in st_select_lex_unit::exec (this=0x7febc00045e0) at /home/elenst/git/10.2-14601/sql/sql_union.cc:990 #15 0x000055c58cc07a54 in mysql_union (thd=0x7febc0000b00, lex=0x7febc0004518, result=0x7febc0130940, unit=0x7febc00045e0, setup_tables_done_option=0) at /home/elenst/git/10.2-14601/sql/sql_union.cc:41 #16 0x000055c58cb64033 in handle_select (thd=0x7febc0000b00, lex=0x7febc0004518, result=0x7febc0130940, setup_tables_done_option=0) at /home/elenst/git/10.2-14601/sql/sql_select.cc:351 #17 0x000055c58cb2ffc8 in execute_sqlcom_select (thd=0x7febc0000b00, all_tables=0x7febc013e188) at /home/elenst/git/10.2-14601/sql/sql_parse.cc:6442 #18 0x000055c58cb25d18 in mysql_execute_command (thd=0x7febc0000b00) at /home/elenst/git/10.2-14601/sql/sql_parse.cc:3457 #19 0x000055c58cb33951 in mysql_parse (thd=0x7febc0000b00, rawbuf=0x7febc0010f38 "<query has been redacted>"..., length=1881, parser_state=0x7fec2c18a250, is_com_multi=false, is_next_command=false) at /home/elenst/git/10.2-14601/sql/sql_parse.cc:7879 #20 0x000055c58cb21725 in dispatch_command (command=COM_QUERY, thd=0x7febc0000b00, packet=0x7febc00086c1 "<query has been redacted>"..., packet_length=1881, is_com_multi=false, is_next_command=false) at /home/elenst/git/10.2-14601/sql/sql_parse.cc:1817 #21 0x000055c58cb2005f in do_command (thd=0x7febc0000b00) at /home/elenst/git/10.2-14601/sql/sql_parse.cc:1362 #22 0x000055c58cc6b402 in do_handle_one_connection (connect=0x55c591217920) at /home/elenst/git/10.2-14601/sql/sql_connect.cc:1354 #23 0x000055c58cc6b182 in handle_one_connection (arg=0x55c591217920) at /home/elenst/git/10.2-14601/sql/sql_connect.cc:1260 #24 0x00007fec2fda46ba in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #25 0x00007fec2f23982d in clone () from /lib/x86_64-linux-gnu/libc.so.6

          There was a missing test in CTE handling if creating a temporary table failed (in this case as a result of out of space). This caused a table handler to be used even if it was not allocated.

          monty Michael Widenius added a comment - There was a missing test in CTE handling if creating a temporary table failed (in this case as a result of out of space). This caused a table handler to be used even if it was not allocated.

          Fix pushed

          monty Michael Widenius added a comment - Fix pushed

          People

            monty Michael Widenius
            GeoffMontee Geoff Montee (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.