Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-11520

Extending an InnoDB data file unnecessarily allocates a large memory buffer on Windows

Details

    Description

      Debug version of 10.2 server has assertion failure while importing 'employees' database(https://github.com/datacharmer/test_db). Server is build using current 10.2 branch. ("current" means commit 7ca1e2abad42a7436e6b668b4568d6fadc2ca165 here)

      2016-12-09 23:37:00 16112 [Note] InnoDB: Buffer pool(s) load completed at 161209 23:37:00
      2016-12-09 23:46:00 0x4624  InnoDB: Assertion failure in thread 17956 in file ut0byte.ic line 89
      InnoDB: Failing assertion: ptr
      

      Occurs in ut_align called from fil_write_zeros. The callstack:

      > mysqld.exe!my_sigabrt_handler(int sig) Line 477 C
        [External Code]
        mysqld.exe!ut_dbg_assertion_failed(const char * expr, const char * file, unsigned long line) Line 68  C++
        mysqld.exe!ut_align(const void * ptr, unsigned long align_no) Line 89 C++
        mysqld.exe!fil_write_zeros(const fil_node_t * node, unsigned long page_size, unsigned __int64 start, unsigned long len, bool read_only_mode) Line 4891  C++
        mysqld.exe!fil_space_extend(fil_space_t * space, unsigned long size) Line 5067  C++
        mysqld.exe!fsp_try_extend_data_file(fil_space_t * space, unsigned char * header, mtr_t * mtr, unsigned long * n_pages_added) Line 1581  C++
        mysqld.exe!fsp_reserve_free_extents(unsigned long * n_reserved, unsigned long space_id, unsigned long n_ext, fsp_reserve_t alloc_type, mtr_t * mtr, unsigned long n_pages) Line 3506  C++
        mysqld.exe!btr_cur_pessimistic_insert(unsigned long flags, btr_cur_t * cursor, unsigned long * * offsets, mem_block_info_t * * heap, dtuple_t * entry, unsigned char * * rec, big_rec_t * * big_rec, unsigned long
      n_ext, que_thr_t * thr, mtr_t * mtr) Line 3501 C++
        mysqld.exe!row_ins_clust_index_entry_low(unsigned long flags, unsigned long mode, dict_index_t * index, unsigned long n_uniq, dtuple_t * entry, unsigned long n_ext, que_thr_t * thr, bool dup_chk_only) Line 2651
       C++
        mysqld.exe!row_ins_clust_index_entry(dict_index_t * index, dtuple_t * entry, que_thr_t * thr, unsigned long n_ext, bool dup_chk_only) Line 3386 C++
        mysqld.exe!row_ins_index_entry(dict_index_t * index, dtuple_t * entry, que_thr_t * thr) Line 3490 C++
        mysqld.exe!row_ins_index_entry_step(ins_node_t * node, que_thr_t * thr) Line 3640 C++
        mysqld.exe!row_ins(ins_node_t * node, que_thr_t * thr) Line 3782  C++
        mysqld.exe!row_ins_step(que_thr_t * thr) Line 3967  C++
        mysqld.exe!row_insert_for_mysql_using_ins_graph(const unsigned char * mysql_rec, row_prebuilt_t * prebuilt) Line 1784 C++
        mysqld.exe!row_insert_for_mysql(const unsigned char * mysql_rec, row_prebuilt_t * prebuilt) Line 1915 C++
        mysqld.exe!ha_innobase::write_row(unsigned char * record) Line 9094 C++
        mysqld.exe!handler::ha_write_row(unsigned char * buf) Line 5924 C++
        mysqld.exe!write_record(THD * thd, TABLE * table, st_copy_info * info) Line 1883  C++
        mysqld.exe!mysql_insert(THD * thd, TABLE_LIST * table_list, List<Item> & fields, List<List<Item> > & values_list, List<Item> & update_fields, List<Item> & update_values, enum_duplicates duplic, bool ignore) Line
       1003  C++
        mysqld.exe!mysql_execute_command(THD * thd) Line 4328 C++
        mysqld.exe!mysql_parse(THD * thd, char * rawbuf, unsigned int length, Parser_state * parser_state, bool is_com_multi, bool is_next_command) Line 7799 C++
        mysqld.exe!dispatch_command(enum_server_command command, THD * thd, char * packet, unsigned int packet_length, bool is_com_multi, bool is_next_command) Line 1808 C++
        mysqld.exe!do_command(THD * thd) Line 1368  C++
        mysqld.exe!threadpool_process_request(THD * thd) Line 319 C++
        mysqld.exe!tp_callback(TP_connection * c) Line 158  C++
        mysqld.exe!tp_callback(_TP_CALLBACK_INSTANCE * instance, void * context) Line 377 C++
        mysqld.exe!work_callback(_TP_CALLBACK_INSTANCE * instance, void * context, _TP_WORK * work) Line 451  C++
        [External Code]
        [Frames below may be incorrect and/or missing, no symbols loaded for ntdll.dll]
      

      After that, mysqld cannot recover, and crashes on start in memcpy because of writing to/reading from NULL pointer. The "after" dump is for that case, but there is not much useful info in it. At least there is no meaningful backtrace. And again, RelWithDebInfo version can start on the same config.

      Attachments

        1. error.log
          70 kB
        2. mysqld_after_noheap.dmp
          91 kB
        3. mysqld_noheap.dmp
          113 kB
        4. mysqld_pdb.7z
          7.40 MB
        5. mysqld.7z
          5.15 MB

        Issue Links

          Activity

            In 10.1 and 10.2 where InnoDB supports page compression via sparse files, we should extend the files using SetEndOfFile() on Windows, and on POSIX, either posix_fallocate() or ftruncate(), depending on whether the file is supposed to be sparse.

            marko Marko Mäkelä added a comment - In 10.1 and 10.2 where InnoDB supports page compression via sparse files, we should extend the files using SetEndOfFile() on Windows, and on POSIX, either posix_fallocate() or ftruncate(), depending on whether the file is supposed to be sparse.

            bb-10.1-marko implements the use of sparse files via ftruncate() on POSIX, for tables that use page_compression. On Windows, SetEndOfFile() is used, but because the files are not being declared as sparse, this will end up physically extending the files (preallocating, wasting space for all-zero, unused pages).

            marko Marko Mäkelä added a comment - bb-10.1-marko implements the use of sparse files via ftruncate() on POSIX, for tables that use page_compression. On Windows, SetEndOfFile() is used, but because the files are not being declared as sparse, this will end up physically extending the files (preallocating, wasting space for all-zero, unused pages).

            bb-10.1-marko employs SetFileInformationByHandle() and FILE_END_OF_FILE_INFO on Windows, and fixes some bugs in the error messages.

            marko Marko Mäkelä added a comment - bb-10.1-marko employs SetFileInformationByHandle() and FILE_END_OF_FILE_INFO on Windows, and fixes some bugs in the error messages.

            It looks like we should implement some retry logic for EINTR, to address a rpl.rpl_domain_id_filter test failure:

            2017-03-02 18:25:53 139979823866624 [ERROR] InnoDB: posix_fallocate(): Failed to preallocate data for file ./test/t3.ibd, desired size 65536 Operating system error number 4. Check that the disk is not full or a disk quota exceeded. Some operating system error numbers are described at http://dev.mysql.com/doc/refman/5.7/en/operating-system-error-codes.html
            

            marko Marko Mäkelä added a comment - It looks like we should implement some retry logic for EINTR, to address a rpl.rpl_domain_id_filter test failure : 2017-03-02 18:25:53 139979823866624 [ERROR] InnoDB: posix_fallocate(): Failed to preallocate data for file ./test/t3.ibd, desired size 65536 Operating system error number 4. Check that the disk is not full or a disk quota exceeded. Some operating system error numbers are described at http://dev.mysql.com/doc/refman/5.7/en/operating-system-error-codes.html

            I added a retry loop if posix_fallocate() returns EINTR.

            marko Marko Mäkelä added a comment - I added a retry loop if posix_fallocate() returns EINTR.

            People

              marko Marko Mäkelä
              Lawrin Lawrin Novitsky
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.