Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-19002

Partition performance optimization

Details

    Description

      Add the partition options of the following function for batch call instead of each partition

      • store_lock
      • external_lock
      • start_stmt
      • extra
      • cond_push
      • info_push
      • top_table

      original description:
      Partition performance optimization: improve the select/insert/delete/update/replace performance of the partition table under multiple partitions by executing prune_partition before the lock table

      Attachments

        3. Test_procedures_delete.txt
          1 kB
          Kentoku Shiba
        5. Test_procedures_insert.txt
          1 kB
          Kentoku Shiba
        6. Test_procedures_select.txt
          1 kB
          Kentoku Shiba
        7. Test_procedures_update.txt
          1 kB
          Kentoku Shiba
        25. test_results_of_spider.txt
          7 kB
          Kentoku Shiba

        Issue Links

          causes

          Bug - A problem which impairs or prevents the functions of the product. MDEV-26158 SIGSEGV in spider_free_mem from ha_spider::open on INSERT

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-26539 SIGSEGV in spider_check_and_set_trx_isolation and I_P_List_iterator from THD::drop_temporary_table (10.5.3 opt only) on ALTER

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-26582 SIGSEGV in spider_db_bulk_insert and spider_db_connect and spider_db_before_query, and hang in "End of update loop" / "Reset for next command" query states

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-27171 Spider: SIGSEGV in spider_set_direct_limit_offset on INSERT

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-27239 Spider: Assertion `thd->transaction->stmt.ha_list == __null || trans == &thd->transaction->stmt' failed in ha_commit_trans on BEGIN WORK after FTWRL

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-27240 SIGSEGV in ha_spider::store_lock on LOCK TABLE

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-30191 SIGSEGV & heap-use-after-free in spider_db_print_item_type, SIGABRT in __cxa_pure_virtual/spider_db_print_item_type, Got error 128 "Out of memory in engine", 56/112 memory not freed, and Assertion `fixed()' failed in Item_sp_variable::val_str on SP call

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-31996 Segfault when setting spider_delete_all_rows to 0 and delete all rows of a spider table, ASAN heap-use-after-free in spider_db_delete_all_rows

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          relates to

          New Feature - A new feature of the product, which has yet to be developed. MDEV-20250 Implement early partition pruning before statement start

          • Critical - Crashes, loss of data, severe memory leak.
          • In Progress

          Activity

            Ascending order - Click to sort in descending order
            Kentoku Kentoku Shiba (Inactive) added a comment -

            When increasing number of partitions, performance is dropping down even if all partitions are quite fast.
            So, I added some options to partition handler for calling at once instead of calling each partitions.
            Please review the following commit especially changing of handler and partition handler.
            ae106f1

            Kentoku Kentoku Shiba (Inactive) added a comment - When increasing number of partitions, performance is dropping down even if all partitions are quite fast. So, I added some options to partition handler for calling at once instead of calling each partitions. Please review the following commit especially changing of handler and partition handler. ae106f1
            Kentoku Kentoku Shiba (Inactive) added a comment -

            gmon_*_without_partition.zip is call stack of MyISAM table without partitioning.
            gmon_*_with_1024_partitions.zip is call stack of MyISAM table with partitioning.
            Above call stacks are for knowing this performance dropping down with partitioning is not only caused by Spider.

            gmon_spider_*_without_partition.zip is call stack of Spider table without partitioning.
            gmon_spider_*_with_1024_partitions_before.zip is call stack of Spider table with partitioning before using new options.
            gmon_spider_*_with_1024_partitions_after.zip is call stack of Spider table with partitioning after using new options.
            Test_procedures_*.txt is test sequences.
            test_results_of_spider.txt is the results of performance testing.

            Kentoku Kentoku Shiba (Inactive) added a comment - gmon_*_without_partition.zip is call stack of MyISAM table without partitioning. gmon_*_with_1024_partitions.zip is call stack of MyISAM table with partitioning. Above call stacks are for knowing this performance dropping down with partitioning is not only caused by Spider. gmon_spider_*_without_partition.zip is call stack of Spider table without partitioning. gmon_spider_*_with_1024_partitions_before.zip is call stack of Spider table with partitioning before using new options. gmon_spider_*_with_1024_partitions_after.zip is call stack of Spider table with partitioning after using new options. Test_procedures_*.txt is test sequences. test_results_of_spider.txt is the results of performance testing.

            People

              holyfoot Alexey Botchkov
              willhan willhan
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.