Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-39491

ParallelQuery: split the table/index into chunks to read: Study

    XMLWordPrintable

Details

    • Q2/2026 Server Development

    Description

      Suppose we want to do a full table/index scan using N parallel workers.

      The idea is to split the index into N approximately equal portions, and then give each portion to a separate worker.

      This should be a storage-engine call. Reverse to what records_in_range() does.
      records_in_range() gets a key (i.e. range endpoint) and produces a percentage "this endpoint is at 0.12345 fraction of the table".
      Here, the idea would be to do the reverse: given a fraction=0.123456, dive into the table and position a read cursor at the specified fraction of the index.

      According to Monty, MySQL has the code to do this?

      Example of how to do this.

      suppose we want to a parallel full table scan with N=10 workers.
      In InnoDB, full table scan is a full index scan on a [hidden] primary key.

      The primary worker requests innodb to provide boundary points for 10%, 20% ... 90% of the table.
      It gets:

      range_endpoints= {  
        pk=val1,   -- this record is located after 10% of the table records
        pk=val2,     -- this record is located after 20% of the table records 
        ...
      }
      

      Then,

      • worker1 will be requested to scan the range: -inf < pk < val1
      • worker2 will be requested to scan the range: val1 <= pk < val2 (note the left endpoint is inclusve, the right is non-inclusive)
      • ...

      worker2 can position at its start location and then scan the rows it needs to scan by doing this:

      h->index_init(clustered_pk);
      h->index_read_map(val2);  // position at 20% of the records.
      then it will call
      h->index_next();
      

      Questions

      Attachments

        Issue Links

          Activity

            People

              oleg.smirnov Oleg Smirnov
              psergei Sergei Petrunia
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.