I filed MDEV-21895 for a task of refactoring ha_innobase::records_in_range() to address the problems that I am aware of. It might be possible to do something in earlier versions, especially if a repeatable test case can be provided.

Note that there are also some strange things around handler::stats.records in handler::info(), as noted in MDEV-20169. That might also affect some query plans.

Related to this, BTR_ESTIMATE and btr_cur_t::path_arr should be removed. They are only being used in the function btr_estimate_n_rows_in_range_low(). That function had better implement a specialized version of btr_cur_search_to_nth_level() to gather the path information.

See the slides to understand pages latching order during estimation.

psergei,

> what are p1 and p2 there?
p1 and p2 are cursors, which are used to find 'aaa' and 'zzz' leaves in b-tree.

> Looking at slides #18 and #19... so, for records_in_range('aaa', 'zzz') the code will obtain latches for all pages that hold values between 'aaa' and 'zzz' ? Even if that is thousands of pages?

No, there is some limit on pages count(9 pages) for per-level walking, if the right page is not reached and the the limit was exceeded, the number of rows is estimated approximately by some formula, otherwise the estimation is exact. See btr_estimate_n_rows_in_range_on_level(), it has quite detailed description.

And, as you can see on the slides, the latches are released when we leave some page. The maximum amount of latched pages(red rectangles in the slides) at the same time is 4.