Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-4430

INDEX Optimization - Virtual Column, Dynamic Column and Index optimization

Details

    • Task
    • Status: Closed (View Workflow)
    • Minor
    • Resolution: Duplicate
    • None
    • None

    Description

      Hi guys,
      i was trying to work around MDEV-4414 with virtual column + index
      but, check my problem... here the table definition:

      create table t (
         linha_digitavel varchar(54) not null default '',
         tmp_substr char(37) as (SUBSTRING(linha_digitavel,1,37)) persistent,
         key linha_key (linha_digitavel),
         key tmp_substr_key (tmp_substr)
      );

      now the problem... i have about 50k rows in this table..

      explain select * from t where tmp_substr='abc'

      result => using index condition, key = 'tmp_substr_key', 1 rows

      explain select * from t where SUBSTRING(linha_digitavel,1,37)='abc'

      result => using where, full table scan


      well i see two kinds of optimization here...
      1)the MDEV-4414, where substring(?,1,?) could use index "key linha_key (linha_digitavel)"
      2)the "SUBSTRING(linha_digitavel,1,37)" could be rewrite as "tmp_substr"

      (2) is the MDEV feature requested here

      it give DBA an oportunity to optimize closed systems, that we can't change source code, just tables...

      think about a sloowww query that could be optimized with a virtual column + index + sql rewrite ! ok many job... but it's a nice feature
      think about determinist functions (md5 for example) that could be optimized just replacing the "determinist_function(some_parameters)" to the virtual column, and the option to index it and make it really faster!

      think about MongoDB like features... a schema less table with index on fields that may or may not exists based on LONGBLOB fields + Dynamic Column + Virtual Index, i could say goodbye mongoDB for small systems (ok the map reduce is a parallel operation with high query performace, but it's not mysql =] )

      Attachments

        Issue Links

          Activity

            rspadim roberto spadim created issue -
            rspadim roberto spadim made changes -
            Field Original Value New Value
            Labels optimizer
            rspadim roberto spadim made changes -
            Summary VIRTUAL COLUMN INDEX optimization INDEX Optimization - Virtual Column, Dynamic Column and Index optimization
            rspadim roberto spadim made changes -
            Description Hi guys,
            i was trying to work around MDEV-4414 with virtual column + index
            but, check my problem... here the table definition:

            {code:sql}
            create table t (
               linha_digitavel varchar(54) not null default '',
               tmp_substr char(37) as (SUBSTRING(linha_digitavel,1,37)) persistent,
               key linha_key (linha_digitavel),
               key tmp_substr_key (tmp_substr)
            );
            {code}

            now the problem... i have about 50k rows in this table..

            {code:sql}
            explain select * from t where tmp_substr='abc'
            {code}

            result => using index condition, key = 'tmp_substr_key', 1 rows

            {code:sql}
            explain select * from t where SUBSTRING(linha_digitavel,1,37)='abc'
            {code}

            result => using where, full table scan

            ---
            well i see two kinds of optimization here...
            1)the MDEV-4414, where substring(?,1,?) could use index "key linha_key (linha_digitavel)"
            2)the "SUBSTRING(linha_digitavel,1,37)" could be rewrite as "tmp_substr"

            here (2), is the idea of this 'issue'
            it give DBA an oportunity to optimize closed systems, that we can't change source code, just tables...

            think about a sloowww query that could be optimized with a virtual column + index + sql rewrite ! ok many job... but it's a nice feature
            think about determinist functions (md5 for example) that could be optimized just replacing the "determinist_function(some_parameters)" to the virtual column, and the option to index it and make it really faster!

            well thanks again guys
            Hi guys,
            i was trying to work around MDEV-4414 with virtual column + index
            but, check my problem... here the table definition:

            {code:sql}
            create table t (
               linha_digitavel varchar(54) not null default '',
               tmp_substr char(37) as (SUBSTRING(linha_digitavel,1,37)) persistent,
               key linha_key (linha_digitavel),
               key tmp_substr_key (tmp_substr)
            );
            {code}

            now the problem... i have about 50k rows in this table..

            {code:sql}
            explain select * from t where tmp_substr='abc'
            {code}

            result => using index condition, key = 'tmp_substr_key', 1 rows

            {code:sql}
            explain select * from t where SUBSTRING(linha_digitavel,1,37)='abc'
            {code}

            result => using where, full table scan

            ---
            well i see two kinds of optimization here...
            1)the MDEV-4414, where substring(?,1,?) could use index "key linha_key (linha_digitavel)"
            2)the "SUBSTRING(linha_digitavel,1,37)" could be rewrite as "tmp_substr"

            here (2), is the idea of this 'issue'
            it give DBA an oportunity to optimize closed systems, that we can't change source code, just tables...

            think about a sloowww query that could be optimized with a virtual column + index + sql rewrite ! ok many job... but it's a nice feature
            think about determinist functions (md5 for example) that could be optimized just replacing the "determinist_function(some_parameters)" to the virtual column, and the option to index it and make it really faster!


            think about MongoDB like features... a schema less table with index on fields that may or may not exists based on LONGBLOB fields + Dynamic Column + Virtual Index, i could say goodbye mongoDB for small systems (ok the map reduce is a parallel operation with high query performace, but it's not mysql =] )
            rspadim roberto spadim made changes -
            Description Hi guys,
            i was trying to work around MDEV-4414 with virtual column + index
            but, check my problem... here the table definition:

            {code:sql}
            create table t (
               linha_digitavel varchar(54) not null default '',
               tmp_substr char(37) as (SUBSTRING(linha_digitavel,1,37)) persistent,
               key linha_key (linha_digitavel),
               key tmp_substr_key (tmp_substr)
            );
            {code}

            now the problem... i have about 50k rows in this table..

            {code:sql}
            explain select * from t where tmp_substr='abc'
            {code}

            result => using index condition, key = 'tmp_substr_key', 1 rows

            {code:sql}
            explain select * from t where SUBSTRING(linha_digitavel,1,37)='abc'
            {code}

            result => using where, full table scan

            ---
            well i see two kinds of optimization here...
            1)the MDEV-4414, where substring(?,1,?) could use index "key linha_key (linha_digitavel)"
            2)the "SUBSTRING(linha_digitavel,1,37)" could be rewrite as "tmp_substr"

            here (2), is the idea of this 'issue'
            it give DBA an oportunity to optimize closed systems, that we can't change source code, just tables...

            think about a sloowww query that could be optimized with a virtual column + index + sql rewrite ! ok many job... but it's a nice feature
            think about determinist functions (md5 for example) that could be optimized just replacing the "determinist_function(some_parameters)" to the virtual column, and the option to index it and make it really faster!


            think about MongoDB like features... a schema less table with index on fields that may or may not exists based on LONGBLOB fields + Dynamic Column + Virtual Index, i could say goodbye mongoDB for small systems (ok the map reduce is a parallel operation with high query performace, but it's not mysql =] )
            Hi guys,
            i was trying to work around MDEV-4414 with virtual column + index
            but, check my problem... here the table definition:

            {code:sql}
            create table t (
               linha_digitavel varchar(54) not null default '',
               tmp_substr char(37) as (SUBSTRING(linha_digitavel,1,37)) persistent,
               key linha_key (linha_digitavel),
               key tmp_substr_key (tmp_substr)
            );
            {code}

            now the problem... i have about 50k rows in this table..

            {code:sql}
            explain select * from t where tmp_substr='abc'
            {code}

            result => using index condition, key = 'tmp_substr_key', 1 rows

            {code:sql}
            explain select * from t where SUBSTRING(linha_digitavel,1,37)='abc'
            {code}

            result => using where, full table scan

            ---
            well i see two kinds of optimization here...
            1)the MDEV-4414, where substring(?,1,?) could use index "key linha_key (linha_digitavel)"
            2)the "SUBSTRING(linha_digitavel,1,37)" could be rewrite as "tmp_substr"

            (2) is the MDEV feature requested here

            it give DBA an oportunity to optimize closed systems, that we can't change source code, just tables...

            think about a sloowww query that could be optimized with a virtual column + index + sql rewrite ! ok many job... but it's a nice feature
            think about determinist functions (md5 for example) that could be optimized just replacing the "determinist_function(some_parameters)" to the virtual column, and the option to index it and make it really faster!


            think about MongoDB like features... a schema less table with index on fields that may or may not exists based on LONGBLOB fields + Dynamic Column + Virtual Index, i could say goodbye mongoDB for small systems (ok the map reduce is a parallel operation with high query performace, but it's not mysql =] )
            rspadim roberto spadim made changes -
            Comment [ maybe it could be optimized in opt_range.cc changing parts of query that are indexed in a virtual index and not indexed outside without it ]
            serg Sergei Golubchik made changes -
            Resolution Duplicate [ 3 ]
            Status Open [ 1 ] Closed [ 6 ]
            serg Sergei Golubchik made changes -
            Workflow defaullt [ 27222 ] MariaDB v2 [ 42662 ]
            rspadim roberto spadim made changes -
            ratzpo Rasmus Johansson (Inactive) made changes -
            Workflow MariaDB v2 [ 42662 ] MariaDB v3 [ 61643 ]
            serg Sergei Golubchik made changes -
            Workflow MariaDB v3 [ 61643 ] MariaDB v4 [ 132111 ]

            People

              Unassigned Unassigned
              rspadim roberto spadim
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.