Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-35152

DATA/INDEX DIRECTORY options are ignored for vector index

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Fixed
    • N/A
    • 11.8.2
    • Vector search
    • None

    Description

      DATA DIRECTORY and INDEX DIRECTORY work for main InnoDB tablespaces and MyISAM data/index files, but not for the vector index ones.
      I don't know whether it is meant to work, but given that vector index files can be very large, I think storing them in a separate location can be a realistic use case. 

      --source include/have_innodb.inc
      		 
       
      		 
      --let $datadir= `select @@datadir`
      		 
      eval create table ti (a vector(1) not null, vector(a)) engine=InnoDB data directory '$MYSQL_TMP_DIR';
      		 
      eval create table tm (a vector(1) not null, vector(a)) engine=MyISAM data directory '$MYSQL_TMP_DIR' index directory '$MYSQL_TMP_DIR';
      		 
       
      		 
      --echo ######### MYSQL_TMP_DIR:
      		 
      --list_files $MYSQL_TMP_DIR/
      		 
      --echo ######### MYSQL_TMP_DIR/test:
      		 
      --list_files $MYSQL_TMP_DIR/test/
      		 
      --echo ######### datadir/test:
      		 
      --list_files $datadir/test/
      		 
       
      		 
      drop table ti, tm;

      bb-11.6-MDEV-32887-vector 764592a4da2a1b490471732fbefe2ce745ce1f32

      		 
      ######### MYSQL_TMP_DIR:
      		 
      check-mysqld_1.log
      		 
      check-mysqld_1.result
      		 
      mysqld.1
      		 
      mysqld.1.sock
      		 
      test
      		 
      tm.MYD
      		 
      tm.MYI
      		 
      ######### MYSQL_TMP_DIR/test:
      		 
      ti.ibd
      		 
      ######### datadir/test:
      		 
      db.opt
      		 
      ti#i#00.ibd
      		 
      ti.frm
      		 
      ti.isl
      		 
      tm#i#00.MYD
      		 
      tm#i#00.MYI
      		 
      tm.MYD
      		 
      tm.MYI
      		 
      tm.frm

      Attachments

        Issue Links

          is blocked by

          Bug - A problem which impairs or prevents the functions of the product. MDEV-36050 DATA/INDEX DIRECTORY handling is inconsistent

          • Major - Major loss of function.
          • Closed
          is caused by

          New Feature - A new feature of the product, which has yet to be developed. MDEV-34939 vector search in 11.7

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Activity

            Ascending order - Click to sort in descending order
            elenst Elena Stepanova created issue -
            elenst Elena Stepanova made changes -
            Field Original Value New Value
            svoj Sergey Vojtovich made changes -
            Status Open [ 1 ] Confirmed [ 10101 ]
            svoj Sergey Vojtovich made changes -
            Assignee Sergei Golubchik [ serg ] Sergey Vojtovich [ svoj ]
            svoj Sergey Vojtovich added a comment -

            serg, please review https://github.com/MariaDB/server/pull/3822

            svoj Sergey Vojtovich added a comment - serg , please review https://github.com/MariaDB/server/pull/3822
            svoj Sergey Vojtovich made changes -
            Assignee Sergey Vojtovich [ svoj ] Sergei Golubchik [ serg ]
            Status Confirmed [ 10101 ] In Review [ 10002 ]
            serg Sergei Golubchik made changes -
            serg Sergei Golubchik added a comment -

            I already tried to fix it before I saw your comment, sorry.
            I thought that both all files of the hlindex should be in the INDEX DIRECTORY, because they're, well, index.

            And then I hit the fact that InnoDB doesn't allow INDEX DIRECTORY internally. I've reported MDEV-36050 to solve that.

            after a fix, either all engines should reject INDEX DIRECTORY, in which case there will be a HA_CAN_INDEX_DIRECTORY capability and the server can check it, or all engines will ignore it

            serg Sergei Golubchik added a comment - I already tried to fix it before I saw your comment, sorry. I thought that both all files of the hlindex should be in the INDEX DIRECTORY, because they're, well, index. And then I hit the fact that InnoDB doesn't allow INDEX DIRECTORY internally. I've reported MDEV-36050 to solve that. after a fix, either all engines should reject INDEX DIRECTORY , in which case there will be a HA_CAN_INDEX_DIRECTORY capability and the server can check it, or all engines will ignore it
            svoj Sergey Vojtovich added a comment -

            Well, then, I guess we can close pull request, right?

            svoj Sergey Vojtovich added a comment - Well, then, I guess we can close pull request, right?
            serg Sergei Golubchik added a comment -

            As you prefer.
            I thought of keeping it open for now, as I didn't compare two fixes to see which one is better.

            I think MDEV-36050 should be fixed first anyway.

            serg Sergei Golubchik added a comment - As you prefer. I thought of keeping it open for now, as I didn't compare two fixes to see which one is better. I think MDEV-36050 should be fixed first anyway.
            svoj Sergey Vojtovich added a comment -

            I have no preference, let's stick to your thinking.

            svoj Sergey Vojtovich added a comment - I have no preference, let's stick to your thinking.
            serg Sergei Golubchik made changes -
            Fix Version/s 11.8 [ 29921 ]
            Fix Version/s 11.7(EOL) [ 29815 ]
            serg Sergei Golubchik made changes -
            Fix Version/s N/A [ 14700 ]
            Fix Version/s 11.8 [ 29921 ]
            Resolution Fixed [ 1 ]
            Status In Review [ 10002 ] Closed [ 6 ]
            serg Sergei Golubchik made changes -
            Resolution Fixed [ 1 ]
            Status Closed [ 6 ] Stalled [ 10000 ]
            serg Sergei Golubchik made changes -
            Status Stalled [ 10000 ] In Testing [ 10301 ]
            serg Sergei Golubchik made changes -
            Fix Version/s 11.8 [ 29921 ]
            Fix Version/s N/A [ 14700 ]
            serg Sergei Golubchik made changes -
            Fix Version/s 11.8.2 [ 30001 ]
            Fix Version/s 11.8 [ 29921 ]
            Resolution Fixed [ 1 ]
            Status In Testing [ 10301 ] Closed [ 6 ]
            serg Sergei Golubchik made changes -
            Assignee Sergei Golubchik [ serg ] Sergey Vojtovich [ svoj ]

            People

              svoj Sergey Vojtovich
              elenst Elena Stepanova
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.