[MDEV-27065] Partitioning tables with custom data directories moves data back to default directory Created: 2021-11-16  Updated: 2022-10-31  Resolved: 2022-04-08

Status: Closed
Project: MariaDB Server
Component/s: Data Definition - Alter Table, Partitioning
Affects Version/s: 10.4.22, 10.5.13, 10.6.5, 10.2, 10.3, 10.7, 10.8
Fix Version/s: 10.2.44, 10.3.35, 10.4.25, 10.5.16, 10.6.8, 10.7.4, 10.8.3

Type: Bug Priority: Critical
Reporter: Juan Assignee: Nayuta Yanagisawa (Inactive)
Resolution: Fixed Votes: 1
Labels: None
Environment:

CentOS 7.7


Issue Links:
Relates
relates to MDEV-28108 add partition should not reset data d... Open

 Description   

Partitioning tables ignores specified data directory and moves tables back to default data directory. This has serious consequences when there is a larger alternate mount bigger partitioned tables and not enough space to hold that data in the default data directory.

When tables are created with an individually specified data directory different from the default, this definition is lost during partitioning, and the engine re-creates the table back in the default data directory. Alter table operations allow the specification of a data directory with partitioning commands, but ignore the specified directory, showing a warning.

Other alter table operations, such as adding a column, do not have this problem.

create table partest(
id int(10) unsigned not null auto_increment,
val int(10) unsigned not null default 0,
primary key (id, val)
) engine=InnoDB
default charset=utf8mb4
data directory='/tmp/maria-partest/';
 
insert into partest(val) values(1);
insert into partest(val) values(1001);
 
~# ls -alh /tmp/maria-partest/test/
total 128K
drwxrwx--- 2 mysql mysql   25 Nov 16 23:01 .
drwxrwx--- 3 mysql mysql   18 Nov 16 22:55 ..
-rw-rw---- 1 mysql mysql 128K Nov 16 23:01 partest.ibd
 
~# ls -alh /var/lib/mysql/test/
total 12K
drwx------ 2 mysql mysql  58 Nov 16 23:22 .
drwxr-xr-x 5 mysql mysql 289 Nov 16 22:49 ..
-rw-rw---- 1 mysql mysql  65 Nov 16 21:39 db.opt
-rw-rw---- 1 mysql mysql 947 Nov 16 23:22 partest.frm
-rw-rw---- 1 mysql mysql  35 Nov 16 23:22 partest.isl
 
alter table partest partition by range(val)(
partition p0 values less than (1000),
partition p1 values less than (2000),
partition pFFFF values less than maxvalue
);
 
~# ls -alh /tmp/maria-partest/test/
total 0
drwxrwx--- 2 mysql mysql  6 Nov 16 23:09 .
drwxrwx--- 3 mysql mysql 18 Nov 16 22:55 ..
 
~# ls -alh /var/lib/mysql/test/
total 300K
drwx------ 2 mysql mysql  133 Nov 16 23:11 .
drwxr-xr-x 5 mysql mysql  289 Nov 16 22:49 ..
-rw-rw---- 1 mysql mysql   65 Nov 16 21:39 db.opt
-rw-rw---- 1 mysql mysql 1.2K Nov 16 23:11 partest.frm
-rw-rw---- 1 mysql mysql   44 Nov 16 23:11 partest.par
-rw-rw---- 1 mysql mysql  96K Nov 16 23:09 partest#P#p0.ibd
-rw-rw---- 1 mysql mysql  96K Nov 16 23:09 partest#P#p1.ibd
-rw-rw---- 1 mysql mysql  96K Nov 16 23:09 partest#P#pFFFF.ibd



 Comments   
Comment by Nayuta Yanagisawa (Inactive) [ 2022-03-16 ]

The data directory cannot be specified by the table option when the table is partitioned.

Note that these table options do not work for partitioned tables (use the partition options instead)

https://mariadb.com/kb/en/create-table/#data-directoryindex-directory
https://mariadb.com/kb/en/create-table/#partitions

Comment by Nayuta Yanagisawa (Inactive) [ 2022-03-29 ]

An ALTER TABLE of a non-partitioned table at least returns a warning. Thus, I will give a few more thoughts on the issue.

Comment by Nayuta Yanagisawa (Inactive) [ 2022-03-29 ]

ralf.gebhardt@mariadb.com (cc: serg) If we raise the warning on the case in the issue description, we need to raise the warning even when a user does not specify DATA DIRECTORY intentionally. That is because we cannot distinguish the case where he forgets to specify DATA DIRECTORY at some partitions and the case where he intentionally does not specify DATA DIRECTORY. Do you think it is acceptable?

Comment by Nayuta Yanagisawa (Inactive) [ 2022-03-29 ]

Naive implementation for reference https://github.com/MariaDB/server/commit/a10facb656b55b483fe8a6638253b73d6c51666c

Comment by Sergei Golubchik [ 2022-03-29 ]

https://lists.launchpad.net/maria-developers/msg13117.html

Comment by Nayuta Yanagisawa (Inactive) [ 2022-03-30 ]

Yes, the error message of the above implementation is confusing (thus I described it as "naive"). I will improve the message.

Comment by Nayuta Yanagisawa (Inactive) [ 2022-03-30 ]

https://github.com/MariaDB/server/commit/b1986e31617d0219e9dea1dfe36acdacdcabfbae

Comment by Nayuta Yanagisawa (Inactive) [ 2022-03-31 ]

holyfoot Please review the following: https://github.com/MariaDB/server/commit/0907c3e4254af13870e010e20ff1cdb56b7f14df

Comment by Alexey Botchkov [ 2022-04-08 ]

ok to push.

Generated at Thu Feb 08 09:50:04 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.