[MDEV-10816] Alter table with zero partitions, to add a partition creates two partitions Created: 2016-09-15  Updated: 2016-11-12  Resolved: 2016-11-12

Status: Closed
Project: MariaDB Server
Component/s: Partitioning
Affects Version/s: 10.1.17
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: Alvin Richards (Inactive) Assignee: Unassigned
Resolution: Not a Bug Votes: 0
Labels: None
Environment:

docker mariadb:lastest image



 Description   

Problem

When adding the first partition with ALTER TABLE, two partitions are created - only one should be created.

Reproduce

1. Create the table

create database if not exists test;
create or replace 
    table test.votes(
        voter_id int, 
        vote varchar(1),
        primary key(voter_id))
    ENGINE=spider
    COMMENT='wrapper "mysql", table "votes"'
    PARTITION BY HASH (voter_id);

2. Confirm table has been created and can be described

MariaDB [(none)]> show create table test.votes;
+-------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table | Create Table                                                                                                                                                                                                                                |
+-------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| votes | CREATE TABLE `votes` (
  `voter_id` int(11) NOT NULL,
  `vote` varchar(1) DEFAULT NULL,
  PRIMARY KEY (`voter_id`)
) ENGINE=SPIDER DEFAULT CHARSET=latin1 COMMENT='wrapper "mysql", table "votes"'
/*!50100 PARTITION BY HASH (voter_id) */ |
+-------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set, 2 warnings (0.04 sec)
 
MariaDB [(none)]> desc test.votes;
+----------+------------+------+-----+---------+-------+
| Field    | Type       | Null | Key | Default | Extra |
+----------+------------+------+-----+---------+-------+
| voter_id | int(11)    | NO   | PRI | NULL    |       |
| vote     | varchar(1) | YES  |     | NULL    |       |
+----------+------------+------+-----+---------+-------+
2 rows in set (0.00 sec)

3. Create the server and add the first partition

CREATE SERVER shard1 FOREIGN DATA WRAPPER mysql 
    OPTIONS( 
    HOST 'spider_shard1_1',
    DATABASE 'test',
    USER 'sharded',
    PASSWORD 'sharding',
    PORT 3306);
 
alter table test.votes add partition (PARTITION part1 comment = 'srv "shard1"');

4. Describe the modified table

MariaDB [(none)]> desc test.votes;
+----------+------------+------+-----+---------+-------+
| Field    | Type       | Null | Key | Default | Extra |
+----------+------------+------+-----+---------+-------+
| voter_id | int(11)    | NO   | PRI | NULL    |       |
| vote     | varchar(1) | YES  |     | NULL    |       |
+----------+------------+------+-----+---------+-------+
2 rows in set (0.00 sec)
 
MariaDB [(none)]> show create table test.votes;
+-------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table | Create Table                                                                                                                                                                                                                                                                                                                          |
+-------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| votes | CREATE TABLE `votes` (
  `voter_id` int(11) NOT NULL,
  `vote` varchar(1) DEFAULT NULL,
  PRIMARY KEY (`voter_id`)
) ENGINE=SPIDER DEFAULT CHARSET=latin1 COMMENT='wrapper "mysql", table "votes"'
/*!50100 PARTITION BY HASH (voter_id)
(PARTITION p0 ENGINE = SPIDER,
 PARTITION part1 COMMENT = 'srv "shard1"' ENGINE = SPIDER) */ |
+-------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set, 6 warnings (15.10 sec)

As can be seen, it appears two partitions are created

  • part1 - this comes from the ALTER TABLE
  • p0 - seems to have been created automatically

Solution

Do not create the additional and unrequested partition (in this case "p0").

Rationale

To automate adding a spider node, after the node starts we will want to script (Docker, Salt, Ansible etc.) adding the partition for the new node. Having this extra Partition breaks queries and other operations because they timeout.

Workaround

After each ALTER TABLE ADD PARTITION, execute a ALTER TABLE ... DROP PARTITION p0;



 Comments   
Comment by Alvin Richards (Inactive) [ 2016-09-15 ]

Additionally, if the table is created with at least one partition to begin with, adding a second results in just two partitions (correctly). This appears to be an edge case for when the table is created with zero partitions.

CREATE SERVER shard1 FOREIGN DATA WRAPPER mysql 
    OPTIONS( 
    HOST 'spider_shard1_1',
    DATABASE 'test',
    USER 'sharded',
    PASSWORD 'sharding',
    PORT 3306);
 
create or replace 
    table test.votes(
        voter_id int, 
        vote varchar(1),
        primary key(voter_id))
    ENGINE=spider
    COMMENT='wrapper "mysql", table "votes"'
    PARTITION BY HASH (voter_id)
      (PARTITION part1 comment = 'srv "shard1"');

MariaDB [(none)]> show create table test.votes;
+-------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table | Create Table                                                                                                                                                                                                                                                                                           |
+-------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| votes | CREATE TABLE `votes` (
  `voter_id` int(11) NOT NULL,
  `vote` varchar(1) DEFAULT NULL,
  PRIMARY KEY (`voter_id`)
) ENGINE=SPIDER DEFAULT CHARSET=latin1 COMMENT='wrapper "mysql", table "votes"'
/*!50100 PARTITION BY HASH (voter_id)
(PARTITION part1 COMMENT = 'srv "shard1"' ENGINE = SPIDER) */ |
+-------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set, 2 warnings (10.04 sec)

CREATE SERVER shard2 FOREIGN DATA WRAPPER mysql 
    OPTIONS( 
    HOST 'spider_shard2_1',
    DATABASE 'test',
    USER 'sharded',
    PASSWORD 'sharding',
    PORT 3306);
 
alter table test.votes add partition (PARTITION part2 comment = 'srv "shard2"');

show create table test.votes;
+-------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table | Create Table                                                                                                                                                                                                                                                                                                                                                      |
+-------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| votes | CREATE TABLE `votes` (
  `voter_id` int(11) NOT NULL,
  `vote` varchar(1) DEFAULT NULL,
  PRIMARY KEY (`voter_id`)
) ENGINE=SPIDER DEFAULT CHARSET=latin1 COMMENT='wrapper "mysql", table "votes"'
/*!50100 PARTITION BY HASH (voter_id)
(PARTITION part1 COMMENT = 'srv "shard1"' ENGINE = SPIDER,
 PARTITION part2 COMMENT = 'srv "shard2"' ENGINE = SPIDER) */ |
+-------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set, 6 warnings (20.18 sec)

Comment by Elena Stepanova [ 2016-09-15 ]

Partition p0 is not added by the ALTER statement.
When you create a new table with PARTITION clause, it is already partitioned, which means it has at least one partition, p0 if you didn't specify the partition name (it could have had more – p0, p1, ... pN if you specified the number of partitions upon creation).

What you might be trying to do is

create or replace 
    table test.votes(
        voter_id int, 
        vote varchar(1),
        primary key(voter_id))
    ENGINE=spider
    COMMENT='wrapper "mysql", table "votes"'
    PARTITION BY HASH (voter_id)
    (PARTITION part1 comment = 'srv "shard1"');

Or, if the workflow is such that you don't want to create this shard1 partition right away, you can create a non-partitioned table first, and then add partitioning via the ALTER statement:

create or replace 
    table test.votes(
        voter_id int, 
        vote varchar(1),
        primary key(voter_id))
    ENGINE=spider
    COMMENT='wrapper "mysql", table "votes"';
    
-- do whatever you need to do here .. 
 
alter table test.votes 
    PARTITION BY HASH (voter_id)
    (PARTITION part1 comment = 'srv "shard1"');

Comment by Alvin Richards (Inactive) [ 2016-09-15 ]

So it looks like "p0" is always being created, just not shown by "show create table test.votes" in the case where no partitions were initially specified. The spec of this default partition looks like:

PARTITION p0 ENGINE = SPIDER

Because of the default definition of this default partition, I cannot query the information schema to confirm:

select * from INFORMATION_SCHEMA.PARTITIONS WHERE TABLE_NAME='votes' AND TABLE_SCHEMA='test';
ERROR 1429 (HY000): Unable to connect to foreign data source: localhost

Comment by Elena Stepanova [ 2016-09-15 ]

Note: we are talking about PARTITION BY HASH now, it's different for other partition types.

Default partition p0 is not always created, it's created if you don't specify the partition:

MariaDB [test]> create or replace table t1 (i int) partition by hash(i);
Query OK, 0 rows affected (0.58 sec)
 
MariaDB [test]> select partition_name, partition_expression, partition_method from information_schema.partitions where table_name = 't1';
+----------------+----------------------+------------------+
| partition_name | partition_expression | partition_method |
+----------------+----------------------+------------------+
| p0             | i                    | HASH             |
+----------------+----------------------+------------------+
1 row in set (0.00 sec)

In a more general case, partitions p0 ... pN are created if you don't specify the list of partitions:

MariaDB [test]> create or replace table t1 (i int) partition by hash(i) partitions 5;
Query OK, 0 rows affected (1.79 sec)
 
MariaDB [test]> select partition_name, partition_expression, partition_method from information_schema.partitions where table_name = 't1';
+----------------+----------------------+------------------+
| partition_name | partition_expression | partition_method |
+----------------+----------------------+------------------+
| p0             | i                    | HASH             |
| p1             | i                    | HASH             |
| p2             | i                    | HASH             |
| p3             | i                    | HASH             |
| p4             | i                    | HASH             |
+----------------+----------------------+------------------+
5 rows in set (0.00 sec)

But if you specify the list of partitions, default ones are not created:

MariaDB [test]> create or replace table t1 (i int) partition by hash(i) (partition my_part1, partition my_part2);
Query OK, 0 rows affected (1.59 sec)
 
MariaDB [test]> select partition_name, partition_expression, partition_method from information_schema.partitions where table_name = 't1';
+----------------+----------------------+------------------+
| partition_name | partition_expression | partition_method |
+----------------+----------------------+------------------+
| my_part1       | i                    | HASH             |
| my_part2       | i                    | HASH             |
+----------------+----------------------+------------------+
2 rows in set (0.01 sec)

In MariaDB/MySQL there is no such thing as "partitioned table without partitions". There is always at least one.

For the error that you're getting, it's just Spider specific. That's why I thought that you might want to create a non-partitioned table first, and then partition it later.

For SHOW CREATE TABLE output, with numerous obscure exceptions it shows the statement which was used to create the table. So, if you specified the list of partitions, it's shown there, if you didn't specify it, it's not. Adding ALTER TABLE to the equation makes it confusing beyond common sense, I wouldn't dare formulate any rules for that.

Comment by Elena Stepanova [ 2016-10-14 ]

alvinr, is there still anything you want to have fixed in the scope of this JIRA entry?

Generated at Thu Feb 08 07:45:09 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.