[MDEV-12117] Crash on import Created: 2017-02-23  Updated: 2017-05-18  Resolved: 2017-05-18

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - InnoDB
Affects Version/s: 10.1.21
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: Ruben de Graaf Assignee: Unassigned
Resolution: Not a Bug Votes: 0
Labels: innodb, need_feedback
Environment:

RHEL 6.8


Attachments: File ligne_problematique.rar    

 Description   

Hello,

I tried to import today a mysqldump in a 10.1.21 MariaDB server that I exported from a 10.0.28 MariaDB server.

Apparently there were characters inside the dumpfile that made MariaDB crash :

[root@s-mysql-pp tmp]# mysql -p -D mydatabse < /tmp/database.sql
ERROR 2006 (HY000) at line 1264: MySQL server has gone away

I just had to remove those lines with sed (luckily thoses lines were web cache which I could spare) and then the import went well.

sed -i '/INSERT INTO `fe_session_data` VALUES.*/d' /tmp/database.sql
sed -i '/INSERT INTO `cf_cache_pages` VALUES.*/d' /tmp/database.sql

So back to the main problem, the first line that triggered the problem has a weight of 17398191 Bytes (which is quite a lot). I zipped that line and added it to the ticket.

That line is inserted in a table that is created this way :

DROP TABLE IF EXISTS `fe_session_data`;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!40101 SET character_set_client = utf8 */;
CREATE TABLE `fe_session_data` (
`hash` varchar(32) NOT NULL DEFAULT '',
`content` mediumblob,
`tstamp` int(11) unsigned NOT NULL DEFAULT '0',
PRIMARY KEY (`hash`),
KEY `tstamp` (`tstamp`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
/*!40101 SET character_set_client = @saved_cs_client */;

So what I suppose is that the MariaDB 10.0.28 server somehow allowed to put too much stuff in the mediumblob (over 17.3 million chars, while the maximum should be 16777215) and that it's now impossible to import it inside a mariadb 10.1.21 database.

If you need the whole dump I can provide it to you, but it's about 800MB...



 Comments   
Comment by Elena Stepanova [ 2017-02-23 ]

I don't see any problem with the provided piece of data. The length of the resulting value is 16334000, which is within the limit. Surely the file itself is bigger, there is a lot of overhead with all escape symbols and such.

Unless you can see an actual crash in the error log, your load fails just because max_allowed_packet is too small, "MySQL server has gone away" hints to that. It's 4M by default, and since you didn't mention that, I assume you didn't increase it. Set it to e.g. 32M, it should work. You might encounter different errors due to other variables being too small, but they should be adjustable accordingly.

Comment by Ruben de Graaf [ 2017-03-13 ]

Hello, sorry for the delay, mails from your new Jira aren't send to me for some weird reason (maybe our antispam or something) and I didn't know you already replied.

So concerning the amout of characters, when I open the file I provided you, the insert into query is 17.328.244 characters long, when removing the INSERT INTO `fe_session_data` VALUES (' and the ); at the end of the query, (which is about 41 characters), there is still above 17.3 millions characters left, how did you count about 16334000 characters ?

(Or maybe Notepad++ has a bug and isn't able to count correctly).

Concerning the max_allowed_packet variable, in our configuration it's set to 128M (useful for longblobs) so this souldn't be an issue

Comment by Elena Stepanova [ 2017-04-20 ]

To make it more obvious, let's take a minimal example.

MariaDB [test]> drop table if exists t;
Query OK, 0 rows affected (0.10 sec)
 
MariaDB [test]> create table t (c varchar(3));
Query OK, 0 rows affected (0.26 sec)
 
MariaDB [test]> insert into t values ('a\\b');
Query OK, 1 row affected (0.04 sec)

As you can see, even though the string is 4-symbol long, and will be counted as such in any text editor, it gets inserted into a 3-character-long column just fine. It happens because one of backslashes is in fact just an escape character, which is not inserted into the field:

MariaDB [test]> select * from t;
+------+
| c    |
+------+
| a\b  |
+------+
1 row in set (0.00 sec)

But if you try to dump this table using mysqldump, the backslash will be escaped again (it has to be, otherwise the value won't be loaded correctly):

DROP TABLE IF EXISTS `t`;
/*!40101 SET @saved_cs_client     = @@character_set_client */;
/*!40101 SET character_set_client = utf8 */;
CREATE TABLE `t` (
  `c` varchar(3) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
/*!40101 SET character_set_client = @saved_cs_client */;
 
--
-- Dumping data for table `t`
--
 
LOCK TABLES `t` WRITE;
/*!40000 ALTER TABLE `t` DISABLE KEYS */;
INSERT INTO `t` VALUES ('a\\b');
/*!40000 ALTER TABLE `t` ENABLE KEYS */;
UNLOCK TABLES;
/*!40103 SET TIME_ZONE=@OLD_TIME_ZONE */;

So, your text file is full of symbols that had to be escaped – backslashes, double quote marks, occasional quote marks. Thus, the actual size of the string in the text file is much longer than the length of the value in the column.

Hope it helps to understand.

Generated at Thu Feb 08 07:55:15 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.