[MDEV-7838] Got error 5 "Input/output error" during COMMIT; Created: 2015-03-25  Updated: 2015-03-27  Resolved: 2015-03-27

Status: Closed
Project: MariaDB Server
Component/s: Replication
Affects Version/s: 10.0.17-galera
Fix Version/s: 10.0.19-galera

Type: Bug Priority: Major
Reporter: Leo Kwan Assignee: Nirbhay Choubey (Inactive)
Resolution: Not a Bug Votes: 0
Labels: None


 Description   

The environment is a mixed of Mysql5.6 and MariaDB 10.0. RBR replication from 5.6 to 10.0 is working fine except we have a large volume of data copy from one innodb table into another innodb table.

example.

INSERT into lgV3_Report.T_Report_Copy
select * from lgV3_Report.T_Report where Date >='2015-2-20' AND Date < '2015-2-25';

Which could be a few million of rows. All MySQL 5.6 salves are committed this query without problem. However, on a MariaDB 10.0.17 salve, the SQL_Slave thread stopped with this error

"Got error 5 "Input/output error" during COMMIT;"

 Comments   
Comment by Elena Stepanova [ 2015-03-25 ]

I tried to reproduce with a simple table and 4M rows, it worked all right.

leomkkwan,
Is it reproducible every time?
If not, did you check for real input/output errors (maybe disk problems or something)?
If yes and there was none, please

  • provide cnf files for master and slave;
  • specify the exact version of MySQL 5.6;
  • paste output of SHOW CREATE TABLE T_Report_Copy or whichever table you insert into;
  • paste output of SHOW TRIGGERS LIKE 'T_Report_Copy' or whichever table you insert into.
  • attach the error log from the slave which covers the time when the problem occurred (preferably the whole log from the server startup till shutdown or till now if it hasn't been shutdown since then).

If any of this information is private, you can upload it to ftp.askmonty.org/private

Thanks.

Comment by Leo Kwan [ 2015-03-26 ]

It can be reproducible every time.

I got the exact error from the error log. 

150325 17:09:55 [Warning] WSREP: transaction size limit (1073741824) exceeded: 1073807360
 
150325 17:09:56 [ERROR] WSREP: rbr write fail, data_len: 0, 2
 
150325 17:11:16 [ERROR] Slave SQL: Got error 5 "Input/output error" during COMMIT, Internal MariaDB error code: 1180
 
150325 17:11:16 [Warning] Slave: Got error 5 "Input/output error" during COMMIT Error_code: 1180
 
150325 17:11:16 [ERROR] Error running query, slave SQL thread aborted. Fix the problem, and restart the slave SQL thread with "SLAVE START". We stopped at log 'DB41-bin.000911' position 920434229

This is a stand alone server running 10.0.17-MariaDB-wsrep-log. Running with the following config. wsrep is enabled, but it's only a one node cluster.

wsrep_provider                  = /usr/lib64/galera/libgalera_smm.so
 
wsrep_provider_options          = "gcache.size=100G; gcache.page_size=1G; gmcast.segment=1; gcs.fc_limit=256; gcs.fc_factor=0.99; gcs.fc_master_slave=yes;"
 
wsrep_cluster_name              = "Dash"
 
wsrep_cluster_address           = "gcomm://"

I am not sure this is a bug or not now.

Comment by Elena Stepanova [ 2015-03-26 ]

Thanks. It's still important that you are using the wsrep version – from the log exceprt, it looks like the problem might have started there.

nirbhay_c,
Could you please advise on this? Will the breach of transaction size limit end up with the 'input/output error', and if so, can it be avoided (e.g. is the limit configurable)?

Comment by Nirbhay Choubey (Inactive) [ 2015-03-27 ]

In galera cluster there is a limit set on the transaction size (1GB by default) that it can handle. It can be controlled by wsrep_max_ws_size system variable: https://mariadb.com/kb/en/mariadb/galera-cluster-system-variables/#wsrep_max_ws_size

Generated at Thu Feb 08 07:22:39 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.