[#CONJ-528] Error executing LOAD DATA LOCAL INFILE when file is larger than max_allowed

java.sql.SQLException: (conn=77811) Could not send query:...

java.sql.SQLException: (conn=77811) Could not send query: query size is >= to max_allowed_packet (16777216)

	at org.mariadb.jdbc.internal.util.exceptions.ExceptionMapper.get(ExceptionMapper.java:179)

	at org.mariadb.jdbc.internal.util.exceptions.ExceptionMapper.getException(ExceptionMapper.java:118)

	at org.mariadb.jdbc.MariaDbStatement.executeExceptionEpilogue(MariaDbStatement.java:235)

	at org.mariadb.jdbc.MariaDbPreparedStatementClient.executeInternal(MariaDbPreparedStatementClient.java:224)

	at org.mariadb.jdbc.MariaDbPreparedStatementClient.execute(MariaDbPreparedStatementClient.java:159)

	at org.mariadb.jdbc.MariaDbPreparedStatementClient.executeUpdate(MariaDbPreparedStatementClient.java:192)

	at com.inditex.distribucion.cloud.mariadb.ClientStandalone.executeUpdate(ClientStandalone.java:48)

	at com.inditex.distribucion.cloud.mariadb.ClientStandalone.main(ClientStandalone.java:116)

Caused by: java.sql.SQLException: Could not send query: query size is >= to max_allowed_packet (16777216)

Query is: LOAD DATA LOCAL INFILE 'c://pub/file.dat' IGNORE INTO TABLE tmp.data CHARACTER SET utf8 FIELDS TERMINATED BY '|'

	at org.mariadb.jdbc.internal.util.LogQueryTool.exceptionWithQuery(LogQueryTool.java:146)

	at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executeQuery(AbstractQueryProtocol.java:217)

	at org.mariadb.jdbc.MariaDbPreparedStatementClient.executeInternal(MariaDbPreparedStatementClient.java:218)

	... 4 more

Caused by: org.mariadb.jdbc.internal.util.exceptions.MaxAllowedPacketException: query size (22021201) is >= to max_allowed_packet (16777216)

	at org.mariadb.jdbc.internal.io.output.AbstractPacketOutputStream.flush(AbstractPacketOutputStream.java:179)

	at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.readLocalInfilePacket(AbstractQueryProtocol.java:1491)

	at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.readPacket(AbstractQueryProtocol.java:1297)

	at org.mariadb.jdbc.internal.protocol.A

I changed class org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol
method readLocalInfilePacket line ~1489 where the fileInputStream is read, to read it in chunks and send it in several packets of 1Mb size instead of sending it in only one.

   writer.write(is, false, false);

   writer.flush();

                byte[] buff = new byte[1024 * 1024];  //arbitrary size

                int read = 0;

                while ((read = is.read(buff)) != -1) {

                    writer.startPacket(seq);

                    writer.write(buff, 0, read);

                    writer.flush();

                    seq ++;

There is a test LocalInfileInputStreamTest test2xMaxAllowedPacketLocalInfileInputStream that explicitally checks that it must fail, I don´t understand why:

try {

            checkBigLocalInfile(maxAllowedPacket * 2);

            fail("must have fail");

        } catch (SQLException sqle) {

            assertTrue(sqle.getMessage().contains("Could not send query: query size ")

                    && sqle.getMessage().contains(" is >= to max_allowed_packet"));

[CONJ-528] Error executing LOAD DATA LOCAL INFILE when file is larger than max_allowed_packet Created: 2017-09-20 Updated: 2017-09-21 Resolved: 2017-09-21
Status:	Closed
Project:	MariaDB Connector/J
Component/s:	Other
Affects Version/s:	2.1.1
Fix Version/s:	1.6.5, 2.1.2

[CONJ-528] Error executing LOAD DATA LOCAL INFILE when file is larger than max_allowed_packet Created: 2017-09-20 Updated: 2017-09-21 Resolved: 2017-09-21