Diego Dupin added a comment - 2017-11-29 16:28 - edited Hi, Thanks for reporting this issue. The patch can solve the issue, but not in a reliable way: using the same thread to send and read using pipelining may result having client socket send buffer full and client socket read buffer full at the same time resulting in a lock. The best solution is that if the thread pool queue is full, the driver must not use pipelining, use standard send query/read result, send next query / read result ... Can you confirm that server is < 10.2.4 ? (driver would then use a dedicated protocol for batch that has better performance results)

María added a comment - 2017-11-29 16:37 Hi, Server version is 10.2.10.

Diego Dupin added a comment - 2017-11-29 16:52 Bad assumption, i've read example to quickly : example is using Statement.addBatch(), not PreparedStatement.addBatch().

María added a comment - 2017-11-30 09:59 And something like this? Enqueueing again the rejected runnables. If the queue is full, threadPool.execute() will block. I don´t know if this would cause blocking due to I/O operations wrong ordering. @Override public ThreadPoolExecutor getBulkScheduler() { final int queueSize = 200 ; //magic number final BlockingQueue<Runnable> queue = new ArrayBlockingQueue<Runnable>(queueSize); final ThreadPoolExecutor tp = new ThreadPoolExecutor( 5 , 100 , 1 , TimeUnit.MINUTES, queue, new MariaDbThreadFactory( "bulk" )); tp.setRejectedExecutionHandler( new RejectedExecutionHandler() { @Override public void rejectedExecution( final Runnable r, final ThreadPoolExecutor executor) { try { tp.getQueue().put(r); } catch ( final InterruptedException ie) { throw new RejectedExecutionException(ie); } } }); return tp; }

Diego Dupin added a comment - 2017-11-30 13:09 wlad indicate a smart idea: the best would be using non-blocking sockets, driver can send until EWOULDBLOCK (buffer is full), that would avoid any need of other thread. That mean using NIO2, possibly NIO. Possible implementation using NIO Channels seem to permit having that information : from https://docs.oracle.com/javase/7/docs/api/java/nio/channels/SocketChannel.html#write(java.nio.ByteBuffer ) A socket channel in non-blocking mode, for example, cannot write any more bytes than are free in the socket's output buffer.

Diego Dupin added a comment - 2017-12-13 17:06 Using non-blocking socket is a solution that needs profound change and cannot be correct in next corrective version. That will be handled in CONJ-447 . Until then, there are different solutions, but the only reliable one is to cancel pipelining use: this permit to avoid any timeout limit issue queuing, and perform better (there is already more than 100 batches running from one client)

María added a comment - 2017-12-26 13:51 Thank you!