[CONJ-553] RejectedExecutionException sending a large amount of concurrent batches Created: 2017-11-29 Updated: 2017-12-26 Resolved: 2017-12-21 |
|
| Status: | Closed |
| Project: | MariaDB Connector/J |
| Component/s: | Other |
| Affects Version/s: | 2.1.1 |
| Fix Version/s: | 2.2.1, 1.7.1 |
| Type: | Bug | Priority: | Minor |
| Reporter: | María | Assignee: | Diego Dupin |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Attachments: |
|
| Description |
|
RejectedExecutionException is thrown in high load environments, while issuing batch operations.
We think that changing the executor rejection policy in class org.mariadb.jdbc.internal.util.scheduler.SchedulerServiceProviderHolder, the problem would be solved. Attached example and patch. |
| Comments |
| Comment by Diego Dupin [ 2017-11-29 ] | |||||||||||||||||
|
Hi, Thanks for reporting this issue. The best solution is that if the thread pool queue is full, the driver must not use pipelining, use standard send query/read result, send next query / read result ... Can you confirm that server is < 10.2.4 ? (driver would then use a dedicated protocol for batch that has better performance results) | |||||||||||||||||
| Comment by María [ 2017-11-29 ] | |||||||||||||||||
|
Hi, | |||||||||||||||||
| Comment by Diego Dupin [ 2017-11-29 ] | |||||||||||||||||
|
Bad assumption, i've read example to quickly : example is using Statement.addBatch(), not PreparedStatement.addBatch(). | |||||||||||||||||
| Comment by María [ 2017-11-30 ] | |||||||||||||||||
|
And something like this? Enqueueing again the rejected runnables. If the queue is full, threadPool.execute() will block.
| |||||||||||||||||
| Comment by Diego Dupin [ 2017-11-30 ] | |||||||||||||||||
|
wlad indicate a smart idea: the best would be using non-blocking sockets, driver can send until EWOULDBLOCK (buffer is full), that would avoid any need of other thread. That mean using NIO2, possibly NIO. Possible implementation using NIO Channels seem to permit having that information :
| |||||||||||||||||
| Comment by Diego Dupin [ 2017-12-13 ] | |||||||||||||||||
|
Using non-blocking socket is a solution that needs profound change and cannot be correct in next corrective version. Until then, there are different solutions, but the only reliable one is to cancel pipelining use: this permit to avoid any timeout limit issue queuing, and perform better (there is already more than 100 batches running from one client) | |||||||||||||||||
| Comment by María [ 2017-12-26 ] | |||||||||||||||||
|
Thank you! |