Type:
Task
Priority:
Minor
Resolution:
Fixed
Affects Version/s:
1.1.7
Component/s:
None
Environment:
Long network delay between client and server.
Mysql JDBC Connector has a rewriteBatchedStatements proptery (http://dev.mysql.com/doc/connector-j/en/connector-j-reference-configuration-properties.html ) that does 2 things:
1. INSERT statements are is rewritten into a long INSERT INTO table VALUES(0, 'name0'), (1, 'name1') … query
2. Try to uses the “multiple queries” feature for things that does not fit (1.).
Would it be possible to implement support for this in MariaDB JDBC connector as well? It could be useful if the distance (ping) between client and server is high.
The performance gain for (1.) is very high. Although to be fair, a workaround is to use LOAD DATA INFILE that has equal or greater performance.
The performance gain for (2.) is medium, because it looks like (at least for me) that the server will reply with one TCP segment per query. At least the transfer of data to the server is more efficient.
JDBCBatchStatements.java
package se.lesc;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.PreparedStatement;
import java.sql.SQLException;
import java.sql.Statement;
import org.junit.Before;
import org.junit.Test;
public class JDBCBatchStatements {
private Connection connection;
@Before
public void setUp() throws SQLException {
String url = "jdbc:mysql://127.0.0.1/test?user=root&password=";
//rewriteBatchedStatements is supported by Mysql JDBC
//http://dev.mysql.com/doc/connector-j/en/connector-j-reference-configuration-properties.html
url += "&rewriteBatchedStatements=true";
connection = DriverManager.getConnection(url);
Statement statement = connection.createStatement();
statement.execute("DROP TABLE IF EXISTS persons");
statement.execute("CREATE TABLE persons (ssn int, name VARCHAR(40))");
}
@Test
public void testInsert() throws SQLException {
PreparedStatement preparedStatement = connection.prepareStatement("INSERT INTO persons VALUES(?, ?)");
for (int i = 0; i < 1000; i++) {
preparedStatement.setInt(1, i);
preparedStatement.setString(2, "name" + i);
preparedStatement.addBatch();
}
//Mysql JDBC driver will rewrite the SQL into:
//INSERT INTO persons VALUES(0, 'name0'),(1, 'name1'),(2, 'name2'),(3, 'name3') ...
preparedStatement.executeBatch();
}
@Test
public void testUpdate() throws SQLException {
testInsert(); //Populate the table
PreparedStatement preparedStatement = connection.prepareStatement("UPDATE persons SET name = ? WHERE SSN = ?");
for (int i = 0; i < 1000; i++) {
preparedStatement.setString(1, "updated name" + i);
preparedStatement.setInt(2, i);
preparedStatement.addBatch();
}
//Mysql JDBC driver will rewrite the SQL into:
//UPDATE persons SET name = 'updated name0' WHERE SSN = 0;UPDATE persons SET name = 'updated name1' WHERE SSN = 1; ...
preparedStatement.executeBatch();
//Unfortunately there will be one response packet per query:
//Rows matched: 1 Changed: 1 Warnings: 0
//It resulted in 1501 TCP segments (including ACK) for me
}
}
Rasmus Johansson (Inactive)
made changes -
2014-06-19 15:09
Field
Original Value
New Value
Assignee
Georg Richter
[ georg
]
Massimo Siani
[ massimo.siani
]
Lennart Schedin
made changes -
2014-09-18 09:41
Affects Version/s
1.1.7
[ 15300
]
Sergei Golubchik
made changes -
2021-12-06 21:18
Workflow
MariaDB v3
[ 70239
]
MariaDB v4
[ 128249
]
{"report":{"fcp":1446,"ttfb":628.8999999761581,"pageVisibility":"visible","entityId":36801,"key":"jira.project.issue.view-issue","isInitial":true,"threshold":1000,"elementTimings":{},"userDeviceMemory":8,"userDeviceProcessors":64,"apdex":0.5,"journeyId":"b52fe255-b222-4d27-9576-53d09d533a99","navigationType":0,"readyForUser":1540.199999988079,"redirectCount":0,"resourceLoadedEnd":1752.3999999761581,"resourceLoadedStart":634.3000000119209,"resourceTiming":[{"duration":337.39999997615814,"initiatorType":"link","name":"https://jira.mariadb.org/s/2c21342762a6a02add1c328bed317ffd-CDN/lu2bu7/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/css/_super/batch.css","startTime":634.3000000119209,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":634.3000000119209,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":971.6999999880791,"responseStart":0,"secureConnectionStart":0},{"duration":337.4000000357628,"initiatorType":"link","name":"https://jira.mariadb.org/s/7ebd35e77e471bc30ff0eba799ebc151-CDN/lu2bu7/820016/12ta74/8679b4946efa1a0bb029a3a22206fb5d/_/download/contextbatch/css/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true","startTime":634.5999999642372,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":634.5999999642372,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":972,"responseStart":0,"secureConnectionStart":0},{"duration":346,"initiatorType":"script","name":"https://jira.mariadb.org/s/fbf975c0cce4b1abf04784eeae9ba1f4-CDN/lu2bu7/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/js/_super/batch.js?locale=en","startTime":634.8000000119209,"connectEnd":634.8000000119209,"connectStart":634.8000000119209,"domainLookupEnd":634.8000000119209,"domainLookupStart":634.8000000119209,"fetchStart":634.8000000119209,"redirectEnd":0,"redirectStart":0,"requestStart":634.8000000119209,"responseEnd":980.8000000119209,"responseStart":980.8000000119209,"secureConnectionStart":634.8000000119209},{"duration":386.89999997615814,"initiatorType":"script","name":"https://jira.mariadb.org/s/099b33461394b8015fc36c0a4b96e19f-CDN/lu2bu7/820016/12ta74/8679b4946efa1a0bb029a3a22206fb5d/_/download/contextbatch/js/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true","startTime":635,"connectEnd":635,"connectStart":635,"domainLookupEnd":635,"domainLookupStart":635,"fetchStart":635,"redirectEnd":0,"redirectStart":0,"requestStart":635,"responseEnd":1021.8999999761581,"responseStart":1021.8999999761581,"secureConnectionStart":635},{"duration":390.30000001192093,"initiatorType":"script","name":"https://jira.mariadb.org/s/94c15bff32baef80f4096a08aceae8bc-CDN/lu2bu7/820016/12ta74/c92c0caa9a024ae85b0ebdbed7fb4bd7/_/download/contextbatch/js/atl.global,-_super/batch.js?locale=en","startTime":635.1999999880791,"connectEnd":635.1999999880791,"connectStart":635.1999999880791,"domainLookupEnd":635.1999999880791,"domainLookupStart":635.1999999880791,"fetchStart":635.1999999880791,"redirectEnd":0,"redirectStart":0,"requestStart":635.1999999880791,"responseEnd":1025.5,"responseStart":1025.5,"secureConnectionStart":635.1999999880791},{"duration":390.60000002384186,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-en/jira.webresources:calendar-en.js","startTime":635.3999999761581,"connectEnd":635.3999999761581,"connectStart":635.3999999761581,"domainLookupEnd":635.3999999761581,"domainLookupStart":635.3999999761581,"fetchStart":635.3999999761581,"redirectEnd":0,"redirectStart":0,"requestStart":635.3999999761581,"responseEnd":1026,"responseStart":1026,"secureConnectionStart":635.3999999761581},{"duration":390.80000001192093,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-localisation-moment/jira.webresources:calendar-localisation-moment.js","startTime":635.6999999880791,"connectEnd":635.6999999880791,"connectStart":635.6999999880791,"domainLookupEnd":635.6999999880791,"domainLookupStart":635.6999999880791,"fetchStart":635.6999999880791,"redirectEnd":0,"redirectStart":0,"requestStart":635.6999999880791,"responseEnd":1026.5,"responseStart":1026.5,"secureConnectionStart":635.6999999880791},{"duration":487.69999998807907,"initiatorType":"link","name":"https://jira.mariadb.org/s/b04b06a02d1959df322d9cded3aeecc1-CDN/lu2bu7/820016/12ta74/a2ff6aa845ffc9a1d22fe23d9ee791fc/_/download/contextbatch/css/jira.global.look-and-feel,-_super/batch.css","startTime":635.8000000119209,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":635.8000000119209,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1123.5,"responseStart":0,"secureConnectionStart":0},{"duration":390.89999997615814,"initiatorType":"script","name":"https://jira.mariadb.org/rest/api/1.0/shortcuts/820016/47140b6e0a9bc2e4913da06536125810/shortcuts.js?context=issuenavigation&context=issueaction","startTime":636,"connectEnd":636,"connectStart":636,"domainLookupEnd":636,"domainLookupStart":636,"fetchStart":636,"redirectEnd":0,"redirectStart":0,"requestStart":636,"responseEnd":1026.8999999761581,"responseStart":1026.8999999761581,"secureConnectionStart":636},{"duration":487.5,"initiatorType":"link","name":"https://jira.mariadb.org/s/3ac36323ba5e4eb0af2aa7ac7211b4bb-CDN/lu2bu7/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/css/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.css?jira.create.linked.issue=true","startTime":636.1999999880791,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":636.1999999880791,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1123.699999988079,"responseStart":0,"secureConnectionStart":0},{"duration":391,"initiatorType":"script","name":"https://jira.mariadb.org/s/3339d87fa2538a859872f2df449bf8d0-CDN/lu2bu7/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/js/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.js?jira.create.linked.issue=true&locale=en","startTime":636.3999999761581,"connectEnd":636.3999999761581,"connectStart":636.3999999761581,"domainLookupEnd":636.3999999761581,"domainLookupStart":636.3999999761581,"fetchStart":636.3999999761581,"redirectEnd":0,"redirectStart":0,"requestStart":636.3999999761581,"responseEnd":1027.3999999761581,"responseStart":1027.3999999761581,"secureConnectionStart":636.3999999761581},{"duration":1096.8999999761581,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-js/jira.webresources:bigpipe-js.js","startTime":642.3000000119209,"connectEnd":642.3000000119209,"connectStart":642.3000000119209,"domainLookupEnd":642.3000000119209,"domainLookupStart":642.3000000119209,"fetchStart":642.3000000119209,"redirectEnd":0,"redirectStart":0,"requestStart":642.3000000119209,"responseEnd":1739.199999988079,"responseStart":1739.199999988079,"secureConnectionStart":642.3000000119209},{"duration":1098.2999999523163,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-init/jira.webresources:bigpipe-init.js","startTime":642.3000000119209,"connectEnd":642.3000000119209,"connectStart":642.3000000119209,"domainLookupEnd":642.3000000119209,"domainLookupStart":642.3000000119209,"fetchStart":642.3000000119209,"redirectEnd":0,"redirectStart":0,"requestStart":642.3000000119209,"responseEnd":1740.5999999642372,"responseStart":1740.5999999642372,"secureConnectionStart":642.3000000119209},{"duration":184.10000002384186,"initiatorType":"xmlhttprequest","name":"https://jira.mariadb.org/rest/webResources/1.0/resources","startTime":1151.3999999761581,"connectEnd":1151.3999999761581,"connectStart":1151.3999999761581,"domainLookupEnd":1151.3999999761581,"domainLookupStart":1151.3999999761581,"fetchStart":1151.3999999761581,"redirectEnd":0,"redirectStart":0,"requestStart":1151.3999999761581,"responseEnd":1335.5,"responseStart":1335.5,"secureConnectionStart":1151.3999999761581},{"duration":343.80000001192093,"initiatorType":"link","name":"https://jira.mariadb.org/s/d5715adaadd168a9002b108b2b039b50-CDN/lu2bu7/820016/12ta74/be4b45e9cec53099498fa61c8b7acba4/_/download/contextbatch/css/jira.project.sidebar,-_super,-project.issue.navigator,-jira.general,-jira.browse.project,-jira.view.issue,-jira.global,-atl.general,-com.atlassian.jira.projects.sidebar.init/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true","startTime":1398.5999999642372,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":1398.5999999642372,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1742.3999999761581,"responseStart":0,"secureConnectionStart":0},{"duration":346,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/e65b778d185daf5aee24936755b43da6/_/download/contextbatch/js/browser-metrics-plugin.contrib,-_super,-project.issue.navigator,-jira.view.issue,-atl.general/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true","startTime":1399.5,"connectEnd":1399.5,"connectStart":1399.5,"domainLookupEnd":1399.5,"domainLookupStart":1399.5,"fetchStart":1399.5,"redirectEnd":0,"redirectStart":0,"requestStart":1399.5,"responseEnd":1745.5,"responseStart":1745.5,"secureConnectionStart":1399.5},{"duration":371.4000000357628,"initiatorType":"script","name":"https://www.google-analytics.com/analytics.js","startTime":1439.8999999761581,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":1439.8999999761581,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1811.300000011921,"responseStart":0,"secureConnectionStart":0},{"duration":352.5,"initiatorType":"script","name":"https://jira.mariadb.org/s/f51ef5507eea4c158f257c66c93b2a3f-CDN/lu2bu7/820016/12ta74/be4b45e9cec53099498fa61c8b7acba4/_/download/contextbatch/js/jira.project.sidebar,-_super,-project.issue.navigator,-jira.general,-jira.browse.project,-jira.view.issue,-jira.global,-atl.general,-com.atlassian.jira.projects.sidebar.init/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true","startTime":1399.8999999761581,"connectEnd":1399.8999999761581,"connectStart":1399.8999999761581,"domainLookupEnd":1399.8999999761581,"domainLookupStart":1399.8999999761581,"fetchStart":1399.8999999761581,"redirectEnd":0,"redirectStart":0,"requestStart":1399.8999999761581,"responseEnd":1752.3999999761581,"responseStart":1752.3999999761581,"secureConnectionStart":1399.8999999761581}],"fetchStart":0,"domainLookupStart":0,"domainLookupEnd":0,"connectStart":0,"connectEnd":0,"requestStart":6,"responseStart":629,"responseEnd":638,"domLoading":632,"domInteractive":1816,"domContentLoadedEventStart":1816,"domContentLoadedEventEnd":1866,"domComplete":2034,"loadEventStart":2034,"loadEventEnd":2035,"userAgent":"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)","marks":[{"name":"bigPipe.sidebar-id.start","time":1749.3999999761581},{"name":"bigPipe.sidebar-id.end","time":1750.199999988079},{"name":"bigPipe.activity-panel-pipe-id.start","time":1750.300000011921},{"name":"bigPipe.activity-panel-pipe-id.end","time":1754},{"name":"activityTabFullyLoaded","time":1891.5}],"measures":[],"correlationId":"e439ba82b6979d","effectiveType":"4g","downlink":10,"rtt":0,"serverDuration":553,"dbReadsTimeInMs":28,"dbConnsTimeInMs":39,"applicationHash":"9d11dbea5f4be3d4cc21f03a88dd11d8c8687422","experiments":[]}}
Isn't the first part of this already implemented? I add rewriteBatchedStatements=true as a parameter and I see multiple batched insert statements rewritten.