Uploaded image for project: 'MariaDB Connector/J'
  1. MariaDB Connector/J
  2. CONJ-1017

Calendar possible race condition, cause wrong timestamp setting

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Fixed
    • 3.0.8
    • 3.0.9
    • Other
    • None

    Description

      For instance, given the following data:

      Id Event_Date_Time
      0 2022-09-01T00:00:00Z
      1 2022-09-01T01:00:00Z
      2 2022-09-01T02:00:00Z
      3 2022-09-01T03:00:00Z
      4 2022-09-01T04:00:00Z
      5 2022-09-02T00:00:00Z
      6 2022-09-02T01:00:00Z
      7 2022-09-02T02:00:00Z
      8 2022-09-02T03:00:00Z
      9 2022-09-02T04:00:00Z

      If I insert these rows with 2 threads in parallel (one thread bulk inserting the 5 first rows and another thread bulk inserting the 5 last rows), when I select the inserted rows, the values are randomly scrambled, for instance:

      Id Event_Date_Time
      0 2022-09-01T00:00:00Z
      1 2022-09-02T01:00:00Z
      2 2022-09-02T03:00:00Z
      3 2022-09-02T04:00:00Z
      4 2022-09-01T04:00:00Z
      5 2022-09-01T00:00:00Z
      6 2022-09-02T01:00:00Z
      7 2022-09-02T02:00:00Z
      8 2022-09-02T03:00:00Z
      9 2022-09-02T04:00:00Z

      If I insert these rows in bulk without concurrence, there's no issue.

      Demo : https://github.com/romainmoreau/mariadb-demo

      Attachments

        Activity

          diego dupin Diego Dupin added a comment -

          In your example, the behavior is expected :
          ORM code

          		for (var demo : demoList) {
          			if (i > 0 && i % BATCH_SIZE == 0) {
          				entityManager.flush();
          				entityManager.clear();
          			}
          			entityManager.persist(demo);
          			i++;
          		}
          

          Will result in hibernate internally executing 5 * JDBC prepareStatement.execute().
          Since there is 2 paralleles runner, this result with INSERT with mixed IDs, like you describe.
          There is nothing wrong in MariaDB Server or connector there.
          This is just how hibernate send the data.

          There is some solutions hibernate side, like for example adding batch_size :

          spring:
            jpa:
              hibernate:
                jdbc:
                  batch_size: 2000
          
          

          In that case, hibernate will not run some PrepareStatement.execute, but will execute those in batch (preparestatement.addBatch() x times, then preparestatement.executeBatch)
          MariaDB connector and Server will then receive only one command with all the insert at a time, then IDs will be orderered.

          diego dupin Diego Dupin added a comment - In your example, the behavior is expected : ORM code for (var demo : demoList) { if (i > 0 && i % BATCH_SIZE == 0 ) { entityManager.flush(); entityManager.clear(); } entityManager.persist(demo); i++; } Will result in hibernate internally executing 5 * JDBC prepareStatement.execute(). Since there is 2 paralleles runner, this result with INSERT with mixed IDs, like you describe. There is nothing wrong in MariaDB Server or connector there. This is just how hibernate send the data. There is some solutions hibernate side, like for example adding batch_size : spring: jpa: hibernate: jdbc: batch_size: 2000 In that case, hibernate will not run some PrepareStatement.execute, but will execute those in batch (preparestatement.addBatch() x times, then preparestatement.executeBatch) MariaDB connector and Server will then receive only one command with all the insert at a time, then IDs will be orderered.
          romainmoreau Romain Moreau added a comment - - edited

          If you debug the demo, you'll see it's not the way you describe because you seem to have missed this line which sets the bach size only for the current hibernate Session instead of setting it globally like you were suggesting:

          entityManager.unwrap(Session.class).setJdbcBatchSize(BATCH_SIZE);
          

          In fact, this part is not executed in the current settings of the demo (4 objects by thread) but I left it to support lists which have a size higher than BATCH_SIZE (50):

          entityManager.flush();
          entityManager.clear();
          

          I added the following to show what commands the MariaDB connector client was sending:

          logging:
            level:
              '[org.mariadb]': DEBUG
          

          And aside from transactions related commands, only two commands are issued by each thread : one prepare (PreparePacket) and one bulk insert (BulkExecutePacket) with no mixed up data when I inspect the batchParameterList field.

          romainmoreau Romain Moreau added a comment - - edited If you debug the demo, you'll see it's not the way you describe because you seem to have missed this line which sets the bach size only for the current hibernate Session instead of setting it globally like you were suggesting: entityManager.unwrap(Session. class ).setJdbcBatchSize(BATCH_SIZE); In fact, this part is not executed in the current settings of the demo (4 objects by thread) but I left it to support lists which have a size higher than BATCH_SIZE (50): entityManager.flush(); entityManager.clear(); I added the following to show what commands the MariaDB connector client was sending: logging: level: '[org.mariadb]': DEBUG And aside from transactions related commands, only two commands are issued by each thread : one prepare ( PreparePacket ) and one bulk insert ( BulkExecutePacket ) with no mixed up data when I inspect the batchParameterList field.
          diego dupin Diego Dupin added a comment - - edited

          sorry, i've made bad assumption. This is indeed an issue connector side : calendar use in prepareStatement.setTimestamp(int parameterIndex, Timestamp x, Calendar cal) is not used in synchronized block, resulting in this issue when use in parallel

          problem is only in 3.0 version

          diego dupin Diego Dupin added a comment - - edited sorry, i've made bad assumption. This is indeed an issue connector side : calendar use in prepareStatement.setTimestamp(int parameterIndex, Timestamp x, Calendar cal) is not used in synchronized block, resulting in this issue when use in parallel problem is only in 3.0 version
          diego dupin Diego Dupin added a comment -

          corrected in 3.0.9-SNAPSHOT version with commit https://github.com/mariadb-corporation/mariadb-connector-j/commit/22388e698edb045ea9e8c560195caa9bf4bb6da3

          available using snapshot repository :

          <repositories>
              <repository>
                  <id>sonatype-nexus-snapshots</id>
                  <name>Sonatype Nexus Snapshots</name>
                  <url>https://oss.sonatype.org/content/repositories/snapshots</url>
              </repository>
          </repositories>
           
          <dependencies>
              <dependency>
                  <groupId>org.mariadb.jdbc</groupId>
                  <artifactId>mariadb-java-client</artifactId>
                  <version>3.0.9-SNAPSHOT</version>
              </dependency>
          </dependencies>
          
          

          diego dupin Diego Dupin added a comment - corrected in 3.0.9-SNAPSHOT version with commit https://github.com/mariadb-corporation/mariadb-connector-j/commit/22388e698edb045ea9e8c560195caa9bf4bb6da3 available using snapshot repository : < repositories > < repository > < id >sonatype-nexus-snapshots</ id > < name >Sonatype Nexus Snapshots</ name > < url >https://oss.sonatype.org/content/repositories/snapshots</ url > </ repository > </ repositories >   < dependencies > < dependency > < groupId >org.mariadb.jdbc</ groupId > < artifactId >mariadb-java-client</ artifactId > < version >3.0.9-SNAPSHOT</ version > </ dependency > </ dependencies >
          romainmoreau Romain Moreau added a comment -

          It works great with the demo and also with a real world application, thanks!

          romainmoreau Romain Moreau added a comment - It works great with the demo and also with a real world application, thanks!

          People

            diego dupin Diego Dupin
            romainmoreau Romain Moreau
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.