Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-28775

Implement connection manager in Spider

Details

    Description

      Implement a connection management mechanism to Spider to make the connection handing easy.

      • Each thread gets/returns connections from/to the connection manager. Allocation and deallocation of connections should be done inside the connection manager code.
      • Each connection got from the manager should solely belong to a single thread. Note that, in the current implementation, a single connection is referenced by both ha_spider::conns and SPIDER_TRX::trx_conn_hash and this results in the heap-use-after-free (see linked issues) in many test cases.
      • Spider 10.3-10.9 has two types of connections. One is for SQL and the other is for handler statements. The latter one has been deprecated from 10.7 and deleted by 10.10. So, we only implement the connection manager for SQL connections.
      • For the ease of the development, I recommend creating a patch for 10.10 first and then backporting it to 10.3.

      Attachments

        Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. MDEV-27902 Spider: Crashes, asserts, hangs, memory corruptions and ASAN heap-use-after-free's

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-28352 Spider: heap-use-after-free in ha_spider::lock_tables(), heap freed by spider_commit()

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-28676 Spider: Got error 12701 when reading table (and possible/previous ASAN: heap-use-after-free in ha_spider::external_lock) when using HANDLER

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-28683 Spider: SIGSEGV in spider_db_direct_delete, SIGSEGV in spider_db_connect, ASAN: heap-use-after-free in spider_db_direct_delete

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-29962 SIGSEGV in ha_spider::lock_tables on BEGIN after table lock

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Activity

            Ascending order - Click to sort in descending order
            nayuta-yanagisawa Nayuta Yanagisawa (Inactive) created issue -
            nayuta-yanagisawa Nayuta Yanagisawa (Inactive) made changes -
            Field Original Value New Value
            nayuta-yanagisawa Nayuta Yanagisawa (Inactive) made changes -
            nayuta-yanagisawa Nayuta Yanagisawa (Inactive) made changes -
            nayuta-yanagisawa Nayuta Yanagisawa (Inactive) made changes -
            nayuta-yanagisawa Nayuta Yanagisawa (Inactive) made changes -
            Description Implement a connection management mechanism to Spider to improve performance and fix memory bugs regarding connections to data nodes.

            Spider maintains connections to data nodes at least in {{ha_spider::conns}} and {{SPIDER_TRX::trx_conn_hash}}. This results in the heap-use-after-free (see linked issues). Also,             		Implement a connection management mechanism to Spider to make the connection handing easy.

            Each thread gets and returns connections from the connection manager. Each connection should be maintained in a single place

            Spider maintains connections to data nodes at least in {{ha_spider::conns}} and {{SPIDER_TRX::trx_conn_hash}}. This results in the heap-use-after-free (see linked issues). Also,
            nayuta-yanagisawa Nayuta Yanagisawa (Inactive) made changes -
            Description Implement a connection management mechanism to Spider to make the connection handing easy.

            Each thread gets and returns connections from the connection manager. Each connection should be maintained in a single place

            Spider maintains connections to data nodes at least in {{ha_spider::conns}} and {{SPIDER_TRX::trx_conn_hash}}. This results in the heap-use-after-free (see linked issues). Also,             		Implement a connection management mechanism to Spider to make the connection handing easy.

            Each thread gets and returns connections from the connection manager. Each connection got by a thread belongs to the thread, not {{ha_spider}} or {{SPIDER_TRX}}. Note that, in the current implementation, a single connection are referenced by both {{ha_spider::conns}} and {{SPIDER_TRX::trx_conn_hash}} and this results in the heap-use-after-free (see linked issues). Also,
            nayuta-yanagisawa Nayuta Yanagisawa (Inactive) made changes -
            nayuta-yanagisawa Nayuta Yanagisawa (Inactive) made changes -
            Description Implement a connection management mechanism to Spider to make the connection handing easy.

            Each thread gets and returns connections from the connection manager. Each connection got by a thread belongs to the thread, not {{ha_spider}} or {{SPIDER_TRX}}. Note that, in the current implementation, a single connection are referenced by both {{ha_spider::conns}} and {{SPIDER_TRX::trx_conn_hash}} and this results in the heap-use-after-free (see linked issues). Also,             		Implement a connection management mechanism to Spider to make the connection handing easy.

            Each thread gets and returns connections from the connection manager. Each connection got by a thread belongs to the thread, not {{ha_spider}} or {{SPIDER_TRX}}. Note that, in the current implementation, a single connection are referenced by both {{ha_spider::conns}} and {{SPIDER_TRX::trx_conn_hash}} and this results in the heap-use-after-free (see linked issues) in many test cases..
            nayuta-yanagisawa Nayuta Yanagisawa (Inactive) made changes -
            Description Implement a connection management mechanism to Spider to make the connection handing easy.

            Each thread gets and returns connections from the connection manager. Each connection got by a thread belongs to the thread, not {{ha_spider}} or {{SPIDER_TRX}}. Note that, in the current implementation, a single connection are referenced by both {{ha_spider::conns}} and {{SPIDER_TRX::trx_conn_hash}} and this results in the heap-use-after-free (see linked issues) in many test cases..             		Implement a connection management mechanism to Spider to make the connection handing easy.
            Each thread gets and returns connections from the connection manager. Each connection got by a thread belongs to the thread, not {{ha_spider}} or {{SPIDER_TRX}}. Note that, in the current implementation, a single connection are referenced by both {{ha_spider::conns}} and {{SPIDER_TRX::trx_conn_hash}} and this results in the heap-use-after-free (see linked issues) in many test cases..
            nayuta-yanagisawa Nayuta Yanagisawa (Inactive) made changes -
            Description Implement a connection management mechanism to Spider to make the connection handing easy.
            Each thread gets and returns connections from the connection manager. Each connection got by a thread belongs to the thread, not {{ha_spider}} or {{SPIDER_TRX}}. Note that, in the current implementation, a single connection are referenced by both {{ha_spider::conns}} and {{SPIDER_TRX::trx_conn_hash}} and this results in the heap-use-after-free (see linked issues) in many test cases..             		Implement a connection management mechanism to Spider to make the connection handing easy.

            Each thread gets and returns connections from the connection manager. Each connection got from the manager should belong to the thread, not {{ha_spider}} or {{SPIDER_TRX}}. Note that, in the current implementation, a single connection are referenced by both {{ha_spider::conns}} and {{SPIDER_TRX::trx_conn_hash}} and this results in the heap-use-after-free (see linked issues) in many test cases..
            nayuta-yanagisawa Nayuta Yanagisawa (Inactive) made changes -
            Summary Implement Spider connection manager Implement connection manager in Spider
            nayuta-yanagisawa Nayuta Yanagisawa (Inactive) made changes -
            Description Implement a connection management mechanism to Spider to make the connection handing easy.

            Each thread gets and returns connections from the connection manager. Each connection got from the manager should belong to the thread, not {{ha_spider}} or {{SPIDER_TRX}}. Note that, in the current implementation, a single connection are referenced by both {{ha_spider::conns}} and {{SPIDER_TRX::trx_conn_hash}} and this results in the heap-use-after-free (see linked issues) in many test cases..             		Implement a connection management mechanism to Spider to make the connection handing easy.

            Each thread gets and returns connections from the connection manager. Allocation and deallocation of connections are done inside the connection manager code.

            Each connection got from the manager should belong to the thread, not {{ha_spider}} or {{SPIDER_TRX}}. Note that, in the current implementation, a single connection are referenced by both {{ha_spider::conns}} and {{SPIDER_TRX::trx_conn_hash}} and this results in the heap-use-after-free (see linked issues) in many test cases..
            nayuta-yanagisawa Nayuta Yanagisawa (Inactive) made changes -
            Description Implement a connection management mechanism to Spider to make the connection handing easy.

            Each thread gets and returns connections from the connection manager. Allocation and deallocation of connections are done inside the connection manager code.

            Each connection got from the manager should belong to the thread, not {{ha_spider}} or {{SPIDER_TRX}}. Note that, in the current implementation, a single connection are referenced by both {{ha_spider::conns}} and {{SPIDER_TRX::trx_conn_hash}} and this results in the heap-use-after-free (see linked issues) in many test cases..             		Implement a connection management mechanism to Spider to make the connection handing easy.

            * Each thread gets and returns connections from the connection manager. Allocation and deallocation of connections are done inside the connection manager code.
            * Each connection got from the manager should belong to the thread, not {{ha_spider}} or {{SPIDER_TRX}}. Note that, in the current implementation, a single connection are referenced by both {{ha_spider::conns}} and {{SPIDER_TRX::trx_conn_hash}} and this results in the heap-use-after-free (see linked issues) in many test cases..
            nayuta-yanagisawa Nayuta Yanagisawa (Inactive) made changes -
            Description Implement a connection management mechanism to Spider to make the connection handing easy.

            * Each thread gets and returns connections from the connection manager. Allocation and deallocation of connections are done inside the connection manager code.
            * Each connection got from the manager should belong to the thread, not {{ha_spider}} or {{SPIDER_TRX}}. Note that, in the current implementation, a single connection are referenced by both {{ha_spider::conns}} and {{SPIDER_TRX::trx_conn_hash}} and this results in the heap-use-after-free (see linked issues) in many test cases..             		Implement a connection management mechanism to Spider to make the connection handing easy.

            * Each thread gets and returns connections from the connection manager. Allocation and deallocation of connections are done inside the connection manager code.
            * Each connection got from the manager should solely belong to the thread. Note that, in the current implementation, a single connection is referenced by both {{ha_spider::conns}} and {{SPIDER_TRX::trx_conn_hash}} and this results in the heap-use-after-free (see linked issues) in many test cases.
            nayuta-yanagisawa Nayuta Yanagisawa (Inactive) made changes -
            Description Implement a connection management mechanism to Spider to make the connection handing easy.

            * Each thread gets and returns connections from the connection manager. Allocation and deallocation of connections are done inside the connection manager code.
            * Each connection got from the manager should solely belong to the thread. Note that, in the current implementation, a single connection is referenced by both {{ha_spider::conns}} and {{SPIDER_TRX::trx_conn_hash}} and this results in the heap-use-after-free (see linked issues) in many test cases.             		Implement a connection management mechanism to Spider to make the connection handing easy.

            * Each thread gets/returns connections from/to the connection manager. Allocation and deallocation of connections are done inside the connection manager code.
            * Each connection got from the manager should solely belong to the thread. Note that, in the current implementation, a single connection is referenced by both {{ha_spider::conns}} and {{SPIDER_TRX::trx_conn_hash}} and this results in the heap-use-after-free (see linked issues) in many test cases.
            nayuta-yanagisawa Nayuta Yanagisawa (Inactive) made changes -
            Description Implement a connection management mechanism to Spider to make the connection handing easy.

            * Each thread gets/returns connections from/to the connection manager. Allocation and deallocation of connections are done inside the connection manager code.
            * Each connection got from the manager should solely belong to the thread. Note that, in the current implementation, a single connection is referenced by both {{ha_spider::conns}} and {{SPIDER_TRX::trx_conn_hash}} and this results in the heap-use-after-free (see linked issues) in many test cases.             		Implement a connection management mechanism to Spider to make the connection handing easy.

            * Each thread gets/returns connections from/to the connection manager. Allocation and deallocation of connections should be done inside the connection manager code.
            * Each connection got from the manager should solely belong to the thread. Note that, in the current implementation, a single connection is referenced by both {{ha_spider::conns}} and {{SPIDER_TRX::trx_conn_hash}} and this results in the heap-use-after-free (see linked issues) in many test cases.
            nayuta-yanagisawa Nayuta Yanagisawa (Inactive) made changes -
            Description Implement a connection management mechanism to Spider to make the connection handing easy.

            * Each thread gets/returns connections from/to the connection manager. Allocation and deallocation of connections should be done inside the connection manager code.
            * Each connection got from the manager should solely belong to the thread. Note that, in the current implementation, a single connection is referenced by both {{ha_spider::conns}} and {{SPIDER_TRX::trx_conn_hash}} and this results in the heap-use-after-free (see linked issues) in many test cases.             		Implement a connection management mechanism to Spider to make the connection handing easy.

            * Each thread gets/returns connections from/to the connection manager. Allocation and deallocation of connections should be done inside the connection manager code.
            * Each connection got from the manager should solely belong to a single thread. Note that, in the current implementation, a single connection is referenced by both {{ha_spider::conns}} and {{SPIDER_TRX::trx_conn_hash}} and this results in the heap-use-after-free (see linked issues) in many test cases.
            nayuta-yanagisawa Nayuta Yanagisawa (Inactive) made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            nayuta-yanagisawa Nayuta Yanagisawa (Inactive) made changes -
            Status In Progress [ 3 ] Stalled [ 10000 ]
            nayuta-yanagisawa Nayuta Yanagisawa (Inactive) made changes -
            Status Stalled [ 10000 ] In Progress [ 3 ]
            nayuta-yanagisawa Nayuta Yanagisawa (Inactive) made changes -
            Description Implement a connection management mechanism to Spider to make the connection handing easy.

            * Each thread gets/returns connections from/to the connection manager. Allocation and deallocation of connections should be done inside the connection manager code.
            * Each connection got from the manager should solely belong to a single thread. Note that, in the current implementation, a single connection is referenced by both {{ha_spider::conns}} and {{SPIDER_TRX::trx_conn_hash}} and this results in the heap-use-after-free (see linked issues) in many test cases.             		Implement a connection management mechanism to Spider to make the connection handing easy.

            * Each thread gets/returns connections from/to the connection manager. Allocation and deallocation of connections should be done inside the connection manager code.
            * Each connection got from the manager should solely belong to a single thread. Note that, in the current implementation, a single connection is referenced by both {{ha_spider::conns}} and {{SPIDER_TRX::trx_conn_hash}} and this results in the heap-use-after-free (see linked issues) in many test cases.
            * Spider 10.3-10.9 has two types of connections. One is for SQL and the other is for handler statements. The latter one has been deprecated from 10.7 and deleted by 10.10. So, we only implement the connection manager for SQL connections.
            nayuta-yanagisawa Nayuta Yanagisawa (Inactive) made changes -
            Description Implement a connection management mechanism to Spider to make the connection handing easy.

            * Each thread gets/returns connections from/to the connection manager. Allocation and deallocation of connections should be done inside the connection manager code.
            * Each connection got from the manager should solely belong to a single thread. Note that, in the current implementation, a single connection is referenced by both {{ha_spider::conns}} and {{SPIDER_TRX::trx_conn_hash}} and this results in the heap-use-after-free (see linked issues) in many test cases.
            * Spider 10.3-10.9 has two types of connections. One is for SQL and the other is for handler statements. The latter one has been deprecated from 10.7 and deleted by 10.10. So, we only implement the connection manager for SQL connections.             		Implement a connection management mechanism to Spider to make the connection handing easy.

            * Each thread gets/returns connections from/to the connection manager. Allocation and deallocation of connections should be done inside the connection manager code.
            * Each connection got from the manager should solely belong to a single thread. Note that, in the current implementation, a single connection is referenced by both {{ha_spider::conns}} and {{SPIDER_TRX::trx_conn_hash}} and this results in the heap-use-after-free (see linked issues) in many test cases.
            * Spider 10.3-10.9 has two types of connections. One is for SQL and the other is for handler statements. The latter one has been deprecated from 10.7 and deleted by 10.10. So, we only implement the connection manager for SQL connections.
            * For the ease of the development, I recommend creating a patch for 10.10 first and then backporting it to 10.3.
            nayuta-yanagisawa Nayuta Yanagisawa (Inactive) made changes -
            Fix Version/s 10.11 [ 27614 ]
            Fix Version/s 10.3 [ 22126 ]
            Fix Version/s 10.4 [ 22408 ]
            Fix Version/s 10.5 [ 23123 ]
            Fix Version/s 10.6 [ 24028 ]
            Fix Version/s 10.7 [ 24805 ]
            Fix Version/s 10.8 [ 26121 ]
            Fix Version/s 10.9 [ 26905 ]
            nayuta-yanagisawa Nayuta Yanagisawa (Inactive) made changes -
            Status In Progress [ 3 ] Stalled [ 10000 ]
            ralf.gebhardt Ralf Gebhardt made changes -
            Fix Version/s 10.12 [ 28320 ]
            Fix Version/s 10.11 [ 27614 ]
            nayuta-yanagisawa Nayuta Yanagisawa (Inactive) made changes -
            Priority Critical [ 2 ] Major [ 3 ]
            ralf.gebhardt Ralf Gebhardt made changes -
            Assignee Nayuta Yanagisawa [ JIRAUSER47117 ]
            ralf.gebhardt Ralf Gebhardt made changes -
            Fix Version/s 11.0 [ 28320 ]
            ycp Yuchen Pei made changes -
            Assignee Yuchen Pei [ JIRAUSER52627 ]
            ralf.gebhardt Ralf Gebhardt made changes -
            Assignee Yuchen Pei [ JIRAUSER52627 ]
            ycp Yuchen Pei made changes -

            People

              Unassigned Unassigned
              nayuta-yanagisawa Nayuta Yanagisawa (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.