Uploaded image for project: 'MariaDB MaxScale'
  1. MariaDB MaxScale
  2. MXS-16

bugzillaId-670: Tee filter: statement router loses statements when other router gets enough ahead

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Unresolved
    • 1.0.2
    • 1.1.0
    • Plugin, tee
    • None
    • Linux

    Description

      This is import of bugzilla item http://bugs.mariadb.com/show_bug.cgi?id=670
      The fix version in Bugzilla is release-1.0GA in commit 9b52f5e6bab629149ca584c510fd4e805085a1e9

      Description Vilho Raatikka 2014-12-30 11:54:52 UTC
      Statement router (readwritesplit) loses pending statement if the other router executes statements faster than it. Statement router assumes that client doesn't send next statement before previous has replied. The only supported exception is session command which doesn't need to reply before client may send the next statement.

      What happens in practice, is, that when 'next' statement arrives router's routeQuery it finds previous, still pending statement. In Debug build process traps. In Release build the pending command is overwritten.

       
      (gdb) bt
      #0  0x00007f050fa56065 in raise () from /lib64/libc.so.6
      #1  0x00007f050fa574e8 in abort () from /lib64/libc.so.6
      #2  0x00007f050fa4ef72 in __assert_fail_base () from /lib64/libc.so.6
      #3  0x00007f050fa4f022 in __assert_fail () from /lib64/libc.so.6
      #4  0x00007f050c06ddbb in route_single_stmt (inst=0x25ea750, rses=0x7f04d4002350, querybuf=0x7f04d400f130)
          at /home/raatikka/src/git/MaxScale/server/modules/routing/readwritesplit/readwritesplit.c:2372
      #5  0x00007f050c06c2ef in routeQuery (instance=0x25ea750, router_session=0x7f04d4002350, querybuf=0x7f04d400f130)
          at /home/raatikka/src/git/MaxScale/server/modules/routing/readwritesplit/readwritesplit.c:1895
      #6  0x00007f04f03ca573 in routeQuery (instance=0x7f04d4001f20, session=0x7f04d4002050, queue=0x7f04d400f130)
          at /home/raatikka/src/git/MaxScale/server/modules/filter/tee.c:597
      #7  0x00007f04f8df700a in gw_read_client_event (dcb=0x7f04cc0009c0)
          at /home/raatikka/src/git/MaxScale/server/modules/protocol/mysql_client.c:867
      #8  0x000000000058b351 in process_pollq (thread_id=4) at /home/raatikka/src/git/MaxScale/server/core/poll.c:858
      #9  0x000000000058a9eb in poll_waitevents (arg=0x4) at /home/raatikka/src/git/MaxScale/server/core/poll.c:608
      #10 0x00007f0511223e0f in start_thread () from /lib64/libpthread.so.0
      #11 0x00007f050fb0a0dd in clone () from /lib64/libc.so.6
      (gdb)

      Attachments

        Activity

          dshjoshi Dipti Joshi (Inactive) added a comment - - edited

          These are the comments imported from bugzilla

          Vilho Raatikka 2014-12-30 12:04:06 UTC
          Created attachment 171 :http://bugs.mariadb.com/attachment.cgi?id=171
          Configuration.

          1. Start MaxScale
          2. feed session command/statement load to port 4008 which belongs to readconnrouter. Statements are then duplicated to rwsplit which starts to lag behind immediately.

          Comment 2 Vilho Raatikka 2014-12-30 12:05:26 UTC
          Created attachment 172 http://bugs.mariadb.com/attachment.cgi?id=172
          Simple script to run session command/statement load

          Requires setmix.sql

          Comment 3 Vilho Raatikka 2014-12-30 12:05:49 UTC
          Created attachment 173 http://bugs.mariadb.com/attachment.cgi?id=173
          List of statements used by run_setmix.sh

          Comment 4 Vilho Raatikka 2014-12-31 19:13:21 UTC
          tee filter doesn't send reply to client before both routers have replied. This required adding upstream processing to tee filter. First reply is routed to client. By this tee ensures that new query is never sent to either router before they have replied to previous one.

          Comment 5 Timofey Turenko 2015-01-08 12:40:34 UTC
          test added, closing

          Comment 6 Timofey Turenko 2015-02-28 18:11:16 UTC
          Reopen: starting from 1.0.5 test for bug670 start to fail with debug assert:

          (gdb) bt
          #0  0x00007f542de05625 in raise () from /lib64/libc.so.6
          #1  0x00007f542de06e05 in abort () from /lib64/libc.so.6
          #2  0x00007f542ddfe74e in __assert_fail_base () from /lib64/libc.so.6
          #3  0x00007f542ddfe810 in __assert_fail () from /lib64/libc.so.6
          #4  0x00007f53fe7a2daf in clientReply (instance=0x7f53ec001970, session=0x7f53ec001aa0, reply=0x7f5404000b70)
              at /usr/local/skysql/maxscale/server/modules/filter/tee.c:973
          #5  0x00007f54292c1b55 in clientReply (instance=0x335ae20, router_session=0x7f53ec000f90, queue=0x7f5404000b70, 
              backend_dcb=0x7f53ec000fd0) at /usr/local/skysql/maxscale/server/modules/routing/readconnroute.c:814
          #6  0x00007f54143f7fb7 in gw_read_backend_event (dcb=0x7f53ec000fd0)
              at /usr/local/skysql/maxscale/server/modules/protocol/mysql_backend.c:577
          #7  0x000000000056962d in process_pollq (thread_id=3) at /usr/local/skysql/maxscale/server/core/poll.c:867
          #8  0x0000000000568508 in poll_waitevents (arg=0x3) at /usr/local/skysql/maxscale/server/core/poll.c:608
          #9  0x00007f542f54a9d1 in start_thread () from /lib64/libpthread.so.0
          #10 0x00007f542debb8fd in clone () from /lib64/libc.so.6

          dshjoshi Dipti Joshi (Inactive) added a comment - - edited These are the comments imported from bugzilla Vilho Raatikka 2014-12-30 12:04:06 UTC Created attachment 171 : http://bugs.mariadb.com/attachment.cgi?id=171 Configuration. 1. Start MaxScale 2. feed session command/statement load to port 4008 which belongs to readconnrouter. Statements are then duplicated to rwsplit which starts to lag behind immediately. Comment 2 Vilho Raatikka 2014-12-30 12:05:26 UTC Created attachment 172 http://bugs.mariadb.com/attachment.cgi?id=172 Simple script to run session command/statement load Requires setmix.sql Comment 3 Vilho Raatikka 2014-12-30 12:05:49 UTC Created attachment 173 http://bugs.mariadb.com/attachment.cgi?id=173 List of statements used by run_setmix.sh Comment 4 Vilho Raatikka 2014-12-31 19:13:21 UTC tee filter doesn't send reply to client before both routers have replied. This required adding upstream processing to tee filter. First reply is routed to client. By this tee ensures that new query is never sent to either router before they have replied to previous one. Comment 5 Timofey Turenko 2015-01-08 12:40:34 UTC test added, closing Comment 6 Timofey Turenko 2015-02-28 18:11:16 UTC Reopen: starting from 1.0.5 test for bug670 start to fail with debug assert: (gdb) bt #0 0x00007f542de05625 in raise () from /lib64/libc.so.6 #1 0x00007f542de06e05 in abort () from /lib64/libc.so.6 #2 0x00007f542ddfe74e in __assert_fail_base () from /lib64/libc.so.6 #3 0x00007f542ddfe810 in __assert_fail () from /lib64/libc.so.6 #4 0x00007f53fe7a2daf in clientReply (instance=0x7f53ec001970, session=0x7f53ec001aa0, reply=0x7f5404000b70) at /usr/local/skysql/maxscale/server/modules/filter/tee.c:973 #5 0x00007f54292c1b55 in clientReply (instance=0x335ae20, router_session=0x7f53ec000f90, queue=0x7f5404000b70, backend_dcb=0x7f53ec000fd0) at /usr/local/skysql/maxscale/server/modules/routing/readconnroute.c:814 #6 0x00007f54143f7fb7 in gw_read_backend_event (dcb=0x7f53ec000fd0) at /usr/local/skysql/maxscale/server/modules/protocol/mysql_backend.c:577 #7 0x000000000056962d in process_pollq (thread_id=3) at /usr/local/skysql/maxscale/server/core/poll.c:867 #8 0x0000000000568508 in poll_waitevents (arg=0x3) at /usr/local/skysql/maxscale/server/core/poll.c:608 #9 0x00007f542f54a9d1 in start_thread () from /lib64/libpthread.so.0 #10 0x00007f542debb8fd in clone () from /lib64/libc.so.6

          People

            markus makela markus makela
            tturenko Timofey Turenko
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.