Uploaded image for project: 'MariaDB MaxScale'
  1. MariaDB MaxScale
  2. MXS-17

bugzillaId-736: Memory leak while doing read/write splitting

    XMLWordPrintable

    Details

      Description

      This is imported from bugzilla item: http://bugs.mariadb.com/show_bug.cgi?id=736

      Yves Trudeau 2015-02-19 16:33:04 UTC
      I found what seems to be a memory leak in MaxScale while doing Read/write splitting. Just running the following load:

      yves@yves-desktop:~/src/$ ( echo 'drop table if exists simpleinsert; create table simpleinsert (id int not null auto_increment, val tinyint not null, primary key (id)) engine=innodb;'; while true; do echo 'rollback;set autocommit=1;set autocommit=0;insert into simpleinsert (val) values (0);commit;'; sleep 0.1; done ) | mysql -h 10.3.1.120 -u tpcc -ptpcc test

      is sufficient. Below is the rate of increase of the RSS.

      root@radian6-maxscale-1:~/git/MaxScale/build# while true; do ps faxu | grep maxscale |grep -v grep; sleep 15; done
      root 8084 1.5 14.0 638896 106632 ? Ssl 10:55 0:22 /usr/local/skysql/maxscale/bin/maxscale -c /usr/local/skysql/maxscale-0 -f /usr/local/skysql/maxscale-0/etc/MaxScale.cnf -lfile
      root 8084 1.6 15.2 648848 115872 ? Ssl 10:55 0:23 /usr/local/skysql/maxscale/bin/maxscale -c /usr/local/skysql/maxscale-0 -f /usr/local/skysql/maxscale-0/etc/MaxScale.cnf -lfile
      root 8084 1.7 16.4 658936 125112 ? Ssl 10:55 0:25 /usr/local/skysql/maxscale/bin/maxscale -c /usr/local/skysql/maxscale-0 -f /usr/local/skysql/maxscale-0/etc/MaxScale.cnf -lfile
      root 8084 1.7 17.6 668888 134088 ? Ssl 10:55 0:26 /usr/local/skysql/maxscale/bin/maxscale -c /usr/local/skysql/maxscale-0 -f /usr/local/skysql/maxscale-0/etc/MaxScale.cnf -lfile

      Going from about 106MB to 134MB in 1min. The relevant parts of my config:

      [maxscale]
      threads=1
      log_debug=0
      log_trace=0
       
      [MySQL Monitor]
      type=monitor
      module=mysqlmon
      servers=server1,server2,server3
      user=maxuser
      passwd=maxpasswd
      detect_replication_lag=1
      detect_stale_master=1
      monitor_interval=2000
       
      [RW Split Router]
      type=service
      router=readwritesplit
      servers=server1,server2,server3
      router_options=slave_selection_criteria=LEAST_CURRENT_OPERATIONS
      max_slave_connections=1
      max_slave_replication_lag=30
      use_sql_variables_in=all
      user=maxuser
      passwd=maxpasswd
       
      [CLI] 
      type=service 
      router=cli
       
      [RW Split Listener]
      type=listener
      service=RW Split Router
      protocol=MySQLClient
      port=4006

      Something interesting is that the memory is reused, if I restart the insert script, the memory for about the time it took originally to reach the current memory usage and then it will rise again. Valgrind shows many entries like:

      ==8131== 25,906,560 bytes in 6,288 blocks are still reachable in loss record 160 of 162
      ==8131== at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
      ==8131== by 0x794BB1: my_malloc (my_malloc.c:41)
      ==8131== by 0x78EF58: reset_root_defaults (my_alloc.c:129)
      ==8131== by 0x669781: THD::init_for_queries() (sql_class.cc:1345)
      ==8131== by 0x572C08: create_embedded_thd (lib_sql.cc:685)
      ==8131== by 0x192FA09D: ???
      ==8131== by 0x192F9E74: ???
      ==8131== by 0x192F9BD4: ???
      ==8131== by 0x190DB445: ???
      ==8131== by 0x190DAE5E: ???
      ==8131== by 0x1F3A30EB: ???
      ==8131== by 0x19F1EF3E: ???
      ==8131==
      ==8131== 51,662,208 bytes in 6,288 blocks are still reachable in loss record 161 of 162
      ==8131== at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
      ==8131== by 0x794BB1: my_malloc (my_malloc.c:41)
      ==8131== by 0x78EF58: reset_root_defaults (my_alloc.c:129)
      ==8131== by 0x669767: THD::init_for_queries() (sql_class.cc:1342)
      ==8131== by 0x572C08: create_embedded_thd (lib_sql.cc:685)
      ==8131== by 0x192FA09D: ???
      ==8131== by 0x192F9E74: ???
      ==8131== by 0x192F9BD4: ???
      ==8131== by 0x190DB445: ???
      ==8131== by 0x190DAE5E: ???
      ==8131== by 0x1F3A30EB: ???
      ==8131== by 0x19F1EF3E: ???
      ==8131==
      ==8131== 115,497,984 bytes in 6,288 blocks are still reachable in loss record 162 of 162
      ==8131== at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
      ==8131== by 0x794BB1: my_malloc (my_malloc.c:41)
      ==8131== by 0x572ACD: create_embedded_thd (sql_list.h:629)
      ==8131== by 0x192FA09D: ???
      ==8131== by 0x192F9E74: ???
      ==8131== by 0x192F9BD4: ???
      ==8131== by 0x190DB445: ???
      ==8131== by 0x190DAE5E: ???
      ==8131== by 0x1F3A30EB: ???
      ==8131== by 0x19F1EF3E: ???
      ==8131== by 0x19F1D1A9: ???
      ==8131== by 0x55CA70: process_pollq (poll.c:858) }}

      Regards,

      Yves

        Attachments

        1. vg.log
          110 kB
        2. vg2.log
          83 kB

          Activity

            People

            Assignee:
            markus makela markus makela
            Reporter:
            ytrudeau Yves Trudeau
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Git Integration