Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-33978

P_S.THREADS is not showing all server threads

Details

    Description

      While actually trying to analyze something else I found that PERFORMANCE_SCHEMA.THREADS does not list all server threads.

      In my case I was originally looking for InnoDB background encryption threads, of which I had four configured, but found that there were actually five threads missing.

      ps showed:

      $ pgrep -w mariadbd | xargs ps -Lo pid,lwp,cmd
      		 
          PID     LWP CMD
      		 
        14728   14728 /usr/sbin/mariadbd
      		 
        14728   14729 /usr/sbin/mariadbd
      		 
        14728   14730 /usr/sbin/mariadbd
      		 
        14728   14731 /usr/sbin/mariadbd
      		 
        14728   14732 /usr/sbin/mariadbd
      		 
        14728   14735 /usr/sbin/mariadbd
      		 
        14728   14736 /usr/sbin/mariadbd
      		 
        14728   14737 /usr/sbin/mariadbd
      		 
        14728   14738 /usr/sbin/mariadbd
      		 
        14728   14739 /usr/sbin/mariadbd
      		 
        14728   14740 /usr/sbin/mariadbd
      		 
        14728   14742 /usr/sbin/mariadbd
      		 
        14728   15299 /usr/sbin/mariadbd
      		 
        14728   15383 /usr/sbin/mariadbd

      But the THREADS table only had:

      MariaDB [test]> select name, thread_os_id from performance_schema.threads order by thread_os_id;
      		 
      +-----------------------------------+--------------+
      		 
      | name                              | thread_os_id |
      		 
      +-----------------------------------+--------------+
      		 
      | thread/sql/main                   |        14728 |
      		 
      | thread/mysys/statement_timer      |        14729 |
      		 
      | thread/aria/checkpoint_background |        14730 |
      		 
      | thread/innodb/page_cleaner_thread |        14732 |
      		 
      | thread/sql/manager                |        14739 |
      		 
      | thread/sql/binlog_background      |        14740 |
      		 
      | thread/sql/signal_handler         |        14742 |
      		 
      | thread/innodb/thread_pool_thread  |        15299 |
      		 
      | thread/sql/one_connection         |        15383 |
      		 
      +-----------------------------------+--------------+

      So LWP ides 14731 and 14835-14738 were missing. The later four are the background encryption threads, and 14731 was running this according to gdb:

      Thread 4 (Thread 0x7fa945118700 (LWP 14731)):
      		 
      #0  syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
      		 
      #1  0x000055faa5f1213d in my_getevents (min_nr=1, nr=256, ev=0x7fa945115d60, ctx=<optimized out>) at ./tpool/aio_linux.cc:105
      		 
      #2  tpool::aio_linux::getevent_thread_routine (aio=0x55faa8c4e650) at ./tpool/aio_linux.cc:105
      		 
      #3  0x00007fa96107bdf4 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
      		 
      #4  0x00007fa961195609 in start_thread (arg=<optimized out>) at pthread_create.c:477
      		 
      #5  0x00007fa960d81353 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

      Not sure whether it is only these threads that do not show, or whether there are also other candidates that were just not running in this setup to begin with.

      Attachments

        Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. MDEV-36189 P_S.THREADS is still not showing all server threads in 11.4.5

          • Major - Major loss of function.
          • Open

          Activity

            Ascending order - Click to sort in descending order
            serg Sergei Golubchik added a comment -

            because encryption threads are created using std::thread which completely bypasses perfschema.
            should be converted to use the thread pool, I presume

            serg Sergei Golubchik added a comment - because encryption threads are created using std::thread which completely bypasses perfschema. should be converted to use the thread pool, I presume
            marko Marko Mäkelä added a comment -

            tpool::aio_linux::getevent_thread_routine is actually using wlad’s thread pool, yet it does not appear in performance_schema. Once that has been sorted out, it should be rather straightforward to make the encryption threads (actually flag-pages-dirty threads) use the thread pool.

            marko Marko Mäkelä added a comment - tpool::aio_linux::getevent_thread_routine is actually using wlad ’s thread pool, yet it does not appear in performance_schema . Once that has been sorted out, it should be rather straightforward to make the encryption threads (actually flag-pages-dirty threads) use the thread pool.
            wlad Vladislav Vaintroub added a comment -

            a2f510fccff23e5011486c240587b8f1a98ecacc makes the aio completion thread to appear as "innodb background thread" in PS
            The rest is now Innodb's

            wlad Vladislav Vaintroub added a comment - a2f510fccff23e5011486c240587b8f1a98ecacc makes the aio completion thread to appear as "innodb background thread" in PS The rest is now Innodb's
            marko Marko Mäkelä added a comment -

            I added thread/innodb/page_encrypt_thread for fil_crypt_thread. Everything else seemed to be covered already; only thread/innodb/thread_pool_thread was apparently missing some cases. These labels but not the thread counts are being checked in the test perfschema.threads_innodb.

            marko Marko Mäkelä added a comment - I added thread/innodb/page_encrypt_thread for fil_crypt_thread . Everything else seemed to be covered already; only thread/innodb/thread_pool_thread was apparently missing some cases. These labels but not the thread counts are being checked in the test perfschema.threads_innodb .

            People

              marko Marko Mäkelä
              hholzgra Hartmut Holzgraefe
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.