Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-31337

MariaDB 10.6.10 hang with lot a "check permissions" processlist.

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Incomplete
    • 10.4.28, 10.6.12
    • N/A
    • None
    • windows (server 2019) and linux ubuntu

    Description

      Mariadb hangs with list of processes in "check permissions" or "open tables", and only restarting the service resolves.
      We have dozens of servers with version 10.6.12, but the problem occurs in 2 of them sporadically

      Attachments

        Activity

          danblack Daniel Black added a comment -

          How long after restarting the service until the problem presents itself again? At the moment it looks like your table_open_cache is insufficient. Can you include your configuration and SHOW GLOBAL STATUS at the time of contention.

          danblack Daniel Black added a comment - How long after restarting the service until the problem presents itself again? At the moment it looks like your table_open_cache is insufficient. Can you include your configuration and SHOW GLOBAL STATUS at the time of contention.
          ANDRE_DBA ANDRE IVANES NIETHAMMER added a comment - - edited

          The problem reoccurs sporadically, sometimes it takes days and sometimes twice a day.
          The table_open_cache parameter is table_open_cache=16384 the database has 3000 tables and we have 200 concurrent sessions.

          my,cnf:

          [mysqld]
          skip-networking=0
          lower_case_table_names=1
          innodb_file_per_table=1
          query_cache_type=OFF
          innodb_log_file_size=1024M
          innodb_log_buffer_size=128M
          innodb_flush_log_at_trx_commit=0
          tmp_table_size=200M
          max_heap_table_size=200M
          performance_schema=1
          key_buffer_size=1M
          table_open_cache=16384
          table_definition_cache=16384
          thread_handling=pool-of-threads
          net_write_timeout=3600
          net_read_timeout=600
          max_allowed_packet=268435456
          connect_timeout=60
          innodb_buffer_pool_size=512M
          port=3306
          socket=/usr/local/mysql/data/mysql.sock
          log_error=/usr/local/mysql/data/log_error.log
          max_connections=700
          log_bin=/usr/local/mysql/binlog/mysql-bin.log
          max_binlog_size=512M
          server_id=1
          expire_logs_days=1
          binlog_format=mixed
          open_files_limit=102400
          log_bin_trust_function_creators=1
          [client]
          socket=/usr/local/mysql/data/mysql.sock
          

          When it crashes again I will post the data of "SHOW GLOBAL STATUS".

          Thanks e best regards !

          ANDRE_DBA ANDRE IVANES NIETHAMMER added a comment - - edited The problem reoccurs sporadically, sometimes it takes days and sometimes twice a day. The table_open_cache parameter is table_open_cache=16384 the database has 3000 tables and we have 200 concurrent sessions. my,cnf: [mysqld] skip-networking=0 lower_case_table_names=1 innodb_file_per_table=1 query_cache_type=OFF innodb_log_file_size=1024M innodb_log_buffer_size=128M innodb_flush_log_at_trx_commit=0 tmp_table_size=200M max_heap_table_size=200M performance_schema=1 key_buffer_size=1M table_open_cache=16384 table_definition_cache=16384 thread_handling=pool-of-threads net_write_timeout=3600 net_read_timeout=600 max_allowed_packet=268435456 connect_timeout=60 innodb_buffer_pool_size=512M port=3306 socket=/usr/local/mysql/data/mysql.sock log_error=/usr/local/mysql/data/log_error.log max_connections=700 log_bin=/usr/local/mysql/binlog/mysql-bin.log max_binlog_size=512M server_id=1 expire_logs_days=1 binlog_format=mixed open_files_limit=102400 log_bin_trust_function_creators=1 [client] socket=/usr/local/mysql/data/mysql.sock When it crashes again I will post the data of "SHOW GLOBAL STATUS". Thanks e best regards !
          danblack Daniel Black added a comment -

          Could this be swap related? innodb_buffer_pool_size looks rather small. How much memory do you have? tmp_table_size/max_heap_table_size is per connection so large in memory operations might end up running out of memory?

          danblack Daniel Black added a comment - Could this be swap related? innodb_buffer_pool_size looks rather small. How much memory do you have? tmp_table_size/max_heap_table_size is per connection so large in memory operations might end up running out of memory?

          The server has only 8gb of memory, and no swap is detected. The crash seems to occur when very long reads are being performed (BI). The tmp_table_size is no longer used so I'm going to remove it, we used tables with ENGINE MEMORY.
          So far, no crash has occurred, if it does I will post a "SHOW GLOBAL STATUS" and a "SHOW ENGINE INNODB STATUS".

          ANDRE_DBA ANDRE IVANES NIETHAMMER added a comment - The server has only 8gb of memory, and no swap is detected. The crash seems to occur when very long reads are being performed (BI). The tmp_table_size is no longer used so I'm going to remove it, we used tables with ENGINE MEMORY. So far, no crash has occurred, if it does I will post a "SHOW GLOBAL STATUS" and a "SHOW ENGINE INNODB STATUS".

          ANDRE_DBA, there was a month with no feedback. I hope it means MariaDB didn't crash for you. I'll tentatively close this issue then, but whenever you want to add more info — just add a comment and I'll reopen it again. Thanks.

          serg Sergei Golubchik added a comment - ANDRE_DBA , there was a month with no feedback. I hope it means MariaDB didn't crash for you. I'll tentatively close this issue then, but whenever you want to add more info — just add a comment and I'll reopen it again. Thanks.

          People

            Unassigned Unassigned
            ANDRE_DBA ANDRE IVANES NIETHAMMER
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.