Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-26636

Race conditions due to attempted updates of InnoDB defragmentation statistics for temporary tables

Details

    Description

      Sporadic. Loop till crash is seen. Two attempts usually is enough. For some reason it seems to affect 10.3 and 10.5-10.7, but not 10.4. Stacks looks a bit odd. Not sure on real priority, if any. The TEMPORARY is required.

      SET GLOBAL innodb_defragment_stats_accuracy=1;
      SET GLOBAL innodb_limit_optimistic_insert_debug=2;
      CREATE TEMPORARY TABLE t (c INT) ENGINE=InnoDB;
      INSERT INTO t VALUES (0);
      INSERT INTO t VALUES (0);
      INSERT INTO t VALUES (0);
      ALTER TABLE t ENGINE=InnoDB;
      ALTER TABLE t ENGINE=InnoDB;
      ALTER TABLE t ENGINE=InnoDB;
      ALTER TABLE t ENGINE=InnoDB;
      DROP TABLE t;
      

      Leads to:

      10.7.0 d552e092c9f3e20da078d1b62b976f629f73d3a4 (Debug)

      Core was generated by `/test/MD180921-mariadb-10.7.0-linux-x86_64-dbg/bin/mysqld --no-defaults --core-'.
      Program terminated with signal SIGSEGV, Segmentation fault.
      #0  dict_table_get_first_index (table=0x0)
          at /test/10.7_dbg/storage/innobase/include/dict0dict.ic:211
      [Current thread is 1 (Thread 0x1455a0364700 (LWP 3119087))]
      (gdb) bt
      #0  dict_table_get_first_index (table=0x0) at /test/10.7_dbg/storage/innobase/include/dict0dict.ic:211
      #1  dict_table_find_index_on_id (table=table@entry=0x0, id=id@entry=23) at /test/10.7_dbg/storage/innobase/dict/dict0dict.cc:1342
      #2  0x00005588fc9881da in dict_stats_process_entry_from_defrag_pool () at /test/10.7_dbg/storage/innobase/include/dict0mem.h:1802
      #3  dict_defrag_process_entries_from_defrag_pool () at /test/10.7_dbg/storage/innobase/dict/dict0defrag_bg.cc:212
      #4  0x00005588fc985ce7 in dict_stats_func () at /test/10.7_dbg/storage/innobase/dict/dict0stats_bg.cc:394
      #5  0x00005588fca28b38 in tpool::thread_pool_generic::timer_generic::run (this=0x5588ff5b4e90) at /test/10.7_dbg/tpool/tpool_generic.cc:313
      #6  tpool::thread_pool_generic::timer_generic::execute (arg=0x5588ff5b4e90) at /test/10.7_dbg/tpool/tpool_generic.cc:333
      #7  0x00005588fca29aa3 in tpool::task::execute (this=0x5588ff5b4ed0) at /test/10.7_dbg/tpool/task.cc:52
      #8  0x00005588fca28653 in tpool::thread_pool_generic::worker_main (this=0x5588ff217e70, thread_var=0x5588ff227890) at /test/10.7_dbg/tpool/tpool_generic.cc:550
      #9  0x00005588fca2898a in std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/9/bits/invoke.h:89
      #10 std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__fn=<optimized out>) at /usr/include/c++/9/bits/invoke.h:95
      #11 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul> (this=<optimized out>) at /usr/include/c++/9/thread:244
      #12 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator() (this=<optimized out>) at /usr/include/c++/9/thread:251
      #13 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run (this=<optimized out>) at /usr/include/c++/9/thread:195
      #14 0x00001455b8776de4 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
      #15 0x00001455b8891609 in start_thread (arg=<optimized out>) at pthread_create.c:477
      #16 0x00001455b847f293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
      

      10.3.32 b112c9dfaacbcb7c3548414c6f402114663223dc (Debug)

      InnoDB: Failing assertion: table->get_ref_count() == 0
      

      10.3.32 b112c9dfaacbcb7c3548414c6f402114663223dc (Debug)

      Core was generated by `/test/MD180921-mariadb-10.3.32-linux-x86_64-dbg/bin/mysqld --no-defaults --core'.
      Program terminated with signal SIGABRT, Aborted.
      #0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
      [Current thread is 1 (Thread 0x14c55c056700 (LWP 2206952))]
      (gdb) bt
      #0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
      #1  0x000014c573312859 in __GI_abort () at abort.c:79
      #2  0x000055bd02efbc79 in ut_dbg_assertion_failed (expr=expr@entry=0x55bd03525f52 "table->get_ref_count() == 0", file=file@entry=0x55bd0355e338 "/test/10.3_dbg/storage/innobase/dict/dict0dict.cc", line=line@entry=1844) at /test/10.3_dbg/storage/innobase/ut/ut0dbg.cc:60
      #3  0x000055bd02fea50e in dict_table_remove_from_cache_low (table=0x14c51400c290, lru_evict=lru_evict@entry=0) at /test/10.3_dbg/storage/innobase/dict/dict0dict.cc:1844
      #4  0x000055bd02feb349 in dict_table_remove_from_cache (table=<optimized out>) at /test/10.3_dbg/storage/innobase/dict/dict0dict.cc:1948
      #5  0x000055bd02e10520 in row_drop_table_for_mysql (name=name@entry=0x14c55c051530 "data/#sql21a851_9_ac", trx=trx@entry=0x14c55e4a6178, sqlcom=sqlcom@entry=SQLCOM_ALTER_TABLE, create_failed=create_failed@entry=false, nonatomic=<optimized out>, nonatomic@entry=true) at /test/10.3_dbg/storage/innobase/row/row0mysql.cc:3413
      #6  0x000055bd02ca017a in ha_innobase::delete_table (this=this@entry=0x14c514012698, name=<optimized out>, name@entry=0x14c51404c2a0 "/test/MD180921-mariadb-10.3.32-linux-x86_64-dbg/data/#sql21a851_9_ac", sqlcom=sqlcom@entry=SQLCOM_ALTER_TABLE) at /test/10.3_dbg/storage/innobase/handler/ha_innodb.cc:13215
      #7  0x000055bd02c95d44 in ha_innobase::delete_table (this=0x14c514012698, name=0x14c51404c2a0 "/test/MD180921-mariadb-10.3.32-linux-x86_64-dbg/data/#sql21a851_9_ac") at /test/10.3_dbg/storage/innobase/handler/ha_innodb.cc:13340
      #8  0x000055bd02a5aab7 in handler::ha_delete_table (this=this@entry=0x14c514012698, name=name@entry=0x14c51404c2a0 "/test/MD180921-mariadb-10.3.32-linux-x86_64-dbg/data/#sql21a851_9_ac") at /test/10.3_dbg/sql/handler.cc:4710
      #9  0x000055bd029ae958 in THD::rm_temporary_table (this=this@entry=0x14c514000d90, base=0x55bd0581f4f0, path=0x14c51404c2a0 "/test/MD180921-mariadb-10.3.32-linux-x86_64-dbg/data/#sql21a851_9_ac") at /test/10.3_dbg/sql/temporary_tables.cc:706
      #10 0x000055bd029af26f in THD::free_tmp_table_share (this=this@entry=0x14c514000d90, share=share@entry=0x14c51404bd50, delete_table=delete_table@entry=true) at /test/10.3_dbg/sql/handler.h:1630
      #11 0x000055bd029b0dd1 in THD::drop_temporary_table (this=this@entry=0x14c514000d90, table=table@entry=0x14c51404c840, is_trans=is_trans@entry=0x0, delete_table=delete_table@entry=true) at /test/10.3_dbg/sql/temporary_tables.cc:669
      #12 0x000055bd028867e1 in mysql_alter_table (thd=thd@entry=0x14c514000d90, new_db=new_db@entry=0x14c514005470, new_name=new_name@entry=0x14c514005858, create_info=create_info@entry=0x14c55c053490, table_list=<optimized out>, table_list@entry=0x14c514011658, alter_info=alter_info@entry=0x14c55c0533d0, order_num=0, order=0x0, ignore=false) at /test/10.3_dbg/sql/sql_table.cc:10200
      #13 0x000055bd028f9754 in Sql_cmd_alter_table::execute (this=0x14c514011cc8, thd=0x14c514000d90) at /test/10.3_dbg/sql/sql_alter.cc:512
      #14 0x000055bd027c65b1 in mysql_execute_command (thd=thd@entry=0x14c514000d90) at /test/10.3_dbg/sql/sql_parse.cc:6075
      #15 0x000055bd027c85b2 in mysql_parse (thd=thd@entry=0x14c514000d90, rawbuf=<optimized out>, length=<optimized out>, parser_state=parser_state@entry=0x14c55c055530, is_com_multi=is_com_multi@entry=false, is_next_command=is_next_command@entry=false) at /test/10.3_dbg/sql/sql_parse.cc:7870
      #16 0x000055bd027cad23 in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x14c514000d90, packet=packet@entry=0x14c5140198f1 "", packet_length=packet_length@entry=27, is_com_multi=is_com_multi@entry=false, is_next_command=is_next_command@entry=false) at /test/10.3_dbg/sql/sql_class.h:1152
      #17 0x000055bd027cdeb3 in do_command (thd=0x14c514000d90) at /test/10.3_dbg/sql/sql_parse.cc:1398
      #18 0x000055bd028f5709 in do_handle_one_connection (connect=connect@entry=0x55bd0635e7e0) at /test/10.3_dbg/sql/sql_connect.cc:1403
      #19 0x000055bd028f5940 in handle_one_connection (arg=0x55bd0635e7e0) at /test/10.3_dbg/sql/sql_connect.cc:1308
      #20 0x000014c5734e8609 in start_thread (arg=<optimized out>) at pthread_create.c:477
      #21 0x000014c57340f293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
      

      Bug confirmed present in:
      MariaDB: 10.3.32 (dbg), 10.5.13 (dbg), 10.6.5 (dbg), 10.7.0 (dbg)

      Bug (or feature/syntax) confirmed not present in:
      MariaDB: 10.2.41 (dbg), 10.2.41 (opt), 10.3.32 (opt), 10.4.22 (dbg), 10.4.22 (opt), 10.5.13 (opt), 10.6.5 (opt), 10.7.0 (opt)
      MySQL: 5.5.62 (dbg), 5.5.62 (opt), 5.6.51 (dbg), 5.6.51 (opt), 5.7.35 (dbg), 5.7.35 (opt), 8.0.26 (dbg), 8.0.26 (opt)

      Attachments

        Issue Links

          Activity

            Roel Roel Van de Paar added a comment - - edited

            Summary of fix; defragmentation statistics of TEMPORARY tables disabled in 10.2+ (and defragmentation of TEMPORARY tables was already disabled)

            Roel Roel Van de Paar added a comment - - edited Summary of fix; defragmentation statistics of TEMPORARY tables disabled in 10.2+ (and defragmentation of TEMPORARY tables was already disabled)
            Roel Roel Van de Paar added a comment - - edited

            I saw this in a 10.7 debug build from 9 October:

            10.7.1 5cc9cf9a05582307abefd8f3b57548945da86b1a (Debug)

            InnoDB: Failing assertion: table->get_ref_count() == 0
            

            10.7.1 5cc9cf9a05582307abefd8f3b57548945da86b1a (Debug)

            Core was generated by `/test/MD091021-mariadb-10.7.1-linux-x86_64-dbg/bin/mysqld --no-defaults --core-'.
            Program terminated with signal SIGABRT, Aborted.
            #0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
            [Current thread is 1 (Thread 0x1525100e1700 (LWP 1625177))]
            (gdb) bt
            #0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
            #1  0x00001525124bd859 in __GI_abort () at abort.c:79
            #2  0x0000563bcd532312 in ut_dbg_assertion_failed (expr=expr@entry=0x563bcdbacbc0 "table->get_ref_count() == 0", file=file@entry=0x563bcdbad1c0 "/test/10.7_dbg/storage/innobase/dict/dict0dict.cc", line=line@entry=1823) at /test/10.7_dbg/storage/innobase/ut/ut0dbg.cc:60
            #3  0x0000563bcd6040a0 in dict_sys_t::remove (this=0x563bce1ba4c0 <dict_sys>, table=table@entry=0x1524c697ba68, lru=lru@entry=false, keep=keep@entry=false) at /test/10.7_dbg/storage/innobase/dict/dict0dict.cc:1823
            #4  0x0000563bcd636928 in trx_t::commit (this=this@entry=0x1524fd755390, deleted=std::vector of length 5, capacity 8 = {{m_file = 125, m_psi = 0x0}, {m_file = 140, m_psi = 0x0}, {m_file = 144, m_psi = 0x0}, {m_file = 146, m_psi = 0x0}, {m_file = 141, m_psi = 0x0}}) at /test/10.7_dbg/storage/innobase/dict/drop.cc:249
            #5  0x0000563bcd2faef4 in ha_innobase::delete_table (this=<optimized out>, name=<optimized out>) at /test/10.7_dbg/storage/innobase/handler/ha_innodb.cc:13600
            #6  0x0000563bccf4a731 in hton_drop_table (hton=<optimized out>, path=<optimized out>) at /test/10.7_dbg/sql/handler.cc:576
            #7  0x0000563bccf53ff8 in ha_delete_table (thd=thd@entry=0x1524c4000db8, hton=hton@entry=0x563bd091f928, path=path@entry=0x1525100dd5d0 "./test/#sql-backup-18c39f-4", db=db@entry=0x1525100df180, alias=alias@entry=0x1525100dd980, generate_warning=generate_warning@entry=false) at /test/10.7_dbg/sql/handler.cc:3100
            #8  0x0000563bccd24232 in quick_rm_table (thd=thd@entry=0x1524c4000db8, base=base@entry=0x563bd091f928, db=db@entry=0x1525100df180, table_name=table_name@entry=0x1525100dd980, flags=3, table_path=table_path@entry=0x0) at /test/10.7_dbg/sql/sql_table.cc:1919
            #9  0x0000563bccd3c216 in mysql_alter_table (thd=thd@entry=0x1524c4000db8, new_db=new_db@entry=0x1524c4013d70, new_name=new_name@entry=0x1524c4013d80, create_info=create_info@entry=0x1525100dffc0, table_list=<optimized out>, table_list@entry=0x1524c4013d58, alter_info=alter_info@entry=0x1525100dfed0, order_num=<optimized out>, order=<optimized out>, ignore=<optimized out>, if_exists=<optimized out>) at /test/10.7_dbg/sql/sql_table.cc:10616
            #10 0x0000563bccc4d381 in mysql_execute_command (thd=thd@entry=0x1524c4000db8, is_called_from_prepared_stmt=is_called_from_prepared_stmt@entry=false) at /test/10.7_dbg/sql/structs.h:568
            #11 0x0000563bccc38d6f in mysql_parse (thd=thd@entry=0x1524c4000db8, rawbuf=<optimized out>, length=<optimized out>, parser_state=parser_state@entry=0x1525100e0400) at /test/10.7_dbg/sql/sql_parse.cc:8028
            #12 0x0000563bccc47968 in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x1524c4000db8, packet=packet@entry=0x1524c55f1ed9 "", packet_length=packet_length@entry=31, blocking=blocking@entry=true) at /test/10.7_dbg/sql/sql_class.h:1358
            #13 0x0000563bccc4ad6e in do_command (thd=0x1524c4000db8, blocking=blocking@entry=true) at /test/10.7_dbg/sql/sql_parse.cc:1402
            #14 0x0000563bccdc0e88 in do_handle_one_connection (connect=<optimized out>, connect@entry=0x563bd0f060b8, put_in_cache=put_in_cache@entry=true) at /test/10.7_dbg/sql/sql_connect.cc:1418
            #15 0x0000563bccdc148d in handle_one_connection (arg=arg@entry=0x563bd0f060b8) at /test/10.7_dbg/sql/sql_connect.cc:1312
            #16 0x0000563bcd22a83e in pfs_spawn_thread (arg=0x563bd0e45d98) at /test/10.7_dbg/storage/perfschema/pfs.cc:2201
            #17 0x00001525129cc609 in start_thread (arg=<optimized out>) at pthread_create.c:477
            #18 0x00001525125ba293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
            

            When testing (looping, so likely highly sporadic) an original longer input testcase of MDEV-26381. TEMPORARY tables are present in the testcase.

            Roel Roel Van de Paar added a comment - - edited I saw this in a 10.7 debug build from 9 October: 10.7.1 5cc9cf9a05582307abefd8f3b57548945da86b1a (Debug) InnoDB: Failing assertion: table->get_ref_count() == 0 10.7.1 5cc9cf9a05582307abefd8f3b57548945da86b1a (Debug) Core was generated by `/test/MD091021-mariadb-10.7.1-linux-x86_64-dbg/bin/mysqld --no-defaults --core-'. Program terminated with signal SIGABRT, Aborted. #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50 [Current thread is 1 (Thread 0x1525100e1700 (LWP 1625177))] (gdb) bt #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50 #1 0x00001525124bd859 in __GI_abort () at abort.c:79 #2 0x0000563bcd532312 in ut_dbg_assertion_failed (expr=expr@entry=0x563bcdbacbc0 "table->get_ref_count() == 0", file=file@entry=0x563bcdbad1c0 "/test/10.7_dbg/storage/innobase/dict/dict0dict.cc", line=line@entry=1823) at /test/10.7_dbg/storage/innobase/ut/ut0dbg.cc:60 #3 0x0000563bcd6040a0 in dict_sys_t::remove (this=0x563bce1ba4c0 <dict_sys>, table=table@entry=0x1524c697ba68, lru=lru@entry=false, keep=keep@entry=false) at /test/10.7_dbg/storage/innobase/dict/dict0dict.cc:1823 #4 0x0000563bcd636928 in trx_t::commit (this=this@entry=0x1524fd755390, deleted=std::vector of length 5, capacity 8 = {{m_file = 125, m_psi = 0x0}, {m_file = 140, m_psi = 0x0}, {m_file = 144, m_psi = 0x0}, {m_file = 146, m_psi = 0x0}, {m_file = 141, m_psi = 0x0}}) at /test/10.7_dbg/storage/innobase/dict/drop.cc:249 #5 0x0000563bcd2faef4 in ha_innobase::delete_table (this=<optimized out>, name=<optimized out>) at /test/10.7_dbg/storage/innobase/handler/ha_innodb.cc:13600 #6 0x0000563bccf4a731 in hton_drop_table (hton=<optimized out>, path=<optimized out>) at /test/10.7_dbg/sql/handler.cc:576 #7 0x0000563bccf53ff8 in ha_delete_table (thd=thd@entry=0x1524c4000db8, hton=hton@entry=0x563bd091f928, path=path@entry=0x1525100dd5d0 "./test/#sql-backup-18c39f-4", db=db@entry=0x1525100df180, alias=alias@entry=0x1525100dd980, generate_warning=generate_warning@entry=false) at /test/10.7_dbg/sql/handler.cc:3100 #8 0x0000563bccd24232 in quick_rm_table (thd=thd@entry=0x1524c4000db8, base=base@entry=0x563bd091f928, db=db@entry=0x1525100df180, table_name=table_name@entry=0x1525100dd980, flags=3, table_path=table_path@entry=0x0) at /test/10.7_dbg/sql/sql_table.cc:1919 #9 0x0000563bccd3c216 in mysql_alter_table (thd=thd@entry=0x1524c4000db8, new_db=new_db@entry=0x1524c4013d70, new_name=new_name@entry=0x1524c4013d80, create_info=create_info@entry=0x1525100dffc0, table_list=<optimized out>, table_list@entry=0x1524c4013d58, alter_info=alter_info@entry=0x1525100dfed0, order_num=<optimized out>, order=<optimized out>, ignore=<optimized out>, if_exists=<optimized out>) at /test/10.7_dbg/sql/sql_table.cc:10616 #10 0x0000563bccc4d381 in mysql_execute_command (thd=thd@entry=0x1524c4000db8, is_called_from_prepared_stmt=is_called_from_prepared_stmt@entry=false) at /test/10.7_dbg/sql/structs.h:568 #11 0x0000563bccc38d6f in mysql_parse (thd=thd@entry=0x1524c4000db8, rawbuf=<optimized out>, length=<optimized out>, parser_state=parser_state@entry=0x1525100e0400) at /test/10.7_dbg/sql/sql_parse.cc:8028 #12 0x0000563bccc47968 in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x1524c4000db8, packet=packet@entry=0x1524c55f1ed9 "", packet_length=packet_length@entry=31, blocking=blocking@entry=true) at /test/10.7_dbg/sql/sql_class.h:1358 #13 0x0000563bccc4ad6e in do_command (thd=0x1524c4000db8, blocking=blocking@entry=true) at /test/10.7_dbg/sql/sql_parse.cc:1402 #14 0x0000563bccdc0e88 in do_handle_one_connection (connect=<optimized out>, connect@entry=0x563bd0f060b8, put_in_cache=put_in_cache@entry=true) at /test/10.7_dbg/sql/sql_connect.cc:1418 #15 0x0000563bccdc148d in handle_one_connection (arg=arg@entry=0x563bd0f060b8) at /test/10.7_dbg/sql/sql_connect.cc:1312 #16 0x0000563bcd22a83e in pfs_spawn_thread (arg=0x563bd0e45d98) at /test/10.7_dbg/storage/perfschema/pfs.cc:2201 #17 0x00001525129cc609 in start_thread (arg=<optimized out>) at pthread_create.c:477 #18 0x00001525125ba293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95 When testing (looping, so likely highly sporadic) an original longer input testcase of MDEV-26381 . TEMPORARY tables are present in the testcase.

            marko A core dump for the issue above is available. Let me know if needed. Also let me know if this bug should be reopened or a new one created.

            Roel Roel Van de Paar added a comment - marko A core dump for the issue above is available. Let me know if needed. Also let me know if this bug should be reopened or a new one created.

            Roel, yes, new tickets should be filed if a bug does not appear to be fixed. This assertion failure has many potential root causes.

            In 10.6 and later, there is a known bug MDEV-29250 that affects tables that contain FULLTEXT INDEX.

            marko Marko Mäkelä added a comment - Roel , yes, new tickets should be filed if a bug does not appear to be fixed. This assertion failure has many potential root causes. In 10.6 and later, there is a known bug MDEV-29250 that affects tables that contain FULLTEXT INDEX .

            Created MDEV-29277: MDEV-26636 Race conditions not fully fixed yet

            Roel Roel Van de Paar added a comment - Created MDEV-29277 : MDEV-26636 Race conditions not fully fixed yet

            People

              marko Marko Mäkelä
              Roel Roel Van de Paar
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.