Uploaded image for project: 'MariaDB ColumnStore'
  1. MariaDB ColumnStore
  2. MCOL-4630

mysqld process in CS 1.2.5 repeatedly restarting until whole CS is shutdown

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Cannot Reproduce
    • 1.2.5
    • N/A
    • MariaDB Server
    • None
    • CentOS Linux release 7.6.1810 (Core) Physical

    Description

      The mysqld process is repeatedly restarting on all 4 CS nodes until the whole CS system is shutdown and restarted.

      From the logs the mysqld process is terminated with a signal 6:
      terminate called after throwing an instance of 'logging::IDBExcept'
      what(): DistributedEngineComm::write: Broken Pipe error
      210315 17:19:35 [ERROR] mysqld got signal 6 ;

      and following is the stack trace: (error and stack trace are identical on all 4 nodes but with a small difference in time)

      Server version: 10.3.16-MariaDB-log
      key_buffer_size=536870912
      read_buffer_size=4194304
      max_used_connections=26
      max_threads=153
      thread_count=28
      It is possible that mysqld could use up to
      key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1781080 K bytes of memory
      Hope that's ok; if not, decrease some variables in the equation.

      Thread pointer: 0x7f26580008c8
      Attempting backtrace. You can use the following information to find out
      where mysqld died. If you see no messages after this, something went
      terribly wrong...
      stack_bottom = 0x7f270cae4d98 thread_stack 0x80000

          • buffer overflow detected ***: /usr/local/mariadb/columnstore/mysql//bin/mysqld terminated
            ======= Backtrace: =========
            /lib64/libc.so.6(__fortify_fail+0x37)[0x7f276dd129e7]
            /lib64/libc.so.6(+0x115b62)[0x7f276dd10b62]
            /lib64/libc.so.6(+0x117947)[0x7f276dd12947]
            /usr/local/mariadb/columnstore/mysql//bin/mysqld(my_addr_resolve+0xda)[0x562794aee31a]
            /usr/local/mariadb/columnstore/mysql//bin/mysqld(my_print_stacktrace+0x1aa)[0x562794adc3fa]
            /usr/local/mariadb/columnstore/mysql//bin/mysqld(handle_fatal_signal+0x30f)[0x562794614b6f]
            /lib64/libpthread.so.0(+0xf5d0)[0x7f276fd555d0]
            /lib64/libc.so.6(gsignal+0x37)[0x7f276dc31207]
            /lib64/libc.so.6(abort+0x148)[0x7f276dc328f8]
            /lib64/libstdc++.so.6(ZN9gnu_cxx27_verbose_terminate_handlerEv+0x165)[0x7f276e5407d5]
            /lib64/libstdc++.so.6(+0x5e746)[0x7f276e53e746]
            /lib64/libstdc++.so.6(+0x5e773)[0x7f276e53e773]
            /lib64/libstdc++.so.6(__cxa_rethrow+0x49)[0x7f276e53e9e9]
            /usr/local/mariadb/columnstore/lib/libexecplan.so.1(_ZN8execplan20CalpontSystemCatalog10getSysDataERNS_26CalpontSelectExecutionPlanERNS0_14NJLSysDataListERKSs+0x9bf)[0x7f2769a8bd6f]
            /usr/local/mariadb/columnstore/lib/libexecplan.so.1(_ZN8execplan20CalpontSystemCatalog9lookupOIDERKNS0_12TableColNameE+0x1cfb)[0x7f2769a8ee9b]
            /usr/local/mariadb/columnstore/lib/libexecplan.so.1(_ZN8execplan12SimpleColumn6setOIDEv+0x133)[0x7f2769aee2b3]
            /usr/local/mariadb/columnstore/lib/libexecplan.so.1(_ZN8execplan12SimpleColumnC2ERKSsS2_S2_j+0x96)[0x7f2769aee9e6]
            /usr/local/mariadb/columnstore/mysql/lib/plugin/libcalmysql.so(_ZN11cal_impl_if17cp_get_table_planEP3THDRN5boost10shared_ptrIN8execplan26CalpontSelectExecutionPlanEEERNS_14cal_table_infoE+0x207)[0x7f276b113207]
            /usr/local/mariadb/columnstore/mysql/lib/plugin/libcalmysql.so(_Z24ha_calpont_impl_rnd_initP5TABLE+0x5151)[0x7f276b0da0a1]
            /usr/local/mariadb/columnstore/mysql//bin/mysqld(_ZN7handler22ha_rnd_init_with_errorEb+0x16)[0x56279461ae66]
            /usr/local/mariadb/columnstore/mysql//bin/mysqld(_Z16init_read_recordP11READ_RECORDP3THDP5TABLEP10SQL_SELECTP9SORT_INFOibb+0x61c)[0x56279472ec8c]
            /usr/local/mariadb/columnstore/mysql//bin/mysqld(_Z21join_init_read_recordP13st_join_table+0xc9)[0x5627944a4b39]
            /usr/local/mariadb/columnstore/mysql//bin/mysqld(_Z10sub_selectP4JOINP13st_join_tableb+0x169)[0x562794494bf9]
            /usr/local/mariadb/columnstore/mysql//bin/mysqld(_ZN4JOIN10exec_innerEv+0xa86)[0x5627944b56c6]
            /usr/local/mariadb/columnstore/mysql//bin/mysqld(_ZN4JOIN4execEv+0x29)[0x5627944b58e9]
            /usr/local/mariadb/columnstore/mysql//bin/mysqld(_Z12mysql_selectP3THDP10TABLE_LISTjR4ListI4ItemEPS4_jP8st_orderS9_S7_S9_yP13select_resultP18st_select_lex_unitP13st_select_lex+0x1ce)[0x5627944b5afe]
            /usr/local/mariadb/columnstore/mysql//bin/mysqld(_Z13handle_selectP3THDP3LEXP13select_resultm+0x1bb)[0x5627944b66ab]
            /usr/local/mariadb/columnstore/mysql//bin/mysqld(+0x58c658)[0x562794457658]
            /usr/local/mariadb/columnstore/mysql//bin/mysqld(_Z21mysql_execute_commandP3THD+0x1d0f)[0x5627944605af]
            /usr/local/mariadb/columnstore/mysql//bin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_statebb+0x1d2)[0x562794465ec2]
            /usr/local/mariadb/columnstore/mysql//bin/mysqld(_Z18idb_vtable_processP3THDyP9Statement+0x1db3)[0x562794468853]
            /usr/local/mariadb/columnstore/mysql//bin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcjbb+0x12b1)[0x56279446a791]
            /usr/local/mariadb/columnstore/mysql//bin/mysqld(_Z10do_commandP3THD+0xd8)[0x56279446bb88]
            /usr/local/mariadb/columnstore/mysql//bin/mysqld(_Z24do_handle_one_connectionP7CONNECT+0x1a4)[0x562794533524]
            /usr/local/mariadb/columnstore/mysql//bin/mysqld(handle_one_connection+0x34)[0x5627945335e4]
            /lib64/libpthread.so.0(+0x7dd5)[0x7f276fd4ddd5]
            /lib64/libc.so.6(clone+0x6d)[0x7f276dcf8ead]

      Attachments

        Activity

          People

            Unassigned Unassigned
            Bongini Luigi
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.