Uploaded image for project: 'MariaDB ColumnStore'
  1. MariaDB ColumnStore
  2. MCOL-5745

Integrate multinode tests into Drone CI

Details

    • Task
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Won't Do
    • None
    • N/A
    • CI
    • None

    Description

      the following process is suggested:

      • after building and publishing server and columnstore packages Drone call scripts on the host to create virtual machines and setup cluster
      • script on the host provides credentials to access the primary node
      • MTR can be executed in the container or on one of the virtual machines (primary?)
      • logs have to be copied from all virtual machines
      • VMs have to be deleted: it should be "always executed step" + MDBCI cloud monitor will report all old VMs (and possibly destroy them automatically after a certain time)
      • scripts can be executed manually from the host

      Communication between container and the Drone host:

      • curl: the server has to be created on the host, and access to the server has to be restricted
      • ssh: only private key is needed on the container side

      To be implemented:

      the first implementation of the scripts to create VMs and cluster https://github.com/mariadb-corporation/mariadb-columnstore-engine/tree/TODO-4505-multinode-scripts/tests/mdbci

      Attachments

        Activity

          AWS and MDBCI is working on ci.columnstore.mariadb.net/ host, next step - add call of MDBCI into drone jsonnet

          tturenko Timofey Turenko added a comment - AWS and MDBCI is working on ci.columnstore.mariadb.net/ host, next step - add call of MDBCI into drone jsonnet

          attempt to run Multinode MTR fro Dorne is done https://github.com/mariadb-corporation/mariadb-columnstore-engine/compare/develop...MCOL-5745_multinode_ci

          results: for 2-3 distros, it works ok, but a larger amount of multinode MTR executed from Drone host causes AWS failures. Possible solution is to use a separate server to execute MDBCI and tests

          tturenko Timofey Turenko added a comment - attempt to run Multinode MTR fro Dorne is done https://github.com/mariadb-corporation/mariadb-columnstore-engine/compare/develop...MCOL-5745_multinode_ci results: for 2-3 distros, it works ok, but a larger amount of multinode MTR executed from Drone host causes AWS failures. Possible solution is to use a separate server to execute MDBCI and tests

          final test are done. Conclusion: running multinode on virtual machines in Drone is too heavy for drone infrastructure.

          tturenko Timofey Turenko added a comment - final test are done. Conclusion: running multinode on virtual machines in Drone is too heavy for drone infrastructure.

          moving multinode to BuildBot due to performance issues in Drone infrastructure

          tturenko Timofey Turenko added a comment - moving multinode to BuildBot due to performance issues in Drone infrastructure

          People

            tturenko Timofey Turenko
            tturenko Timofey Turenko
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.