Details
-
Task
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Fixed
-
None
-
None
Description
About K8s operator
A Kubernetes Operator is a controller that encodes human operational knowledge: how do I run and manage a specific piece of complex software.
Operator is an custom resources in K8s involving specific knowledge of application (in our case MariaDB) in order to automate lifecycle.
This task should create MariaDB operator used to create the MariaDB statefull application in a cluster.
Project scope
The goal of this project is to create a Kubernetes Operator. As a Minimum Viable Product (MVP), the operator needs to know how to:
- Start a MariaDB Server
- Set up the initial database (mariadb_install_db, authentication methods, networking ports)
- Set up a complete and secure networking configuration (service) between nodes.
- Configuration method for specifying which machines to be used as nodes.
- Configuration method for specifying data directory locations.
- Set up a replication topology - primary server (master) & at least 2 replicas (slaves) (Asynchronous replication)
- Provides HA.
- Monitor the status of all MariaDB nodes in the cluster and restart in case nodes go down.
- Easy management of cluster; Change the size parameter to add/remove members from cluster
Things that are not part of MVP, but are to be considered as future development:
- Ability to take backups (on demand or scheduled backup - locally or to object storage -S3, restore DB from existing backup)
- Provisioning of slaves from backups.
- Ability to fine tune which machines get assigned to which nodes. (For example master node might need to have more CPUs).
- Detect master failure and automatically promote one of the slaves to become the new master.
- Proxying. Automatically route writes to Primary/ies and distribute reads between all members (example haproxy, proxysql).
- Encryption in transit and Data-At-Rest
- Usage of private docker registries
- Helm chart to deploy the MariaDB operator (instead of editing yaml file).
- Change from master-slave replication to Galera replication topology.
Attachments
Issue Links
- is part of
-
-
- Closed
-
Activity
Commit: https://github.com/an3l/mariadb-k8s/commit/9fb056cd71ba80c84a6e574d676078bebebf1cde
$ kubectl get mariadbs -w
|
NAME MARIADB STATE PORT AGE
|
mariadb-sample 3306 0s
|
$ kubectl get mariadbs -w -o wide
|
NAME MARIADB STATE PORT IMAGE AGE
|
mariadb-sample 3306 quay.io/mariadb-foundation/mariadb-devel:10.5 9s |
|
|
|
|
Note that spec attributes are not expected to change during reconcilation - Spec.ShowState
|
|
|
|
|
$ kubectl get all
|
NAME READY STATUS RESTARTS AGE
|
pod/mariadb-sample-server-68bc5c6f9f-45v94 1/1 Running 0 6m9s |
pod/mariadb-sample-server-68bc5c6f9f-dm6b7 1/1 Running 0 6m9s |
pod/mariadb-sample-server-deployment-68bc5c6f9f-lf84g 1/1 Running 0 4s |
pod/mariadb-sample-server-deployment-68bc5c6f9f-mntlm 1/1 Running 0 4s |
|
|
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
|
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 105d |
service/mariadb-sample ClusterIP 10.106.75.88 <none> 3306/TCP 6m9s |
service/mariadb-sample-server-service ClusterIP 10.110.35.216 <none> 3306/TCP 4s |
service/mariadb-sample-service ClusterIP 10.103.95.33 <none> 3306/TCP 2m8s |
|
|
NAME READY UP-TO-DATE AVAILABLE AGE
|
deployment.apps/mariadb-sample-server 2/2 2 2 6m9s |
deployment.apps/mariadb-sample-server-deployment 2/2 2 2 4s |
|
|
NAME DESIRED CURRENT READY AGE
|
replicaset.apps/mariadb-sample-server-68bc5c6f9f 2 2 2 6m9s |
replicaset.apps/mariadb-sample-server-deployment-68bc5c6f9f 2 2 2 4s |
|
|
|
|
$ kubectl exec -it svc/mariadb-sample-server-service -- mariadb -uexample-user -pmy_cool_secret -e "show databases;" |
+--------------------+
|
| Database |
|
+--------------------+
|
| information_schema |
|
| testDB-operator |
|
+--------------------+
|
Note for Anel: spec attributes are not expected to change during reconcilation - Spec.ShowState
Added secret creation
https://github.com/an3l/mariadb-k8s/commits/mariadb-operator-secrets
Todo: Edit secrets should work on the fly, find out about API call for this
Switching from go 1.16 to go 1.18 I'm facing the problem that cannot start the operator and had to install in GOBIN relative to project (although GOBIN doesn't accept relative paths) controller-gen and kustomize.
$ GOBIN=/home/anel/mariadb/k8s/mariadb-k8s/bin go install sigs.k8s.io/kustomize/kustomize/v4@latest
|
$ GOBIN=/home/anel/mariadb/k8s/mariadb-k8s/bin go install sigs.k8s.io/controller-tools/cmd/controller-gen@latest # note here I used @v0.3.0 that couldn't install the manifests
|
$ make manifests
|
$ make install
|
# Custom resource definitions CRD installed
|
$ kubectl get crds
|
NAME CREATED AT
|
mariadbs.mariak8g.mariadb.org 2022-06-14T12:25:33Z
|
|
|
# Apply sample to create custom resource CR mariadb
|
$ kubectl apply -f config/samples/mariak8g_v1alpha1_mariadb.yaml
|
mariadb.mariak8g.mariadb.org/mariadb-sample created
|
|
|
$ kubectl get mariadb
|
NAME MARIADB STATE PORT AGE
|
mariadb-sample 3306 39s
|
Done with https://github.com/an3l/mariadb-k8s/tree/mariadb-refactoring
Future work will be contribution to the Percona operator
https://github.com/percona/percona-server-mysql-operator
$ kubectl get svcNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE$ make run/home/anel/operators/mariadb/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases/home/anel/operators/mariadb/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."go fmt ./...go vet ./...go run ./main.go2021-12-22T02:33:51.198-0800 INFO controller-runtime.metrics metrics server is starting to listen {"addr": ":8080"}2021-12-22T02:33:51.198-0800 INFO setup starting manager2021-12-22T02:33:51.198-0800 INFO starting metrics server {"path": "/metrics"}2021-12-22T02:33:51.199-0800 INFO controller.mariadb Starting EventSource {"reconciler group": "mariak8g.mariadb.org", "reconciler kind": "MariaDB", "source": "kind source: /, Kind="}2021-12-22T02:33:51.199-0800 INFO controller.mariadb Starting EventSource {"reconciler group": "mariak8g.mariadb.org", "reconciler kind": "MariaDB", "source": "kind source: /, Kind="}2021-12-22T02:33:51.199-0800 INFO controller.mariadb Starting EventSource {"reconciler group": "mariak8g.mariadb.org", "reconciler kind": "MariaDB", "source": "kind source: /, Kind="}2021-12-22T02:33:51.199-0800 INFO controller.mariadb Starting Controller {"reconciler group": "mariak8g.mariadb.org", "reconciler kind": "MariaDB"}2021-12-22T02:33:51.300-0800 INFO controller.mariadb Starting workers {"reconciler group": "mariak8g.mariadb.org", "reconciler kind": "MariaDB", "worker count": 1}2021-12-22T02:33:51.300-0800 INFO controllers.MariaDB1 Reconciling MariaDB kind {"MariaDB: ": "default/mariadb-sample", "mariadb": ""}2021-12-22T02:33:51.332-0800 INFO controllers.MariaDB1 Reconciled MariaDB kind {"MariaDB: ": "default/mariadb-sample", "mariadb": "mariadb-sample", "status": {"desiredReplicas":0,"lastMessage":"","dbState":"RUNNING"}}2021-12-22T02:33:51.332-0800 INFO controllers.MariaDB1 Reconciling MariaDB kind {"MariaDB: ": "default/mariadb-sample", "mariadb": ""}2021-12-22T02:33:51.364-0800 INFO controllers.MariaDB1 Reconciled MariaDB kind {"MariaDB: ": "default/mariadb-sample", "mariadb": "mariadb-sample", "status": {"desiredReplicas":0,"lastMessage":"","dbState":"RUNNING"}}$ kubectl get svc -wNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGECommit - https://github.com/an3l/mariadb-k8s/commit/234e3bde413ecceed7bc37bd022e094f6961973d