A DBA holds a big responsibility for keeping mission critical databases in the organization to be always up and running and meeting all the SLAs assigned to the database, such as availability, recovery, security, performance, and more. This is really a difficult and risky job to maintain mission critical databases without proper tools providing monitoring data. In real life, it is very important for DBAs to be equipped with good database monitoring tools to ensure they can quickly identify, resolve and fix any potential problems before those problems even impact operations.
MySQL Enterprise Monitor (MEM) is a comprehensive, centralized, real-time MySQL database monitoring tool that provides a complete range of database monitoring aspects: from availability to performance. It is able to monitor all MySQL database instances in the environment and present information in a web-based interface which visually displays a global and detailed view of all monitored databases, including MySQL cluster (NDB Cluster and InnoDB Cluster), replication, backup, all security features on MySQL Enterprise Edition, etc. More information about MEM can be found here: https://dev.mysql.com/doc/mysql-monitor/8.0/en/
This article discusses on how to run the MEM on Kubernetes to monitor closely all MySQL Enterprise Edition database that running on the same Kubernetes cluster. We will start by creating a docker image from scratch. (Disclaimer: Please note, this is unofficial and DIY way to create docker image for MEM. If you require your MEM to run on container, you can follow below steps, test those, but support is by your own in case of issues).
1. Obtain MySQL Enterprise Monitor 8.0.21 binary
Download mysql-enterprise-monitor-8.0.21 for Linux x86_64 from edelivery.oracle.com (file name: V999555-01.zip). Unzip V999555-01.zip to get installation file: mysqlmonitor-8.0.21.1248-linux-x86_64-installer.bin
2. Prepare and run docker-build to build image
FROM oraclelinux:7-slim
RUN groupadd mysql
RUN useradd -g mysql mysql
RUN mkdir /mem
COPY mysqlmonitor-8.0.21.1248-linux-x86_64-installer.bin docker-entrypoint.sh /mem/
WORKDIR /mem
RUN chmod u+x /mem/docker-entrypoint.sh
RUN yum install -y libaio
RUN yum install -y numactl-libs
RUN yum install -y hostname
RUN yum install -y procps-ng
ENTRYPOINT ["/mem/docker-entrypoint.sh"]
Create docker-entrypoint.sh file on the same directory:
#! /bin/bash
if [ "$ADMINUSER" = "" ]; then
ADMINUSER="admin"
fi
if [ "$ADMINPASSWORD" = "" ]; then
ADMINPASSWORD="admin"
fi;
if [ "$SYSTEMSIZE" = "" ]; then
SYSTEMSIZE="small"
fi;
echo "Admin user is set to $ADMINUSER"
echo "Admin password is set to $ADMINPASSWORD"
echo "System size is set to $SYSTEMSIZE"
if [ `ls -l /opt/mysql/enterprise/monitor | grep total | awk '{print $2}'` -eq 0 ]; then
echo "Install new Service Manager"
/mem/mysqlmonitor-8.0.21.1248-linux-x86_64-installer.bin --mode unattended --system_size $SYSTEMSIZE --adminuser $ADMINUSER --adminpassword $ADMINPASSWORD --mysql_installation_type bundled
/opt/mysql/enterprise/monitor/mysqlmonitorctl.sh status
fi;
while true; do
echo "Checking if MySQL is Up"
if [ `/opt/mysql/enterprise/monitor/mysqlmonitorctl.sh status mysql | grep "not running" | wc -l` -gt 0 ]; then
echo "MySQL is not running, starting up"
/opt/mysql/enterprise/monitor/mysqlmonitorctl.sh start mysql
else
echo "MySQL is already running"
fi
echo "Checking if Tomcat is Up"
if [ `/opt/mysql/enterprise/monitor/mysqlmonitorctl.sh status tomcat | grep "not running" | wc -l` -gt 0 ]; then
echo "Tomcat is not running, starting up"
/opt/mysql/enterprise/monitor/mysqlmonitorctl.sh start tomcat
else
echo "Tomcat is already running"
fi
/opt/mysql/enterprise/monitor/mysqlmonitorctl.sh status
sleep 15;
done;
Check our installed docker images:
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
mysql/enterprise-server 8.0 106673d79487 8 weeks ago 452MB
mysql/mysql-cluster latest ca2cf722118d 3 months ago 415MB
k8s.gcr.io/kube-proxy v1.16.2 8454cbe08dc9 10 months ago 86.1MB
k8s.gcr.io/kube-apiserver v1.16.2 c2c9a0406787 10 months ago 217MB
k8s.gcr.io/kube-controller-manager v1.16.2 6e4bffa46d70 10 months ago 163MB
k8s.gcr.io/kube-scheduler v1.16.2 ebac1ae204a2 10 months ago 87.3MB
quay.io/kubernetes-ingress-controller/nginx-ingress-controller 0.26.1 29024c9c6e70 10 months ago 483MB
k8s.gcr.io/etcd 3.3.15-0 b2756210eeab 11 months ago 247MB
kubernetesui/dashboard v2.0.0-beta4 6802d83967b9 11 months ago 84MB
k8s.gcr.io/coredns 1.6.2 bf261d157914 12 months ago 44.1MB
k8s.gcr.io/kube-addon-manager v9.0.2 bd12a212f9dc 12 months ago 83.1MB
kubernetesui/metrics-scraper v1.0.1 709901356c11 13 months ago 40.1MB
k8s.gcr.io/kube-addon-manager v9.0 119701e77cbc 19 months ago 83.1MB
k8s.gcr.io/kubernetes-dashboard-amd64 v1.10.1 f9aed6605b81 20 months ago 122MB
k8s.gcr.io/k8s-dns-sidecar-amd64 1.14.13 4b2e93f0133d 23 months ago 42.9MB
k8s.gcr.io/k8s-dns-kube-dns-amd64 1.14.13 55a3c5209c5e 23 months ago 51.2MB
k8s.gcr.io/k8s-dns-dnsmasq-nanny-amd64 1.14.13 6dc8ef8287d3 23 months ago 41.4MB
k8s.gcr.io/pause 3.1 da86e6ba6ca1 2 years ago 742kB
gcr.io/k8s-minikube/storage-provisioner v1.8.1 4689081edb10 2 years ago 80.8MB
Execute the following command to build the image:
$ set -x
$ set -e
$ docker build -m 2g -c 4 -t private/mem:8.0.21 .
Check again our installed docker images:
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
private/mem 8.0.21 57ddbccb6758 39 seconds ago 1.74GB
oraclelinux 7-slim 153f8d73287e 4 weeks ago 131MB
mysql/enterprise-server 8.0 106673d79487 8 weeks ago 452MB
mysql/mysql-cluster latest ca2cf722118d 3 months ago 415MB
k8s.gcr.io/kube-proxy v1.16.2 8454cbe08dc9 10 months ago 86.1MB
k8s.gcr.io/kube-controller-manager v1.16.2 6e4bffa46d70 10 months ago 163MB
k8s.gcr.io/kube-scheduler v1.16.2 ebac1ae204a2 10 months ago 87.3MB
k8s.gcr.io/kube-apiserver v1.16.2 c2c9a0406787 10 months ago 217MB
quay.io/kubernetes-ingress-controller/nginx-ingress-controller 0.26.1 29024c9c6e70 10 months ago 483MB
k8s.gcr.io/etcd 3.3.15-0 b2756210eeab 11 months ago 247MB
kubernetesui/dashboard v2.0.0-beta4 6802d83967b9 11 months ago 84MB
k8s.gcr.io/coredns 1.6.2 bf261d157914 12 months ago 44.1MB
k8s.gcr.io/kube-addon-manager v9.0.2 bd12a212f9dc 12 months ago 83.1MB
kubernetesui/metrics-scraper v1.0.1 709901356c11 13 months ago 40.1MB
k8s.gcr.io/kube-addon-manager v9.0 119701e77cbc 19 months ago 83.1MB
k8s.gcr.io/kubernetes-dashboard-amd64 v1.10.1 f9aed6605b81 20 months ago 122MB
k8s.gcr.io/k8s-dns-sidecar-amd64 1.14.13 4b2e93f0133d 23 months ago 42.9MB
k8s.gcr.io/k8s-dns-kube-dns-amd64 1.14.13 55a3c5209c5e 23 months ago 51.2MB
k8s.gcr.io/k8s-dns-dnsmasq-nanny-amd64 1.14.13 6dc8ef8287d3 23 months ago 41.4MB
k8s.gcr.io/pause 3.1 da86e6ba6ca1 2 years ago 742kB
gcr.io/k8s-minikube/storage-provisioner v1.8.1 4689081edb10 2 years ago 80.8MB
3. Prepare YAML
Deploying MEM on Kubernetes requires the following sample YAML (mem.yaml). Please note the pod needs to run as privileged container, or else tomcat service will fail to start. We will configure db repository with user memadmin:manager. MEM will be for small number of monitored database instances (up to 5 - 10 databases). If there’re more number of instances, consider to change SYSTEMSIZE parameter from “small” to “medium” (up to 100 databases) or “large” (more than 100 databases).
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: mem
namespace: mysql-cluster
spec:
replicas: 1
serviceName: mem
selector:
matchLabels:
app: mem
template:
metadata:
labels:
app: mem
spec:
containers:
- image: private/mem:8.0.21
name: mem
env:
- name: ADMINUSER
value: memadmin
- name: ADMINPASSWORD
value: manager
- name: SYSTEMSIZE
value: small
resources:
limits:
cpu: "2"
memory: "4096Mi"
requests:
cpu: "2"
memory: "2048Mi"
volumeMounts:
- name: mem-vol
mountPath: /opt/mysql/enterprise/monitor
securityContext:
privileged: true
volumeClaimTemplates:
- metadata:
name: mem-vol
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 1Gi
---
4. Deploy MEM
Deploy the YAML file to Kubernetes
$ kubectl apply -f mem.yaml
Check Pod Status
$ kubectl -n mysql-cluster get pod
NAME READY STATUS RESTARTS AGE
mem-0 1/1 Running
0 3s
Wait for 1 min to give time for MEM configuration process. Then, check the status of our MEM on Kubernetes
$ kubectl -n mysql-cluster exec -it mem-0 -- /opt/mysql/enterprise/monitor/mysqlmonitorctl.sh status
MySQL Enterprise MySQL is running
MySQL Enterprise Tomcat is running
Both MySQL instance and Tomcat are running!
5. Access our MEM
In this article, I just simply used port-forward to test if my MEM is running:
$ kubectl -n mysql-cluster port-forward pods/mem-0 18443:18443
Then, I opened my web browser and accessed https://127.0.0.1:18443
The MEM initial setup page appeared after few minutes.
It's working !
