Schedule Elasticsearch Curator Tasks Natively in Kubernetes

As of Kubernetes 1.21, CronJobs are a stable native Kubernetes feature. Using Kubernetes CronJobs and Elasticsearch Curator, we can schedule periodic changes on an Elasticsearch instance.

Unfortunately, Elasticsearch does not currently publish a native Elasticsearch Curator Docker image. Luckily, however, Bitnami provides such a Docker image, available on DockerHub.

To use the Elasticsearch Curator with a Kubernetes CronJob we need to define two Kubernetes objects, a CronJob and a ConfigMap. We use the ConfigMap to save the configuration of Curator and the task steps we want Curator to take. I have included both the Curator tasks and Curator configuration in the same ConfigMap as separate keys to simplify the setup. However, if it contains sensitive values, the Curator configuration could also be stored as a secret.

An example of the ConfigMap, containing both the Curator configuration and tasks:

apiVersion: v1
kind: ConfigMap
metadata:
    name: curator
    namespace: logging
data:
    config.yml: |
        client:
        hosts:
            - elasticsearch-master
        port: 9200
        url_prefix:
        use_ssl: False
        certificate:
        client_cert:
        client_key:
        ssl_no_validate: False
        username:
        password:
        timeout: 30
        master_only: False

        logging:
        loglevel: INFO
        logfile:
        logformat: default
        blacklist: ['elasticsearch', 'urllib3']
    actions.yml: |
        actions:
            1:
                action: delete_indices
                description: "Delete indices older than 20 days"
                options:
                ignore_empty_list: True
                filters:
                - filtertype: age
                    source: creation_date
                    direction: older
                    unit: days
                    unit_count: 20

The CronJob that we need to define will need to mount both the Curator configuration and the task file values stored in our ConfigMap. I have mounted both Curator files in the /curator directory in this example, which is scheduled to run at midnight on the first day of each month:

apiVersion: batch/v1
kind: CronJob
metadata:
  name: curator
  namespace: logging
spec:
  schedule: "0 0 1 * *"
  jobTemplate:
    spec:
      template:
          spec:
            containers:
              - name: curator
                image: bitnami/elasticsearch-curator:latest
                command: ["/bin/sh", "-c"]
                args:
                - curator --config /curator/config.yml /curator/actions.yml;
                volumeMounts:
                  - name: curator
                    mountPath: /curator
                    readOnly: true
            volumes:
            - name: curator
                configMap:
                name: curator
                items:
                  - key: "config.yml"
                    path: "config.yml"
                  - key: "actions.yml"
                    path: "actions.yml"
            restartPolicy: OnFailure

And that is all! The CronJob and ConfigMap are easily customisable to suit whatever you need Curator to achieve in your Kubernetes cluster.

Documentation on Kubernetes CronJobs, and Elasticsearch Curator.