Performance Certification of Couchbase Autonomous Operator on Kubernetes

At Couchbase, we take performance very seriously, and with the launch of our new product, Couchbase Autonomous Operator 1.0, we wanted to make sure it’s Enterprise-grade and production ready for customers.

In this post, we will discuss the detailed performance results from running YCSB Performance Benchmark tests on Couchbase Server 5.5 using the Autonomous Operator to deploy on Kubernetes platform. One of the big concerns for Enterprises planning to run a database on Kubernetes is "performance."

This document gives a quick comparison of two workloads, namely YCSB A & E with Couchbase Server 5.5 on Kubernetes vs. bare metal.

YCSB Workload A: This workload has a mix of 50/50 reads and writes. An application example is a session store recording recent actions.

Workload E: Short ranges: In this workload, short ranges of records are queried, instead of individual records. Application example: threaded conversations, where each scan is for the posts in a given thread (assumed to be clustered by thread id).

In general, we observed no significant performance degradation in running Couchbase Cluster on Kubernetes, Workload A had on par performance compared to bare metal and Workload E had approximately less than 10% degradation.

Setup

For the setup, Couchbase was installed using the Operator deployment as stated below. For more details on the setup, please refer here.

Files

Operator deployment: deployment.yaml (See Appendix)

Couchbase deployment: couchbase-cluster-simple-selector.yaml (See Appendix)

Client / workload generator deployment: pillowfight-ycsb.yaml (See Appendix) (Official pillowfight docker image from dockerhub and installed java and YCSB manually on top of it)

Hardware

7 servers

24 CPU x 64GB RAM per server

Couchbase Setup

4 servers: 2 data nodes, 2 index+query nodes

40GB RAM quota for data service

40GB RAM quota for index services

1 data/bucket replica

1 primary index replica

Tests

YCSB WorkloadA and WorkloadE

10M docs

Workflow after new empty k8s cluster is initialized on 7 servers:

Running the workloads:

Test results:

Conclusions

Couchbase Server 5.5 is production ready to be deployed on Kubernetes with the Autonomous Operator. Performance of Couchbase Server 5.5 on Kubernetes comparable to running on bare metal. There is little performance penalty in running Couchbase Server on Kubernetes platform. Looking at the results Workload A had on par performance compared to bare metal and Workload E had approximately less than 10% degradation.

References

1. YCSB Workloads https://github.com/brianfrankcooper/YCSB/wiki/Core-Workloads
2. Couchbase Kubernetes page https://www.couchbase.com/products/cloud/kubernetes
3. Download Couchbase Autonomous Operator https://www.couchbase.com/downloads
4. Introducing Couchbase Operator https://blog.couchbase.com/couchbase-autonomous-operator-1-0-for-kubernetes-and-openshift/

Appendix

My deployment.yaml file:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: couchbase-operator
spec:
  replicas: 1
  template:
    metadata:
      labels:
        name: couchbase-operator
    spec:
      nodeSelector:
        type: power
      containers:
      - name: couchbase-operator
        image: couchbase/couchbase-operator-internal:1.0.0-292
        command:
        - couchbase-operator
        # Remove the arguments section if you are installing the CRD manually
        args:
        - -create-crd
        - -enable-upgrades=false
        env:
        - name: MY_POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: MY_POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        ports:
          - name: readiness-port
            containerPort: 8080
        readinessProbe:
          httpGet:
            path: /readyz
            port: readiness-port
          initialDelaySeconds: 3
          periodSeconds: 3
          failureThreshold: 19

My couchbase-cluster-simple-selector.yaml file:

apiVersion: couchbase.database.couchbase.com/v1

kind: CouchbaseCluster

metadata:

 name: cb-example

spec:

 baseImage: couchbase/server

 version: enterprise-5.5.0

 authSecret: cb-example-auth

 exposeAdminConsole: true

 antiAffinity: true

 exposedFeatures:

   - xdcr

 cluster:

   dataServiceMemoryQuota: 40000

   indexServiceMemoryQuota: 40000

   searchServiceMemoryQuota: 1000

   eventingServiceMemoryQuota: 1024

   analyticsServiceMemoryQuota: 1024

   indexStorageSetting: memory_optimized

   autoFailoverTimeout: 120

   autoFailoverMaxCount: 3

   autoFailoverOnDataDiskIssues: true

   autoFailoverOnDataDiskIssuesTimePeriod: 120

   autoFailoverServerGroup: false

 buckets:

   - name: default

     type: couchbase

     memoryQuota: 20000

     replicas: 1

     ioPriority: high

     evictionPolicy: fullEviction

     conflictResolution: seqno

     enableFlush: true

     enableIndexReplica: false

 servers:

   - size: 2

     name: data

     services:

       - data

     pod:

       nodeSelector:

         type: kv

       resources:

         limits:

           cpu: 22000m

           memory: 48Gi

         requests:

          cpu: 22000m

          memory: 48Gi

   - size: 2

     name: qi

     services:

       - index

       - query

     pod:

       nodeSelector:

         type: kv

       resources:

         limits:

           cpu: 22000m

           memory: 48Gi

         requests:

           cpu: 22000m

           memory: 48Gi

My pillowfight-ycsb.yaml file:

apiVersion: batch/v1

kind: Job

metadata:

 name: pillowfight

spec:

 template:

   metadata:

     name: pillowfight

   spec:

     containers:

     - name: pillowfight

       image: sequoiatools/pillowfight:v5.0.1

       command: ["sh", "-c", "tail -f /dev/null"]

     restartPolicy: Never

     nodeSelector:

       type: client