The best way to get Haystack running as a workload in a container orchestrator like Kubernetes is to create a service to expose one or more Hayhooks instances.

Create a Haystack Service

A single Hayhooks instance can serve multiple Pipelines. As a first step, it might be beneficial to start with a simple service running a single pod:

kind: Pod
apiVersion: v1
metadata:
  name: hayhooks
  labels:
    app: haystack
spec:
  containers:
  # We use the unstable release to keep this example fresh and relevant.
  - image: deepset/hayhooks:main
    name: hayhooks
    # Since we're using the moving tag `main`, we have to always pull
    # to be sure we're getting the latest.
    imagePullPolicy: Always

---

kind: Service
apiVersion: v1
metadata:
  name: haystack-service
spec:
  selector:
    app: haystack
  ports:
  # Default port used by the Hayhooks Docker image
  - port: 1416

If you have an ingress properly configured, haystack-service will expose the Hayhooks API. From there, you can manage and run Pipelines. See the Hayhooks docs for more details.

Auto-Run Pipelines at Pod Start

Hayhooks can load Haystack Pipelines at startup, making them readily available when the server starts. We can leverage this mechanism to have our pods immediately serve one or more Pipelines when they start.

Since Pipelines YAML definitions are rather small, we can use a ConfigMap to make them available to our pods. In this example, create the config map directly from files, assuming you have a local folder ./pipelines containing one or more Pipeline definitions:

kubectl create configmap pipelines --from-file=pipelines

You can double-check if the config map was correctly created with:

kubectl describe configmap pipelines

The next step is to mount the config map pipelines as a volume in our pods (see the Kubernetes docs for more details) by changing the Pod resource definition from the previous example like this:

kind: Pod
apiVersion: v1
metadata:
  name: hayhooks
  labels:
    app: haystack
spec:
  containers:
  # We use the unstable release to keep this example fresh and relevant.
  - image: deepset/hayhooks:main
    name: hayhooks
    # Since we're using the moving tag `main`, we have to always pull
    # to be sure we're getting the latest.
    imagePullPolicy: Always
    # Mount the ConfigMap containing the pipelines under
    # /opt/pipelines in the container
    volumeMounts:
      - name: config-volume
        mountPath: /opt/pipelines
    # Instruct Hayhooks that the pipelines we want to run at startup
    # will be found under /opt/pipelines
    env:
    - name: HAYHOOKS_PIPELINES_DIR
      value: /opt/pipelines
  volumes:
    - name: config-volume
      configMap:
        name: pipelines

Roll Out Multiple Pods

Haystack Pipelines are usually stateless – a perfect use case for distributing the requests to multiple pods running the same set of Pipelines. The easiest way to tell Kubernetes to roll out multiple replicas is through a Deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: haystack-deployment
spec:
  replicas: 3
  selector:
    matchLabels:
      app: haystack
  template:
    metadata:
      labels:
        app: haystack
    spec:
      containers:
      # We use the unstable release to keep this example fresh and relevant.
      - image: deepset/hayhooks:main
        name: hayhooks
        # Since we're using the moving tag `main`, we have to always pull
        # to be sure we're getting the latest.
        imagePullPolicy: Always
        # Mount the ConfigMap containing the pipelines under
        # /opt/pipelines in the container
        volumeMounts:
          - name: config-volume
            mountPath: /opt/pipelines
        # Instruct Hayhooks that the pipelines we want to run at startup
        # will be found under /opt/pipelines
        env:
        - name: HAYHOOKS_PIPELINES_DIR
          value: /opt/pipelines
      volumes:
        - name: config-volume
          configMap:
            name: pipelines

Implementing the above configuration will create three pods. Each pod will run a different instance of Hayhooks, all serving the same two Pipelines provided by the config map we created in the previous example.

Deploy With Helm

Haystack comes with a Helm chart that can be used to deploy a fully functional application on a Kubernetes cluster. The chart is distributed through the deepset repository, so the first thing to do is add it to Helm:

helm repo add deepset https://deepset-ai.github.io/charts/

To install the chart on a local cluster (such as KIND or Minikube) with the default values, you need to give a name to your application (for example, full-coral) and invoke Helm like this:

helm install full-coral deepset/hayhooks

The chart will create a ConfigMap you can use to store Pipeline definitions that will be automatically served when a Hayhooks pod starts. To update the config map when the application has already started, you can run:

kubectl create configmap full-coral-default-pipelines --from-file=your/pipelines/dir -o yaml --dry-run=client | kubectl replace -f -

Then, restart the application so it will pick up the Pipelines:

kubectl rollout restart deployment full-coral-haystack-deployment

Chart Values

Value	Type	Default
`replicaCount`	int	`1`
`image.repository`	string	`deepset/hayhooks`
`image.pullPolicy`	string	`Always`
`image.tag`	string	`main`
`pipelinesDirMount`	string	`/opt/pipelines`
`nameOverride`	string	`“”`
`fullnameOverride`	string	`“”`
`podAnnotations`	object	`{}`
`podLabels`	object	`{}`
`podSecurityContext`	object	`{}`
`securityContext`	object	`{}`
`service.port`	int	`1416`
`ingress.enabled`	bool	`true`
`ingress.className`	string	`“”`
`ingress.annotations`	string	`nginx.ingress.kubernetes.io/rewrite-target: /$2`
`ingress.hosts`	list	`[ host: localhost paths: - path: /haystack(/\|$)(.\*) pathType: ImplementationSpecific ]`
`ingress.tls`	list	`[]`
`resources`	object	`{}`
`livenessProbe.httpGet.path`	string	`/status`
`livenessProbe.httpGet.port`	string	`http`
`readinessProbe.httpGet.path`	string	`/status`
`readinessProbe.httpGet.port`	string	`http`
`autoscaling.enabled`	bool	`false`
`autoscaling.minReplicas`	int	`1`
`autoscaling.maxReplicas`	int	`100`
`autoscaling.targetCPUUtilizationPercentage`	int	`80`
`volumes`	list	`[]`
`volumeMounts`	list	`[]`
`nodeSelector`	object	`{}`
`tolerations`	list	`[]`
`affinity`	object	`{}`