Ingress
After first installing the LlamaCloud helm chart into your kubernetes environment, you will be able to test the deployment immediately by port-forwarding the frontend server to your local machine using the following command:
kubectl --namespace <your-namespace> port-forward svc/llamacloud-frontend 3000:3000Once that command is running, you will be able to visit the LlamaCloud UI at http://localhost:3000.
While this may be sufficient for initial testing of your deployment, you will eventually need to setup the an ingress when taking your deployment to production and leveraging LlamaCloud as an API based service.
Requirements
Section titled âRequirementsâ- A
super-cool.domain - An ingress controller deployed in your kubernetes cluster
- A popular choice is
ingress-nginx
- A popular choice is
Helm Chart Configuration
Section titled âHelm Chart ConfigurationâAs of version 0.1.47, the LlamaCloud helm chart supports the ability to configure and create an ingress resource for your deployment. The chart supports the following configuration:
ingress: enabled: true create: true labels: {} annotations: {} host: "" # your desired hostname scheme: https # or http tlsSecretName: "" # important if you are serving LlamaCloud over HTTPS ingressClassName: "" # different ingress controllers may require a different class name. if unset, the default ingress class will be used.We recommend setting up the ingress resource using the helm chart configuration above.
Once your ingress endpoint is setup, you can visit the LlamaCloud UI at https://<your-domain-name>. And, you can check the status of your ingress resource:
kubectl -n <your-llamacloud-namespace> get ingress
# Example outputNAME CLASS HOSTS ADDRESS PORTS AGEllamacloud-ingress nginx llamacloud.example <ip-or-load-balancer-address> 80, 443 10mIngress Route Details
Section titled âIngress Route Detailsâ/api: route all incoming requests with a path prefix of/apito the Backend service/: route all other requests to the Frontend service
TLS Configuration
Section titled âTLS ConfigurationâDepending on your ingress controller, you may need to add a TLS secret to your ingress resource. You can specify that in the .Values.ingress.tlsSecretName field. Currently, we only support a single host and a single TLS secret. For more information on TLS Secrets, see the Kubernetes Ingress documentation.
Self-Managed Ingress
Section titled âSelf-Managed IngressâIf you prefer to manage the ingress resource yourself, you can disable the ingress resource creation by setting ingress.create to false and then apply the following ingress resource to your cluster:
For information about configuring an ingress resource, see the Kubernetes Ingress documentation.
apiVersion: networking.k8s.io/v1kind: Ingressmetadata: name: llamacloud-nginx-ingress namespace: <your-llamacloud-namespace>spec: ingressClassName: <your-ingress-class-name> rules: - http: paths: - backend: service: name: llamacloud-backend # or whatever your backend service is named port: number: 8000 path: /api pathType: Prefix - backend: service: name: llamacloud-frontend # or whatever your frontend service is named port: number: 3000 path: / pathType: PrefixBACKEND_URL Configuration
Section titled âBACKEND_URL ConfigurationâThe BACKEND_URL environment variable is used by the Frontend service to know where to send /api requests to. When you self-manage your ingress resource, it is recommended to explicitly set this environment variable so the Frontend service doesnât have to rely on the internal NextJS proxy to route requests to the Backend service.
# (recommended) values.yaml# Enabling ingress via the charts, but not creating the ingress resource yourself. The helm charts will automatically configure the BACKEND_URL environment variable for the Frontend service to point to the ingress host.ingress: enabled: true create: false host: <your-domain-name>
# alternative values.yamlfrontend: extraEnvVariables: - name: BACKEND_URL value: "https://<your-domain-name>"Common Issues
Section titled âCommon Issuesâ-
When an ingress resource is created, sometimes the frontend service may not be able to resolve the certificate. Users may experience slowness or failure during the login flow if this happens and may see an
UNABLE_TO_VERIFY_LEAF_SIGNATUREerror in the frontend logs.- There are easy and hard ways to resolve this issue.
- The easy way is to set
NODE_TLS_REJECT_UNAUTHORIZED=0in the Frontend deployment with.Values.frontend.extraEnvVariables. This will tell the Frontend to ignore the certificate error. One drawback of this approach is that it may not be accepted by your organizationâs security policies. - The harder way is enable the Frontend pod to resolve the certificate you used to sign the ingress host. You can do this by mounting the certificate into the Frontend pod.
- If you are using a self-signed certificate, you can use the following to generate a certificate and key:
Terminal window # create a configmap with the Certificate Authoritykubectl create configmap ca-cert-config --from-file=<certificate-authority-name>.pem=/local/path/to/your/<certificate-authority-name>.pem - Then, you can mount the configmap into the Frontend pod and add the NODE_EXTRA_CA_CERTS environment variable:
# In the values.yaml file:frontend:config:tls:enabled: truecaCertSecretName: "ca-cert-secret"# or caCertConfigMapName: ca-cert-configcaCertMountPath: /mount/path/to/your/certificate-mount-path # default is /etc/llamacloud-ssl/certscaCertKey: <certificate-authority-name>.pem # default is cert.pem
-
If you do not have an automated DNS setup, you will need to provide the IP address of the ingress resource to your DNS provider.
If you require assistance with setting up an ingress resource, please reach out to us on support at llamaindex.ai and we will be happy to help you!