Skip to content

litellm-helm-overrides.adoc

Latest commit

 

History

History
88 lines (71 loc) · 2.81 KB

litellm-helm-overrides.adoc

File metadata and controls

88 lines (71 loc) · 2.81 KB

../common/generic-attributes.adoc = Examples of {litellm} {helm} chart override files

Example 1. Basic override file with {postgresql} deployment and master key automatically generated.
# litellm_custom_overrides.yaml
global:
  imagePullSecrets:
    - application-collection (1)
    - suse-ai-registry (2)
postgresql:
  persistence:
    storageClassName: "local-path" (3)

  1. Specify the secret containing the credentials for the {sappco} in global.imagePullSecrets. For instructions on how to configure the image pull secrets for the {sappco}, refer to [suse-ai-deploy-suse-ai].

  2. Specify the secret containing the credentials for the {sregistry} in global.imagePullSecrets. For instructions on how to configure the image pull secrets for the {sregistry}, refer to [suse-ai-deploy-suse-ai].

  3. Use local-path storage only for testing purposes. For production use, we recommend using a storage solution suitable for persistent storage, such as {sstorage}.

The {litellm} chart from {sregistry} makes use of subcharts such as {redis} and {postgresql} from the {sappco} registry. Therefore, both registry secrets must be configured.

Validating the installation

You can monitor the status of the {litellm} installation by running the following command:

{prompt_user}kubectl get pods -n <SUSE_AI_NAMESPACE>

NAME                         READY   STATUS    RESTARTS   AGE
[...]
litellm-5d8c4c864f-9v7q5      1/1     Running   1          2m33s
litellm-postgresql-0          1/1     Running   0          2m33s

Pods for the {litellm} deployment should transition to the Ready and Running states.

Validation

By default, the configuration does not enable {ingress}. To expose the service on localhost, run:

{prompt_user}kubectl \
  -n <SUSE_AI_NAMESPACE> port-forward service/litellm 4000:4000

Your {litellm} Proxy Server is now running at http://127.0.0.1:4000.

Admin UI access

You can access the administration page at http://127.0.0.1:4000/ui and log in using the administrator account. Retrieve the password for the administrator user from the secret resource:

{prompt_user}kubectl get secret litellm-masterkey \
  -n <SUSE_AI_NAMESPACE> -o jsonpath='{.data.masterkey}' | base64 --decode