File tree Expand file tree Collapse file tree
Expand file tree Collapse file tree Original file line number Diff line number Diff line change @@ -111,7 +111,6 @@ The following {vllm} override file includes basic configuration options.
111111* Access to a {huggingface} token (`HF_TOKEN` ).
112112* The model `meta-llama/Llama-3.1-8B-Instruct` from this example is a gated model that requires you to accept the agreement to access it.
113113For more information, see link:https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct[].
114- * The `runtimeClassName` specified here is `nvidia` .
115114* Update the `storageClass:` entry for each `modelSpec` .
116115
117116[source,yaml]
@@ -121,7 +120,7 @@ global:
121120 imagePullSecrets:
122121 - application-collection
123122servingEngineSpec:
124- modelSpec:
123+ modelSpec:
125124 - name: "llama3" <.>
126125 registry: "dp.apps.rancher.io" <.>
127126 repository: "containers/vllm-openai" <.>
@@ -262,7 +261,7 @@ global:
262261 imagePullSecrets:
263262 - application-collection
264263servingEngineSpec:
265- modelSpec:
264+ modelSpec:
266265 - name: "llama3"
267266 registry: "dp.apps.rancher.io"
268267 repository: "containers/vllm-openai"
@@ -381,7 +380,7 @@ global:
381380 imagePullSecrets:
382381 - application-collection
383382servingEngineSpec:
384- modelSpec:
383+ modelSpec:
385384 - name: "mistral"
386385 registry: "dp.apps.rancher.io"
387386 repository: "containers/lmcache-vllm-openai"
@@ -429,7 +428,7 @@ global:
429428 imagePullSecrets:
430429 - application-collection
431430servingEngineSpec:
432- modelSpec:
431+ modelSpec:
433432 - name: "mistral"
434433 registry: "dp.apps.rancher.io"
435434 repository: "containers/lmcache-vllm-openai"
Original file line number Diff line number Diff line change @@ -164,7 +164,7 @@ metadata:
164164 namespace: default
165165spec:
166166 restartPolicy: OnFailure
167- containers:
167+ containers:
168168 - name: cuda-container
169169 image: nvcr.io/nvidia/k8s/cuda-sample:nbody
170170 args: ["nbody", "-gpu", "-benchmark"]
You can’t perform that action at this time.
0 commit comments