Skip to content

Restructure remaining GKE training recipes to align with standard naming convention #226

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
tonyjohnchen merged 1 commit into from
May 11, 2026
Merged

Restructure remaining GKE training recipes to align with standard naming convention #226

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .../gpt3-175b/nemo-pretraining-gke/README.md → ...3_175b/nemo-gke/nemo2507/recipe/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,7 @@ recipe folder.
git clone https://github.com/ai-hypercomputer/gpu-recipes.git
cd gpu-recipes
export REPO_ROOT=`git rev-parse --show-toplevel`
export RECIPE_ROOT=$REPO_ROOT/training/a3mega/gpt3-175b/nemo-pretraining-gke
export RECIPE_ROOT=$REPO_ROOT/training/a3mega/gpt3_175b/nemo-gke/nemo2507/recipe
```

### Get cluster credentials
Expand Down
0 ...pt3-175b/nemo-pretraining-gke/values.yaml → ...175b/nemo-gke/nemo2507/recipe/values.yaml
View file
Open in desktop
File renamed without changes.
2 changes: 1 addition & 1 deletion ...-1-70b/nemo-pretraining-gke-gcs/README.md → ...0b/nemo-gke-gcs/nemo2507/recipe/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@ From your client, clone the `gpu-recipes` repository and set a reference to the
git clone https://github.com/ai-hypercomputer/gpu-recipes.git
cd gpu-recipes
export REPO_ROOT=`git rev-parse --show-toplevel`
export RECIPE_ROOT=$REPO_ROOT/training/a3mega/llama3-1-70b/nemo-pretraining-gke-gcs
export RECIPE_ROOT=$REPO_ROOT/training/a3mega/llama3_70b/nemo-gke-gcs/nemo2507/recipe
```

### Get cluster credentials
Expand Down
0 ...-70b/nemo-pretraining-gke-gcs/values.yaml → .../nemo-gke-gcs/nemo2507/recipe/values.yaml
View file
Open in desktop
File renamed without changes.
2 changes: 1 addition & 1 deletion ...nemo-pretraining-gke-resiliency/README.md → ...-gke-resiliency/nemo2507/recipe/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,7 @@ From your client, clone the `gpu-recipes` repository and set a reference to the
git clone https://github.com/ai-hypercomputer/gpu-recipes.git
cd gpu-recipes
export REPO_ROOT=`git rev-parse --show-toplevel`
export RECIPE_ROOT=$REPO_ROOT/training/a3mega/llama3-1-70b/nemo-pretraining-gke-resiliency
export RECIPE_ROOT=$REPO_ROOT/training/a3mega/llama3_70b/nemo-gke-resiliency/nemo2507/recipe
```

### Get cluster credentials
Expand Down
2 changes: 1 addition & 1 deletion ...etraining-gke-resiliency/goodput-guide.md → ...siliency/nemo2507/recipe/goodput-guide.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ Achieving high GoodPut can be challenging due to several factors common in large
| **Stragglers and Performance Bottlenecks** | Slower nodes delay the entire job, underutilizing resources. | 3-7% |
| **Lack of Rapid Failure Detection and Diagnosis** | Longer detection/diagnosis time increases downtime. | 2-5% |

This guide provides a general overview of techniques and tools to address these common challenges and maximize ML GoodPut. While the principles discussed are broadly applicable, we will use the [Llama 3.1 70B pretraining recipe](https://github.com/AI-Hypercomputer/gpu-recipes/tree/main/training/a3mega/llama3-1-70b/nemo-pretraining-gke-resiliency) as a concrete case study to illustrate how these components can be implemented and customized for large-scale training workloads on Google Cloud. The goal is to showcase a "DIY" style product, where users can understand and selectively adopt these "Lego blocks" to build resilient and efficient training pipelines.
This guide provides a general overview of techniques and tools to address these common challenges and maximize ML GoodPut. While the principles discussed are broadly applicable, we will use the [Llama 3.1 70B pretraining recipe](https://github.com/AI-Hypercomputer/gpu-recipes/tree/main/training/a3mega/llama3_70b/nemo-gke-resiliency/nemo2507/recipe) as a concrete case study to illustrate how these components can be implemented and customized for large-scale training workloads on Google Cloud. The goal is to showcase a "DIY" style product, where users can understand and selectively adopt these "Lego blocks" to build resilient and efficient training pipelines.

## TLDR: Recommended Lego Blocks for Your Deployment
For customers looking to improve GoodPut on their own ML training workloads, here’s a concise guide to the key strategies discussed in this document, presented as 'Lego blocks' you can implement:
Expand Down
0 ...pretraining-gke-resiliency/ksa-setup.yaml → ...resiliency/nemo2507/recipe/ksa-setup.yaml
View file
Open in desktop
File renamed without changes.
0 ...ing-gke-resiliency/kueue-merge-patch.yaml → ...cy/nemo2507/recipe/kueue-merge-patch.yaml
View file
Open in desktop
File renamed without changes.
0 .../nemo-pretraining-gke-resiliency/train.py → ...o-gke-resiliency/nemo2507/recipe/train.py
View file
Open in desktop
File renamed without changes.
0 ...retraining-gke-resiliency/values-gcs.yaml → ...esiliency/nemo2507/recipe/values-gcs.yaml
View file
Open in desktop
File renamed without changes.
0 ...ing-gke-resiliency/values-supervisor.yaml → ...cy/nemo2507/recipe/values-supervisor.yaml
View file
Open in desktop
File renamed without changes.
0 ...mo-pretraining-gke-resiliency/values.yaml → ...ke-resiliency/nemo2507/recipe/values.yaml
View file
Open in desktop
File renamed without changes.
2 changes: 1 addition & 1 deletion ...llama3-70b/nemo-pretraining-gke/README.md → ...ke/nemo2507/128gpus-bf16/recipe/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ recipe folder.
git clone https://github.com/ai-hypercomputer/gpu-recipes.git
cd gpu-recipes
export REPO_ROOT=`git rev-parse --show-toplevel`
export RECIPE_ROOT=$REPO_ROOT/training/a3mega/llama3-70b/nemo-pretraining-gke
export RECIPE_ROOT=$REPO_ROOT/training/a3mega/llama3_70b/nemo-gke/nemo2507/256gpus-bf16/recipe/old_llama3_70b
```

### Get cluster credentials
Expand Down
0 ...ama3-70b/nemo-pretraining-gke/values.yaml → .../nemo2507/128gpus-bf16/recipe/values.yaml
View file
Open in desktop
File renamed without changes.
2 changes: 1 addition & 1 deletion ...ama3-1-70b/nemo-pretraining-gke/README.md → ...ke/nemo2507/256gpus-bf16/recipe/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ recipe folder.
git clone https://github.com/ai-hypercomputer/gpu-recipes.git
cd gpu-recipes
export REPO_ROOT=`git rev-parse --show-toplevel`
export RECIPE_ROOT=$REPO_ROOT/training/a3mega/llama3-1-70b/nemo-pretraining-gke
export RECIPE_ROOT=$REPO_ROOT/training/a3mega/llama3_70b/nemo-gke/nemo2507/256gpus-bf16/recipe
```

### Get cluster credentials
Expand Down
0 ...a3-1-70b/nemo-pretraining-gke/values.yaml → .../nemo2507/256gpus-bf16/recipe/values.yaml
View file
Open in desktop
File renamed without changes.
2 changes: 1 addition & 1 deletion ...xtral-8x7b/nemo-pretraining-gke/README.md → ...l_8x7b/nemo-gke/nemo2507/recipe/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ recipe folder.
git clone https://github.com/ai-hypercomputer/gpu-recipes.git
cd gpu-recipes
export REPO_ROOT=`git rev-parse --show-toplevel`
export RECIPE_ROOT=$REPO_ROOT/training/a3mega/mixtral-8x7b/nemo-pretraining-gke
export RECIPE_ROOT=$REPO_ROOT/training/a3mega/mixtral_8x7b/nemo-gke/nemo2507/recipe
```

### Get cluster credentials
Expand Down
0 ...ral-8x7b/nemo-pretraining-gke/values.yaml → ...8x7b/nemo-gke/nemo2507/recipe/values.yaml
View file
Open in desktop
File renamed without changes.
0 ...602/64gpus_bf16_1280gbs/recipe/Chart.yaml → ...602/64gpus-bf16-gbs1280/recipe/Chart.yaml
View file
Open in desktop
File renamed without changes.
2 changes: 1 addition & 1 deletion ...2602/64gpus_bf16_1280gbs/recipe/README.md → ...2602/64gpus-bf16-gbs1280/recipe/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ Clone the `gpu-recipes` repository and set a reference to the recipe folder.
git clone https://github.com/ai-hypercomputer/gpu-recipes.git
cd gpu-recipes
export REPO_ROOT=`git rev-parse --show-toplevel`
export RECIPE_ROOT=$REPO_ROOT/training/a3ultra/gpt-oss-120b/megatron-bridge-pretraining-gke/8node-BF16-GBSunknown/recipe
export RECIPE_ROOT=$REPO_ROOT/training/a3ultra/gpt_oss_120b/nemo-gke/nemo2602/64gpus-bf16-gbs1280/recipe
cd $RECIPE_ROOT
```

Expand Down
0 ...1280gbs/recipe/custom_setup_experiment.py → ...gbs1280/recipe/custom_setup_experiment.py
View file
Open in desktop
File renamed without changes.
0 ...02/64gpus_bf16_1280gbs/recipe/launcher.sh → ...02/64gpus-bf16-gbs1280/recipe/launcher.sh
View file
Open in desktop
File renamed without changes.
0 .../templates/workload-config-configmap.yaml → .../templates/workload-config-configmap.yaml
View file
Open in desktop
File renamed without changes.
0 ...280gbs/recipe/templates/workload-job.yaml → ...bs1280/recipe/templates/workload-job.yaml
View file
Open in desktop
File renamed without changes.
0 ...emplates/workload-launcher-configmap.yaml → ...emplates/workload-launcher-configmap.yaml
View file
Open in desktop
File renamed without changes.
0 ...280gbs/recipe/templates/workload-svc.yaml → ...bs1280/recipe/templates/workload-svc.yaml
View file
Open in desktop
File renamed without changes.
0 ...02/64gpus_bf16_1280gbs/recipe/values.yaml → ...02/64gpus-bf16-gbs1280/recipe/values.yaml
View file
Open in desktop
File renamed without changes.
2 changes: 1 addition & 1 deletion ...nemo-pretraining-gke-resiliency/README.md → ...-gke-resiliency/nemo2412/recipe/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,7 @@ From your client, clone the `gpu-recipes` repository and set a reference to the
git clone https://github.com/ai-hypercomputer/gpu-recipes.git
cd gpu-recipes
export REPO_ROOT=`git rev-parse --show-toplevel`
export RECIPE_ROOT=$REPO_ROOT/training/a3ultra/llama3-1-405b/nemo-pretraining-gke-resiliency
export RECIPE_ROOT=$REPO_ROOT/training/a3ultra/llama31_405b/nemo-gke-resiliency/nemo2412/recipe
```

### Get cluster credentials
Expand Down
2 changes: 1 addition & 1 deletion ...etraining-gke-resiliency/goodput-guide.md → ...siliency/nemo2412/recipe/goodput-guide.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ Achieving high GoodPut can be challenging due to several factors common in large
| **Stragglers and Performance Bottlenecks** | Slower nodes delay the entire job, underutilizing resources. | 3-7% |
| **Lack of Rapid Failure Detection and Diagnosis** | Longer detection/diagnosis time increases downtime. | 2-5% |

This guide provides a general overview of techniques and tools to address these common challenges and maximize ML GoodPut. While the principles discussed are broadly applicable, we will use the [Llama 3.1 405B pretraining recipe](https://github.com/AI-Hypercomputer/gpu-recipes/tree/main/training/a3ultra/llama3-1-405b/nemo-pretraining-gke-resiliency) as a concrete case study to illustrate how these components can be implemented and customized for large-scale training workloads on Google Cloud. The goal is to showcase a "DIY" style product, where users can understand and selectively adopt these "Lego blocks" to build resilient and efficient training pipelines.
This guide provides a general overview of techniques and tools to address these common challenges and maximize ML GoodPut. While the principles discussed are broadly applicable, we will use the [Llama 3.1 405B pretraining recipe](https://github.com/AI-Hypercomputer/gpu-recipes/tree/main/training/a3ultra/llama31_405b/nemo-gke-resiliency/nemo2412/recipe) as a concrete case study to illustrate how these components can be implemented and customized for large-scale training workloads on Google Cloud. The goal is to showcase a "DIY" style product, where users can understand and selectively adopt these "Lego blocks" to build resilient and efficient training pipelines.

## TLDR: Recommended Lego Blocks for Your Deployment
For customers looking to improve GoodPut on their own ML training workloads, here’s a concise guide to the key strategies discussed in this document, presented as 'Lego blocks' you can implement:
Expand Down
0 ...pretraining-gke-resiliency/ksa-setup.yaml → ...resiliency/nemo2412/recipe/ksa-setup.yaml
View file
Open in desktop
File renamed without changes.
0 ...ing-gke-resiliency/kueue-merge-patch.yaml → ...cy/nemo2412/recipe/kueue-merge-patch.yaml
View file
Open in desktop
File renamed without changes.
0 .../nemo-pretraining-gke-resiliency/train.py → ...o-gke-resiliency/nemo2412/recipe/train.py
View file
Open in desktop
File renamed without changes.
0 ...retraining-gke-resiliency/values-gcs.yaml → ...esiliency/nemo2412/recipe/values-gcs.yaml
View file
Open in desktop
File renamed without changes.
0 ...ing-gke-resiliency/values-supervisor.yaml → ...cy/nemo2412/recipe/values-supervisor.yaml
View file
Open in desktop
File renamed without changes.
0 ...mo-pretraining-gke-resiliency/values.yaml → ...ke-resiliency/nemo2412/recipe/values.yaml
View file
Open in desktop
File renamed without changes.
2 changes: 1 addition & 1 deletion ...ma3-1-405b/nemo-pretraining-gke/README.md → ...1_405b/nemo-gke/nemo2412/recipe/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ Clone the `gpu-recipes` repository and set a reference to the recipe folder.
git clone https://github.com/ai-hypercomputer/gpu-recipes.git
cd gpu-recipes
export REPO_ROOT=`git rev-parse --show-toplevel`
export RECIPE_ROOT=$REPO_ROOT/training/a3ultra/llama3-1-405b/nemo-pretraining-gke
export RECIPE_ROOT=$REPO_ROOT/training/a3ultra/llama31_405b/nemo-gke/nemo2412/recipe
cd $RECIPE_ROOT
```

Expand Down
0 ...3-1-405b/nemo-pretraining-gke/values.yaml → ...405b/nemo-gke/nemo2412/recipe/values.yaml
View file
Open in desktop
File renamed without changes.
2 changes: 1 addition & 1 deletion ...ama3-1-70b/nemo-pretraining-gke/README.md → ...a3_70b/nemo-gke/nemo2407/recipe/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ Clone the `gpu-recipes` repository and set a reference to the recipe folder.
git clone https://github.com/ai-hypercomputer/gpu-recipes.git
cd gpu-recipes
export REPO_ROOT=`git rev-parse --show-toplevel`
export RECIPE_ROOT=$REPO_ROOT/training/a3ultra/llama3-1-70b/nemo-pretraining-gke
export RECIPE_ROOT=$REPO_ROOT/training/a3ultra/llama3_70b/nemo-gke/nemo2407/recipe
cd $RECIPE_ROOT
```

Expand Down
0 ...a3-1-70b/nemo-pretraining-gke/values.yaml → ..._70b/nemo-gke/nemo2407/recipe/values.yaml
View file
Open in desktop
File renamed without changes.
2 changes: 1 addition & 1 deletion ...nemo-pretraining-gke-resiliency/README.md → ...-gke-resiliency/nemo2407/recipe/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -157,7 +157,7 @@ From your client, clone the `gpu-recipes` repository and set a reference to the
git clone https://github.com/ai-hypercomputer/gpu-recipes.git
cd gpu-recipes
export REPO_ROOT=`git rev-parse --show-toplevel`
export RECIPE_ROOT=$REPO_ROOT/training/a3ultra/mixtral-8x7b/nemo-pretraining-gke-resiliency
export RECIPE_ROOT=$REPO_ROOT/training/a3ultra/mixtral_8x7b/nemo-gke-resiliency/nemo2407/recipe
```

### Get cluster credentials
Expand Down
2 changes: 1 addition & 1 deletion ...etraining-gke-resiliency/goodput-guide.md → ...siliency/nemo2407/recipe/goodput-guide.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ Achieving high GoodPut can be challenging due to several factors common in large
| **Stragglers and Performance Bottlenecks** | Slower nodes delay the entire job, underutilizing resources. | 3-7% |
| **Lack of Rapid Failure Detection and Diagnosis** | Longer detection/diagnosis time increases downtime. | 2-5% |

This guide provides a general overview of techniques and tools to address these common challenges and maximize ML GoodPut. While the principles discussed are broadly applicable, we will use the [Mixtral 8x7B pretraining recipe](https://github.com/AI-Hypercomputer/gpu-recipes/tree/main/training/a3ultra/mixtral-8x7b/nemo-pretraining-gke-resiliency) as a concrete case study to illustrate how these components can be implemented and customized for large-scale training workloads on Google Cloud. The goal is to showcase a "DIY" style product, where users can understand and selectively adopt these "Lego blocks" to build resilient and efficient training pipelines.
This guide provides a general overview of techniques and tools to address these common challenges and maximize ML GoodPut. While the principles discussed are broadly applicable, we will use the [Mixtral 8x7B pretraining recipe](https://github.com/AI-Hypercomputer/gpu-recipes/tree/main/training/a3ultra/mixtral_8x7b/nemo-gke-resiliency/nemo2407/recipe) as a concrete case study to illustrate how these components can be implemented and customized for large-scale training workloads on Google Cloud. The goal is to showcase a "DIY" style product, where users can understand and selectively adopt these "Lego blocks" to build resilient and efficient training pipelines.

## TLDR: Recommended Lego Blocks for Your Deployment
For customers looking to improve GoodPut on their own ML training workloads, here’s a concise guide to the key strategies discussed in this document, presented as 'Lego blocks' you can implement:
Expand Down
0 ...pretraining-gke-resiliency/ksa-setup.yaml → ...resiliency/nemo2407/recipe/ksa-setup.yaml
View file
Open in desktop
File renamed without changes.
0 ...ing-gke-resiliency/kueue-merge-patch.yaml → ...cy/nemo2407/recipe/kueue-merge-patch.yaml
View file
Open in desktop
File renamed without changes.
0 .../nemo-pretraining-gke-resiliency/train.py → ...o-gke-resiliency/nemo2407/recipe/train.py
View file
Open in desktop
File renamed without changes.
0 ...retraining-gke-resiliency/values-gcs.yaml → ...esiliency/nemo2407/recipe/values-gcs.yaml
View file
Open in desktop
File renamed without changes.
0 ...ing-gke-resiliency/values-supervisor.yaml → ...cy/nemo2407/recipe/values-supervisor.yaml
View file
Open in desktop
File renamed without changes.
0 ...mo-pretraining-gke-resiliency/values.yaml → ...ke-resiliency/nemo2407/recipe/values.yaml
View file
Open in desktop
File renamed without changes.
2 changes: 1 addition & 1 deletion ...xtral-8x7b/nemo-pretraining-gke/README.md → ...l_8x7b/nemo-gke/nemo2407/recipe/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ Clone the `gpu-recipes` repository and set a reference to the recipe folder.
git clone https://github.com/ai-hypercomputer/gpu-recipes.git
cd gpu-recipes
export REPO_ROOT=`git rev-parse --show-toplevel`
export RECIPE_ROOT=$REPO_ROOT/training/a3ultra/mixtral-8x7b/nemo-pretraining-gke
export RECIPE_ROOT=$REPO_ROOT/training/a3ultra/mixtral_8x7b/nemo-gke/nemo2407/recipe
cd $RECIPE_ROOT
```

Expand Down
0 ...ral-8x7b/nemo-pretraining-gke/values.yaml → ...8x7b/nemo-gke/nemo2407/recipe/values.yaml
View file
Open in desktop
File renamed without changes.
0 ...602/16gpus_bf16_1024gbs/recipe/Chart.yaml → ...602/16gpus-bf16-gbs1024/recipe/Chart.yaml
View file
Open in desktop
File renamed without changes.
2 changes: 1 addition & 1 deletion ...2602/16gpus_bf16_1024gbs/recipe/README.md → ...2602/16gpus-bf16-gbs1024/recipe/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ Clone the `gpu-recipes` repository and set a reference to the recipe folder.
git clone https://github.com/ai-hypercomputer/gpu-recipes.git
cd gpu-recipes
export REPO_ROOT=`git rev-parse --show-toplevel`
export RECIPE_ROOT=$REPO_ROOT/training/a3ultra/qwen3-30b-a3b/megatron-bridge-pretraining-gke/2node-BF16-GBSunknown/recipe
export RECIPE_ROOT=$REPO_ROOT/training/a3ultra/qwen3_30b_a3b/nemo-gke/nemo2602/16gpus-bf16-gbs1024/recipe
cd $RECIPE_ROOT
```

Expand Down
0 ...1024gbs/recipe/custom_setup_experiment.py → ...gbs1024/recipe/custom_setup_experiment.py
View file
Open in desktop
File renamed without changes.
0 ...02/16gpus_bf16_1024gbs/recipe/launcher.sh → ...02/16gpus-bf16-gbs1024/recipe/launcher.sh
View file
Open in desktop
File renamed without changes.
0 .../templates/workload-config-configmap.yaml → .../templates/workload-config-configmap.yaml
View file
Open in desktop
File renamed without changes.
0 ...024gbs/recipe/templates/workload-job.yaml → ...bs1024/recipe/templates/workload-job.yaml
View file
Open in desktop
File renamed without changes.
0 ...emplates/workload-launcher-configmap.yaml → ...emplates/workload-launcher-configmap.yaml
View file
Open in desktop
File renamed without changes.
0 ...024gbs/recipe/templates/workload-svc.yaml → ...bs1024/recipe/templates/workload-svc.yaml
View file
Open in desktop
File renamed without changes.
0 ...02/16gpus_bf16_1024gbs/recipe/values.yaml → ...02/16gpus-bf16-gbs1024/recipe/values.yaml
View file
Open in desktop
File renamed without changes.
0 ...02/16gpus_fp8cs_1024gbs/recipe/Chart.yaml → ...02/16gpus-fp8cs-gbs1024/recipe/Chart.yaml
View file
Open in desktop
File renamed without changes.
2 changes: 1 addition & 1 deletion ...602/16gpus_fp8cs_1024gbs/recipe/README.md → ...602/16gpus-fp8cs-gbs1024/recipe/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ Clone the `gpu-recipes` repository and set a reference to the recipe folder.
git clone https://github.com/ai-hypercomputer/gpu-recipes.git
cd gpu-recipes
export REPO_ROOT=`git rev-parse --show-toplevel`
export RECIPE_ROOT=$REPO_ROOT/training/a3ultra/qwen3-30b-a3b/megatron-bridge-pretraining-gke/2node-FP8CS-GBSunknown/recipe
export RECIPE_ROOT=$REPO_ROOT/training/a3ultra/qwen3_30b_a3b/nemo-gke/nemo2602/16gpus-fp8cs-gbs1024/recipe
cd $RECIPE_ROOT
```

Expand Down
0 ...1024gbs/recipe/custom_setup_experiment.py → ...gbs1024/recipe/custom_setup_experiment.py
View file
Open in desktop
File renamed without changes.
0 ...2/16gpus_fp8cs_1024gbs/recipe/launcher.sh → ...2/16gpus-fp8cs-gbs1024/recipe/launcher.sh
View file
Open in desktop
File renamed without changes.
0 .../templates/workload-config-configmap.yaml → .../templates/workload-config-configmap.yaml
View file
Open in desktop
File renamed without changes.
0 ...024gbs/recipe/templates/workload-job.yaml → ...bs1024/recipe/templates/workload-job.yaml
View file
Open in desktop
File renamed without changes.
0 ...emplates/workload-launcher-configmap.yaml → ...emplates/workload-launcher-configmap.yaml
View file
Open in desktop
File renamed without changes.
0 ...024gbs/recipe/templates/workload-svc.yaml → ...bs1024/recipe/templates/workload-svc.yaml
View file
Open in desktop
File renamed without changes.
0 ...2/16gpus_fp8cs_1024gbs/recipe/values.yaml → ...2/16gpus-fp8cs-gbs1024/recipe/values.yaml
View file
Open in desktop
File renamed without changes.
2 changes: 1 addition & 1 deletion ...ma3-8b/megatron-bridge-gke/nemo2602/8gpus-fp8cs-seq8192-gbs128/recipe/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ Clone the `gpu-recipes` repository and set a reference to the recipe folder.
git clone https://github.com/ai-hypercomputer/gpu-recipes.git
cd gpu-recipes
export REPO_ROOT=`git rev-parse --show-toplevel`
export RECIPE_ROOT=$REPO_ROOT/training/a4/llama3-8b/megatron-bridge-pretraining-gke/1node-FP8CS-GBSunknown/recipe
export RECIPE_ROOT=$REPO_ROOT/training/a4/llama3-8b/megatron-bridge-gke/nemo2602/8gpus-fp8cs-seq8192-gbs128/recipe
cd $RECIPE_ROOT
```

Expand Down
0 ...2507/128gpus-fp8-gbs128/recipe/Chart.yaml → ...8gpus-fp8-gbs128/recipe/recipe/Chart.yaml
View file
Open in desktop
File renamed without changes.
0 ...o2507/128gpus-fp8-gbs128/recipe/README.md → ...28gpus-fp8-gbs128/recipe/recipe/README.md
View file
Open in desktop
File renamed without changes.
0 ...507/128gpus-fp8-gbs128/recipe/launcher.sh → ...gpus-fp8-gbs128/recipe/recipe/launcher.sh
View file
Open in desktop
File renamed without changes.
0 ...ma3-1-405b-seq8192-gbs128-mbs1-gpus128.py → ...ma3-1-405b-seq8192-gbs128-mbs1-gpus128.py
View file
Open in desktop
File renamed without changes.
0 .../templates/workload-config-configmap.yaml → .../templates/workload-config-configmap.yaml
View file
Open in desktop
File renamed without changes.
0 ...gbs128/recipe/templates/workload-job.yaml → ...recipe/recipe/templates/workload-job.yaml
View file
Open in desktop
File renamed without changes.
0 ...emplates/workload-launcher-configmap.yaml → ...emplates/workload-launcher-configmap.yaml
View file
Open in desktop
File renamed without changes.
0 ...gbs128/recipe/templates/workload-svc.yaml → ...recipe/recipe/templates/workload-svc.yaml
View file
Open in desktop
File renamed without changes.
0 ...507/128gpus-fp8-gbs128/recipe/values.yaml → ...gpus-fp8-gbs128/recipe/recipe/values.yaml
View file
Open in desktop
File renamed without changes.
0 ...2507/256gpus-fp8-gbs256/recipe/Chart.yaml → ...6gpus-fp8-gbs256/recipe/recipe/Chart.yaml
View file
Open in desktop
File renamed without changes.
0 ...o2507/256gpus-fp8-gbs256/recipe/README.md → ...56gpus-fp8-gbs256/recipe/recipe/README.md
View file
Open in desktop
File renamed without changes.
0 ...507/256gpus-fp8-gbs256/recipe/launcher.sh → ...gpus-fp8-gbs256/recipe/recipe/launcher.sh
View file
Open in desktop
File renamed without changes.
0 ...ma3-1-405b-seq8192-gbs256-mbs1-gpus256.py → ...ma3-1-405b-seq8192-gbs256-mbs1-gpus256.py
View file
Open in desktop
File renamed without changes.
0 .../templates/workload-config-configmap.yaml → .../templates/workload-config-configmap.yaml
View file
Open in desktop
File renamed without changes.
0 ...gbs256/recipe/templates/workload-job.yaml → ...recipe/recipe/templates/workload-job.yaml
View file
Open in desktop
File renamed without changes.
0 ...emplates/workload-launcher-configmap.yaml → ...emplates/workload-launcher-configmap.yaml
View file
Open in desktop
File renamed without changes.
0 ...gbs256/recipe/templates/workload-svc.yaml → ...recipe/recipe/templates/workload-svc.yaml
View file
Open in desktop
File renamed without changes.
0 ...507/256gpus-fp8-gbs256/recipe/values.yaml → ...gpus-fp8-gbs256/recipe/recipe/values.yaml
View file
Open in desktop
File renamed without changes.
0 ...o2507/64gpus-fp8-gbs256/recipe/Chart.yaml → ...4gpus-fp8-gbs256/recipe/recipe/Chart.yaml
View file
Open in desktop
File renamed without changes.
0 ...mo2507/64gpus-fp8-gbs256/recipe/README.md → ...64gpus-fp8-gbs256/recipe/recipe/README.md
View file
Open in desktop
File renamed without changes.
0 ...2507/64gpus-fp8-gbs256/recipe/launcher.sh → ...gpus-fp8-gbs256/recipe/recipe/launcher.sh
View file
Open in desktop
File renamed without changes.
0 ...ma3-1-405b-seq8192-gbs2048-mbs1-gpus64.py → ...ma3-1-405b-seq8192-gbs2048-mbs1-gpus64.py
View file
Open in desktop
File renamed without changes.
0 .../templates/workload-config-configmap.yaml → .../templates/workload-config-configmap.yaml
View file
Open in desktop
File renamed without changes.
0 ...gbs256/recipe/templates/workload-job.yaml → ...recipe/recipe/templates/workload-job.yaml
View file
Open in desktop
File renamed without changes.
0 ...emplates/workload-launcher-configmap.yaml → ...emplates/workload-launcher-configmap.yaml
View file
Open in desktop
File renamed without changes.
0 ...gbs256/recipe/templates/workload-svc.yaml → ...recipe/recipe/templates/workload-svc.yaml
View file
Open in desktop
File renamed without changes.
0 ...2507/64gpus-fp8-gbs256/recipe/values.yaml → ...gpus-fp8-gbs256/recipe/recipe/values.yaml
View file
Open in desktop
File renamed without changes.
2 changes: 1 addition & 1 deletion training/a4/llama31_405b/nemo-gke/nemo2602/256gpus-fp8cs-gbs256/recipe/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ Clone the `gpu-recipes` repository and set a reference to the recipe folder.
git clone https://github.com/ai-hypercomputer/gpu-recipes.git
cd gpu-recipes
export REPO_ROOT=`git rev-parse --show-toplevel`
export RECIPE_ROOT=$REPO_ROOT/training/a4/llama31-405b/megatron-bridge-pretraining-gke/32node-FP8CS-GBSunknown/recipe
export RECIPE_ROOT=$REPO_ROOT/training/a4/llama31_405b/nemo-gke/nemo2602/256gpus-fp8cs-gbs256/recipe
cd $RECIPE_ROOT
```

Expand Down
2 changes: 1 addition & 1 deletion training/a4/llama3_70b/nemo-gke/nemo2507/256gpus-bf16-gbs256/recipe/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ Clone the `gpu-recipes` repository and set a reference to the recipe folder.
git clone https://github.com/ai-hypercomputer/gpu-recipes.git
cd gpu-recipes
export REPO_ROOT=`git rev-parse --show-toplevel`
export RECIPE_ROOT=$REPO_ROOT/training/a4/llama3_70b/nemo-gke/nemo2507/256gpus-bf16-gbs256/recipe/recipe
export RECIPE_ROOT=$REPO_ROOT/training/a4/llama3_70b/nemo-gke/nemo2507/256gpus-bf16-gbs256/recipe
cd $RECIPE_ROOT
```

Expand Down
2 changes: 1 addition & 1 deletion ...xtral-8x7b/nemo-pretraining-gke/README.md → ...l_8x7b/nemo-gke/nemo2507/recipe/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ Clone the `gpu-recipes` repository and set a reference to the recipe folder.
git clone https://github.com/ai-hypercomputer/gpu-recipes.git
cd gpu-recipes
export REPO_ROOT=`git rev-parse --show-toplevel`
export RECIPE_ROOT=$REPO_ROOT/training/a4/mixtral-8x7b/nemo-pretraining-gke
export RECIPE_ROOT=$REPO_ROOT/training/a4/mixtral_8x7b/nemo-gke/nemo2507/recipe
cd $RECIPE_ROOT
```

Expand Down
0 ...ral-8x7b/nemo-pretraining-gke/values.yaml → ...8x7b/nemo-gke/nemo2507/recipe/values.yaml
View file
Open in desktop
File renamed without changes.
2 changes: 1 addition & 1 deletion .../a4/qwen3_30b_a3b/nemo-gke/nemo2602/8gpus-fp8mx-seq4096-gbs512/recipe/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ Clone the `gpu-recipes` repository and set a reference to the recipe folder.
git clone https://github.com/ai-hypercomputer/gpu-recipes.git
cd gpu-recipes
export REPO_ROOT=`git rev-parse --show-toplevel`
export RECIPE_ROOT=$REPO_ROOT/training/a4/qwen3-30b-a3b/megatron-bridge-pretraining-gke/1node-FP8MX-GBSunknown/recipe
export RECIPE_ROOT=$REPO_ROOT/training/a4/qwen3_30b_a3b/nemo-gke/nemo2602/8gpus-fp8mx-seq4096-gbs512/recipe
cd $RECIPE_ROOT
```

Expand Down
2 changes: 1 addition & 1 deletion training/a4x/llama3_70b/nemo-gke/nemo2602/64gpus-bf16-gbs256/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ Clone the `gpu-recipes` repository and set a reference to the recipe folder.
git clone https://github.com/ai-hypercomputer/gpu-recipes.git
cd gpu-recipes
export REPO_ROOT=`git rev-parse --show-toplevel`
export RECIPE_ROOT=$REPO_ROOT/training/a4x/llama3-70b/megatron-bridge-pretraining-gke/16node-FP8CS-GBSunknown/recipe
export RECIPE_ROOT=$REPO_ROOT/training/a4x/llama3_70b/nemo-gke/nemo2602/64gpus-bf16-gbs256
cd $RECIPE_ROOT
```

Expand Down
2 changes: 1 addition & 1 deletion training/a4x/llama3_70b/nemo-gke/nemo2602/64gpus-fp8cs-gbs256/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ Clone the `gpu-recipes` repository and set a reference to the recipe folder.
git clone https://github.com/ai-hypercomputer/gpu-recipes.git
cd gpu-recipes
export REPO_ROOT=`git rev-parse --show-toplevel`
export RECIPE_ROOT=$REPO_ROOT/training/a4x/llama3-70b/megatron-bridge-pretraining-gke/16node-FP8CS-GBSunknown/recipe
export RECIPE_ROOT=$REPO_ROOT/training/a4x/llama3_70b/nemo-gke/nemo2602/64gpus-fp8cs-gbs256
cd $RECIPE_ROOT
```

Expand Down
2 changes: 1 addition & 1 deletion training/a4x/llama3_70b/nemo-gke/nemo2602/64gpus-fp8mx-gbs256/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ Clone the `gpu-recipes` repository and set a reference to the recipe folder.
git clone https://github.com/ai-hypercomputer/gpu-recipes.git
cd gpu-recipes
export REPO_ROOT=`git rev-parse --show-toplevel`
export RECIPE_ROOT=$REPO_ROOT/training/a4x/llama3-70b/megatron-bridge-pretraining-gke/16node-FP8MX-GBSunknown/recipe
export RECIPE_ROOT=$REPO_ROOT/training/a4x/llama3_70b/nemo-gke/nemo2602/64gpus-fp8mx-gbs256
cd $RECIPE_ROOT
```

Expand Down
0 .../nemo-finetuning-gke/4gpu-bf16/Chart.yaml → ...mo2507/4gpus-bf16-gbs32/recipe/Chart.yaml
View file
Open in desktop
File renamed without changes.
2 changes: 1 addition & 1 deletion ...b/nemo-finetuning-gke/4gpu-bf16/README.md → ...emo2507/4gpus-bf16-gbs32/recipe/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ Clone the `gpu-recipes` repository and set a reference to the recipe folder.
git clone https://github.com/ai-hypercomputer/gpu-recipes.git
cd gpu-recipes
export REPO_ROOT=`git rev-parse --show-toplevel`
export RECIPE_ROOT=$REPO_ROOT/training/g4/llama3-1-70b/nemo-pretraining-gke/4gpu-bf16
export RECIPE_ROOT=$REPO_ROOT/training/g4/llama3_70b/nemo-finetuning-gke/nemo2507/4gpus-bf16-gbs32/recipe
cd $RECIPE_ROOT
```

Expand Down
0 ...nemo-finetuning-gke/4gpu-bf16/launcher.sh → ...o2507/4gpus-bf16-gbs32/recipe/launcher.sh
View file
Open in desktop
File renamed without changes.
0 ...gke/4gpu-bf16/llama3-1-70b-fine-tuning.py → ...-gbs32/recipe/llama3-1-70b-fine-tuning.py
View file
Open in desktop
File renamed without changes.
0 .../templates/workload-config-configmap.yaml → .../templates/workload-config-configmap.yaml
View file
Open in desktop
File renamed without changes.
0 ...gke/4gpu-bf16/templates/workload-job.yaml → ...-gbs32/recipe/templates/workload-job.yaml
View file
Open in desktop
File renamed without changes.
0 ...emplates/workload-launcher-configmap.yaml → ...emplates/workload-launcher-configmap.yaml
View file
Open in desktop
File renamed without changes.
0 ...gke/4gpu-bf16/templates/workload-svc.yaml → ...-gbs32/recipe/templates/workload-svc.yaml
View file
Open in desktop
File renamed without changes.
0 ...nemo-finetuning-gke/4gpu-bf16/values.yaml → ...o2507/4gpus-bf16-gbs32/recipe/values.yaml
View file
Open in desktop
File renamed without changes.
0 .../nemo-finetuning-gke/8gpu-bf16/Chart.yaml → ...mo2507/8gpus-bf16-gbs32/recipe/Chart.yaml
View file
Open in desktop
File renamed without changes.
2 changes: 1 addition & 1 deletion ...b/nemo-finetuning-gke/8gpu-bf16/README.md → ...emo2507/8gpus-bf16-gbs32/recipe/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ Clone the `gpu-recipes` repository and set a reference to the recipe folder.
git clone https://github.com/ai-hypercomputer/gpu-recipes.git
cd gpu-recipes
export REPO_ROOT=`git rev-parse --show-toplevel`
export RECIPE_ROOT=$REPO_ROOT/training/g4/llama3-1-70b/nemo-finetuning-gke/8gpu-bf16
export RECIPE_ROOT=$REPO_ROOT/training/g4/llama3_70b/nemo-finetuning-gke/nemo2507/8gpus-bf16-gbs32/recipe
cd $RECIPE_ROOT
```

Expand Down
0 ...nemo-finetuning-gke/8gpu-bf16/launcher.sh → ...o2507/8gpus-bf16-gbs32/recipe/launcher.sh
View file
Open in desktop
File renamed without changes.
0 ...gke/8gpu-bf16/llama3-1-70b-fine-tuning.py → ...-gbs32/recipe/llama3-1-70b-fine-tuning.py
View file
Open in desktop
File renamed without changes.
0 .../templates/workload-config-configmap.yaml → .../templates/workload-config-configmap.yaml
View file
Open in desktop
File renamed without changes.
0 ...gke/8gpu-bf16/templates/workload-job.yaml → ...-gbs32/recipe/templates/workload-job.yaml
View file
Open in desktop
File renamed without changes.
0 ...emplates/workload-launcher-configmap.yaml → ...emplates/workload-launcher-configmap.yaml
View file
Open in desktop
File renamed without changes.
0 ...gke/8gpu-bf16/templates/workload-svc.yaml → ...-gbs32/recipe/templates/workload-svc.yaml
View file
Open in desktop
File renamed without changes.
0 ...nemo-finetuning-gke/8gpu-bf16/values.yaml → ...o2507/8gpus-bf16-gbs32/recipe/values.yaml
View file
Open in desktop
File renamed without changes.