Document Float8DynamicActivationFloat8WeightConfig as inference-only

[Ashutosh0x]

Summary

Users attempting LoRA training on fp8-quantized bases hit an opaque RuntimeError: derivative for aten::_scaled_mm is not implemented with no guidance. The docstring did not mention that this config only supports inference.

Added a .. note:: block making the inference-only limitation explicit and pointing users to convert_to_float8_training for the training path.

Before

Users discover the limitation only at runtime via a cryptic error.

After

The docstring clearly states:

This config is inference-only. Calling loss.backward() on the output of a model quantized with this config will raise RuntimeError: derivative for aten::_scaled_mm is not implemented. For float8 training, use torchao.float8.convert_to_float8_training instead.

Addresses #4376

Test Plan

Documentation-only change. Verified RST note syntax renders correctly by inspection.

This change was authored with the assistance of an AI coding assistant.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/4495

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• [ROCm] MI350 CI jobs will have longer queue times due to CI migration

❌ 1 New Failure, 1 Pending, 2 Unrelated Failures

As of commit ff9d86a with merge base 5647df9 ():

NEW FAILURE - The following job has failed:

• PR Label Check / Check PR Labels (gh)
  Process completed with exit code 1.

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Run Regression Tests / test (CPU 2.12, linux.4xlarge, torch==2.12.0 torchvision==0.27.0 --index-url https://download.pyt... / linux-job (gh) (trunk failure)
  test/quantization/pt2e/test_x86inductor_fusion.py::DynamicShapesCppWrapperCpuTests::test_qconv2d_maxpool2d_linear_dynamic_cpu_dynamic_shapes_cpp_wrapper
• Run Regression Tests / test (CUDA 2.12, linux.g5.12xlarge.nvidia.gpu, torch==2.12.0 torchvision==0.27.0 --index-url http... / linux-job (gh) (trunk failure)
  test/quantization/pt2e/test_x86inductor_fusion.py::DynamicShapesCppWrapperCpuTests::test_qconv2d_maxpool2d_linear_dynamic_cpu_dynamic_shapes_cpp_wrapper

This comment was automatically generated by Dr. CI and updates every 15 minutes.