huggingface/accelerate
🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support
https://skillcdn.ai/gh/huggingface/accelerateConnect this address to your AI to use these skills. How to connect
- Unverified
- Default branch
main - Commit
01c73fb - License
Apache-2.0
Explore this repository
Searches the original text of skills and documents; display translations are not searched.
Documents
- AcceleratorThe [Accelerator] is the main class for enabling distributed training on any type of training setup. Read the Add Accelerator to your code tutorial to learn more about how to add the [Accelerator] to…docs/source/package_reference/accelerator.md
- Working with large models[[autodoc]] big_modeling.init_empty_weightsdocs/source/package_reference/big_modeling.md
- The Command LineBelow is a list of all the available commands 🤗 Accelerate with their parametersdocs/source/package_reference/cli.md
- DeepSpeed utilities[[autodoc]] utils.get_active_deepspeed_plugindocs/source/package_reference/deepspeed.md
- FP8Below are functions and classes relative to the underlying FP8 implementationdocs/source/package_reference/fp8.md
- Fully Sharded Data Parallel utilities[[autodoc]] utils.enable_fsdp_ram_efficient_loadingdocs/source/package_reference/fsdp.md
- Pipeline parallelismAccelerate supports pipeline parallelism for large-scale training with the PyTorch torch.distributed.pipelining API.docs/source/package_reference/inference.md
- Kwargs handlersThe following objects can be passed to the main [Accelerator] to customize how some PyTorch objects related to distributed training or mixed precision are created.docs/source/package_reference/kwargs.md
- LaunchersFunctions for launching training on distributed processes.docs/source/package_reference/launchers.md
- LoggingRefer to the Troubleshooting guide or to the example below to learn how to use Accelerate's logger.docs/source/package_reference/logging.md
- Megatron-LM utilities[[autodoc]] utils.MegatronLMPlugindocs/source/package_reference/megatron_lm.md
- Stateful ClassesBelow are variations of a singleton class in the sense that all instances share the same state, which is initialized on the first instantiation.docs/source/package_reference/state.md
- DataLoaders, Optimizers, and SchedulersThe internal classes Accelerate uses to prepare objects for distributed training when calling [~Accelerator.prepare].docs/source/package_reference/torch_wrappers.md
- Experiment Trackers[[autodoc]] tracking.GeneralTrackerdocs/source/package_reference/tracking.md
- Utility functions and classesBelow are a variety of utility functions that 🤗 Accelerate provides, broken down by use-case.docs/source/package_reference/utilities.md