Triton Inference Server
Deploy AI models across frameworks with dynamic batching, real-time support, and cloud integration.
NVIDIA Dynamo-Triton, previously known as Triton Inference Server, facilitates the deployment of AI models across major frameworks like TensorRT, PyTorch, and ONNX. Experience high performance with features like dynamic batching, concurrent execution, and optimized configurations. It supports diverse workloads, including real-time and batched operations, and runs on NVIDIA GPUs, non-NVIDIA accelerators, x86, and ARM CPUs.
Open-source and DevOps-friendly, Dynamo-Triton integrates with Kubernetes for scaling and Prometheus for monitoring, making it ideal for both cloud and on-premises AI platforms. It offers a secure, production-ready environment with stable APIs for AI deployment.
For large language model (LLM) use cases, NVIDIA Dynamo complements Dynamo-Triton with LLM-specific optimizations, enhancing inference performance. Access resources like self-paced training, quick-start guides, and tutorials to get started. Explore the potential of AI deployment with NVIDIA Dynamo-Triton today.
Similar to Triton Inference Server
