Tools tagged with "Inference"

Customize and deploy AI assistants and agents with open models, ensuring privacy and control.

Boost AI coding efficiency with local context processing. No cloud, no data leaves your device, 70% token reduction.

Create efficient AI agents using a lightweight, frameworkless approach. Supports ReAct and CodeAct paradigms for versatile applications.

Explore Qwen3's powerful language models for deep reasoning and multilingual capabilities.

MLC LLM is a universal solution that enables native deployment of any large language models with native APIs with compiler acceleration.

LMDeploy is a toolkit for compressing, deploying, and serving LLMs. It provides efficient inference with TurboMind engine.

Transcribe audio in seconds with a powerful CLI tool using Whisper and Flash Attention.

Boost ML model serving with dynamic batching and CPU/GPU pipelines for maximum efficiency.

Deliver low-latency, high-throughput inference for AI models. Supports diverse hardware and model types with extensive community backing.

Cross-platform tensor library with no dependencies, supporting quantization and diverse hardware.

Fast inference library for local LLMs on consumer GPUs. Supports dynamic batching, smart caching, and more.

Enhance LLM performance with Python API and NVIDIA GPU optimizations for efficient inference.

Deploy AI models across frameworks with dynamic batching, real-time support, and cloud integration.

Run AI models like GPT-OSS and Llama privately on your computer. Free for home and work.

A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp, and many other backends.

Experience rapid, efficient transcription using CTranslate2 for faster results and reduced memory usage.

Achieve high-performance LLM inference with minimal setup using C/C++. Supports diverse hardware and quantization for optimal efficiency.