Modal Review 2026

Together AI

200+ open-source models via one API, from fine-tuning to raw GPU clusters

Together AI hosts over 200 open-source models — Llama, DeepSeek, Qwen, Mistral, Kimi, Gemma and more — behind a single OpenAI-compatible endpoint, letting teams swap between models without rewriting application code, while also offering LoRA and full fine-tuning infrastructure and direct GPU cluster access (H100, H200, Blackwell B200) for teams that outgrow serverless token pricing. That breadth — model variety plus fine-tuning depth plus raw GPU access under one roof — is its core differentiator from narrower competitors. Token pricing ranges from $0.10 to $9 per million tokens depending on model size and type; on Llama 3.3 70B specifically, Together sits mid-pack at $1.04/M input and output — pricier than Groq's $0.59/$0.79 or Fireworks's flat $0.90, but with meaningfully broader model selection and fine-tuning capability neither directly matches. Against proprietary frontier models, the gap is substantial: Llama 3.3 70B's $0.88/$0.88 output rate runs roughly 4x cheaper than comparable proprietary model output, which at genuine scale (50 million output tokens/month) works out to hundreds of thousands of dollars annualized in savings. For direct GPU access rather than per-token billing, dedicated inference runs $6.49/hour, on-demand clusters $5.49/hour, and longer 91-180-day reserved clusters $3.99/hour — all more expensive per raw hour than Modal's roughly $3.95/hour base H100 rate, reflecting that Together's rates include the managed hosting and orchestration layer Modal leaves to you. Realistically, most production AI teams in 2026 run multi-provider architectures rather than picking one: Together AI for bulk open-model inference, a frontier provider like OpenAI or Anthropic for complex reasoning, and Modal for anything requiring custom weights or isolated execution — Together's breadth makes it a strong default for the open-model-hosting piece of that stack specifically.

Groq

Trending

The fastest LLM inference on custom LPU chips — now backed by a $20B NVIDIA deal

Groq built custom LPU (Language Processing Unit) hardware specifically optimized for LLM inference speed, and it shows in the numbers: Llama 3.3 70B runs at roughly 394 tokens/second on Groq's chips, with smaller models like Llama 3.1 8B hitting around 840 tokens/second — speeds that make it the go-to choice for latency-sensitive applications like real-time voice agents or interactive chat where every millisecond of response time is felt by the user. The commonly cited "10-20x cheaper than OpenAI" figure is roughly accurate but needs a caveat: that's comparing Groq's open-model pricing against OpenAI's proprietary models, not identical capability tiers. A major corporate development reshapes how to think about Groq's future: NVIDIA announced a $20 billion deal for Groq in the period following Christmas 2025, though as of mid-2026 the GroqCloud platform remains live and its public pricing page unchanged. Groq pricing starts at $0.05/M input tokens for Llama 3.1 8B Instant, scaling up to $0.90/M tokens for larger vision models — and in direct model-for-model comparisons against Together AI, Groq wins on price for 6 of 11 shared models, while Together wins on 3 and 2 tie, reflecting genuine competitive parity rather than one clearly cheaper provider. The honest limitation: Groq serves a fixed catalog of models (Llama, GPT-OSS, Qwen, Kimi K2, DeepSeek and others) optimized specifically for its LPU hardware — it doesn't host GPT-5, Claude or Gemini, so apps depending on those specific proprietary models need to stay with their original provider or route through an aggregator like OpenRouter. For latency-critical applications running a model Groq's catalog covers, its speed advantage is genuinely difficult for GPU-based competitors to match; for anything requiring proprietary frontier models or custom fine-tuned weights, Groq isn't the right tool.

Replicate

Run any open-source AI model via a simple API, billed per second

Replicate's core value proposition is removing infrastructure complexity from running open-source AI models: browse a library of thousands of community and proprietary models (Stable Diffusion, Flux, Llama and many others), call one via a simple API, and pay only for the seconds your prediction actually runs — no idle GPU charges, no server management. Pricing varies by hardware: CPU-based models cost fractions of a cent per second, while GPU models range from $0.000225/second (T4) up to $0.003525/second (A100 80GB), so cost scales directly with the compute a given model actually needs. Developers can also package and deploy their own custom models using Replicate's Cog containerization format — define setup and prediction functions in a Python file, build a Cog container, and push it live, typically taking 30-60 minutes for a first deployment. That combination of a huge pre-built model catalog plus straightforward custom deployment is what differentiates it from a pure inference API like Together AI or Groq, which host a curated model list rather than accepting arbitrary custom weights. Against its closest comparison, Hugging Face, the trade-off is fairly clean: Replicate offers a simpler, cleaner API experience with especially strong image-generation ergonomics (optimized cold starts, clean output handling for models like Stable Diffusion and Flux), while Hugging Face wins on sheer model breadth (800,000+ models) and community size. For teams that want the fastest path to calling an open-source model via API without managing Docker or GPU infrastructure themselves, Replicate remains a strong, low-friction default.

Hugging Face

Trending

The largest open-source AI hub — 800,000+ models, free to browse

Hugging Face is the closest thing the AI industry has to GitHub: a hub hosting over 800,000 open-source models, 100,000+ datasets, and Spaces (hosted demo apps), all free to browse, download and use, with a thriving community driving much of its content. Beyond the hub itself, its Inference offerings split into three distinct products, a naming overlap that trips up newcomers: the Serverless Inference API (free tier with rate limits, best for prototyping), Inference Endpoints (dedicated GPU instances you spin up per model, starting around $0.50-0.60/hour with scale-to-zero), and Inference Providers (a unified OpenAI-compatible gateway routing to 15+ third-party providers including Groq, Together AI, Fireworks and Replicate). Inference Providers is the more recent addition, maturing through 2025-2026 as Hugging Face's answer to "which inference partner should I use" — rather than picking one provider directly, you call Hugging Face's single endpoint and it routes to whichever underlying provider hosts the model, with pricing passed through at each provider's own published rate. Free users get a small monthly credit allocation ($0.10) toward this routed inference, while PRO subscribers ($9/month) get $2.00 in monthly credits plus faster rate limits and private Spaces — a meaningful upgrade for anyone doing more than light experimentation. For teams choosing between Hugging Face and Replicate specifically: Hugging Face wins decisively on sheer model breadth and community size, while Replicate offers a somewhat simpler, more image-generation-optimized API experience. For teams choosing between Hugging Face and LangChain: they're not really competitors — Hugging Face is model hosting and inference, LangChain is an application-building framework — and many production stacks genuinely use both together.

OpenAI API

Trending

Build with GPT-5.6 — pricing that shifts with a genuine AI price war

OpenAI's API gives developers programmatic access to the GPT model family for building chatbots, agents, content generation and any application needing language understanding, billed per token rather than a flat subscription. The current flagship lineup, GPT-5.6, reached general availability on July 9, 2026 across three tiers — Sol (frontier reasoning and complex agent tasks), Terra (balanced mid-tier) and Luna (high-volume, budget) — each sharing a 1.05-million-token context window. Pricing here moves fast enough that any figure needs a date attached: on July 30, 2026, OpenAI cut Terra's price by 20% and Luna's by a striking 80%, the largest single price move since GPT-5 launched, in what multiple industry analysts are now openly calling a price war among frontier AI labs. As of that cut, Luna runs $0.20/M input and $1.20/M output tokens — roughly 4x cheaper than the previous-generation GPT-5.4 Mini despite sitting in the current flagship family — while Sol remains at $5/M input, $30/M output for the hardest reasoning and agentic workloads. Cached input reads bill at just 10% of standard rates, and the Batch API halves both input and output costs for asynchronous jobs completed within 24 hours. The practical implication for anyone budgeting an OpenAI-powered feature: tier choice, not just vendor choice, is now the single biggest lever on your bill, and prices that were accurate even a month or two ago may already be stale. For simple classification, summarization or routing tasks, Luna or GPT-5 Nano deliver adequate results at a small fraction of Sol's cost; reserve the expensive frontier tier specifically for genuinely difficult reasoning, coding or multi-step agent work where the capability gap actually matters.

Anthropic API

Trending

Build with Claude — Haiku, Sonnet and Opus at per-token rates

The Anthropic API gives developers programmatic access to the Claude model family — Haiku (fast, low-cost), Sonnet (balanced), and Opus (most capable) — for building applications ranging from customer support bots to autonomous coding agents, billed per million tokens with output priced roughly 5x higher than input across the lineup. As of mid-2026, current rates run Haiku 4.5 at $1/$5 per million input/output tokens, Sonnet 4.6 at $3/$15, and Opus 4.8 at $5/$25 — with all three supporting a full 1-million-token context window at standard rates and no long-context surcharge, a meaningful simplification for teams that previously had to budget for context-length premiums. A genuinely significant pricing shift happened with Opus: the current $5/$25 rate represents roughly a 3x cut from the older Opus 3 generation's $15/$75, making Anthropic's most capable model dramatically more accessible than it was a year or two earlier. Prompt caching cuts cached-input costs by 90% (Sonnet 4.6 cached reads drop to $0.30/M from $3.00/M), and Batch processing offers a flat 50% discount across every model for asynchronous, non-urgent workloads — stacking both optimizations is where most production teams find the biggest savings. There's no ongoing free API tier — new accounts receive a modest trial credit ($5 in some documented cases) to test integration before committing to paid usage, separate entirely from Claude.ai's consumer subscription plans, which don't include API access. For teams choosing between models, the practical guidance holds steady: route routine, latency-sensitive work (extraction, classification, short replies) to Haiku, reserve Opus for genuinely complex reasoning or high-stakes tasks, and lean on caching aggressively wherever system prompts or tool definitions repeat across requests.

LangChain

The free, open-source framework for LLM apps — LangSmith observability costs extra

LangChain is a completely free, open-source Python and JavaScript framework for building LLM-powered applications — chains, agents, retrieval-augmented generation (RAG), and memory management, all through modular, composable components that work across virtually any model provider (OpenAI, Anthropic, Hugging Face and others) via a unified interface. This is worth stating plainly because pricing confusion around LangChain is common: the core framework itself has never charged anything and never will, since it's genuinely open source with community contributions. What does cost money is LangSmith, Anthropic's — sorry, LangChain's — separate observability and debugging platform for tracing, monitoring and evaluating LLM application behavior in production: free for up to 5,000 traces/month, Plus at $39/month for higher volume and team features, and custom Enterprise pricing above that. LangGraph, a related framework for building more structured, stateful multi-agent workflows, is also open source and free, distinct from both the core LangChain library and LangSmith. The practical cost of running a LangChain-based application, then, is really the sum of your underlying LLM API costs (OpenAI, Anthropic, etc.) plus, optionally, LangSmith if you want production observability — LangChain itself contributes zero licensing cost either way. It remains one of the most widely adopted frameworks for prototyping and building LLM applications specifically because of that combination: genuinely free tooling, broad multi-provider support, and an extensive integration ecosystem, though developers still handle their own deployment, infrastructure and scaling rather than getting a managed, turnkey hosting layer.

Ollama

Featured

Run open LLMs locally with one command

Ollama makes local language models genuinely easy. A single command pulls a quantised model — Llama, Mistral, Gemma, Qwen, DeepSeek and hundreds more — and starts serving it with an OpenAI-compatible HTTP endpoint on your own machine. That compatibility is the reason it spread so quickly: existing code written against the OpenAI SDK usually works by changing the base URL, so developers can prototype against a local model at zero marginal cost, then swap to a hosted provider for production. Modelfiles let you bake a system prompt, parameters and adapters into a reusable named model. Because nothing leaves the device, Ollama is also the default answer for regulated work, offline environments and anyone experimenting with fine-tunes on consumer hardware.

OpenRouter

Featured

One API and one bill for hundreds of AI models

OpenRouter is a unified gateway to hundreds of large language models from dozens of providers, exposed through a single OpenAI-compatible API. Switch from one model to another by changing a string, with no new SDK, contract or billing relationship. For teams it solves three practical problems: price comparison across providers, automatic failover when an upstream provider goes down, and consolidated spend tracking. Routing rules let you prefer the cheapest or fastest provider for a given model, and per-key limits keep experiments from burning budget. OpenRouter also publishes live rankings of model usage and pricing, which has made it a de facto benchmark of what developers are actually shipping with.

OpenAI API

Trending

Build with GPT-5.6 — pricing that shifts with a genuine AI price war

OpenAI's API gives developers programmatic access to the GPT model family for building chatbots, agents, content generation and any application needing language understanding, billed per token rather than a flat subscription. The current flagship lineup, GPT-5.6, reached general availability on July 9, 2026 across three tiers — Sol (frontier reasoning and complex agent tasks), Terra (balanced mid-tier) and Luna (high-volume, budget) — each sharing a 1.05-million-token context window. Pricing here moves fast enough that any figure needs a date attached: on July 30, 2026, OpenAI cut Terra's price by 20% and Luna's by a striking 80%, the largest single price move since GPT-5 launched, in what multiple industry analysts are now openly calling a price war among frontier AI labs. As of that cut, Luna runs $0.20/M input and $1.20/M output tokens — roughly 4x cheaper than the previous-generation GPT-5.4 Mini despite sitting in the current flagship family — while Sol remains at $5/M input, $30/M output for the hardest reasoning and agentic workloads. Cached input reads bill at just 10% of standard rates, and the Batch API halves both input and output costs for asynchronous jobs completed within 24 hours. The practical implication for anyone budgeting an OpenAI-powered feature: tier choice, not just vendor choice, is now the single biggest lever on your bill, and prices that were accurate even a month or two ago may already be stale. For simple classification, summarization or routing tasks, Luna or GPT-5 Nano deliver adequate results at a small fraction of Sol's cost; reserve the expensive frontier tier specifically for genuinely difficult reasoning, coding or multi-step agent work where the capability gap actually matters.

Anthropic API

Trending

Build with Claude — Haiku, Sonnet and Opus at per-token rates

The Anthropic API gives developers programmatic access to the Claude model family — Haiku (fast, low-cost), Sonnet (balanced), and Opus (most capable) — for building applications ranging from customer support bots to autonomous coding agents, billed per million tokens with output priced roughly 5x higher than input across the lineup. As of mid-2026, current rates run Haiku 4.5 at $1/$5 per million input/output tokens, Sonnet 4.6 at $3/$15, and Opus 4.8 at $5/$25 — with all three supporting a full 1-million-token context window at standard rates and no long-context surcharge, a meaningful simplification for teams that previously had to budget for context-length premiums. A genuinely significant pricing shift happened with Opus: the current $5/$25 rate represents roughly a 3x cut from the older Opus 3 generation's $15/$75, making Anthropic's most capable model dramatically more accessible than it was a year or two earlier. Prompt caching cuts cached-input costs by 90% (Sonnet 4.6 cached reads drop to $0.30/M from $3.00/M), and Batch processing offers a flat 50% discount across every model for asynchronous, non-urgent workloads — stacking both optimizations is where most production teams find the biggest savings. There's no ongoing free API tier — new accounts receive a modest trial credit ($5 in some documented cases) to test integration before committing to paid usage, separate entirely from Claude.ai's consumer subscription plans, which don't include API access. For teams choosing between models, the practical guidance holds steady: route routine, latency-sensitive work (extraction, classification, short replies) to Haiku, reserve Opus for genuinely complex reasoning or high-stakes tasks, and lean on caching aggressively wherever system prompts or tool definitions repeat across requests.

Hugging Face

Trending

The largest open-source AI hub — 800,000+ models, free to browse

Hugging Face is the closest thing the AI industry has to GitHub: a hub hosting over 800,000 open-source models, 100,000+ datasets, and Spaces (hosted demo apps), all free to browse, download and use, with a thriving community driving much of its content. Beyond the hub itself, its Inference offerings split into three distinct products, a naming overlap that trips up newcomers: the Serverless Inference API (free tier with rate limits, best for prototyping), Inference Endpoints (dedicated GPU instances you spin up per model, starting around $0.50-0.60/hour with scale-to-zero), and Inference Providers (a unified OpenAI-compatible gateway routing to 15+ third-party providers including Groq, Together AI, Fireworks and Replicate). Inference Providers is the more recent addition, maturing through 2025-2026 as Hugging Face's answer to "which inference partner should I use" — rather than picking one provider directly, you call Hugging Face's single endpoint and it routes to whichever underlying provider hosts the model, with pricing passed through at each provider's own published rate. Free users get a small monthly credit allocation ($0.10) toward this routed inference, while PRO subscribers ($9/month) get $2.00 in monthly credits plus faster rate limits and private Spaces — a meaningful upgrade for anyone doing more than light experimentation. For teams choosing between Hugging Face and Replicate specifically: Hugging Face wins decisively on sheer model breadth and community size, while Replicate offers a somewhat simpler, more image-generation-optimized API experience. For teams choosing between Hugging Face and LangChain: they're not really competitors — Hugging Face is model hosting and inference, LangChain is an application-building framework — and many production stacks genuinely use both together.

Groq

Trending

The fastest LLM inference on custom LPU chips — now backed by a $20B NVIDIA deal

Groq built custom LPU (Language Processing Unit) hardware specifically optimized for LLM inference speed, and it shows in the numbers: Llama 3.3 70B runs at roughly 394 tokens/second on Groq's chips, with smaller models like Llama 3.1 8B hitting around 840 tokens/second — speeds that make it the go-to choice for latency-sensitive applications like real-time voice agents or interactive chat where every millisecond of response time is felt by the user. The commonly cited "10-20x cheaper than OpenAI" figure is roughly accurate but needs a caveat: that's comparing Groq's open-model pricing against OpenAI's proprietary models, not identical capability tiers. A major corporate development reshapes how to think about Groq's future: NVIDIA announced a $20 billion deal for Groq in the period following Christmas 2025, though as of mid-2026 the GroqCloud platform remains live and its public pricing page unchanged. Groq pricing starts at $0.05/M input tokens for Llama 3.1 8B Instant, scaling up to $0.90/M tokens for larger vision models — and in direct model-for-model comparisons against Together AI, Groq wins on price for 6 of 11 shared models, while Together wins on 3 and 2 tie, reflecting genuine competitive parity rather than one clearly cheaper provider. The honest limitation: Groq serves a fixed catalog of models (Llama, GPT-OSS, Qwen, Kimi K2, DeepSeek and others) optimized specifically for its LPU hardware — it doesn't host GPT-5, Claude or Gemini, so apps depending on those specific proprietary models need to stay with their original provider or route through an aggregator like OpenRouter. For latency-critical applications running a model Groq's catalog covers, its speed advantage is genuinely difficult for GPU-based competitors to match; for anything requiring proprietary frontier models or custom fine-tuned weights, Groq isn't the right tool.