Anthropic has introduced its Message Batches API, designed to provide developers with a cost-efficient method for processing large volumes of queries directed at Claude asynchronously.

With this new API, developers can send batches containing up to 10,000 queries, which are processed within 24 hours, all at a 50% reduction in cost compared to traditional API calls. This advancement allows for more efficient management of non-urgent tasks.

Currently available in public beta, the API supports Claude 3.5 Sonnet, Claude 3 Opus, and Claude 3 Haiku on the Anthropic API. Users of Claude via Amazon Bedrock can already take advantage of batch inference, with plans to extend support to Google Cloud’s Vertex AI in the near future.

The demand for high-throughput processing is crucial for developers employing Claude for applications such as customer feedback analysis and language translation, where immediate responses are less critical.

Instead of dealing with complicated queuing systems or rate limits, the API enables developers to submit batches of up to 10,000 queries simultaneously, facilitating seamless processing at a reduced cost. Notably, batches are often processed well before the 24-hour limit.

This API not only improves throughput and increases rate limits but also allows for the handling of large volumes of requests without compromising standard API usage. Additionally, it provides scalability for extensive data processing tasks, such as dataset analysis and model evaluations, without the need for complex infrastructure.

The introduction of the Message Batches API opens doors to large-scale data processing that was previously either impractical or too expensive. By taking advantage of the batching discount, businesses can feasibly analyze vast repositories of corporate documents, potentially spanning millions of files.

Pricing Structure for the Message Batches API

The Message Batches API enables users to enjoy infrastructure cost savings, offering a 50% discount on both input and output tokens. Here’s a breakdown of the pricing for each version:

• Claude 3.5 Sonnet
  • Input: $1.50 per 1,000 tokens
  • Output: $7.50 per 1,000 tokens
• Claude 3 Opus
  • Input: $7.50 per 1,000 tokens
  • Output: $37.50 per 1,000 tokens
• Claude 3 Haiku
  • Input: $0.125 per 1,000 tokens
  • Output: $0.625 per 1,000 tokens

Quora, a user-driven Q&A platform, is leveraging Anthropic’s Batches API for tasks like summarization and highlight extraction, improving features for its users.

Andy Edmonds, Product Manager at Quora, remarked: “Anthropic’s Batches API not only saves costs but also simplifies the execution of numerous queries that don’t require real-time processing.

“It’s incredibly convenient to submit a batch and receive results within 24 hours, rather than managing multiple parallel queries. This allows our engineers to focus on more complex challenges.”

Developers interested in utilizing the Message Batches API during its public beta can find detailed documentation available online.