As more companies utilize large language models, demanding access to GPUs, the most popular choice, especially Nvidia GPUs, is both costly and often in limited supply. Renting a long-term instance from a cloud provider when you only require these expensive resources for a single job might not be practical.
To address this issue, AWS has introduced Amazon Elastic Compute Cloud (EC2) Capacity Blocks for ML. This service enables customers to purchase GPU access for a specified duration, typically for AI-related tasks such as training machine learning models or conducting experiments with existing models.
Channy Yun introduced this innovative approach for scheduling GPU instances, allowing you to reserve the required number of instances for a specific future date and only for the necessary duration,” as mentioned in the blog post.
Flexible GPU Reservation System for Efficient AI Computing
This product provides customers with access to NVIDIA H100 Tensor Core GPU instances, available in cluster sizes ranging from one to 64 instances, each with 8 GPUs. Users can reserve time for up to 14 days in 1-day increments, up to 8 weeks in advance. Once the reserved timeframe concludes, the instances will automatically shut down.
The new product allows users to book a specific number of instances for a set duration, similar to reserving a hotel room for a certain number of days, as described by the company. From the customer’s perspective, this means they will have a clear understanding of the job’s duration, the number of GPUs they’ll utilize, and the upfront cost, providing cost predictability.
For Amazon, this approach enables them to make efficient use of these highly sought-after resources, creating a quasi-auction environment that guarantees revenue (provided customers engage). The pricing for accessing these resources will be genuinely flexible, fluctuating in response to supply and demand, according to the company.
When users subscribe to the service, it shows them the total cost for the specified timeframe and resources. Users have the flexibility to adjust this according to their resource requirements and budget before confirming their purchase.
This new feature is now accessible to all users, starting today in the AWS US East (Ohio) region.
CHECK THESE OUT:
