Amazon EC2 P4 Instances

With the latest-generation NVIDIA A100 Tensor Core GPUs, each P4d instance delivers on average 2.5x better DL performance compared to previous-generation P3 instances. EC2 UltraClusters of P4d instances help everyday developers, data scientists, and researchers run their most complex ML and HPC workloads by giving access to supercomputing-class performance without any upfront costs or long-term commitments. The reduced training time with P4d instances boosts productivity, helping developers focus on their core mission of building ML intelligence into business applications.

Developers can seamlessly scale to up to thousands of GPUs with EC2 UltraClusters of P4d instances. High-throughput, low-latency networking with support for 400 Gbps instance networking, Elastic Fabric Adapter (EFA), and GPUDirect RDMA technology help rapidly train ML models using scale-out/distributed techniques. EFA uses the NVIDIA Collective Communications Library (NCCL) to scale to thousands of GPUs, and GPUDirect RDMA technology enables low-latency GPU-to-GPU communication between P4d instances.

P4d instances deliver up to 60% lower cost to train ML models compared to P3 instances. Additionally, P4d instances are available for purchase as Spot Instances. Spot Instances take advantage of unused EC2 instance capacity and can lower your EC2 costs significantly with up to a 90% discount from On-Demand prices. With the lower cost of ML training with P4d instances, budgets can be reallocated to build more ML intelligence into business applications.

AWS Deep Learning AMIs (DLAMIs) and Amazon Deep Learning Containers make it easier to deploy P4d DL environments in minutes as they contain the required DL framework libraries and tools. You can also more easily add your own libraries and tools to these images. P4d instances support popular ML frameworks, such as TensorFlow, PyTorch, and MXNet. Additionally, P4d instances are supported by major AWS services for ML, management, and orchestration, such as Amazon SageMaker, Amazon Elastic Kubernetes Service (Amazon EKS), Amazon Elastic Container Service (Amazon ECS), AWS Batch, and AWS ParallelCluster.

NVIDIA A100 Tensor Core GPUs deliver unprecedented acceleration at scale for ML and HPC. NVIDIA A100’s third-generation Tensor Cores accelerate every precision workload, speeding time to insight and time to market. Each A100 GPU offers over 2.5x the compute performance compared to the previous-generation V100 GPU and comes with 40 GB HBM2 (in P4d instances) or 80 GB HBM2e (in P4de instances) of high-performance GPU memory. Higher GPU memory particularly benefits those workloads training on large datasets of high-resolution data. NVIDIA A100 GPUs use NVSwitch GPU interconnect throughput so each GPU can communicate with every other GPU in the same instance at the same 600 GB/s bidirectional throughput and with single-hop latency.

P4d instances provide 400 Gbps networking to help customers better scale out their distributed workloads such as multinode training more efficiently with high-throughput networking between P4d instances as well as between a P4d instance and storage services such as Amazon Simple Storage Service (Amazon S3) and FSx for Lustre. EFA is a custom network interface designed by AWS to help scale ML and HPC applications to thousands of GPUs. To further reduce latency, EFA is coupled with NVIDIA GPUDirect RDMA to enable low-latency GPU-to-GPU communication between servers with OS bypass.

Access petabyte-scale high-throughput, low-latency storage with FSx for Lustre or virtually unlimited cost-effective storage with Amazon S3 at 400 Gbps speeds. For workloads that need fast access to large datasets, each P4d instance also includes 8 TB NVMe-based SSD storage with 16 GB/sec read throughput.

The P4d instances are built on the AWS Nitro System, which is a rich collection of building blocks that offloads many of the traditional virtualization functions to dedicated hardware and software to deliver high performance, high availability, and high security while also reducing virtualization overhead.

Amazon SageMaker is a fully managed service for building, training, and deploying ML models. When used together with P4d instances, customers can easily scale to tens, hundreds, or thousands of GPUs to train a model quickly at any scale without worrying about setting up clusters and data pipelines.

DLAMI provides ML practitioners and researchers with the infrastructure and tools to accelerate DL in the cloud, at any scale. Deep Learning Containers are Docker images preinstalled with DL frameworks to make it easier to deploy custom ML environments quickly by letting you skip the complicated process of building and optimizing your environments from scratch.

If you prefer to manage your own containerized workloads through container orchestration services, you can deploy P4d instances with Amazon EKS or Amazon ECS.