High Performance Computing
- The cloud is the perfect place to perform HPC
- Can create a very high number of resources in no time
- Can speed up time to results by adding more resources
- Can pay only for the systems you have used
- Perform genomics, computational chemistry, financial risk modeling, weather prediction, machine learning, deep learning, autonomous driving
- AWS Direct Connect
- Move GB/s of data to the cloud, over a private secure network
- Snowball & Snowmobile
- Move PB of data to the cloud
- AWS DataSync
- Move large amount of data between on-premises and S3, EFS, FSx for Windows
Compute and Networking
- EC2 Instances
- CPU optimized, GPU optimized
- Spot Instances / Spot Fleets for cost svaings + Auto Scaling
- EC2 Placement Groups : Cluster for good network performance
- EC2 Enhanced Networking
- Higher bandwidth, higher PPS, low latency
- Elastic Fabric Adapter
- Improved ENA for HPC, only works for Linux
- Great for inter-node communications, tightly coupled workloads
- Leveraged Message Passing Interface standard
- Bypasses the underlying Linux OS to provide low-latency, reliable transport
Storage
- Instance-attached storage :
- EBS : scale up to 256,000 IOPS with io2 Block Express
- Instance Store : scale to millions of IOPS, linked to EC2 instance, low latency
- Network Storage :
- Amazon S3 : large blob, not a file system
- Amazon EFS : scale IOPS based on total size, or use provisioned IOPS
- Amazon FSx for Lustre :
- HPC optimized distributed file system, millions of IOPS
- Backed by S3
Automation and Orchestration
- AWS Batch
- AWS Batch supports multi-node parallel jobs, which enables you to run single jobs that span multiple EC2 instances
- Easily schedule jobs and launch EC2 instances accordingly
- AWS ParallelCluster
- Open-source cluster management tool to deploy HPC on AWS
- Configure with text files
- Automate creatin of VPC, Subnet, cluster type and instance types
- Ability to enable EFA on the cluster
'aws' 카테고리의 다른 글
| Architectures (1) | 2024.04.20 |
|---|---|
| Other Services (1) | 2024.04.20 |
| Disaster Recovery & Migrations (1) | 2024.04.19 |
| Amazon VPC (2) | 2024.04.19 |
| AWS Security & Encryption (1) | 2024.04.18 |