During the assessment phase, the AWS MAP helped Heirloom Carbon understand their readiness for the migration. It offered a clear view of the costs, potential risks, and the numerous benefits associated with the migration. With AWS MAP, Heirloom Carbon was able to chart out an effective migration strategy that aligned well with their business objectives.
In the mobilization phase, the AWS MAP equipped Heirloom Carbon's team with necessary tools and training to ensure a seamless transition. This included understanding the AWS environment, the implementation of best practices, and hands-on training to manage the new infrastructure effectively.
The migration and modernization phase was the most crucial. AWS MAP's resources and best practices were pivotal in ensuring a seamless transition of Heirloom's computational workloads and data to the AWS cloud. The tools provided by AWS MAP aided Heirloom Carbon in re-platforming and re-architecting their applications for the AWS cloud.
For Heirloom Carbon, the use of OpenFOAM and Ansys Fluent is critical to their goal of reducing carbon dioxide (CO2) in the atmosphere. These software packages enable Heirloom Carbon to simulate the performance of different carbon capture technologies and evaluate their effectiveness in reducing atmospheric carbon levels. Through simulations Heirloom Carbon is able to develop more effective carbon capture technologies to deploy at scale, reducing the carbon footprint of industrial processes.
OpenFOAM and Ansys Fluent are widely used computational fluid dynamics (CFD) software packages commonly used to simulate fluid flow and heat transfer in engineering and environmental applications. By deploying these software packages on AWS, it allows researchers and engineers at Heirloom Carbon to model and analyze complex systems difficult to study using traditional experimental methods.
Slurm is a workload manager that is used to manage and schedule jobs on a compute cluster and is particularly beneficial when used in conjunction with OpenFOAM and Ansys Fluent jobs. Slurm allows efficient compute resource management for jobs that are computationally intensive, and requiring significant memory and processing power. Cloud303 engineers implemented Slurm using Amazon EC2 instances to optimize and automate the allocation of AWS resources as needed. This ensures that jobs are processed quickly and efficiently. Additionally, Slurm provides a fair share scheduling algorithm, ensuring that all users get a fair share of compute resources. It prevents any one user from monopolizing resources which could cause delays for others. Ultimately, these improvements result in faster job turnaround times and improved overall efficiency.
The Slurm head node was deployed in a private subnet alongside the compute nodes in a cluster placement group to optimize performance. AWS ParallelClusters were used to provision and manage cluster compute resources such as the Slurm head node, compute queues, and auto-scaling infrastructure.
ParallelCluster is an open-source cluster management tool for deploying and managing High-Performance Computing (HPC) clusters on AWS. AWS ParallelCluster uses a simple text file to model and provision all the resources needed for the HPC application(s) in an automated, secure manner. It supports multiple instance types, job submission queues, and job schedulers like Slurm.
Cloud303 leveraged Amazon FsX for Lustre - a high-performing file system - to ensure Heirloom Carbon’s simulations were executed in a timely and efficient manner while maintaining durability and availability of data. This is a critical aspect of the architectural design: OpenFOAM jobs often involve large and complex simulations that generate a significant amount of data.
Slurm was also configured to manage the powering of nodes when jobs were submitted and shutting off nodes when jobs were complete. An Ansys Fluent license server was hosted on Heirloom Carbon’s corporate network. IP whitelisting controlled access, thus eliminating the need for a site-to-site VPN.
The Slurm head node had Paraview installed to visualize the solution and/or mesh. NICE DCV was used to create Remote Desktop sessions into the SLURM Head Node to use the Paraview GUI. Amazon Simple Storage Service (s3) was used to store job inputs and outputs. All S3 objects utilized a lifecycle policy to transfer infrequently used data into cheaper storage classes. A CLI script was written to submit jobs. This CLI uploaded the required inputs to the S3 bucket before initiating the job.
Cloud303 also provided a notification system to notify users when jobs reached certain key states. This was necessary because of the long duration of OpenFOAM and Ansys Fluent jobs. Cloud303 provided Slurm accounting capability to collect accounting information for every job and job step executed, cluster resource utilization monitoring, multiple users per cluster support with NICE DCV session isolation, and restricted access to all compute resources via OpenVPN, durable log storage, and all using AWS Best Practices. Cloud303 also conducted a Well-Architected Review and provided a Success Criteria Report to Heirloom Carbon.