This is My Architecture: Speed HPC Data Sets Quickly, Securely

Each week AWS features innovative cloud architectures from its partners and customers in a video feature it calls, This is My Architecture. We’re excited to share that this week’s installment features our very own DevOps Engineer, Arthur Mandel, and David Fluck from Toyota Research Institute. In the episode, they highlight how two technologies — AWS P3 GPU instances and AWS Service Catalog — enable TRI data scientists to more quickly progress their research.

Specifically, Mandel and Fluck share how they:

• Deployed AWS P3 GPU instances to deliver high performance compute in the cloud to increase the number of simulations completed for TRI’s high performance computing. Amazon reports that these instance types can increase simulations completed by 3-4x and have been proven to reduce machine learning training times from days to minutes.
• Created an on-demand self-service portal for TRI data scientists with AWS Service Catalog, Amazon’s service for creating and managing catalogs of IT services. TRI’s data scientists use the Service Catalog-powered portal to create and process data sets both quickly and securely.

In the episode, Fluck and Mandel walk viewers through the process from the user’s perspective, illustrating how Python Scripts, Service Catalog, S3 buckets, FSX for Lustre, GPU clusters, Machine Learning models and more all work together to help TRI autonomous vehicle efforts. Watch the episode today to learn how to use AWS Service Catalog products and trigger them on-demand to create P3 compute clusters to process Machine Learning data sets.

Interested in additional HPC Resources? Check out our HPC resource page or contact us today to learn how our HPC Consulting Practice can help address your technology and business needs.