NVIDIA NeMo Megatron Integrations

Amazon SageMaker Model Training simplifies the training and fine-tuning of machine learning (ML) models at scale, significantly reducing both time and costs while removing the burden of infrastructure management. This platform enables users to tap into some of the cutting-edge ML computing resources available, with the flexibility of scaling infrastructure seamlessly from a single GPU to thousands to ensure peak performance. By adopting a pay-as-you-go pricing structure, maintaining training costs becomes more manageable. To boost the efficiency of deep learning model training, SageMaker offers distributed training libraries that adeptly spread large models and datasets across numerous AWS GPU instances, while also allowing the integration of third-party tools like DeepSpeed, Horovod, or Megatron for enhanced performance. The platform facilitates effective resource management by providing a wide range of GPU and CPU options, including the P4d.24xl instances, which are celebrated as the fastest training instances in the cloud environment. Users can effortlessly designate data locations, select suitable SageMaker instance types, and commence their training workflows with just a single click, making the process remarkably straightforward. Ultimately, SageMaker serves as an accessible and efficient gateway to leverage machine learning technology, removing the typical complications associated with infrastructure management, and enabling users to focus on refining their models for better outcomes.

BioNeMo is a cloud-based platform designed for drug discovery that harnesses artificial intelligence and employs NVIDIA NeMo Megatron to enable the training and deployment of large biomolecular transformer models at an impressive scale. This service provides users with access to pre-trained large language models (LLMs) and supports multiple file formats pertinent to proteins, DNA, RNA, and chemistry, while also offering data loaders for SMILES to represent molecular structures and FASTA for sequences of amino acids and nucleotides. In addition, users have the flexibility to download the BioNeMo framework for local execution on their own machines. Among the notable models available are ESM-1, which is based on Meta AI’s state-of-the-art ESM-1b, and ProtT5, both fine-tuned transformer models aimed at protein language tasks that assist in generating learned embeddings for predicting protein structures and properties. Furthermore, the platform will incorporate OpenFold, an innovative deep learning model specifically focused on forecasting the 3D structures of new protein sequences, which significantly boosts its capabilities in biomolecular exploration. Overall, this extensive array of tools establishes BioNeMo as an invaluable asset for researchers navigating the complexities of drug discovery in modern science. As such, BioNeMo not only streamlines research processes but also empowers scientists to make significant advancements in the field.