Learn how GE Healthcare uses AWS to build a new AI model that interprets MRIs

MRI images are understandably complex and data-heavy.

For this reason, developers of large language model (LLM) training for MRI analysis have had to slice captured images in 2D. But this results in approximations of the original image, limiting the model's ability to analyze complex anatomical structures. This poses challenges in complex cases involving brain tumors, skeletal disorders or cardiovascular disease.

but GE Healthcare This major hurdle appears to have been overcome, introducing the industry's first full-body 3D MRI Research Foundation Model (FM) this year. AWS re: innovation. For the first time, models can use full 3D images of the entire body.

GE Healthcare's FM was built from the ground up on AWS — there are very few models designed specifically for medical imaging like MRI — and is based on more than 173,000 images from more than 19,000 studies. The developers say they were able to train the model with five times less compute than previously required.

GE Healthcare has not yet commercialized the foundation model; It is still in an evolutionary research stage. A primary assessor, Mass. General BrighamIt is set to start testing soon

“Our vision is to put these models into the hands of technical teams working in healthcare systems, giving them powerful tools to develop research and clinical applications faster and more cost-effectively,” GE Healthcare chief AI officer Pari Bhatia told VentureBeat.

Enabling real-time analysis of complex 3D MRI data

Although this is a groundbreaking development, generative AI and LLM are not new territory for companies. The team has been working on the advanced technology for more than 10 years, Bhatia explained.

One of its flagship products AIR RECON DLA deep learning-based reconstruction algorithm that allows radiologists to acquire sharper images faster. The algorithm removes noise from the raw image and improves the signal-to-noise ratio, reducing scan time by up to 50%. As of 2020, 34 million patients have been scanned with the AIR Recon DL.

GE Healthcare begins work on its MRI FM in early 2024. Because the model is multimodal, it can support image-to-text searching, image and word linking, and disease category and classification. The goal is to give healthcare professionals more detail in a scan than ever before, Bhatia said, leading to faster, more accurate diagnosis and treatment.

“The model has significant potential to enable real-time analysis of 3D MRI data, which could improve medical procedures such as biopsies, radiation therapy and robotic surgery,” Dan Sheeran, GM of healthcare and life sciences at AWS, told VentureBeat.

Already, it has outperformed other publicly available research models, including classifications for prostate cancer and Alzheimer's disease. It demonstrated up to 30% accuracy in image retrieval when matching MRI scans with text descriptions — which may not sound all that impressive, but it's a big improvement over the 3% ability demonstrated by similar models.

“It's gotten to a point where it's really producing some strong results,” Bhatia said. “The implications are huge.”

Doing more with (much less) data

Bhatia explained that the MRI process requires several different types of datasets to support the various techniques that map the human body.

What is known as a T1-weighted imaging technique, for example, highlights fat tissue and reduces water signal, while T2-weighted imaging increases water signal. The two methods are complementary and create a complete picture of the brain to help clinicians detect abnormalities such as tumors, trauma or cancer.

“MRI images come in different shapes and sizes, how would you have books of different formats and sizes, right?” Bhatia said.

To overcome the challenges presented by different datasets, the developers introduced a “resize and adapt” technique to allow the model to process and respond to variations. Also, in some cases the data might be missing – for example, an image might be incomplete – so they taught the model to simply ignore those instances.

“Instead of getting stuck, we taught the model to skip the gaps and focus on what's available,” Bhatia said. “Think of it as solving a puzzle with some missing pieces.”

The developers have also employed semi-supervised student-teacher learning, which is particularly helpful when there is limited data. With this approach, two different neural networks are trained on both labeled and unlabeled data, the teacher generates labels that help the learner learn and predict future labels.

“We are now using many self-supervised techniques, which do not require large amounts of data or labels to train large models,” Bhatia said. “It reduces dependency, where you can learn more from these raw images than in the past.”

This helps ensure that the model works well in hospitals with fewer resources, older machines and different types of datasets, Bhatia explained.

He also highlights the importance of the model's multimodality. “A lot of technology in the past was unimodal,” says Bhatia. “It would just look between images, between text. But now they're becoming multi-model, they can go from image to text, text to image, so you can bring a lot of things that were done with separate models in the past and really integrate the workflow. ”

He insists that researchers only use datasets they own; GE Healthcare has partners who license de-identified data sets and are careful to adhere to compliance standards and policies.

AWS uses SageMaker to address computational, data challenges

Undoubtedly, there are many challenges when creating such sophisticated models — such as limited computing power for gigabyte-sized 3D images.

“It's a huge 3D volume of data,” Bhatia said. “You have to bring it into the memory of the model, which is a really complicated problem.”

To help overcome this, GE Healthcare was created Amazon Sagemakerwhich provides high-speed networking and distributes training power across multiple GPUs and leverages the Nvidia A100 and Tensor Core GPUs for large-scale training.

“Because of the data size and the model size, they can't send it to a single GPU,” Bhatia explained. SageMaker allowed them to customize and scale operations across multiple GPUs that could communicate with each other.

Also used by developers Amazon FSX in Amazon S3 Object storage, which allows fast reading and writing for datasets.

Bhatia points out that another challenge is cost optimization; With Amazon's Elastic Compute Cloud (EC2), developers were able to move unused or infrequently used data to low-cost storage tiers.

“Employing SageMaker to train these large models—mainly for efficient, distributed training across multiple high-performance GPU clusters—was a key element that helped us move forward quickly,” said Bhatia.

He stressed that all components were developed from a data integrity and compliance perspective that took into account HIPAA and other regulatory regulations and frameworks.

Ultimately, “these technologies can really streamline, help us innovate faster, as well as improve overall operational efficiency by reducing administrative load and ultimately drive better patient care – because now you're providing more personalized care.”

Serving as a basis for other special fine-tuned models

Although the current model is specific to the MRI domain, the researchers see great opportunities to expand it to other areas of medicine.

Sheeran noted that, historically, AI in medical imaging has been limited by the need to develop custom models for specific conditions of specific organs, requiring expert annotation for each image used in training.

But this approach is “inherently limited” because diseases manifest in different ways in individuals and introduce generalization challenges.

“We really need thousands of such models and the ability to quickly create new ones as we encounter new information,” he said. High-quality labeled datasets are also essential for each model.

Now with generative AI, instead of training separate models for each disease/organ combination, developers can pre-train a single foundation model that can serve as the basis for other specialized fine-tuned models.

For example, GE Healthcare's model could be extended to areas such as radiation therapy, where radiologists spend significant time manually identifying organs at risk. It could also help reduce scan times during X-rays and other procedures that currently require patients to sit in a machine for long periods of time, Bhatia said.

Sheeran marveled that “we're not just expanding access to medical imaging data through cloud-based tools; We are changing how that data can be used to drive AI advances in healthcare.”