Scientists develop an AI pathology model, creating a new tool for pathological d

"Nature editors quickly realized that this was a milestone scientific achievement, and thus expedited the manuscript processing. Ultimately, this paper was accepted within 5 months, far faster than Nature's average acceptance time of 268 days," said Professor Wang Sheng from the University of Washington in the United States, when talking about his and his co-authors' latest paper.

The reviewers also stated that in the field of digital pathology, this is an unprecedented work and they are very much looking forward to seeing it change the research and clinical paradigms in pathology.

In the study, Wang Sheng and his collaborators developed an ultra-high parameter pathological model, which is also the world's first model capable of modeling and classifying "entire" pathology images.

They trained and validated this model on 30,000 patient data from 28 cancer centers.

The results showed that the model achieved the best results in 25 out of 26 tasks, proving its effectiveness and universality.

Wang Sheng stated that this achievement is a powerful assistant for doctors, good news for ordinary people, and a supplementary teaching material for medical colleges.The success of this study fills us with confidence in the application of AI methods in cancer treatment and will mark a new beginning in the field of cancer therapy, he said.

Advertisement

With the further development of technology, this AI model is expected to play an increasingly important role in cancer diagnosis and treatment, bringing new hope to millions of cancer patients worldwide.

It is anticipated that it will bring the following applications:

Firstly, it will become a powerful assistant to doctors.

Firstly, AI pathology models will be directly applied to major hospitals, becoming one of the preliminary screening tools for doctors to conduct pathological diagnoses.This model can quickly analyze pathological images, provide preliminary diagnostic opinions, and help pathologists make accurate diagnoses more efficiently.

Not only can this improve the speed of diagnosis, but it can also reduce the workload of pathologists, allowing them more time to handle complex cases.

Secondly, it becomes a boon for ordinary people.

In areas or countries lacking highly skilled pathologists, the AI pathology model will serve as an alternative solution.

General users can directly use this model to analyze their pathological images and obtain reliable diagnostic results.This will greatly improve the diagnostic level in areas with insufficient medical resources, ensuring that more patients can receive timely and accurate diagnoses and treatment recommendations.

Thirdly, it serves as auxiliary teaching materials for medical schools.

Cultivating a qualified pathologist requires a significant amount of time and money, and AI models can act as auxiliary teaching materials for medical schools.

By providing a large number of annotated learning samples, AI models can expose medical students to a wider range of pathology cases, enhancing their learning efficiency and practical experience.

In summary, AI pathology models demonstrate broad application prospects in three aspects: doctors, the general public, and medical students. It will promote the development of the field of pathology and improve the efficiency and accuracy of cancer diagnosis.Make the Model No Longer "Forgetting the Past After Seeing the Future"

It is reported that more than ten million people worldwide die from cancer each year, making rapid and accurate cancer diagnosis extremely important.

Pathological section examination is the most intuitive and reliable method for diagnosing tumors and is one of the essential means for every cancer patient to be diagnosed with cancer.

Pathological examination involves making sections from the diseased organ's lesion, observing them under a microscope, and then determining whether they are malignant tumors, predicting the corresponding pathological types, and thus deciding on the relevant treatment methods.

Considering the high demand and tedious workload of pathological testing, designing accurate AI models to automate the above process will bring great value.The process of pathological section analysis can be modeled as an image classification problem, where an AI model is used to categorize a given pathological section image into malignant or benign pathological types.

However, unlike traditional image classification models (such as classifying animal images into cats or dogs), pathological images are very large.

A typical pathological image can even reach a size of 100,000 by 100,000 pixels, while traditional AI image classifiers are generally designed for images of 256 by 256 pixels.

Such an entire pathological image occupies a large amount of memory space and cannot be fully loaded into the AI model for training using a graphics processing unit (GPU, Graphics Processing Unit).

Therefore, existing AI models are unable to model "whole-slide" pathological images.To address this issue, Wang Sheng and his colleagues found that modeling an entire pathology image with 100,000 by 100,000 pixels requires tackling the core technical challenge of modeling long texts.

This means enabling the model to model an extremely long sequence of text as a whole without the situation of "seeing the latter and forgetting the former."

The data scale corresponding to a pathology image is approximately equivalent to a long text of 150,000 characters. For example, asking a person to read a 150,000-character article continuously will inevitably result in forgetting the former after seeing the latter, and it is impossible to understand this long text from a global perspective.

For AI models, the difficulty is the same, and AI models cannot fully understand and digest this long text.

Therefore, long text modeling is a long-standing problem in the field of natural language processing and is also one of the most core issues.In recent years, the trend of large models sparked by ChatGPT and GPT4 has garnered widespread attention and research on the problem of long article modeling.

Unlike traditional Q&A systems, when answering user questions, ChatGPT takes into account the previous conversations between the user and ChatGPT, and these conversations form a long article.

The success of ChatGPT largely stems from new natural language processing technologies for long article modeling.

Therefore, Wang Sheng and others have ingeniously improved and applied these technologies for modeling long articles in natural language processing to the classification of pathological images, thus solving the problem of modeling and classifying large whole pathological images.

The three parties have joined forces, and former teachers and students now serve as corresponding authors together.Wang Sheng stated that this project was jointly completed by the University of Washington where he is based, Microsoft Research, and one of the largest medical institutions in the United States, the Providence Cancer Institute.

In June 2023, Wang Sheng's doctoral student, Xu Hanwen, undertook a summer internship at Microsoft Research, focusing on the research topic of how to model large-scale pathological images.

At that time, large model technologies such as ChatGPT were beginning to gain prominence in various fields, but no one had yet applied them to medical pathological images.

After in-depth discussions, the three parties discovered that the key challenge in applying large model technology was in long article modeling.

At this point, the LongNet model released by Microsoft Research caught their attention. Although this model had never been used in the medical field, it showed excellent performance in long article modeling.Later, Xu Hanwen and his Microsoft Research intern mentor preliminarily judged that Microsoft Research's LongNet model could solve the problem of large-scale pathological image modeling. Subsequently, Xu Hanwen carried out a three-month experiment.

In September 2023, they obtained a preliminary solution, that is, the problem of large-scale pathological image modeling can be solved by long article modeling techniques in natural language processing.

After the preliminary plan was determined, they carried out more in-depth verification. And they tested it on data from 31 types of cancer from 28 cancer centers.

The experimental results show that this model has achieved the best results in multiple tasks, indicating that it is a general and accurate pathological modeling solution.

Wang Sheng said: "The success of the project is inseparable from the interdisciplinary team composed of the world's top AI experts and pathologists."In this collaboration, the Providence Cancer Institute in the United States provided core data and medical technology, Microsoft Research contributed the most advanced AI technology and computing resources, and the University of Washington, where Wang Sheng is based, brought cutting-edge AI medical research techniques and experience.

Recently, the related paper was published in Nature[1] under the title "A whole-slide foundation model for digital pathology from real-world data."

Xu Hanwen is the first author, Professor Carlo Bifulco from the Providence Cancer Institute in the United States, Wang Sheng, and Dr. Pan Haifeng from Microsoft Research are the co-corresponding authors.

Wang Sheng said: "My collaboration with Dr. Pan began in 2014 when I was a first-year doctoral student, doing a summer internship at Microsoft Research in Seattle, where Dr. Pan was my internship supervisor."

At that time, they were both researching Natural Language Processing (NLP), with Wang Sheng being a doctoral student in the Department of Computer Science at the University of Illinois at Urbana-Champaign, and Dr. Pan Haifeng being a researcher in the NLP group at Microsoft Research.At that time, deep learning algorithms had just begun to emerge, and the trend of AI had not yet taken off, with no researchers applying AI or NLP methods to medical research.

NLP and medicine seemed to be completely unrelated fields at the time.

However, Pan Haifeng suggested that Wang Sheng explore how to apply NLP technology to solve medical data problems.

In the summer of 2014, their collaborative project utilized the common belief propagation method in machine learning and NLP for causal inference and prediction of the effects of cancer drugs.

After AlphaFold was introduced in 2020, AI drug development has become one of the hottest applications in the field of AI, and they were already researching similar issues as early as 2014.The paper published in Nature this time continues this line of thought. They have used the long article modeling technology in ChatGPT to solve the modeling problem of ultra-large pathology images in the medical field.

"It can be said that we have applied the most cutting-edge generative AI technology to a new field, which is also the biggest innovation point of our paper," said Wang Sheng.

Next, they plan to extend this AI model to image data in other cancer diagnoses, such as computed tomography (CT, Computed Tomography), magnetic resonance imaging, and X-rays.

Because what is proposed this time is a universal medical image model architecture, they believe that this model architecture is also suitable for other types of medical image data.

Specifically, they plan to build a large model for each type of image data (such as CT, magnetic resonance imaging, X-rays) to fully utilize the information of these different types of images.In addition to this, they will also construct corresponding large models for other important types of medical data such as genetic data and clinical diagnosis reports.

Ultimately, their goal is to integrate these independent large models to create a comprehensive cancer diagnosis AI system.

This system will be able to combine image, genetic, and clinical data to provide comprehensive support for cancer diagnosis and treatment.

It is expected that this cross-disciplinary, multi-data source AI model will become a powerful tool for cancer diagnosis and treatment, providing doctors with more comprehensive information support and promoting further development in the field of medical research.