Medical imaging, a common non-invasive technique used in physical examinations, enables the detection of physiological structures and pathological conditions within organs or tissues. Physicians employ auxiliary tools like x-rays, ultrasound, and magnetic resonance imaging (MRI) to diagnose existing or potential lesions. Macao Polytechnic University is at the vanguard of integrating academia, research, and industry within the realm of artificial intelligence (AI) technologies and applications. The research team on AI in medical imaging at the Faculty of Applied Sciences lives out the mission of harnessing smart technology for enhanced healthcare services. Engaging in extensive dialogues with medical professionals, in collaboration with international experts across myriad fields, the team has been relentlessly innovating, developing AI-driven solutions that prioritise clinical needs in medical imaging, stimulating the evolution of the smart healthcare industries.
Predictive Technology for Breast Cancer
Breast cancer, one of the three leading causes of cancer-related deaths in women, demands prompt and precise diagnosis to enhance treatment outcomes and improve patient prognosis. The research team worked together with the Netherlands Cancer Institute (NKI), Maastricht University, Radboud University Medical Centre, Hospital Group Twente (ZGT), University of Twente, Haga Teaching Hospital, and Cancer Centre Amsterdam on AI solutions helpful for breast cancer diagnosis. Their study explores the prediction of cancer types going beyond the molecular level using the multimodal deep learning approach. By seamlessly fusing technology with medical science, this study paves the way for more precise and early detection of cancer.
“Multimodal” refers to an analytical approach that amalgamates diverse data sources in machine learning. Traditionally, cancer research has focused predominantly on examining alterations at the molecular level within tumour cells. Scientists analyse genetic mutations, expression profiles, and other molecular characteristics of cancer cells to classify tumour subtypes and identify potential targeted therapies. Leveraging intelligent technology, the research team enhances conventional research methods by integrating diverse data types to formulate a holistic model. This model broadens the data scope to include microenvironmental factors such as the tumour’s surrounding blood vessels and immune cells. It provides a multidimensional perspective on the molecular complexity of breast cancer and facilitates the analysis of the tumour within its microenvironment. This insight led to a more thorough characterisation of breast cancer phenotypes, consequently enabling a more precise, targeted, and swift evaluation of tumour types.
From a technical standpoint, the research team developed a multimodal image classification model that leverages diagnostic mammography and ultrasound images. By training with deep learning algorithms, it automatically learns and evaluates image features for classifying and predicting molecular subtypes of breast cancer. The model integrates internal and cross-modal attention mechanisms, enhancing the assimilation of cross-modal image features. This integration allows the model to evaluate images from various perspectives, thereby elevating its predictive efficacy. Clinically validated, this model’s predictive capabilities significantly streamline the interpretation of radiological images. As the depth of data computation intensifies, the model’s accuracy and specificity in classifying breast cancer is expected to continually improve. This advancement will facilitate earlier diagnosis and prognosis of breast cancer, paving the way for a broader range of treatment possibilities.
Automation of Brain Tumour Segmentation
Brain tumour segmentation is a vital part of diagnosing and treating brain tumours as it involves identifying and outlining tumour regions within MRI scans. The process is labour-intensive, time-consuming, and requires a high level of expertise. The research team, together with the Harbin Institute of Technology, Qingdao University of Science and Technology, Osaka University, and NBL Technovator Co., developed MimicNet, automating the complex process of brain tumour segmentation from multimodal MRI scans.
MimicNet employs state-of-the-art deep learning techniques to emulate the manual delineation of human experts and generate automatic delineation of tumour areas in MRI scans. The core of this system is a vast dataset of MRI scans that have been painstakingly segmented by experts. Powered by the deep learning techniques, intricate human expertise is translated into an automated process that sifts through the complex layers of images, accurately identifying and segmenting the tumour regions. This process mirrors the meticulous work of human experts with the benefits of speed, efficiency, and consistency that AI brings.
Training MimicNet involves multiple MRI modalities, including T1-weighted, T1-weighted with contrast, T2-weighted, and T2-Flair images. Each modality provides a different perspective and a unique slice of information about the brain tumour. By integrating these multiple modalities, MimicNet is able to capture a comprehensive view of the brain tumour, resulting in a higher accuracy of brain tumour segmentation that facilitates effective treatment planning.
Intelligent Evaluation of Foetal Lung Development
A newborn’s first cry is what people eagerly await in the delivery room. This loud cry signifies the beginning of the newborn’s breathing, indicating that the lung function has kicked into gear. The successful birth of a new life largely depends on the development and maturity of the lungs of the fetus in the mother’s womb. Starting in the realm of medical imaging, the research team integrated intelligent technology into foetal ultrasound monitoring to look for non-invasive detection methods that can evaluate foetal lung development and maturity, thereby optimising prenatal care and ensuring the health of both mothers and infants.
Accurately assessing foetal lung maturity can be complex. Traditional methods often necessitate invasive procedures, which can induce anxiety in expectant mothers and pose potential risks. The research team used algorithms that can learn from ultrasound image analyses to assess foetal lung maturity. Technically, a deep learning model is employed together with graph theory to transform ultrasound data into graphs and analyse complex patterns and relationships among variables thus identified in the images. This approach leads to better-informed foetal lung evaluation results while mitigating the risks associated with traditional prenatal foetal lung testing, both physically and psychologically.
This study was conducted by the research team in collaboration with various other universities, research centres and medical institutions. Their partners include East China Normal University, the Engineering Research Centre of Traditional Chinese Medicine Intelligent Rehabilitation, Xi’an Jiaotong-Liverpool University, the University of Liverpool, Shanghai Sixth People’s Hospital Affiliated to Shanghai Jiaotong University School of Medicine, Nanjing Medical University Affiliated Suzhou Hospital, the Artificial Intelligence Innovation Centre (AIIC), and Naval Medical University. Experts from fields ranging from AI and mathematics to medicine and ultrasound technology work together, taking a step forward to smart innovations through interdisciplinary research.
Aids for Clinical Diagnosis and Prognosis
Interpreting medical images can be time-consuming and energy-draining as medical professionals visually examine each image to spot any abnormalities. The research team employs AI technologies to develop auxiliary tools for medical image interpretation. Tailored to fit seamlessly into clinical workflows, such tools help streamline regular routine tasks enhancing the efficacy of healthcare resource management.
Working together with experts from Shanghai Jiaotong University, Cardiff University, the University of Tokyo, and Tianjin University, the research team studied existing approaches to medical image analyses using AI and machine learning techniques. By synthesising vast amounts of imaging data, lab test results, demographic data, family histories, treatment response rates, and other variables influencing disease progression, these AI-driven systems are able to obtain enhanced analytical performance. Medical professionals could have access to patient profiles that are more comprehensive, better informing their diagnosis and prognosis.
These AI systems are adept at automated analyses and preliminary disease classification based on medical images. Operating 24/7, it could considerably reduce medical resources devoted to repetitive tasks. Diagnosis and prognosis outcomes, derived from automated image analyses, can promptly reach medical professionals for more in-depth interpretation, facilitating earliest possible treatment of diseases. The findings of this study underscore the affordances of AI in enhancing healthcare efficiency and providing a technical reference point for optimising medical services, particularly in regions with limited healthcare resources.
Dr Tan Tao is devoted to research on AI-driven clinical aids in medical imaging
Enhancing Intelligent Interpretation of Medical Images
Associate Professor Dr Tan Tao, the principal investigator of the research team on AI in medical imaging at the Faculty of Applied Sciences, is devoted to research on AI-driven clinical aids. “The power of AI lies in its capacity for rapid, large-scale data analysis and the detection of subtle patterns, potentially heralding application-focused breakthroughs in medical imaging,” he says. Medicine knowledge and clinical data constitute invaluable resources for training AI models and systems. Underpinned by clinical expertise and data, these systems can perform tasks beyond individual human capabilities, such as the simultaneous analysis of millions of medical images and the identification of subtle patterns that may escape human observation. The research team works to support medical professionals in enhancing diagnostic and treatment efficiency. They aspire that expert-level medical image analytical capabilities could be aggregated for use worldwide via AI systems, thus contributing to the United Nations’ sustainable development goals for global health.
MPU fosters young researchers on AI medical imaging
Fostering Young Researchers on Smart Healthcare
The fusion of teaching and research can catalyse a student’s quest for knowledge and spark curiosity to delve into the unexplored. Wang Rongsheng, a Master’s student at MPU in Big Data and Internet of Things, is a shining example. Wang made his mark in the 2023 RSNA Screening Mammography Breast Cancer Detection AI Challenge by winning the silver medal among nearly 2,000 competitors globally. His mentor is Dr Tan Tao.
“Dr Tan has been engaging us in many insightful discussions and exchanges, helping us to comprehend the intricacies and diagnostic rules of mammography images. He guided us through the design of algorithmic frameworks for multitask and multi-information fusion, providing valuable advices on data processing and feature extraction,” Wang shares. Participating in the competition is a platform to showcase and validate research outcomes. He adds, “The competition enables me to collaborate with globally-recognised universities, medical laboratories and professionals, giving me access to rich mammography image datasets for my research and practice.”
Reflecting on the early stages of his master’s studies in machine learning, Wang recalls his inaugural exposure to the competition platform. “We had to complete practical assignments and submit them via the platform in this learning module. Through this module, I came to understand that this globally accessible platform convenes leading organisations to data science challenges. This platform hosted the medical imaging competition organised by the Radiological Society of North America. The competition required participants to develop a machine learning model that augments the precision and productivity of breast cancer screening via medical imaging. It consisted of public and private testing phases. We proudly attained the 50th position in the public phase and the 56th in the private phase, demonstrating the considerable generalisation and reliability of our algorithm. This achievement merited us the silver medal standing globally,” he proudly shares.
Wang Rongsheng (right) won the silver medal in RSNA Screening Mammography Breast Cancer Detection AI Challenge with his mentor Dr Tan Tao (left)
The Promising Future of AI Medical Imaging
The pioneering role of the research team in healthcare-related innovation emphasises the complimentary relationship between AI and medical science. By harnessing advanced technologies such as AI and machine learning, the research team fuels progress in medical science, cultivating a unique synergy between technology and clinical sciences that deepens our understanding of diseases and enhances clinical management efficiency. This collaborative endeavour showcases the pivotal role of interdisciplinary research in shaping healthcare, from accurately predicting cancer subtypes to leveraging AI for improved predictions and diagnostics in medical imaging and even advancing prenatal care. The seamless integration of technology and healthcare is crucial to steering the future course of medicine. Our ongoing exploration of AI’s role in medical imaging charts a course towards better health outcomes via collaborative efforts, and we look forward to witnessing the continued evolution of this exciting field.
“Today's machine learning models are typically built to perform a single task in a single scenario. However, I foresee a future where we will have more integrated machine learning models capable of simultaneously accomplishing multiple tasks across various scenarios. The emergence of ChatGPT indeed suggests such a possibility,” Wang expresses. His fascination doesn’t stop at the large language models for medical dialogues but extends to active participation in research at the crossroads of large-scale models and medical science. “Under Dr Tan’s guidance, we created IvyGPT, MPU’s first large-scale medical language model. This venture has exceeded ChatGPT’s performance in the medical evaluation benchmark list, achieving impressive results. We are committed to further deepening our research on these related topics in the future,” he affirms.
Encouraging the advancement of high-tech industries is pivotal in bolstering the growth of health-related sectors, including medical services, health management, biotechnology, and pharmaceutical research and development. Dr Tan Tao and Wang Rongsheng are resolute in their belief in the enormous potential of artificial intelligence in the field of medical image analysis. They posit that this technology can aid physicians in their diagnoses and formulate tailored healthcare treatment plans, thereby realising precise healthcare for the populace.
The full articles of relevant research at MPU:
Predicting breast cancer types on and beyond molecular level in a multi-modal fashion
Artificial intelligence-based medical image automatic diagnosis and prognosis prediction
Ultrasonic evaluation of fetal lung development using deep learning with graph
