Accéder directement au contenu Accéder directement à la navigation
Nouvelle interface
Communication dans un congrès

Reinforcement Learning Driven Intra-modal and Inter-modal Representation Learning for 3D Medical Image Classification

Abstract : Multi-modality 3D medical images play an important role in the clinical practice. Due to the effectiveness of exploring the complementary information among different modalities, multi-modality learning has attracted increased attention recently, which can be realized by Deep Learning (DL) models. However, it remains a challenging task for two reasons. First, the prediction confidence of multi-modality learning network cannot be guaranteed when the model is trained with weakly-supervised volume-level labels. Second, it is difficult to effectively exploit the complementary information across modalities and also preserve the modality-specific properties when fusion. In this paper, we present a novel Reinforcement Learning (RL) driven approach to comprehensively address these challenges, where two Recurrent Neural Networks (RNN) based agents are utilized to choose reliable and informative features within modality (intra-learning) and explore complementary representations across modalities (inter-learning) with the guidance of dynamic weights. These agents are trained via Proximal Policy Optimization (PPO) with the confidence increment of the prediction as the reward. We take the 3D image classification as an example and conduct experiments on a multi-modality brain tumor MRI data. Our approach outperforms other methods when employing the proposed RL-based multi-modality representation learning.
Type de document :
Communication dans un congrès
Liste complète des métadonnées
Contributeur : Administrateur IMT - Mines Alès Connectez-vous pour contacter le contributeur
Soumis le : mardi 4 octobre 2022 - 09:54:27
Dernière modification le : mercredi 2 novembre 2022 - 13:42:13


 Accès restreint
Fichier visible le : 2023-03-16

Connectez-vous pour demander l'accès au fichier



Zhonghang Zhu, Liansheng Wang, Baptiste Magnier, Lei Zhu, Defu Zhang, et al.. Reinforcement Learning Driven Intra-modal and Inter-modal Representation Learning for 3D Medical Image Classification. MICCAI 2022 - The 25th International Conference on Medical Image Computing and Computer Assisted Intervention, Sep 2022, Singapour, Singapore. pp.604-613, ⟨10.1007/978-3-031-16437-8_58⟩. ⟨hal-03790158⟩



Consultations de la notice