TY - JOUR

T1 - MFDA: Unified Multi-Task Architecture for Cross-Scene Sea Ice Classification

AU - Chen, Yuhan

AU - Yan, Qingyun

AU - Башмачников, Игорь Львович

PY - 2024/11/7

Y1 - 2024/11/7

N2 - Although extensive research has been conducted on retrieving sea ice variables from synthetic aperture radar (SAR) and multimodal remote sensing data, cross-scene retrieval using regional training models remains a significant challenge. Previous studies have employed multi-task learning but have not sufficiently explored the interplay between network architectures and multi-task performance. Moreover, self-supervised learning (SSL) has shown promise in improving tasks with limited training samples, though its potential in sea ice variable retrieval requires further study. To address the challenge of cross-scene retrieval of sea ice variables, we introduce a novel and effective method called Multimodal Fusion Domain Adaptive (MFDA), which combines three key strategies: 1) Employ SSL methods for multimodal data to pre-train the model, improving its noise sensitivity and promoting a hierarchical understanding of multimodality. 2) Propose a unified convolutional and Transformer-based data fusion architecture to enhance the integration of multimodal data and improve semantic understanding. 3) Incorporate a domain adaptation module between the multimodal encoder and the multi-task decoding predictor to facilitate the model's understanding of the semantic gaps between different regional environments. The performance of the proposed MFDA has been extensively evaluated on the Ai4Arctic dataset. The experimental results demonstrate that MFDA achieves superior performance compared to other state-of-the-art sea ice classification approaches for the task of cross-scene sea ice retrieval.

AB - Although extensive research has been conducted on retrieving sea ice variables from synthetic aperture radar (SAR) and multimodal remote sensing data, cross-scene retrieval using regional training models remains a significant challenge. Previous studies have employed multi-task learning but have not sufficiently explored the interplay between network architectures and multi-task performance. Moreover, self-supervised learning (SSL) has shown promise in improving tasks with limited training samples, though its potential in sea ice variable retrieval requires further study. To address the challenge of cross-scene retrieval of sea ice variables, we introduce a novel and effective method called Multimodal Fusion Domain Adaptive (MFDA), which combines three key strategies: 1) Employ SSL methods for multimodal data to pre-train the model, improving its noise sensitivity and promoting a hierarchical understanding of multimodality. 2) Propose a unified convolutional and Transformer-based data fusion architecture to enhance the integration of multimodal data and improve semantic understanding. 3) Incorporate a domain adaptation module between the multimodal encoder and the multi-task decoding predictor to facilitate the model's understanding of the semantic gaps between different regional environments. The performance of the proposed MFDA has been extensively evaluated on the Ai4Arctic dataset. The experimental results demonstrate that MFDA achieves superior performance compared to other state-of-the-art sea ice classification approaches for the task of cross-scene sea ice retrieval.

KW - Synthetic Aperture Radar

KW - arctic sea ice

KW - domain adaptation

KW - masked image modeling

KW - multi-task learning

KW - self-supervised learning

UR - https://ieeexplore.ieee.org/document/10746462

UR - https://www.mendeley.com/catalogue/9901c014-27b8-3b4f-9c88-80d1445784da/

UR - https://www.scopus.com/pages/publications/85208975830

U2 - 10.1109/TGRS.2024.3491190

DO - 10.1109/TGRS.2024.3491190

M3 - Article

VL - 62

JO - IEEE Transactions on Geoscience and Remote Sensing

JF - IEEE Transactions on Geoscience and Remote Sensing

SN - 0196-2892

ER -