TY - JOUR

T1 - Real-time and multimodal brain slice-to-volume registration using CNN

AU - Chen, Zixuan

AU - Xu, Zekai

AU - Yi, Weiwei

AU - Yang, Xin

AU - Hou, Wenguang

AU - Ding, Mingyue

AU - Granichin, Oleg

PY - 2019/11/1

Y1 - 2019/11/1

N2 - Brain slice-to-volume registration is an essential problem in numerous clinical applications; it is aimed at determining the rigid transformation parameters for locating intraoperative 2D slices in the preoperative 3D volume. This task has several challenges: (1) it has to be carried out in real-time; (2) it should not require manual intervention; (3) it requires multimodal operation; (4) it should not require landmark; and (5) it should not require initialization. However, available methods are incapable of overcoming these obstacles simultaneously. In this study, a discrete strategy is introduced, and the slice-to-volume registration is transformed into a global multilabel classification problem. An automatic system using a convolutional neural network (CNN) is applied to implement multilabel classification. The network is trained by slices extracted from the volume and applied in a multibranch manner. Moreover, silhouettes are utilized to unify the multimodal images into the same metric. Centralization and spindle transformation are applied to reduce the dimension of the parameter space, resulting in fewer training samples and less computational complexity. This system exhibits the advantages of not requiring initialization, landmark, or manual intervention; moreover, it can satisfy the real-time and multimodal requirements simultaneously. Experiments on five datasets demonstrate that our system can achieve highly accurate registration and strong robustness; moreover, it is observed to be superior to the state-of-the-art methods by an average of approximately 4.5 mm in terms of the mean target registration error. (C) 2019 Elsevier Ltd. All rights reserved.

AB - Brain slice-to-volume registration is an essential problem in numerous clinical applications; it is aimed at determining the rigid transformation parameters for locating intraoperative 2D slices in the preoperative 3D volume. This task has several challenges: (1) it has to be carried out in real-time; (2) it should not require manual intervention; (3) it requires multimodal operation; (4) it should not require landmark; and (5) it should not require initialization. However, available methods are incapable of overcoming these obstacles simultaneously. In this study, a discrete strategy is introduced, and the slice-to-volume registration is transformed into a global multilabel classification problem. An automatic system using a convolutional neural network (CNN) is applied to implement multilabel classification. The network is trained by slices extracted from the volume and applied in a multibranch manner. Moreover, silhouettes are utilized to unify the multimodal images into the same metric. Centralization and spindle transformation are applied to reduce the dimension of the parameter space, resulting in fewer training samples and less computational complexity. This system exhibits the advantages of not requiring initialization, landmark, or manual intervention; moreover, it can satisfy the real-time and multimodal requirements simultaneously. Experiments on five datasets demonstrate that our system can achieve highly accurate registration and strong robustness; moreover, it is observed to be superior to the state-of-the-art methods by an average of approximately 4.5 mm in terms of the mean target registration error. (C) 2019 Elsevier Ltd. All rights reserved.

KW - Brain slice-to-volume registration

KW - Convolutional neural network

KW - Multimodal

KW - Real-time

KW - System

KW - MEDICAL IMAGE REGISTRATION

KW - OF-THE-ART

UR - http://www.scopus.com/inward/record.url?scp=85065744945&partnerID=8YFLogxK

UR - http://www.mendeley.com/research/realtime-multimodal-brain-slicetovolume-registration-using-cnn

U2 - 10.1016/j.eswa.2019.05.016

DO - 10.1016/j.eswa.2019.05.016

M3 - статья

AN - SCOPUS:85065744945

VL - 133

SP - 86

EP - 96

JO - Expert Systems with Applications

JF - Expert Systems with Applications

SN - 0957-4174

ER -