Abstract: 

Image segmentation using multi-view images offers substantial advantages in capturing detailed structural and spatial information from complex environments. By incorporating multiple perspectives, such as various angles or imaging planes, segmentation models can integrate complementary data resulting in improved accuracy and robustness. This thesis focuses on the development of sophisticated multi-view segmentation networks, leveraging multi-view constraints to significantly enhance the performance of segmentation in different domains, particularly in medical imaging and remote sensing. These advancements aim to address the challenges of intricate data structures and provide more reliable results. In the medical imaging domain, this thesis proposes a robust segmentation framework for cardiac MRI, designed to enhance the quantification of cardiac function across multiple datasets from various vendors, centers, and diseases. By utilizing convolutional neural networks (CNNs), particularly an optimized version of the Isensee UNet, this framework addresses segmentation variability across vendors like Philips, Siemens, Canon, and GE. Experiments reveal that incorporating a polynomial growth learning rate schedule and the Adam optimizer significantly improved results. Additionally, a novel multi-view SA-LA model is introduced, enabling simultaneous segmentation of the right ventricle in cardiac MRI (CMRI). This multi-encoder, multi-decoder architecture leverages multi-view images acquired at spatial angles, i.e., short-axis and long-axis MRI images. The proposed architecture SA-LA net outer performs on the MICCAI 2021 dataset, demonstrated by high Dice scores and low Hausdorff distances, with strong generalization across unseen cardiac pathologies. In the remote sensing domain, this thesis addresses the challenge of accurate building height estimation from stereo-optical imagery by introducing two novel deep learning architectures. Stereoential Net, the first architecture, employs a multi-scale differential decoding feature fusion mechanism with a modified stereo U-Net (mSUNet) architecture. This approach enhances height estimation by preserving fine structural details in densely populated urban scenes. Using only freely available optical imagery, Stereoential Net significantly outperforms state-of-the-art (SOTA) methods that rely on multi-spectral data, reducing root-mean-square error (RMSE) on both the IEEE Data Fusion Contest 2018 (DFC2018) dataset and the 42-Cities dataset. Building on this, Stereollax Net introduces a stereo parallax-based deep learning model for building height estimation. This architecture integrates key parameters-differential parallax (ΔP), satellite height (h_s), and photo-base (b) within a learning framework, augmented by a Multi-Scale Differential Shortcut Connection (MSDSC) module to refine high-frequency components during decoding. Stereollax Net is evaluated on the 42-Cities and DFC2018 datasets, achieving state-of-the-art performance with fewer data and training parameters compared to existing methods using panchromatic and multispectral imagery. This thesis presents significant advancements in both medical imaging and remote sensing. The proposed multi-view segmentation networks improve the accuracy and generalizability of cardiac MRI segmentation, while the innovative architectures for height estimation push the boundaries of urban analysis using only optical imagery. Together, these contributions demonstrate the potential of multi-view approaches to advance both clinical diagnostics and urban planning applications.

Graphical Abstract:

Final Thesis Defense Committee:   

1. Dr. Hassan-Mohy-Ud-Din (Assistant Professor, PhD Supervisor), Department of Electrical Engineering, Lahore University of Management Sciences (LUMS), Pakistan 

2. Dr. Murtaza Taj(Associate Professor, PhD Supervisor), Department of Computer Science, Lahore University of Management Sciences (LUMS), Pakistan 

3. Dr. Zubair Khalid (Associate Professor, PhD Co-Supervisor), Department of Electrical Engineering, Lahore University of Management Sciences (LUMS), Pakistan 

4. Dr. Imdad Ullah Khan  (Associate Professor, PhD Committee Member), Department of Computer Science, Lahore University of Management Sciences

5. Dr. Mohsen Ali (Associate Professor, External Examiner), Department of Artificial Intelligence,  Information Technology University (ITU), Lahore, Pakistan

List of Publications: 

1. S. Jabbar, S.T. Bukhari, and H. Mohy-ud-Din, Multi-view SA-LA Net: A framework for simultaneous segmentation of RV on multi-view cardiac MR Images. International Workshop on Statistical Atlases and Computational Models of the Heart, Springer, 277–286, 2021.

2. S. Jabbar and M. Taj. Stereoential Net: Deep Network for Learning Building Height Using Stereo Imagery. International Conference on Neural Information Processing, Springer, 478–489, 2023.

3. S. Jabbar and M. Taj. StereollaxNet: Stereo Parallax Based Deep Learning Network For Building Height Estimation. IEEE Transactions on Geoscience and Remote Sensing, 2024.

4. F. Rasool, S. Jabbar, and M. Taj. Build Poly: Structural-Constrained Corner Point Prediction for Building Footprint Extraction. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing (Revisions have been submitted)