This dissertation proposes novel solutions for image segmentation problems that vary from macro to micro scale in the medical imaging field. Different segmentation tasks require different approaches and two important categories of optimization based image segmentation methods are Graphical Models and Deformable Models. Each model has its own strengths and weaknesses. In this talk, I will present how we improve these individual models and further, how we combine them to take advantage of each method.

In particular, we propose a novel method of incorporating shape prior into deformable models to segment multiple anatomical regions from brain Magnetic Resonance (MR) images. We hierarchically model the shape ensemble of multiple structures at both global and local levels. We propose a graphical model based segmentation framework for automated and interactive segmentation of neuron structures from Electron Microscopy (EM) images. We use Conditional Random Fields to learn how to segment neuronal structures that are in arbitrary shapes and sizes. Finally, we design a hybrid segmentation method that integrates deformable models and graphical models into a joint framework. Specifically, we show that region-based deformable models can be integrated with Markov Random Fields (MRF). This approach generates accurate segmentation of complex anatomical regions such as cardiac ventricles, and knee joint bones from Computed Tomography (CT) and MR images.