On segmentation model for vector valued images and fast iterative solvers

Abstract

In this paper, we propose a new convex variational model for segmentation of vector valued images. The data term of the proposed model is based on the coefficient of variation, which works well in vector valued images having intensity inhomogeneity. Due to convexity of the model, it is independent of the placement of initial contour. Better performance of the proposed model can be seen from experimental results qualitatively and quantitatively. Images in practice are of large sizes, which makes numerical methods more important. In this paper, we also develop fast and stable numerical methods for solution of partial differential equation arisen from the minimization of the proposed model. We have developed a novel multigrid method based on a locally supported smoother. The proposed method is compared with the existing methods in terms of iterations and CPU time for vector valued images having large sizes.

1 Introduction

Segmentation of images refers to dividing an image into disjoint subdomains, which are homogeneous in some sense, i.e., of the same intensity, color, or texture. To detect objects of interest in an image is the basic objective of image segmentation. Many variational models, like edge-based [1], region-based [2], and active contour models [3–5], have already been developed in connection to image segmentation. Our main focus in this paper is on active contour models. The concept of active contours has been applied for detection of objects in a given image $F_{0}$ by applying the techniques of curve evolution. In this approach, an initial curve C is evolved towards the edges of objects in a given image under some conditions/constraints.

In classical snake and active contours models [6], gradient of the given image $F_{0}$ is used to locate edges. The evolving curve is stopped at the object’s boundary by using edge detector function defined in Eq. (1). The function gives minimum value on the edges and maximum value in homogeneous or similar regions. The most common edge detector function is

$$ g\bigl( \vert \nabla F_{0} \vert \bigr)= \frac{1}{1+ \vert \nabla (G_{\sigma}\ast F_{0} ) \vert ^{2}}, $$

(1)

where $G_{\sigma}(x,y)=\frac{1}{2\pi\sigma^{2}}e^{-\frac{x^{2}+y^{2}}{2\sigma ^{2}}}$ is a Gaussian filter. In the case when there is noise in the image, the given image $F_{0}$ is convolved first with Gaussian filter $G_{\sigma}(x,y)$ to make it smooth. Although the function g as given in Eq. (1) is clearly a positive and decreasing function, it cannot be zero on the edges in practice. It may not be possible, therefore, to stop the curve on edges.

To resolve this problem, Chan et al. [4, 7] proposed region based models that do not apply the edge detector function g to detect edges. Instead, the stopping term of the evolving curve depends on the Mumford Shah [2] region-based energy functional. The energy functional of the Chan et al. [4] active contour model (CV model) uses region information which is based on the variance of each region. This model can detect objects in an image whose boundaries are not defined by gradient. It also works well in noisy images and can handle different types of topologies. This model is non-convex, so it may be stuck at local minima for some initial guess. Also this model uses variance as a statistic, so it may not work in images having intensity inhomogeneity. To avoid local extreme of the CV model, Bresson et al. [8] proposed a convex model, which also uses variance as a region statistic. Due to the convexity of the model, it is independent of the initialization and can lead to the global minima. The proposed model of Bresson et al. [8] is given by

$$\begin{aligned} E^{\mathit{FGM}}(\mathrm{c}_{1}, \mathrm{c}_{2}, \xi)=\mu \int_{\Omega} \vert \nabla\xi \vert \,dx\,dy +\gamma \int_{\Omega}r_{1}\bigl((x,y),\mathrm{c}_{1}, \mathrm{c}_{2}\bigr) \xi \,dx\,dy, \end{aligned}$$

(2)

where $\xi\in[-1,1]$ is an extra constraint. Eq. (2) is homogeneous in ξ of degree 1, so it has no stationary point. As a region statistic of this model is variance, so it may not work well in images having intensity inhomogeneity. For segmentation of images having intensity inhomogeneity, Badshah et al. [9] proposed a model that uses a squared coefficient of variation as a region statistic. This model is specially designed for selective segmentation of images having intensity inhomogeneity. Energy functional of Badshah et al. model is given as follows:

$$\begin{aligned} E(\xi,\mathrm{c}_{1},\mathrm{c}_{2})={}&\mu \int_{\Omega}\, d(x,y)g\bigl( \vert \nabla F_{0} \vert \bigr) \bigl\vert \nabla H(\xi) \bigr\vert \\ &{}+\gamma_{1} \int_{\Omega}\frac{ \vert F_{0}-\mathrm{c}_{1} \vert ^{2}}{{\mathrm {c}_{1}}^{2}}H(\xi) +\gamma_{2} \int_{\Omega}\frac{ \vert F_{0}-\mathrm{c}_{2} \vert ^{2}}{{\mathrm {c}_{2}}^{2}}H(-\xi), \end{aligned}$$

(3)

where $d(x,y)$ is the distance metric which incorporates the geometrical constraints and $g(|\nabla F_{0}|)$ is the edge detector function defined in Eq. (1). All the models discussed above have been developed for segmentation of gray level images (for scalar valued images). In this paper, we propose a convex model based on the coefficient of variation for segmentation of vector valued images.

Minimization of the above discussed model leads towards a highly nonlinear partial differential equation, whose solution is always challenging. Most of the methods for solution of these PDEs are based on explicit discretization, which is conditionally stable (see [10] and the references therein). This method requires very small time step, which in result increases the number of iterations and consequently causes the increase of computational cost. Another approach for solution of these PDEs is the semi-implicit (SI) scheme which is unconditionally stable. The SI method works well in 1D problems, while in higher dimension problems it becomes very slow in convergence and computational cost increases. Additive Operator Splitting (AOS) method was developed in [10] for diffusion problem and was implemented for segmentation models in [9, 11, 12]. The AOS method is fast in convergence as compared to the SI method. However, real images (medical images) are usually of large sizes and in such a case these methods are very slow in convergence. To overcome this problem, the multigrid method based on novel smoothers was proposed in [11] for a two-phase segmentation model (CV model) of gray valued images and in [12] for a multi-phase segmentation model of gray valued images. In this paper, we develop AOS and multigrid methods for solution of PDEs arisen from minimization of the proposed model for two-phase segmentation of vector valued images. The multigrid method is based on a new smoother which is supported locally by freezing the differential coefficients locally. Results of the proposed methods are compared with the existing methods (explicit and implicit), and our methods outperformed the existing methods.

Organization of the rest of the paper is as follows: In Sect. 2, related work is discussed. In Sect. 3, our proposed model is described in detail. In Sect. 4, details of the proposed numerical methods for the solution of partial differential equations are given. In Sect. 5, experimental results and comparison with the existing literature are discussed, and in the last section the conclusion of the paper is given.

2 Related work

In this section, we discuss image segmentation models for vector valued images.

2.1 Chan–Vese model for vector valued images (M1)

Segmentation of vector valued images is always challenging. To address this, Chan et al. [7] proposed the following functional:

$$\begin{aligned} E(\mathrm{C},{\mathbf{c}_{1}},{\mathbf{c}_{2}}) ={}& \mu .\mathit{length}(\mathrm{C}) +\frac{1}{N}\sum ^{N}_{l=1} \int_{\mathit{inside}(\mathrm{C})}\gamma^{+}_{l} \vert F_{0,l}-\mathrm{c}_{1,l} \vert ^{2} \\ &{}+\frac{1}{N}\sum^{N}_{l=1} \int_{\mathit{outside}(\mathrm{C})}\gamma^{-}_{l} \vert F_{0,l}-\mathrm {c}_{2,l} \vert ^{2}, \end{aligned}$$

(4)

where C is an evolving curve. $F_{0,l}:{\Omega}\rightarrow \Re^{2}$ is the lth channel of the given image, where $l=1,2,\ldots,N$ denotes the number of channels. ${\mathbf{c}_{1}}= (\mathrm {c}_{11},\mathrm{c}_{12}, \ldots ,\mathrm{c}_{1N})$ and ${\mathbf{c}_{2}}= (\mathrm{c}_{21},\mathrm{c}_{22}, \ldots ,\mathrm{c}_{2N})$ are the average intensity vectors of both sides, i.e., inner and outer of contour C, respectively. $\mu\geq0$, $\nu\geq0$, $\gamma ^{+}_{l}> 0$, $\gamma^{-}_{l}> 0$ are parameters for each channel.

In a level set formulation, the above functional in Eq. (4) may be written as

$$\begin{aligned} E(\xi,\mathbf{c}_{1},\mathbf{c}_{2})={}&\mu \int_{\Omega}\delta\bigl(\xi(x,y)\bigr) \vert \nabla\xi \vert + \int_{\Omega}\frac{1}{N}\sum^{N}_{l=1} \gamma^{+}_{l} \vert F_{0,l}- \mathrm{c}_{1,l} \vert ^{2}H(\xi) \\ &{}+ \int_{\Omega}\frac {1}{N}\sum^{N}_{l=1} \gamma^{-}_{l} \vert F_{0,l}- \mathrm{c}_{2,l} \vert ^{2} \bigl(H(-\xi) \bigr), \end{aligned}$$

(5)

where $H(\xi)$ and $\delta(\xi)$ are Heaviside and Dirac-delta functions, respectively. As $H(\xi)$ is not differentiable at the origin, we consider the regularized $H_{\epsilon}(\xi)$ and corresponding $\delta_{\epsilon}(\xi)$ as given in [4, 11–13] and defined by

$$ H_{\epsilon}(x)=\frac{\pi+2\arctan(\frac{x}{\epsilon})}{2\pi},\qquad\delta _{\epsilon}(x)=\frac{\epsilon}{\pi(\epsilon^{2}+x^{2})}. $$

(6)

Thus the regularized form of Eq. (5) becomes

$$\begin{aligned} E_{\epsilon}(\xi,\mathbf{c}_{1},\mathbf{ c}_{2})={}&\mu \int _{\Omega}\delta_{\epsilon}(\xi) \vert \nabla\xi \vert +\frac{1}{N}\sum^{N}_{l=1} \int_{\Omega}\gamma^{+}_{l} \vert F_{0,l}-\mathrm{c}_{1,l} \vert ^{2}H_{\epsilon}( \xi) \\ &{}+\frac{1}{N}\sum^{N}_{l=1} \int_{\Omega}\gamma^{-}_{l} \vert F_{0,l}-\mathrm{c}_{2,l} \vert ^{2} \bigl(H_{\epsilon}(-\xi) \bigr). \end{aligned}$$

(7)

For optimal values of $\mathrm{c}_{1,l}$ and $\mathrm{c}_{2,l}$, Eq. (7) is minimized with respect to $\mathrm{c}_{1,l}$ and $\mathrm{c}_{2,l}$ by keeping ξ fixed to get the following:

$$\mathrm{c}_{1,l}=\frac{\int_{\Omega}F_{0,l}H_{\epsilon}(\xi)\,dx\,dy}{\int _{\Omega}H_{\epsilon}(\xi)\,dx\,dy},\qquad \mathrm{c}_{2,l}= \frac{\int_{\Omega}F_{0,l} (H_{\epsilon}(-\xi) )\,dx\,dy}{\int_{\Omega}(H_{\epsilon}(-\xi) )\,dx\,dy}. $$

Minimization of Eq. (7) with respect to ξ is obtained by keeping $\mathrm{c}_{1,l}$ and $\mathrm{c}_{2,l}$ fixed, we get the following Euler–Lagrange equation:

$$\begin{aligned} \mu\delta_{\epsilon}(\xi)\nabla\cdot \biggl(\frac{\nabla\xi}{ \vert \nabla\xi \vert } \biggr)-\delta_{\epsilon}(\xi) \frac{1}{N}\sum ^{N}_{l=1} \bigl\{ \gamma^{+}_{l} \vert F_{0,l}-\mathrm{c}_{1,l} \vert ^{2} -\gamma^{-}_{l} \vert F_{0,l}-\mathrm{c}_{2,l} \vert ^{2} \bigr\} =0, \end{aligned}$$

(8)

with Neumann boundary conditions. For implementation of a time marching scheme (semi-implicit scheme), the following unsteady evolution equation is considered:

$$\begin{aligned} \frac{\partial\xi}{\partial t}= \delta_{\epsilon}(\xi) \Biggl\{ \mu\nabla \cdot \biggl(\frac{\nabla\xi}{ \vert \nabla\xi \vert } \biggr) -\frac{1}{N}\sum ^{N}_{l=1}\gamma^{+}_{l} \vert F_{0,l}-\mathrm {c}_{1,l} \vert ^{2} + \frac{1}{N}\sum^{N}_{l=1} \gamma^{-}_{l} \vert F_{0,l}-\mathrm {c}_{2,l} \vert ^{2} \Biggr\} . \end{aligned}$$

(9)

This model can segment vector valued images having homogeneous intensities in different regions and may not give satisfactory results in images having intensity inhomogeneity.

2.2 X. Cai joint model for image restoration and segmentation (M2)

In [14], X. Cai proposed a joint model for restoration and segmentation of vector valued images. For an observed vector valued image $F_{0}=(F_{0, 1}, F_{0, 2},\ldots,F_{0,N})$, they proposed the following functional (we consider in particular a two-phase case):

$$\begin{aligned} E(\nu_{l},c_{l},g_{l})={}&\mu\sum _{l=1}^{N} \int_{\Omega} \vert A_{l}g_{l}-F_{0,l} \vert ^{2}\omega \,dx\,dy +\sum_{k=1}^{2} \int_{\Omega} \vert \nabla\nu_{k} \vert \,dx\,dy \\ &{}+\sum_{k=1}^{2}\sum _{l=1}^{N}\gamma_{l} \int_{\Omega} \vert g_{l} -\mathrm{c}_{k,l} \vert ^{2}\omega\nu_{k} \,dx\,dy, \end{aligned}$$

(10)

with the constraint that

$$\begin{aligned} \sum_{k=1}^{2}\nu_{k}(x)=1,\quad \nu_{k}(x)\in\{0, 1\},\quad\textrm{and}\quad \omega(x)= \textstyle\begin{cases} 1,&\mbox{if } x\in\Omega\backslash\Omega',\\ 0,&\textrm{otherwise}, \end{cases}\displaystyle \end{aligned}$$

(11)

$g_{l}=(g_{1}, g_{2}, \ldots,g_{N})\in L^{2}(\Omega)$ and A is a blur operator. For denoising, $A=I$ is the identity operator. This model is solved by using an alternating minimization algorithm in the following way: To find $c_{k,l}$ as a minimizer of Eq. (10) by keeping $g_{l}$ and $\nu_{k}$ fixed:

$$c_{k,l}=\frac{\int_{\Omega}g_{l}\omega\nu_{k} \,dx}{\int_{\Omega}\omega\nu _{k}},\quad\textrm{where } l=1,2,\ldots,N \textrm{ and } k=1,2. $$

Further, minimization of Eq. (10) with respect to $\nu_{k}$ will give an optimal value of $\nu_{k}$, where $k=1,2$, details can be found in [14]. Minimization of Eq. (10) with respect to $g_{l}$ gives us the following optimal value of $g_{l}$:

$$g_{l}=\bigl(\mu A^{t}A+\lambda\bigr)^{-1} \Biggl( \mu A^{t}F_{0,l}+\lambda\sum_{k=1}^{2}c_{k,l} \nu_{k} \Biggr),\quad\textrm{where } k=1,2. $$

This model jointly restores the noisy image and then segments it. Due to non usage of TV norm, in restoration of intensity inhomogeneous vector valued images, we may loose some information, due to which segmentation results will be affected. From experimental results it can be seen that this model may not give satisfactory segmentation results in images having intensity inhomogeneity. In this paper, we propose a model which will segment vector valued images having intensity inhomogeneity without prior restoration.

3 Proposed model

For segmentation of vector valued images having intensity inhomogeneity, we propose a novel model based on the coefficient of variation. To discuss the proposed model in detail, we first define coefficient of variation (CoV). Data terms based on CoV are used for segmentation of gray images [9, 15]. We first define CoV as follows.

Definition 1

(Coefficient of variation)

Coefficient of variation can be defined as

$$ \operatorname{CoV}=\frac{\mathit{variance}}{\mathit{mean}}. $$

The coefficient of variation gives high values at the edges and low values in the homogeneous regions. Therefore, based on squared CoV, we propose the following energy functional:

$$\begin{aligned} E(\mathrm{C},\mathbf{c}_{1},\mathbf{c}_{2}) ={}& \mu \mathit{length}(\mathrm{C}) +\frac{1}{N}\sum ^{N}_{l=1} \int_{\mathit{inside}(\mathrm{C})}\gamma^{+}_{l} \frac{ \vert F_{0,l}-\mathrm{c}_{1,l} \vert ^{2}}{\mathrm{c}_{1,l}^{2}}\,dx\,dy \\ &{}+\frac{1}{N}\sum^{N}_{l=1} \int_{\mathit{outside}(\mathrm{C})}\gamma^{-}_{l} \frac { \vert F_{0,l}-\mathrm{c}_{2,l} \vert ^{2}}{\mathrm{c}_{2,l}^{2}}\,dx\,dy. \end{aligned}$$

(12)

For the regularized functional in terms of a level set function ξ, we have

$$\begin{aligned} E_{\epsilon}(\xi,\mathbf{c}_{1},\mathbf{c}_{2})={}&\mu \int_{\Omega}\delta_{\epsilon}(\xi) \vert \nabla\xi \vert + \frac{1}{N}\sum^{N}_{l=1} \int_{\Omega}\gamma^{+}_{l} \frac{ \vert F_{0,l}-\mathrm{c}_{1,l} \vert ^{2}}{\mathrm{c}_{1,l}^{2}}H_{\epsilon}(\xi ) \\ &{}+\frac{1}{N}\sum^{N}_{l=1} \int_{\Omega}\gamma^{-}_{l}\frac{ \vert F_{0,l}-\mathrm{c}_{2,l} \vert ^{2}}{\mathrm{c}_{2,l}^{2}} \bigl(H_{\epsilon}(-\xi ) \bigr). \end{aligned}$$

(13)

Optimal values of $\mathrm{c}_{1,l}$ and $\mathrm{c}_{2,l}$ will be the solution of minimization of Eq. (13) with respect to $\mathrm {c}_{1,l}$ and $\mathrm{c}_{2,l}$ and keeping ξ fixed respectively. The values of $\mathrm{c}_{1,l}$ and $\mathrm{c}_{2,l}$ can be updated in the following way:

$$\begin{aligned} &\mathrm{c}_{1,l}=\frac{\int_{\Omega}F_{0,l}^{2}H_{\epsilon}(\xi)\,dx\,dy}{\int _{\Omega}F_{0,l}H_{\epsilon}(\xi)\,dx\,dy},\\ & \mathrm{c}_{2,l}= \frac{\int_{\Omega}F_{0,l}^{2}(H_{\epsilon}(-\xi ))\,dx\,dy}{\int_{\Omega}F_{0,l}(H_{\epsilon}(-\xi))\,dx\,dy}. \end{aligned}$$

An optimal value of ξ is the solution of the following partial differential equation:

$$\begin{aligned} \mu\delta_{\epsilon}(\xi)\nabla\cdot \biggl(\frac{\nabla\xi}{ \vert \nabla\xi \vert } \biggr)-\delta_{\epsilon}(\xi) \frac{1}{N}\sum ^{N}_{l=1} \biggl\{ \gamma^{+}_{l} \frac{ \vert F_{0,l}-\mathrm{c}_{1,l} \vert ^{2}}{\mathrm {c}_{1,l}^{2}} - \gamma^{-}_{l}\frac{ \vert F_{0,l}-\mathrm{c}_{2,l} \vert ^{2}}{\mathrm{c}_{2,l}^{2}} \biggr\} =0, \end{aligned}$$

(14)

with Neumann boundary conditions. The corresponding unsteady state evolution equation is of the form

$$\begin{aligned} \frac{\partial\xi}{\partial t}= \delta_{\epsilon}(\xi) \Biggl\{ \mu\nabla \cdot \biggl(\frac{\nabla\xi}{ \vert \nabla\xi \vert } \biggr) -\frac{1}{N}\sum ^{N}_{l=1}\gamma^{+}_{l}\frac{ \vert F_{0,l}-\mathrm {c}_{1,l} \vert ^{2}}{\mathrm{c}_{1,l}^{2}} + \frac{1}{N}\sum^{N}_{l=1} \gamma^{-}_{l}\frac{ \vert F_{0,l}-\mathrm {c}_{2,l} \vert ^{2}}{\mathrm{c}_{2,l}^{2}} \Biggr\} . \end{aligned}$$

(15)

The partial differential equations (14) and 15) are solved by using different numerical methods which are discussed in section (4).

Convex formulation of the model

The proposed model is non-convex. To develop an alternate convex model, let us consider Eq. (14):

$$\begin{aligned} \mu\delta_{\epsilon}(\xi)\nabla\cdot \biggl(\frac{\nabla\xi}{ \vert \nabla\xi \vert } \biggr)-\delta_{\epsilon}(\xi) \frac{1}{N}\sum ^{N}_{l=1} \biggl\{ \gamma^{+}_{l} \frac{ \vert F_{0,l}-\mathrm{c}_{1,l} \vert ^{2}}{\mathrm {c}_{1,l}^{2}} - \gamma^{-}_{l}\frac{ \vert F_{0,l}-\mathrm{c}_{2,l} \vert ^{2}}{\mathrm{c}_{2,l}^{2}} \biggr\} =0. \end{aligned}$$

(16)

Since $\delta_{\epsilon}(\xi)$ is a non-compactly supported strictly monotonic smooth function [8], we have the following steady state evolution equation:

$$\begin{aligned} \mu\nabla\cdot \biggl(\frac{\nabla\xi}{ \vert \nabla\xi \vert } \biggr)- \frac {1}{N}\sum^{N}_{l=1} \biggl\{ \gamma^{+}_{l}\frac{ \vert F_{0,l}-\mathrm {c}_{1,l} \vert ^{2}}{\mathrm{c}_{1,l}^{2}} - \gamma^{-}_{l} \frac{ \vert F_{0,l}-\mathrm{c}_{2,l} \vert ^{2}}{\mathrm{c}_{2,l}^{2}} \biggr\} =0. \end{aligned}$$

(17)

This is the Euler–Lagrange equation of the following functional:

$$\begin{aligned} E(\xi,\mathbf{c}_{1},\mathbf{c}_{2})={}&\mu \int_{\Omega}\bigl\vert \nabla\xi(x,y) \bigr\vert \,dx\,dy \\ &{}+\frac{1}{N}\sum^{N}_{l=1} \int_{\Omega}\gamma_{l} \biggl(\frac{ \vert F_{0,l}-\mathrm{c}_{1,l} \vert ^{2}}{\mathrm{c}_{1,l}^{2}} - \frac{ \vert F_{0,l}-\mathrm{c}_{2,l} \vert ^{2}}{\mathrm {c}_{2,l}^{2}} \biggr)\xi \,dx\,dy. \end{aligned}$$

(18)

This functional is homogeneous in ξ of degree one, so it has no stationary point, so we restrict the minimization to $-1\le\xi\le1$. Thus we have the following constrained minimization problem:

$$\begin{aligned} \min_{-1\le\xi\le1}E(\xi,\mathbf{c}_{1}, \mathbf{c}_{2})={}&\mu \int_{\Omega}\bigl\vert \nabla\xi(x,y) \bigr\vert \,dx\,dy \\ &{}+\frac{1}{N}\sum^{N}_{l=1} \int_{\Omega}\gamma_{l} \biggl(\frac{ \vert F_{0,l}-\mathrm{c}_{1,l} \vert ^{2}}{\mathrm{c}_{1,l}^{2}} - \frac{ \vert F_{0,l}-\mathrm{c}_{2,l} \vert ^{2}}{\mathrm {c}_{2,l}^{2}} \biggr)\xi \,dx\,dy. \end{aligned}$$

(19)

Minimizers of the constraint functional in Eq. (19) have the same minimizers as of the following unconstraint functional:

$$\begin{aligned} \min_{\xi}E(\xi,\mathbf{c}_{1}, \mathbf{c}_{2})={}&\mu \int_{\Omega} \vert \nabla\xi \vert \,d\Omega + \lambda_{1} \int_{\Omega}p(\xi)\,d\Omega \\ &{}+\frac{1}{N}\sum^{N}_{l=1} \int_{\Omega}\gamma_{l} \biggl(\frac{ \vert F_{0,l}-\mathrm{c}_{1,l} \vert ^{2}}{\mathrm{c}_{1,l}^{2}} -\frac{ \vert F_{0,l}-\mathrm{c}_{2,l} \vert ^{2}}{\mathrm {c}_{2,l}^{2}} \biggr)\xi \,d\Omega , \end{aligned}$$

(20)

where $p(\xi)=\max\{0,2|\xi-0.5|-1\}$ is an exact penalty function and $d\Omega =dx\,dy$. Minimization of the unconstraint functional with respect to ξ gives the following unsteady state evolution equation:

$$\begin{aligned} \frac{\partial\xi}{\partial t}= \mu\nabla\cdot \biggl(\frac{\nabla\xi }{ \vert \nabla\xi \vert } \biggr)- \frac{1}{N}\sum^{N}_{l=1} \gamma_{l} \biggl(\frac{ \vert F_{0,l} -\mathrm{c}_{1,l} \vert ^{2}}{\mathrm{c}_{1,l}^{2}}-\frac{ \vert F_{0,l} -\mathrm{c}_{2,l} \vert ^{2}}{\mathrm{c}_{2,l}^{2}} \biggr)+q(\xi), \end{aligned}$$

(21)

where $q(\xi)$ is the gradient of $p(\xi)$. In the next section we discuss numerical methods for solution of PDEs (15) and (21). These methods are not used for solution of PDE arisen from minimization of models for vector valued images.

4 Numerical methods

In this section we discuss numerical methods for solution of partial differential equations (15) and (21). We describe semi-implicit and additive operator splitting methods for Eq. (15) and the same can be extended to Eq. (21).

4.1 Semi-implicit method

Equation (15) can be written as

$$ \frac{\partial\xi}{\partial t}=\delta_{\epsilon}(\xi) \biggl(\mu\nabla \cdot \biggl(\frac{\nabla\xi}{ \vert \nabla\xi \vert } \biggr) + f(x,y) \biggr), $$

(22)

where

$$ f(x,y)=-\frac{1}{N}\sum^{N}_{l=1} \gamma^{+}_{l}\frac{ \vert F_{0,l}-\mathrm {c}_{1,l} \vert ^{2}}{\mathrm{c}_{1,l}^{2}}+\frac{1}{N}\sum ^{N}_{l=1}\gamma ^{-}_{l}\frac{ \vert F_{0,l}-\mathrm{c}_{2,l} \vert ^{2}}{\mathrm{c}_{2,l}^{2}}. $$

(23)

Consider $m_{1}\times n_{1}$ as size of the given image $F_{0,l}$ and $h_{1}\times h_{2}$ as size of the pixel where $h_{1}=1/m_{1}$ and $h_{2}=1/n_{1}$. A pixel position is represented as $(x_{i},y_{j})= ( (i-\frac{1}{2} )h_{1}, (j-\frac{1}{2} )h_{2} )$. The corresponding difference equation for Eq. (22) is

$$\begin{aligned} \frac{\xi^{(k+1)}_{i,j}-\xi^{(k)}_{i,j}}{\Delta t}= {}&\delta_{\epsilon}\bigl( \xi^{(k)}_{i,j} \bigr) \biggl\{ \mu\frac{1}{h^{2}_{1}} \triangle^{x}_{-} \biggl(\frac{\triangle^{x}_{+}\xi^{(k+1)}_{i,j}}{\sqrt{ (\triangle^{x}_{+}\xi ^{(k)}_{i,j}/h_{1} )^{2}+ (\triangle^{y}_{+}\xi^{(k)}_{i,j}/h_{2} )^{2}+\varepsilon_{1}}} \biggr) \\ &{}+ \mu\frac {1}{h^{2}_{2}}\triangle^{y}_{-} \biggl( \frac{\triangle^{y}_{+}\xi^{(k+1)}_{i,j}}{\sqrt { (\triangle^{x}_{+}\xi^{(k)}_{i,j}/h_{1} )^{2} + (\triangle^{y}_{+}\xi^{(k)}_{i,j}/h_{2} )^{2}+\varepsilon_{1}}} \biggr) \biggr\} +\delta_{\epsilon}\bigl( \xi^{(k)}_{i,j} \bigr)f_{i,j}, \end{aligned}$$

(24)

where the spatial forward and backward operators are defined as follows:

$$ \begin{aligned} &\triangle^{x}_{+} \xi^{(k)}_{i,j}= \xi^{(k)}_{i+1,j}-\xi^{(k)}_{i,j}, \qquad \triangle^{x}_{-} \xi^{(k)}_{i,j}=\xi^{(k)}_{i,j}- \xi^{(k)}_{i-1,j}, \\ &\triangle^{y}_{+} \xi^{(k)}_{i,j}= \xi^{(k)}_{i,j+1}-\xi^{(k)}_{i,j}, \qquad \triangle^{y}_{-} \xi^{(k)}_{i,j}=\xi^{(k)}_{i,j}- \xi^{(k)}_{i,j-1}. \end{aligned} $$

(25)

The matrix form of the difference equation (24) is

$$ \xi^{(k+1)}=\xi^{(k)}+\Delta t\sum _{l=\{x,y\}}A_{l}\bigl(\xi^{(k)}\bigr) \xi ^{(k+1)}+\Delta t b, $$

(26)

where $A_{l}(\xi^{(k)})$ are coefficient matrices for $l=\{x,y\}$ and $b=\delta_{\epsilon}(\xi^{(k)}_{i,j} )f_{i,j}$. To solve Eq. (26) for $\xi^{(k+1)}$, this equation can be written as

$$ \biggl(I-\Delta t \sum_{l=\{x,y\}}A_{l} \bigl(\xi^{(k)}\bigr) \biggr)\xi^{(k+1)}=\xi ^{(k)}+ \Delta tb, $$

(27)

where I is the identity matrix. The system matrix is strictly diagonally dominant, so we have

$$ \xi^{(k+1)}= \biggl(I-\Delta t \sum _{l=\{x,y\}}A_{l}\bigl(\xi^{(k)}\bigr) \biggr)^{-1} \bigl[\xi^{(k)}+\Delta tb \bigr]. $$

(28)

This method is unconditionally stable, i.e., it converges for large time step $(\Delta t)$. Firstly, increasing the time step also increases the condition number of the system matrix, which results in slow convergence. Secondly, in images of higher dimensions, the bandwidth of the system matrix increases, which results in slow convergence and high computational cost. This method also converges very slowly in images of large sizes or in some cases does not converge.

4.2 Additive Operator Splitting (AOS) method

Semi-implicit method for one-dimensional problem results in a tri-diagonal system of equations, which can be easily solved by any iterative method. The Additive Operator Splitting (AOS) method divides an m-dimensional differential operator into m one-dimensional problems, then each one is solved by using a semi-implicit method, and final solution is obtained by taking their average. In this method we solve m one-dimensional problems through the semi-implicit method. Mathematically, this can be written as follows:

$$ \xi^{(k+1)}=\frac{1}{m}\sum _{l=\{x,y\}} \bigl(I-\Delta tm A_{l}\bigl(\xi ^{(k)}\bigr) \bigr)^{-1} \bigl[\xi^{(k)}+\Delta tb \bigr]. $$

(29)

The AOS method is more efficient than the semi-implicit method in higher dimension and is unconditionally stable. This method works well in images of moderate size, but in images having large size it converges very slowly. To overcome this problem, we develop a multigrid method for solution of PDE (14).

4.3 Multigrid (MG) method

In this section we discuss the multigrid method [11, 12, 16, 17] for solution of PDE given in Eq. (21). The steady state of Eq. (21) can be written as

$$\begin{aligned} \mu\nabla\cdot \biggl(\frac{\nabla\xi}{ \vert \nabla\xi \vert } \biggr)- \frac {1}{N}\sum ^{N}_{l=1}\gamma_{l} \biggl( \frac{ \vert F_{0,l} -\mathrm{c}_{1,l} \vert ^{2}}{\mathrm{c}_{1,l}^{2}}-\frac{ \vert F_{0,l} -\mathrm{c}_{2,l} \vert ^{2}}{\mathrm{c}_{2,l}^{2}} \biggr)+q(\xi)=0, \end{aligned}$$

(30)

which may be written as

$$\begin{aligned} \mu\nabla\cdot \biggl(\frac{\nabla\xi}{ \vert \nabla\xi \vert } \biggr)= \frac {1}{N}\sum ^{N}_{l=1}\gamma_{l} \biggl( \frac{ \vert F_{0,l} -\mathrm{c}_{1,l} \vert ^{2}}{\mathrm{c}_{1,l}^{2}}-\frac{ \vert F_{0,l} -\mathrm{c}_{2,l} \vert ^{2}}{\mathrm{c}_{2,l}^{2}} \biggr)+q(\xi). \end{aligned}$$

(31)

The corresponding difference equation is as follows:

$$\begin{aligned} & \frac{1}{h^{2}_{1}}\triangle^{x}_{-} \biggl( \frac{\triangle^{x}_{+} \xi_{i,j}}{\sqrt{ (\triangle^{x}_{+}\xi_{i,j}/h_{1} )^{2} + (\triangle^{y}_{+}\xi_{i,j}/h_{2} )^{2}+\varepsilon_{1}}} \biggr) \\ &\quad {}+ \frac{1}{h^{2}_{2}}\triangle^{y}_{-} \biggl(\frac{\triangle^{y}_{+} \xi_{i,j}}{\sqrt{ (\triangle^{x}_{+}\xi_{i,j}/h_{1} )^{2} + (\triangle^{y}_{+}\xi_{i,j}/h_{2} )^{2}+\varepsilon_{1}}} \biggr) =\bar{f}_{i,j}, \end{aligned}$$

(32)

where

$$\bar{f}_{i,j}=\frac{1}{\mu} \Biggl(\frac{1}{N}\sum ^{N}_{l=1}\gamma _{l} \biggl( \frac{ \vert F_{0,l} -\mathrm{c}_{1,l} \vert ^{2}}{\mathrm{c}_{1,l}^{2}}-\frac{ \vert F_{0,l} -\mathrm{c}_{2,l} \vert ^{2}}{\mathrm{c}_{2,l}^{2}} \biggr)+q(\xi) \Biggr)_{i,j}. $$

Freezing coefficients as done in [11, 12], we have the following system of equations:

$$ N(\xi)=b, $$

(33)

where $N(\xi)$ is the coefficient matrix of the left-hand side of Eq. (32) after it is linearized.

V-cycle of a multigrid algorithm

For the system of equations given in (33), the V-cycle multigrid is described in Algorithm 1. The smoother is performed $\nu_{1}$ and $\nu_{1}$ number of pre- and post-smoothing steps respectively, $I_{h}^{2h}$ is the restriction operator, $I_{2h}^{h}$ is the prolongation operator, $N^{2h}$ is the coarse grid operator.

Algorithm 1

V-cycle multigrid algorithm

Choice of smoother

In the multigrid method, a smoother is the main ingredient, so it is important to discuss it. As discussed above, to get Eq. (33), the coefficients in Eq. (32) are frozen. In detail, Eq. (32) can be written as follows:

$$\begin{aligned} & \biggl(\frac{\triangle^{x}_{+} \xi_{i,j}}{\sqrt{ (\triangle^{x}_{+}\xi_{i,j} )^{2} +\gamma (\triangle^{y}_{+}\xi_{i,j} )^{2}+\overline{\varepsilon}_{1}}} -\frac{\triangle^{x}_{+} \xi_{i-1,j}}{\sqrt{ (\triangle^{x}_{+}\xi_{i-1,j} )^{2} +\gamma (\triangle^{y}_{+}\xi_{i-1,j} )^{2}+\overline{\varepsilon }_{1}}} \biggr) \\ &\quad {}+\gamma^{2} \biggl(\frac{\triangle^{y}_{+} \xi_{i,j}}{\sqrt{ (\triangle^{x}_{+}\xi_{i,j} )^{2} +\gamma (\triangle^{y}_{+}\xi_{i,j} )^{2}+\overline{\varepsilon}_{1}}} -\frac{\triangle^{y}_{+} \xi_{i,j-1}}{\sqrt{ (\triangle^{x}_{+}\xi_{i,j-1} )^{2} +\gamma (\triangle^{y}_{+}\xi_{i,j-1} )^{2}+\overline{\varepsilon }_{1}}} \biggr) = \bar{f}_{i,j}, \end{aligned}$$

(34)

where $\gamma=h_{1}/h_{2}, \overline{\varepsilon}_{1}=h_{1}^{2}\varepsilon_{1}$. The coefficients that are frozen in the local linearization are given below:

$$\begin{aligned} &D(\xi_{i,j})=\frac{1}{\sqrt{ (\triangle^{x}_{+}\xi_{i,j} )^{2} +\gamma (\triangle^{y}_{+}\xi_{i,j} )^{2}+\overline{\varepsilon }_{1}}}, \\ &D(\xi_{i-1,j})=\frac{1}{\sqrt{ (\triangle^{x}_{+}\xi_{i-1,j} )^{2} +\gamma (\triangle^{y}_{+}\xi_{i-1,j} )^{2}+\overline{\varepsilon }_{1}}}, \\ &D(\xi_{i,j-1})=\frac{1}{\sqrt{ (\triangle^{x}_{+}\xi_{i,j-1} )^{2} +\gamma (\triangle^{y}_{+}\xi_{i,j-1} )^{2}+\overline{\varepsilon}_{1}}}. \end{aligned}$$

(35)

The following form is thus obtained:

$$\begin{aligned} & \bigl\{ D(\xi_{i,j}) (\xi_{i+1,j}- \xi_{i,j} )-D(\xi_{i-1,j}) (\xi_{i,j}-\xi_{i-1,j} ) \bigr\} \\ &\quad {}+\gamma^{2} \bigl\{ D(\xi_{i,j}) (\xi_{i,j+1}- \xi_{i,j} )-D(\xi _{i,j-1}) (\xi_{i,j}-\xi_{i,j-1} ) \bigr\} =\bar{f}_{i,j}. \end{aligned}$$

(36)

Let ζ be an approximation to ξ in the previous iteration, then Eq. (36) has only one local unknown $\xi_{i,j}$. For clarity, we have shown it in bold.

$$\begin{aligned} & \bigl\{ D(\zeta_{i,j}) (\zeta_{i+1,j}- \xi_{i,j} )-D(\zeta _{i-1,j}) (\xi_{i,j}- \zeta_{i-1,j} ) \bigr\} \\ &\quad {}+\gamma^{2} \bigl\{ D(\zeta_{i,j}) (\zeta_{i,j+1}- \xi_{i,j} )-D(\zeta _{i,j-1}) (\xi_{i,j}- \zeta_{i,j-1} ) \bigr\} =\bar{f}_{i,j}. \end{aligned}$$

(37)

Our proposed method solves the above equation for $\xi_{i,j}$ to update $\zeta_{i,j}$, which leads us to updated coefficients (35) and further iterations. See Algorithm 2.

Algorithm 2

Smoother

Here, the coefficients are first updated locally and are stored for relaxation use. In this way Eq. (37) becomes linear and easy to solve.

5 Experimental results and discussion

In this section, we give experimental comparison of the proposed model and method with the existing models and methods qualitatively and quantitatively.

Qualitative comparison

We first give qualitative comparison of the proposed model by testing it on different synthetic and real images having intensity inhomogeneity. It can be seen from the experimental results that the proposed model outperforms the existing models.

In Fig. 1, the experimental results of CV model for vector valued images (M1), Cai model (M2), and proposed model are presented. All the models are tested on a synthetic color image having different objects with different levels of intensities. The proposed model has outperformed the other two models. In Fig. 1(a), the original image with initial contour is given. Figure 1(b) is the segmented result of M1, Fig. 1(c) is the final segmented result of M2, and Fig. 1(d) is the final segmented result of the proposed model.

Figure 1

(a) Given image with initial contour, (b) Result of model M1 [7], Jaccard similarity index (JSI) = 0.7883, (c) Result of Cai model M2 [14], JSI = 0.7882, (d) Result of the proposed model, JSI = 1

In Fig. 2, the experimental results of M1, M2, and the proposed model are given. All models are implemented on a color image with multi objects having different intensity variations. Segmentation results of the proposed model are far better than the results of other two existing models. Fig. 2(a) is the original image with initial contour. Figure 2(b) is the final segmented result of M1 after 500 iterations, and clearly the results are not satisfactory; Fig. 2(c) is the final result of M2 after 500 iterations, the image is not properly segmented; and Fig. 2(d) is the final segmented result of the proposed model after 33 iterations, clearly the image is properly segmented.

Figure 2

(a) Given image with initial contour, (b) Result of model M1 [7], JSI = 0.8842, (c) Result of Cai model M2 [14], JSI = 0.6785, (d) Result of the proposed model, JSI = 1

In Fig. 3, the experimental results of M1, M2, and the proposed model are given. All models are implemented on a real biological color cell image which has multi objects having inhomogeneous intensities within the objects. Figure 3(a) is the original image with initial contour. Final segmented result of M1 is given in Fig. 3(b), which is obtained after 700 iterations, and the results are not satisfactory. Figure 3(c) shows final segmented result of M2 after 500 iterations, which are better than the results of M1 but not that much satisfactory. The final segmented result of the proposed model is given in Fig. 3(d), where the image is segmented properly. From all these results it can be seen that the proposed model works well in color images having intensity inhomogeneity. We remark that the proposed model may not work very well in noisy images with intensity inhomogeneity.

Figure 3

(a) Given image with initial contour, (b) Result of model M1 [7], JSI = 0.7684, (c) Result of Cai model M2 [14], JSI = 0.6828, (d) Result of the proposed model JSI = 1

Quantitative comparison

Here we give quantitative comparison of our proposed model through Jaccard similarity index (JSI) with the existing CV model (M1) and Cai model (M2). In Table 1, quantitative results of the proposed model are given in terms of the number of iterations, CPU time, and JSI. Clearly the proposed model is taking fewer iterations and converges fast with better JSI. Image size used in all experiments is $256\times256$ and the numerical method used for the solution of PDE in the table is additive operator splitting (AOS).

Table 1 Quantitative comparison of the proposed model with CV vector valued [7] (M1) and Cai model [14] (M2). The solution method used for PDE is Additive Operator Splitting (AOS). Size of each image used here is ($256\times256$)

We observe that all the methods become slow in convergence as we increase the size of an image. To tackle this issue, we have proposed multigrid (MG) method, whose results are given in Table 2. In Table 2, we have given the number of iterations and corresponding CPU time for images having different sizes and different methods. A real color image given in Fig. 4(c) is used for all computations given in Table 2. From the table, it can be observed that the MG method has produced very good results in terms of the number of cycles and CPU time.

Figure 4

Original images are in column 1 and column 3 and their respective segmented results using our proposed model are given in column 2 and column 4. Results of real images are given in 1st row, while those of synthetic, biological cells, and medical images are given in 2nd, 3rd, and 4th rows, respectively

Table 2 Comparison of the proposed multigrid method (4.3) with SI (4.1), AOS (4.2) in terms of iterations and CPU time for an image with different sizes

Furthermore, in Fig. 4, we have tested the proposed model on different types of synthetic and real images. These results show effectiveness of the proposed model in different types of synthetic and real images. Fig. 4(b), (d) are final segmented results of natural images, Fig. 4(f), (h) are segmented results of synthetic or artificial images, Fig. 4(j), (l) are segmented results of biological cell images, and Fig. 4(n), (p) are segmented results of medical MR images. We observe that our model segments images with intensity variation or inhomogeneity efficiently in the objects like in Fig. 4(e), (g), (p). It can also segment images having inhomogeneity in their background as in Fig. 4(k).

6 Conclusion

The proposed model is based on the coefficient of variation, which works well in images having intensity inhomogeneity. The model is then formulated in convex framework to make it independent of initial contour. The model is minimized through variation to get a partial differential equation which is solved by using AOS method for images of moderate sizes. For images of large sizes, we have proposed a multigrid method based on locally supported smoother. The proposed convex model and multigrid method are compared with the exiting models and methods, and it was found that the proposed model and method outperformed the exiting models and methods.