- Research
- Open access
- Published:

# On segmentation model for vector valued images and fast iterative solvers

*Advances in Difference Equations*
**volumeÂ 2018**, ArticleÂ number:Â 221 (2018)

## Abstract

In this paper, we propose a new convex variational model for segmentation of vector valued images. The data term of the proposed model is based on the coefficient of variation, which works well in vector valued images having intensity inhomogeneity. Due to convexity of the model, it is independent of the placement of initial contour. Better performance of the proposed model can be seen from experimental results qualitatively and quantitatively. Images in practice are of large sizes, which makes numerical methods more important. In this paper, we also develop fast and stable numerical methods for solution of partial differential equation arisen from the minimization of the proposed model. We have developed a novel multigrid method based on a locally supported smoother. The proposed method is compared with the existing methods in terms of iterations and CPU time for vector valued images having large sizes.

## 1 Introduction

Segmentation of images refers to dividing an image into disjoint subdomains, which are homogeneous in some sense, i.e., of the same intensity, color, or texture. To detect objects of interest in an image is the basic objective of image segmentation. Many variational models, like edge-based [1], region-based [2], and active contour models [3â€“5], have already been developed in connection to image segmentation. Our main focus in this paper is on active contour models. The concept of active contours has been applied for detection of objects in a given image \(F_{0}\) by applying the techniques of curve evolution. In this approach, an initial curve C is evolved towards the edges of objects in a given image under some conditions/constraints.

In classical snake and active contours models [6], gradient of the given image \(F_{0}\) is used to locate edges. The evolving curve is stopped at the objectâ€™s boundary by using edge detector function defined in Eq.Â (1). The function gives minimum value on the edges and maximum value in homogeneous or similar regions. The most common edge detector function is

where \(G_{\sigma}(x,y)=\frac{1}{2\pi\sigma^{2}}e^{-\frac{x^{2}+y^{2}}{2\sigma ^{2}}}\) is a Gaussian filter. In the case when there is noise in the image, the given image \(F_{0}\) is convolved first with Gaussian filter \(G_{\sigma}(x,y)\) to make it smooth. Although the function *g* as given in Eq.Â (1) is clearly a positive and decreasing function, it cannot be zero on the edges in practice. It may not be possible, therefore, to stop the curve on edges.

To resolve this problem, Chan *et al.* [4, 7] proposed region based models that do not apply the edge detector function *g* to detect edges. Instead, the stopping term of the evolving curve depends on the Mumford Shah [2] region-based energy functional. The energy functional of the Chan *et al.* [4] active contour model (CV model) uses region information which is based on the variance of each region. This model can detect objects in an image whose boundaries are not defined by gradient. It also works well in noisy images and can handle different types of topologies. This model is non-convex, so it may be stuck at local minima for some initial guess. Also this model uses variance as a statistic, so it may not work in images having intensity inhomogeneity. To avoid local extreme of the CV model, Bresson *et al.* [8] proposed a convex model, which also uses variance as a region statistic. Due to the convexity of the model, it is independent of the initialization and can lead to the global minima. The proposed model of Bresson *et al.* [8] is given by

where \(\xi\in[-1,1]\) is an extra constraint. Eq.Â (2) is homogeneous in *Î¾* of degree 1, so it has no stationary point. As a region statistic of this model is variance, so it may not work well in images having intensity inhomogeneity. For segmentation of images having intensity inhomogeneity, Badshah *et al.* [9] proposed a model that uses a squared coefficient of variation as a region statistic. This model is specially designed for selective segmentation of images having intensity inhomogeneity. Energy functional of Badshah *et al.* model is given as follows:

where \(d(x,y)\) is the distance metric which incorporates the geometrical constraints and \(g(|\nabla F_{0}|)\) is the edge detector function defined in Eq.Â (1). All the models discussed above have been developed for segmentation of gray level images (for scalar valued images). In this paper, we propose a convex model based on the coefficient of variation for segmentation of vector valued images.

Minimization of the above discussed model leads towards a highly nonlinear partial differential equation, whose solution is always challenging. Most of the methods for solution of these PDEs are based on explicit discretization, which is conditionally stable (see [10] and the references therein). This method requires very small time step, which in result increases the number of iterations and consequently causes the increase of computational cost. Another approach for solution of these PDEs is the semi-implicit (SI) scheme which is unconditionally stable. The SI method works well in 1D problems, while in higher dimension problems it becomes very slow in convergence and computational cost increases. Additive Operator Splitting (AOS) method was developed in [10] for diffusion problem and was implemented for segmentation models in [9, 11, 12]. The AOS method is fast in convergence as compared to the SI method. However, real images (medical images) are usually of large sizes and in such a case these methods are very slow in convergence. To overcome this problem, the multigrid method based on novel smoothers was proposed in [11] for a two-phase segmentation model (CV model) of gray valued images and in [12] for a multi-phase segmentation model of gray valued images. In this paper, we develop AOS and multigrid methods for solution of PDEs arisen from minimization of the proposed model for two-phase segmentation of vector valued images. The multigrid method is based on a new smoother which is supported locally by freezing the differential coefficients locally. Results of the proposed methods are compared with the existing methods (explicit and implicit), and our methods outperformed the existing methods.

Organization of the rest of the paper is as follows: In Sect. 2, related work is discussed. In Sect.Â 3, our proposed model is described in detail. In Sect.Â 4, details of the proposed numerical methods for the solution of partial differential equations are given. In Sect.Â 5, experimental results and comparison with the existing literature are discussed, and in the last section the conclusion of the paper is given.

## 2 Related work

In this section, we discuss image segmentation models for vector valued images.

### 2.1 Chanâ€“Vese model for vector valued images (M1)

Segmentation of vector valued images is always challenging. To address this, Chan *et al.* [7] proposed the following functional:

where C is an evolving curve. \(F_{0,l}:{\Omega}\rightarrow \Re^{2}\) is the *l*th channel of the given image, where \(l=1,2,\ldots,N\) denotes the number of channels. \({\mathbf{c}_{1}}= (\mathrm {c}_{11},\mathrm{c}_{12}, \ldots ,\mathrm{c}_{1N})\) and \({\mathbf{c}_{2}}= (\mathrm{c}_{21},\mathrm{c}_{22}, \ldots ,\mathrm{c}_{2N})\) are the average intensity vectors of both sides, i.e., inner and outer of contour C, respectively. \(\mu\geq0\), \(\nu\geq0\), \(\gamma ^{+}_{l}> 0\), \(\gamma^{-}_{l}> 0\) are parameters for each channel.

In a level set formulation, the above functional in Eq.Â (4) may be written as

where \(H(\xi)\) and \(\delta(\xi)\) are Heaviside and Dirac-delta functions, respectively. As \(H(\xi)\) is not differentiable at the origin, we consider the regularized \(H_{\epsilon}(\xi)\) and corresponding \(\delta_{\epsilon}(\xi)\) as given in [4, 11â€“13] and defined by

Thus the regularized form of Eq.Â (5) becomes

For optimal values of \(\mathrm{c}_{1,l}\) and \(\mathrm{c}_{2,l}\), Eq.Â (7) is minimized with respect to \(\mathrm{c}_{1,l}\) and \(\mathrm{c}_{2,l}\) by keeping *Î¾* fixed to get the following:

Minimization of Eq.Â (7) with respect to *Î¾* is obtained by keeping \(\mathrm{c}_{1,l}\) and \(\mathrm{c}_{2,l}\) fixed, we get the following Eulerâ€“Lagrange equation:

with Neumann boundary conditions. For implementation of a time marching scheme (semi-implicit scheme), the following unsteady evolution equation is considered:

This model can segment vector valued images having homogeneous intensities in different regions and may not give satisfactory results in images having intensity inhomogeneity.

### 2.2 X. Cai joint model for image restoration and segmentation (M2)

In [14], X. Cai proposed a joint model for restoration and segmentation of vector valued images. For an observed vector valued image \(F_{0}=(F_{0, 1}, F_{0, 2},\ldots,F_{0,N})\), they proposed the following functional (we consider in particular a two-phase case):

with the constraint that

\(g_{l}=(g_{1}, g_{2}, \ldots,g_{N})\in L^{2}(\Omega)\) and *A* is a blur operator. For denoising, \(A=I\) is the identity operator. This model is solved by using an alternating minimization algorithm in the following way: To find \(c_{k,l}\) as a minimizer of Eq.Â (10) by keeping \(g_{l}\) and \(\nu_{k}\) fixed:

Further, minimization of Eq.Â (10) with respect to \(\nu_{k}\) will give an optimal value of \(\nu_{k}\), where \(k=1,2\), details can be found in [14]. Minimization of Eq.Â (10) with respect to \(g_{l}\) gives us the following optimal value of \(g_{l}\):

This model jointly restores the noisy image and then segments it. Due to non usage of TV norm, in restoration of intensity inhomogeneous vector valued images, we may loose some information, due to which segmentation results will be affected. From experimental results it can be seen that this model may not give satisfactory segmentation results in images having intensity inhomogeneity. In this paper, we propose a model which will segment vector valued images having intensity inhomogeneity without prior restoration.

## 3 Proposed model

For segmentation of vector valued images having intensity inhomogeneity, we propose a novel model based on the coefficient of variation. To discuss the proposed model in detail, we first define coefficient of variation (CoV). Data terms based on CoV are used for segmentation of gray images [9, 15]. We first define CoV as follows.

### Definition 1

(Coefficient of variation)

Coefficient of variation can be defined as

The coefficient of variation gives high values at the edges and low values in the homogeneous regions. Therefore, based on squared CoV, we propose the following energy functional:

For the regularized functional in terms of a level set function *Î¾*, we have

Optimal values of \(\mathrm{c}_{1,l}\) and \(\mathrm{c}_{2,l}\) will be the solution of minimization of Eq.Â (13) with respect to \(\mathrm {c}_{1,l}\) and \(\mathrm{c}_{2,l}\) and keeping *Î¾* fixed respectively. The values of \(\mathrm{c}_{1,l}\) and \(\mathrm{c}_{2,l}\) can be updated in the following way:

An optimal value of *Î¾* is the solution of the following partial differential equation:

with Neumann boundary conditions. The corresponding unsteady state evolution equation is of the form

The partial differential equations (14) and 15) are solved by using different numerical methods which are discussed in section (4).

*Convex formulation of the model*

The proposed model is non-convex. To develop an alternate convex model, let us consider Eq.Â (14):

Since \(\delta_{\epsilon}(\xi)\) is a non-compactly supported strictly monotonic smooth function [8], we have the following steady state evolution equation:

This is the Eulerâ€“Lagrange equation of the following functional:

This functional is homogeneous in *Î¾* of degree one, so it has no stationary point, so we restrict the minimization to \(-1\le\xi\le1\). Thus we have the following constrained minimization problem:

Minimizers of the constraint functional in Eq.Â (19) have the same minimizers as of the following unconstraint functional:

where \(p(\xi)=\max\{0,2|\xi-0.5|-1\}\) is an exact penalty function and \(d\Omega =dx\,dy\). Minimization of the unconstraint functional with respect to *Î¾* gives the following unsteady state evolution equation:

where \(q(\xi)\) is the gradient of \(p(\xi)\). In the next section we discuss numerical methods for solution of PDEs (15) and (21). These methods are not used for solution of PDE arisen from minimization of models for vector valued images.

## 4 Numerical methods

In this section we discuss numerical methods for solution of partial differential equations (15) and (21). We describe semi-implicit and additive operator splitting methods for Eq.Â (15) and the same can be extended to Eq.Â (21).

### 4.1 Semi-implicit method

EquationÂ (15) can be written as

where

Consider \(m_{1}\times n_{1}\) as size of the given image \(F_{0,l}\) and \(h_{1}\times h_{2}\) as size of the pixel where \(h_{1}=1/m_{1}\) and \(h_{2}=1/n_{1}\). A pixel position is represented as \((x_{i},y_{j})= ( (i-\frac{1}{2} )h_{1}, (j-\frac{1}{2} )h_{2} )\). The corresponding difference equation for Eq.Â (22) is

where the spatial forward and backward operators are defined as follows:

The matrix form of the difference equation (24) is

where \(A_{l}(\xi^{(k)})\) are coefficient matrices for \(l=\{x,y\}\) and \(b=\delta_{\epsilon}(\xi^{(k)}_{i,j} )f_{i,j}\). To solve Eq.Â (26) for \(\xi^{(k+1)}\), this equation can be written as

where *I* is the identity matrix. The system matrix is strictly diagonally dominant, so we have

This method is unconditionally stable, i.e., it converges for large time step \((\Delta t)\). Firstly, increasing the time step also increases the condition number of the system matrix, which results in slow convergence. Secondly, in images of higher dimensions, the bandwidth of the system matrix increases, which results in slow convergence and high computational cost. This method also converges very slowly in images of large sizes or in some cases does not converge.

### 4.2 Additive Operator Splitting (AOS) method

Semi-implicit method for one-dimensional problem results in a tri-diagonal system of equations, which can be easily solved by any iterative method. The Additive Operator Splitting (AOS) method divides an *m*-dimensional differential operator into *m* one-dimensional problems, then each one is solved by using a semi-implicit method, and final solution is obtained by taking their average. In this method we solve *m* one-dimensional problems through the semi-implicit method. Mathematically, this can be written as follows:

The AOS method is more efficient than the semi-implicit method in higher dimension and is unconditionally stable. This method works well in images of moderate size, but in images having large size it converges very slowly. To overcome this problem, we develop a multigrid method for solution of PDE (14).

### 4.3 Multigrid (MG) method

In this section we discuss the multigrid method [11, 12, 16, 17] for solution of PDE given in Eq.Â (21). The steady state of Eq.Â (21) can be written as

which may be written as

The corresponding difference equation is as follows:

where

Freezing coefficients as done in [11, 12], we have the following system of equations:

where \(N(\xi)\) is the coefficient matrix of the left-hand side of Eq.Â (32) after it is linearized.

*V-cycle of a multigrid algorithm*

For the system of equations given in (33), the V-cycle multigrid is described in AlgorithmÂ 1. The smoother is performed \(\nu_{1}\) and \(\nu_{1}\) number of pre- and post-smoothing steps respectively, \(I_{h}^{2h}\) is the restriction operator, \(I_{2h}^{h}\) is the prolongation operator, \(N^{2h}\) is the coarse grid operator.

*Choice of smoother*

In the multigrid method, a smoother is the main ingredient, so it is important to discuss it. As discussed above, to get Eq.Â (33), the coefficients in Eq.Â (32) are frozen. In detail, Eq.Â (32) can be written as follows:

where \(\gamma=h_{1}/h_{2}, \overline{\varepsilon}_{1}=h_{1}^{2}\varepsilon_{1}\). The coefficients that are frozen in the local linearization are given below:

The following form is thus obtained:

Let *Î¶* be an approximation to *Î¾* in the previous iteration, then Eq.Â (36) has only one local unknown \(\xi_{i,j}\). For clarity, we have shown it in bold.

Our proposed method solves the above equation for \(\xi_{i,j}\) to update \(\zeta_{i,j}\), which leads us to updated coefficients (35) and further iterations. See Algorithm 2.

Here, the coefficients are first updated locally and are stored for relaxation use. In this way Eq.Â (37) becomes linear and easy to solve.

## 5 Experimental results and discussion

In this section, we give experimental comparison of the proposed model and method with the existing models and methods qualitatively and quantitatively.

*Qualitative comparison*

We first give qualitative comparison of the proposed model by testing it on different synthetic and real images having intensity inhomogeneity. It can be seen from the experimental results that the proposed model outperforms the existing models.

In Fig.Â 1, the experimental results of CV model for vector valued images (M1), Cai model (M2), and proposed model are presented. All the models are tested on a synthetic color image having different objects with different levels of intensities. The proposed model has outperformed the other two models. In Fig.Â 1(a), the original image with initial contour is given. FigureÂ 1(b) is the segmented result of M1, Fig.Â 1(c) is the final segmented result of M2, and Fig.Â 1(d) is the final segmented result of the proposed model.

In Fig.Â 2, the experimental results of M1, M2, and the proposed model are given. All models are implemented on a color image with multi objects having different intensity variations. Segmentation results of the proposed model are far better than the results of other two existing models. Fig.Â 2(a) is the original image with initial contour. FigureÂ 2(b) is the final segmented result of M1 after 500 iterations, and clearly the results are not satisfactory; Fig.Â 2(c) is the final result of M2 after 500 iterations, the image is not properly segmented; and Fig.Â 2(d) is the final segmented result of the proposed model after 33 iterations, clearly the image is properly segmented.

In Fig.Â 3, the experimental results of M1, M2, and the proposed model are given. All models are implemented on a real biological color cell image which has multi objects having inhomogeneous intensities within the objects. FigureÂ 3(a) is the original image with initial contour. Final segmented result of M1 is given in Fig.Â 3(b), which is obtained after 700 iterations, and the results are not satisfactory. FigureÂ 3(c) shows final segmented result of M2 after 500 iterations, which are better than the results of M1 but not that much satisfactory. The final segmented result of the proposed model is given in Fig.Â 3(d), where the image is segmented properly. From all these results it can be seen that the proposed model works well in color images having intensity inhomogeneity. We remark that the proposed model may not work very well in noisy images with intensity inhomogeneity.

*Quantitative comparison*

Here we give quantitative comparison of our proposed model through Jaccard similarity index (JSI) with the existing CV model (M1) and Cai model (M2). In TableÂ 1, quantitative results of the proposed model are given in terms of the number of iterations, CPU time, and JSI. Clearly the proposed model is taking fewer iterations and converges fast with better JSI. Image size used in all experiments is \(256\times256\) and the numerical method used for the solution of PDE in the table is additive operator splitting (AOS).

We observe that all the methods become slow in convergence as we increase the size of an image. To tackle this issue, we have proposed multigrid (MG) method, whose results are given in TableÂ 2. In TableÂ 2, we have given the number of iterations and corresponding CPU time for images having different sizes and different methods. A real color image given in Fig.Â 4(c) is used for all computations given in TableÂ 2. From the table, it can be observed that the MG method has produced very good results in terms of the number of cycles and CPU time.

Furthermore, in Fig.Â 4, we have tested the proposed model on different types of synthetic and real images. These results show effectiveness of the proposed model in different types of synthetic and real images. Fig.Â 4(b), (d) are final segmented results of natural images, Fig.Â 4(f), (h) are segmented results of synthetic or artificial images, Fig.Â 4(j), (l) are segmented results of biological cell images, and Fig.Â 4(n), (p) are segmented results of medical MR images. We observe that our model segments images with intensity variation or inhomogeneity efficiently in the objects like in Fig.Â 4(e), (g), (p). It can also segment images having inhomogeneity in their background as in Fig.Â 4(k).

## 6 Conclusion

The proposed model is based on the coefficient of variation, which works well in images having intensity inhomogeneity. The model is then formulated in convex framework to make it independent of initial contour. The model is minimized through variation to get a partial differential equation which is solved by using AOS method for images of moderate sizes. For images of large sizes, we have proposed a multigrid method based on locally supported smoother. The proposed convex model and multigrid method are compared with the exiting models and methods, and it was found that the proposed model and method outperformed the exiting models and methods.

## References

Caselles, V., Kimmel, R., Sapiro, G.: Geodesic active contours. Int. J. Comput. Vis.

**22**(1), 61â€“79 (1997)Mumford, D., Shah, J.: Optimal approximations by piecewise smooth functions and associated variational problems. Commun. Pure Appl. Math.

**42**(5), 577â€“685 (1989)Cremers, D., Rousson, M., Deriche, R.: A review of statistical approaches to level set segmentation: integrating color, texture, motion and shape. Int. J. Comput. Vis.

**72**(2), 195â€“215 (2007)Chan, T.F., Vese, L.A.: Active contours without edges. IEEE Trans. Image Process.

**10**(2), 266â€“277 (2001)He, L., Peng, Z., Everding, B.: A comparative study of deformable contour methods on medical image segmentation. Image Vis. Comput.

**26**(2), 141â€“163 (2008)Caselles, V., CattÃ©, F., Coll, T., Dibos, F.: A geometric model for active contours in image processing. Numer. Math.

**66**(1), 1â€“31 (1993)Chan, T.F., Sandberg, B.Y., Vese, L.A.: Active contours without edges for vector-valued images. J. Vis. Commun. Image Represent.

**11**(2), 130â€“141 (2000)Bresson, X., Esedoglu, S., Vandergheynst, P., Thiran, J.P., Osher, S.: Fast global minimization of the active contour/snake model. J. Math. Imaging Vis.

**28**(2), 151â€“167 (2007)Badshah, N., Chen, K., Ali, H., Murtaza, G.: Coefficient of variation based image selective segmentation model using active contours. East Asian J. Appl. Math.

**2**(2), 150â€“169 (2012)Weickert, J., Romeny, B.T.H., Viergever, A.: An efficient local Chanâ€“Vese model for image segmentation. Pattern Recognit.

**43**(3), 603â€“618 (2010)Badshah, N., Chen, K.: Multigrid method for the Chanâ€“Vese model in variational segmentation. Commun. Comput. Phys.

**4**(2), 294â€“316 (2008)Badshah, N., Chen, K.: On two multigrid algorithms for modeling variational multiphase image segmentation. IEEE Trans. Image Process.

**18**(5), 1097â€“1106 (2009)Zhang, K., Zhang, L., Song, H., Zhou, W.: Active contours with selective local or global segmentation: a new formulation and level set method. Image Vis. Comput.

**28**(4), 668â€“676 (2010)Cai, X.: Variational image segmentation model coupled with image restoration achievements. Pattern Recognit.

**48**(6), 2029â€“2042 (2015)Mora, M., Tauber, C., Batatia, H.: Robust level set for heart cavities detection in ultrasound images. In: Computers in Cardiology, pp.Â 235â€“238. IEEE Comput. Soc., Los Alamitos (2005)

Ghaffar, F., Badshah, N., Islam, S.: Multigrid method for solution of 3d Helmholtz equation based on hoc schemes. In: Abstract and Applied Analysis, vol.Â 2014. Hindawi Publishing Corporation, New York (2014)

Ghaffar, F., Badshah, N., Islam, S., Khan, M.A.: Multigrid method based on transformation-free high-order scheme for solving 2d Helmholtz equation on nonuniform grids. Adv. Differ. Equ.

**2016**(1), 19 (2016)

### Acknowledgements

The authors would like to express their gratitude to the editors and anonymous reviewers for their valuable suggestions, which substantially improved the standard of the paper.

### Availability of data and materials

Not applicable.

## Funding

The work is supported financially by the Higher Education Commission (HEC) Islamabad, Pakistan.

## Author information

### Authors and Affiliations

### Contributions

The authors equally contributed in the paper. All authors have read and approved the final manuscript.

### Corresponding author

## Ethics declarations

### Competing interests

The authors declare that they have no competing interests.

## Additional information

### Publisherâ€™s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

## Rights and permissions

**Open Access** This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

## About this article

### Cite this article

Badshah, N., Ullah, F. & Matiullah On segmentation model for vector valued images and fast iterative solvers.
*Adv Differ Equ* **2018**, 221 (2018). https://doi.org/10.1186/s13662-018-1669-9

Received:

Accepted:

Published:

DOI: https://doi.org/10.1186/s13662-018-1669-9