Background-Foreground Modeling Based on Spatiotemporal Sparse Subspace Clustering

Sajid Javed, Arif Mahmood, Thierry Bouwmans, Soon Ki Jung

Research output: Contribution to journalArticlepeer-review

110 Scopus citations


Background estimation and foreground segmentation are important steps in many high-level vision tasks. Many existing methods estimate background as a low-rank component and foreground as a sparse matrix without incorporating the structural information. Therefore, these algorithms exhibit degraded performance in the presence of dynamic backgrounds, photometric variations, jitter, shadows, and large occlusions. We observe that these backgrounds often span multiple manifolds. Therefore, constraints that ensure continuity on those manifolds will result in better background estimation. Hence, we propose to incorporate the spatial and temporal sparse subspace clustering into the robust principal component analysis (RPCA) framework. To that end, we compute a spatial and temporal graph for a given sequence using motion-aware correlation coefficient. The information captured by both graphs is utilized by estimating the proximity matrices using both the normalized Euclidean and geodesic distances. The low-rank component must be able to efficiently partition the spatiotemporal graphs using these Laplacian matrices. Embedded with the RPCA objective function, these Laplacian matrices constrain the background model to be spatially and temporally consistent, both on linear and nonlinear manifolds. The solution of the proposed objective function is computed by using the linearized alternating direction method with adaptive penalty optimization scheme. Experiments are performed on challenging sequences from five publicly available datasets and are compared with the 23 existing state-of-the-art methods. The results demonstrate excellent performance of the proposed algorithm for both the background estimation and foreground segmentation.

Original languageBritish English
Article number8017547
Pages (from-to)5840-5854
Number of pages15
JournalIEEE Transactions on Image Processing
Issue number12
StatePublished - Dec 2017


  • Background modeling
  • foreground detection
  • graph regularization
  • robust principal component analysis
  • subspace clustering


Dive into the research topics of 'Background-Foreground Modeling Based on Spatiotemporal Sparse Subspace Clustering'. Together they form a unique fingerprint.

Cite this