Research

RepFinder: Finding Approximately Repeated Scene Elements for Image Editing

Ming-Ming Cheng1  Fang-Lue Zhang1  Niloy J. Mitra2  Xiaolei Huang3  Shi-Min Hu1

1TNList, Tsinghua University, Beijing      2KAUST/IIT Delhi     3Lehigh University

Abstract

Repeated elements are ubiquitous and abundant in both manmade and natural scenes. Editing such images while preserving the repetitions and their relations is nontrivial due to overlap, missing parts, deformation between instances, illumination variation, etc. Manually enforcing such relations is laborious and error prone. We propose a novel framework where simple user input in the form of scribbles are used to guide detection and extraction of such repeated elements. Our detection process is based on a novel boundary band method, and robustly extracts the repetitions along with their mutual depth relations. We then use topological sorting to establish a partial depth ordering of overlapping repeated instances. Missing parts on occluded instances are completed using information from other instances. The extracted repeated instances can then be seamlessly edited and manipulated for a variety of high level tasks that are otherwise difficult to perform. We demonstrate the versatility of our framework on a large set of inputs of varying complexity, showing applications to image rearrangement, edit transfer, deformation propagation, and instance replacement.

Paper

RepFinder: Finding Approximately Repeated Scene Elements for Image Editing [Bib] [pdf] [PPT Zip File]
ACM Transactions on Graphics, 29, 4, 83:1-8, 2010.
Ming-Ming Cheng, Fang-Lue Zhang, Niloy J. Mitra, Xiaolei Huang, Shi-Min Hu

System Pipeline

Results

Video (download)

Supplementary Materials

1. Source code

Our system is a huge project with many components. To help readers implementing our system, we supply some pointers to important source code components of our system. These components are:

2. FAQs:

  • Edge-based vs. point based matching: Our first attempt to solve this problem was based on employing a state-of-the-art point-based matching method to establish correspondences. However, in images with repeated elements, SIFT matching leads to much ambiguity due to the presence of a large number of points having similar texture. Even spectral correspondence methods fail to resolve the ambiguities. Thus, we moved to an edge-based approach that uses shape information, to resolve ambiguity, and to enforce local coherence of correspondence. We have found it to be simple, effective and robust. [An earlier experiment]

Links

  1. Repeated Structure Extraction papers in Princeton course: “Advanced Topics in Computer Science: Geometric Modeling and Analysis “. Related paper list is here.
Locations of visitors to this page Locations of visitors to this page
(Visited 3,072 times, 1 visits today)
Subscribe
Notify of
guest

0 Comments
Inline Feedbacks
View all comments