Links

Home

Publications

Research

Gallery

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

SIFT and ASIFT --- online demo : try if your images match!

 

Jean-Michel Morel           Guoshen Yu

 

morel[AT]cmla.ens-cachan.fr    yu[AT]cmap.polytechnique.fr

 

 

News: The ASIFT source code and online demo are now published in the journal IPOL! 2011.02.24

 

 

News: frequently asked questions on ASIFT. --- 2009.11.22

 

 

 

Summary:

 

A fully affine invariant image comparison method, Affine-SIFT (ASIFT) is introduced. While SIFT is fully invariant with respect to only four parameters namely zoom, rotation and translation, the new method treats the two left over parameters : the angles defining the camera axis orientation. Against any prognosis, simulating all views depending on these two parameters is feasible. The method permits to reliably identify features that have undergone very large affine distortions measured by a new parameter, the transition tilt. State-of-the-art methods hardly exceed transition tilts of 2 (SIFT), 2.5 (Harris-Affine and Hessian-Affine) and 10 (MSER). ASIFT can handle transition tilts up 36 and higher.

 

References:

Additional Materials:

 

Online demo:  

An online demo (version beta) that allows you to try ASIFT with your own images is available here. If you are interested in ASIFT and want to try it with your own images, we suggest that you start with the user friendly online demo.

 

Source code : NEW!!!

  • Standalone program in C++

  • Ready to be complied

  • Reproduce exactly the same results as in the online demo

The ASIFT source code and online demo are now published in the journal IPOL.

 

 

 

Frequently asked questions:

We have received many correspondences from the ASIFT readers and users. Some typical questions are posted here.

 

Dataset:

An image dataset for systematic evaluation of robustness to absolute and transition tilt of the image matching algorithms is available here.

 

When does it work?

The SIFT method works to compare 2D objects or 3D objects with flat enough details, taken from similar view angles but at arbitrary distances.

 

The typical failure cases are:

  • The illumination conditions are different (for instance daylight/nightlight).

  • The object has a reflecting surface (typically cars are mirrors ; they change completely aspect under different view angles).

  • The object has a strong 3D structure: in that case a change of view angle alters drastically its aspect.

  • The object has a self similar or periodic structure: then "true" mismatches occur.

  • The view angle is too different.

See our failure case study.

 

ASIFT corrects the last problem : if the object is under view has similar illumination conditions, has rather flat surface, and is not a mirror, then ASIFT retrieves the object even under extreme changes of angle. In technical terms, ASIFT is more affine invariant than SIFT.

 

Other popular methods addressing this problem exist: MSER and Hessian-Affine and Harris-Affine. All these methods will be compared below.

 

Examples:

ASIFT is compared with the four state-of-the-art algorithms the SIFT, Harris-Affine, Hessian-Affine and MSER detectors, all coded with the SIFT descriptor. Various types of images (size 600〜450) were used for the experiments. The SIFT software is from D. Lowe. The Harris-Affine, Hessian-Affine and MSER programs are from the web site of K. Mikolajczyk.

In the examples below, the powerful Moisan-Stival procedure is applied to eliminate matches incoherent with epipolar geometry. (download .pdf) Matches are connected by white segments.

 

 

Video Tracking

 

ASIFT is compared with SIFT on video tracking . In each video, ASIFT and SIFT tracking are shown respectively on the top and bottom. If you prefer high-resolution video, please download the video files by clicking the links below.

Facade (click here if you do not see the video)

Magazine (click here if you do not see the video)

Download high-resolution video file (13.5 MB)

Download high-resolution video file (18.9 MB)


 

 
1. Planar objects

Adam taken from short distance (zoom 〜1) at frontal view and at 75 degree angle. Absolute tilt t = 4 (middle), <4 (left), >4 (right)

Not shown Harris-Affine: 3 matches. Hessian-Affine: 1 match

ASIFT: 202 matches

SIFT: 15 matches

MSER: 5 matches

Adam taken from short distance (zoom 〜10) at frontal view and at 65 degree angle. Absolute tilt t = 2.4

Not shown Harris-Affine: 3 matches. Hessian-Affine: 0 match.

ASIFT: 341 matches

SIFT: 5 matches

MSER: 4 matches

Adam taken from short distance (zoom 〜10) at frontal view and at 80 degree angle. Absolute tilt t = 5.8

Not shown Harris-Affine: 1 match. Hessian-Affine: 0 match.

ASIFT: 75 matches

SIFT: 1 match

MSER: 2 matches

Magazine taken from middle distance (zoom 〜4) at frontal view and at 80 degree angle. Absolute tilt t = 5.8

Not shown Harris-Affine: 0 match. Hessian-Affine: 0 match.

ASIFT: 349 matches

SIFT: 0 match

MSER: 17 matches

Magazine taken with absolute tilt t1= t2= 2, with longitude angles Φ1= 0 deg, Φ2= 50 deg.  Transition tilt  τ = 3

Not shown Harris-Affine: 0 match. Hessian-Affine: 0 match.

ASIFT: 881 matches

SIFT: 3 matches

MSER: 87 matches

Magazine taken with absolute tilt t1= t2= 4, with longitude angles Φ1= 0 deg, Φ2= 90 deg.  Transition tilt  τ = 16

Not shown Harris-Affine: 0 match. Hessian-Affine: 0 match.

ASIFT: 88 matches

SIFT: 1 match

MSER: 9 matches

Facade at frontal view and at 75 degree angle. Absolute tilt  t = 3.8.

Not shown SIFT: 0 match. Harris-Affine: 1 match. 

ASIFT: 68 matches

Hessian-Affine: 1 match

MSER: 2 matches

Graffiti No.1 vs No. 6 (images from K. Mikolajczyk).  Transition tilt  τ ~ 3.2.

Not shown SIFT: 0 match. Hessian-Affine: 1 match. 

ASIFT: 721 matches

Harris-Affine: 3 matches

MSER: 70 matches

Direction Transition tilt τ ~ 2.6.

Not shown Harris-Affine: 0 match. Hessian-Affine: 0 match. 

ASIFT: 50 matches

SIFT: 0 match

MSER: 1 match

Parkings Transition tilt τ ~ 15.

Not shown Harris-Affine: 0 match. Hessian-Affine: 0 match. 

ASIFT: 78 matches

SIFT: 0 match

MSER: 0 match

Stump Transition tilt τ ~ 2.6

Not shown Harris-Affine: 2 matches. Hessian-Affine: 1 match.

ASIFT: 168 matches

SIFT: 1 match

MSER: 6 matches

Back to top

2. Monuments and constructions

Notre Dame Transition tilt τ ~ 1.4.

Not shown Hessian-Affine: 1 match. MSER: 0 match. 

ASIFT: 35 matches

SIFT: 10 matches

Harris-Affine: 1 match

 

Pentagon

Not shown SIFT: 6 matches.  Harris-Affine: 2 matches.

ASIFT: 28 matches

Hessian-Affine: 8 matches

MSER: 17 matches

Round building Transition tilt τ ~ [1.8, ±)

Not shown Harris-Affine: 5 matches. Hessian-Affine: 7 matches.

ASIFT: 139 matches

SIFT: 19 matches

MSER: 13 matches

Palace of Versailles Transition tilt τ ~ 1.8

Not shown Harris-Affine: 2 matches. Hessian-Affine: 1 match.

ASIFT: 67 matches

SIFT: 26 matches

MSER: 4 matches

Ecole Polytechnique taken at frontal view and at 65 degree angle. Absolute tilt t = 2.4

Not shown Harris-Affine: 2 matches. Hessian-Affine: 2 matches.

ASIFT: 101 matches

SIFT: 13 matches

MSER: 4 matches

Back to top

3. 3D objects

Statue of Liberty  τ ~ [1.3, ±).

Not shown Hessian-Affine: 1 match. MSER: 1 match. 

ASIFT: 32 matches

SIFT: 9 matches

Harris-Affine: 1 match

Statue Transition tilt τ ~ [1.6, ±).

Not shown Harris-Affine: 7 matches. Hessian-Affine: 2 matches.

ASIFT: 67 matches

SIFT: 26 matches

MSER: 4 matches

Car

Not shown Harris-Affine: 0 match.  MSER: 1 match.

ASIFT: 28 matches

SIFT: 1 match

Harris-Affine: 1 match

Can Transition tilt τ ~ [2.3, ±).

Not shown SIFT: 0 match. Hessian-Affine: 3 matches. 

ASIFT: 287 matches

Harris-Affine: 6 matches

MSER: 22 matches

Back to top

4. Complex scenes

Office Transition tilt  τ ~ 3

Not shown SIFT: 0 match. Harris-Affine: 0 match. 

ASIFT: 88 matches

Hessian-Affine: 1 match

MSER: 3 matches

Bottles (proposed by the authors of MSER)  Transition tilt τ ~ [1.6, 3.0].

Not shown SIFT: 10 matches. Hessian-Affine: 11 matches. 

ASIFT: 254 matches

Harris-Affine: 23 matches

MSER: 22 matches

Coffee room Transition tilt τ ~ [1.5, 3.3].

Not shown Harris-Affine: 0 match. Hessian-Affine: 3 matches. 

ASIFT: 125 matches

SIFT: 13 matches

MSER: 5 matches

Back to top

 
5. Object deformation (images from Ling and Jacobs)

SpongeBob

Not shown Harris-Affine: 6 matches.  MSER: 4 matches.

ASIFT: 370 matches

SIFT: 75 matches

Harris-Affine: 8 matches

 

CVPR

Not shown Harris-Affine: 25 matches.  MSER: 17 matches.

ASIFT: 528 matches

SIFT: 160 matches

Hessian-Affine: 55 matches

 

Flag

Not shown Hessian-Affine: 10 matches.  MSER: 2 matches.

ASIFT: 141 matches

SIFT: 31 matches

Harris-Affine: 15 matches

 

Girl

Not shown Harris-Affine: 45 matches (1 on the cloth).  Hessian-Affine: 22 matches (1 on the cloth). 

ASIFT: 659 matches

SIFT: 304 matches

MSER: 35 matches

Toy

Not shown Harris-Affine: 0 match. MSER: 0 match. 

ASIFT: 33 matches

SIFT: 4 matches

Hessian-Affine: 1 match

 

Back to top

 
Failure: day-and-night illumination change --- all methods fail!

Six images of Notre-Dame under different illumination conditions are compared. The number of matches of ASIFT and SIFT are shown. (Harris-Affine, Hessian-Affine and MSER find less matches than SIFT.) Little view angle change is presented. The red arrows imply recognition failure.

 

In general, matching succeeds between day images and between night images. However, under day-and-night illumination change, all methods fail.

 

Description:

All methods fail under day-and-night illumination change (pairs 1-2, 1-6, 2-3, 2-4, 2-5, 3-6, 4-6, 5-6).

- Matching succeeds between night images, with a substantial scale change (pair 2-6).

- Matching succeeds between day images under same weather condition, with or without a substantial scale change (pairs 1-4, 3-5).

- Matching succeeds between day images under different weather conditions, without a substantial scale change (pairs 1-3, 3-4).

- Matching fails between day images under different weather conditions, with a substantial scale change (pairs 1-5, 4-5).

 

 

 

 

 

Failure: strong relief effect --- all methods fail!

ASIFT, SIFT, Harris-Affine, Hessian-Affine, MSER: 0 match.

The image on the right in close-up shows strong relief effect.

Failure: Repetitive shapes --- "good" false matches

Matches can be arbitrary among repetitive shapes.

ASIFT: 169 matches, many are "good" false matches (for example the matches between the left-most window in the image above and the second window in the image below).

 

Due to the viewpoint change, SIFT, Harris-Affine, Hessian-Affine and MSER find much less matches (respectively 30, 4, 9 and 4), among which many are "good" false matches as well. 

Back to top