Efficient Image Retrieval Using Vantage Objects

1. Vantage Objects Dr. Rolf Lakaemper Dept. of Computer and Information Sciences Temple University

2. The Application: ISS Database Task: Create Image Database Problem: Response Time Comparison of 2 Shapes: 23ms on Pentium1Ghz ISS contains 15,000 images: Response Time about 6 min. Clustering not possible (not a metric)

3. Vantage Objects Solution: Full search on entire database using a simpler comparison Vantage Objects (Vleugels / Veltkamp, 1999) provide a simple comparison of n- dimensional vectors (n typically < 100) Paper: Vleugels/Veltkamp: Efficient Image Retrieval through Vantage Objects (1999)

4. Vantage Objects The Idea: Compare the query-shape q to a predefined subset S of the shapes in the database D The result is an n-dimensional Vantage Vector V, n = |S| s1 v1 s2 v2 q s3 v3 … sn vn

5. Vantage Objects • - Each shape can be represented by a single Vantage Vector • - The computation of the Vantage Vector calls the ASR – comparison only n times • - ISS uses 54 Vantage Objects, reducing the comparison time (needed to create the Vantage Vector) to < 1.5s • - How to compare the query object to the database ?

6. Vantage Objects • - Create the Vantage Vector vi for every shape di in the database D • - Create the Vantage Vector vq for the query-shape q • - compute the euclidean distance between vq and vi • - best response is minimum distance • Note: computing the Vantage Vectors for the database objects is an offline process !

7. Vantage Objects • How to define the set S of Vantage Objects ?

8. Vantage Objects • Algorithm 1 (Vleugels / Veltkamp 2000): • Predefine the number n of Vantage Objects • S0 = { } • Iteratively add shapes di  D\Si-1 to Si-1 such that • Si = Si-1  di • and • k=1..i-1e(di , sk)maximal. (e = eucl. dist.) Stop if i = n.

9. Vantage Objects • Result: • Did not work for ISS.

10. Vantage Objects • Algorithm 2 (Latecki / Henning / Lakaemper): • Def.: • A(s1,s2): ASR distance of shapes s1,s2 • q: query shape • ‘Vantage Query’ : determining the result r by minimizing e(vq , vi ) vi = Vantage Vector to si • ‘ASR Query’: determining the result r by minimizing A(q,di ) • Vantage Query has certain loss of retrieval quality compared to ASR query. • Define a loss function l to model the extent of retrieval performance

11. Vantage Objects • Given a Database D and a set V of Vantage Vectors, the loss of retrieval performance for a single query by shape q is given by: • lV,D (q) = A(q,r), • Where r denotes the resulting shape of the vantage query to D using q. • Property: • lV,D (q) is minimal if r is the result of the ASR-Query.

12. Vantage Objects • Now define retrieval error function L(S) of set • S={s1 ,…, sn }  D of Vantage Vectors of Database D: • L(S) = 1/n  lS,D\{si} (si) • Task: • Find subset S  D such that L(S) is minimal.

13. Vantage Objects Algorithm: V0={ } iteratively determine sj in D\Sj-1 such that Sj =Sj-1 sj and L(Vj) minimal. Stop if improvement is low

14. Vantage Objects Result: Worked fine for ISS, though handpicked objects stil performed better. Handpicked Algorithm 2 L(S) Number of Vantage Objects

15. Vantage Objects …some of the Vantage Objects used in ISS:

16. Vantage Objects Vantage Objects helped in times of need, but discussion is required !