Collaborative Camera Control Algorithms Evaluation

1. Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song* A. Frank van der Stappen† Ken Goldberg* * UC Berkeley, USA † Utrecht University, Netherlands

2. Dezhen Song

3. Webcams

5. n users 1 pan, tilt, zoom robotic camera

6. “ShareCam”

7. Example input: 7 requested frames:

8. OneOptimal Frame ShareCam Problem: Given n requests, find optimal frame

9. Taxonomy (Tanie, Matsuhira, Chong 00) Single Operator, Single Robot (SOSR): Single Operator, Multiple Robot (MOSR): Multiple Operator, Single Robot (MOSR):

10. Related Work • MOSR systems • Cinematrix (91) • Cannon, McDonald, et al. (97) • Goldberg, Chen, et al. (00, 01) • Goldberg, song, et al. (02) • Internet robots • Tanie, K., Chong, N. Et al(01) • Jia, S. And K. Takase (01) • Hu, H., Yu, L., Tsui, P., Zhou, Q (01) • Safaric, R. Et al. (01) • Goldberg and Siegwart (02)

11. Related Work • Facilities Location Problems • Megiddo and Supowit [84] • Eppstein [97] • Halperin et al. [02] • Rectangle Fitting • Grossi and Italiano [99,00] • Agarwal and Erickson [99] • Mount et al [96] • Kapelio et al [95]

12. Related Work • Similarity Measures • Kavraki [98] • Broder et al [98, 00] • Veltkamp and Hagedoorn [00] • Distributed robot algorithms • Sagawa et al [01], Safaric[01] • Parker[02], Bulter et al. [01] • Mumolo et al [00], Hayes et al [01] • Agassounon et al [01], Chen [99]

13. Problem Definition Requested frames: i=[xi, yi, zi], i=1,…,n

14. 3z (x, y) Problem Definition • Assumptions • Camera has fixed aspect ratio: 4 x 3 • Candidate frame  = [x, y, z] t • (x, y)  R2(continuous set) • z  Z (discrete set) 4z

15. Problem Definition • “Satisfaction” for user i: 0  Si  1  =   i  = i Si = 0 Si = 1

16. Satisfaction Metrics • Symmetric Difference • Intersection-Over-Union Nonlinear functions of (x,y)

17. Satisfaction Metrics • Intersection over Maximum: Requested frame i , Area= ai Candidate frame  Area = a pi

18. Intersection over Maximum: si( ,i) Requested frame i Candidate frame  si = 0.20 0.21 0.53

19. (for fixed z) Requested frame i Candidate frame (x,y)

20. Satisfaction Function • si(x,y) is a plateau • One top plane • Four side planes • Quadratic surfaces at corners • Critical boundaries: 4 horizontal, 4 vertical

21. Objective Function • Global Satisfaction: for fixed z ShareCam problem: Find * = arg max S()

22. Properties of Global Satisfaction S(x,y) is non-differentiable, non-convex, but piecewise linear along axis-parallel lines.

23. ShareCam Algorithms Bruteforce Algorithm • Compute S at each pixel (x,y) • O(whmn): • w, h: width and height of panoramic image • m: number of zoom levels • n: # users

24. y x Approximation Algorithm Compute S(x,y) at lattice of sample points: d

25. Approximation Algorithm * : Optimal frame : Smallest frame at lattice that encloses* • Run Time: • O(w h m n / d2) : Optimal at lattice

26. x Exact Algorithm • Define “Virtual corners” • Consider a pair of requested frames and • Their critical boundaries y

27. x Exact Algorithm • Virtual corner: Intersection between boundaries • Self intersection: • Frame intersection: y

28. Exact Algorithm • Claim: An optimal point occurs at a virtual corner. Proof: • Along vertical boundary, S(y) is a 1D piecewise linear function: extrema must occur at x boundaries

29. Exact Algorithm Exact Algorithm: Check all virtual corners • (mn2) virtual corners • (n) time to evaluate S for each • (mn3) total runtime

30. Improved Exact Algorithm • Sweep horizontally: solve at each vertical • Sort critical points along y axis: O(n log n) • 1D problem at each vertical boundary O(nm) • O(n) 1D problems • O(mn2) total runtime O(n) 1D problems

31. Distributed Algorithm More users  More computers available

32. Distributed Algorithm • At the Server • Sort horiz. boundaries • O(n log n) • At the Client • Solve 1D problem for own vertical boundaries. • O(nm) • O(n(m+ log n)) Total Four 1D problems

33. Examples

34. Examples

35. www.tele-actor.net/ sharecam

36. Current Work • New Approx Algorithms: • With Har-Peled, Koltun • Stair-like approximation • Dynamic segment tree • O(n log n) • Weighted Requests

37. Future Work • Continuous zoom (m=) • Multiple outputs: • p cameras • p views from one camera • “Temporal” version: fairness • Integrate si over time: minimize accumulated dissatisfaction for any user • Network / Client Variability: load balancing • Obstacle Avoidance

38. The Tele-Actor Server Operators

39. Summary • Satisfaction Metric: • Intersection over Maximum • ShareCam Problem : find * = arg max S() • Critical Points at Virtual Corners • Exact Algorithms: • Distributed Algorithm: • tele-actor.net/sharecam O(mn2) O(mn)

41. Summary • A collaborative camera control system • Satisfaction metric • Virtual corner based algorithms • Distributed algorithm • www.tele-actor.net/sharecam

42. Regular web-cam Collaborative camera control Introduction Queue

44. Internet Interface:

45. Results & Discussion • Speed of naive (B) and fast (V): • AMD K7 950Mhz • 1.2 GB memory • JDK 1.3.1 • For a fixed z