Improving NEST Performance Using Surrogates

1. Improving NEST Performance Using Surrogates Project Status: Dec 16, 2003 David W. Etherington Andrew J. Parkes Matt Ginsberg University of Oregon C I R L

2. Administrative • Project Title:Improving NEST Performance Using Surrogates • Program Manager: Vijay Raghavan • PI Names:David W. Etherington, Matthew L. Ginsberg, Andrew J. Parkes • PI Phone Numbers: 541-346-{0472, 0471, 0434} • PI E-Mail Addresses:{ether, ginsberg, parkes}@cirl.uoregon.edu • Company/Institution:CIRL/University of Oregon • Contract Number:F33615-02-C-4032 • AO Number: • Award Start Date: 9/12/2002 • Award End Date: 9/11/2005 • Agent Name/Organization: Ed DePalma, AFRL

3. Subcontractors and Collaborators • Subcontractors • none • Collaborators • none

4. Etherington, Parkes CIRL, University of Oregon full property param estimation estimation + phase xition approx-imation pure surrogates Improving NEST PerformanceUsing Surrogates Problem and Challenge New Ideas Find manageable surrogates for critical properties • Application of surrogates to distributed reasoning • easy-to-measure standins for properties of interest • Transition behaviors can be used to justify use of surrogates for system behavior prediction/control • Extend notion of relaxation to approximate relaxations • Explore the interaction between structure and transitions cost quality Avoidance of dangerous regions; develop tractable mechanisms for design/control/prediction FY04 Schedule Impact • Simplified design and control of performance • Simplified performance prediction/modeling with uncertain configuration information • e.g., prediction of capabilities under various attrition/failure models • Predict/determine maintenance/replenishment requirements under uncertain field conditions • e.g., model tradeoffs of capabilities vs costs • 1QFY04 • Demonstrate use of local surrogates in random NESTs • Determine thresholds for static global surrogates • 2QFY04 • Discover local surrogates for static properties of interest • Use static global surrogates to predict/control structured NESTs • 3QFY04 • Develop temporally structured NEST testbed/generator • Determine thresholds for static local surrogates • 4QFY04 • Identify functional dynamic properties in temporally structured NESTs • Use static local surrogates to predict/control structured NESTs

5. Problem Description/Objective • Problem: NEST systems induce difficult design and control problems • these impair our ability to achieve expected benefits like robustness and speed • Goal: develop the ability to predict, analyze, and bound NEST performance “in the large” • Approach: use surrogates: properties that are easy to measure/control yet strongly correlated with real objectives • identify “hard” properties of interest • exploit structure to find surrogates

6. Current Technical Approach • Identify hard problems: • model technical problems of interest • experiment to determine transition points/control variables • discrete/continuous analysis of identified parameters • predict scaling behavior • Apply appropriate tools to find surrogates • simple control behavior • thresholds & experimental bounds on control variables • complex behavior • group theory (ZAP) • constraint weakening/strengthening

7. Changes in Technical Approach • Deëmphasized development of synthetic generators/surrogates to more quickly connect to NEST demonstration platforms. • Reëmphasized computational hardness of target problems, as well as utility for NEST. • Reëmphasized general modeling as opposed to search for specific surrogates.

8. Progress Since Last PI Meeting • Built NEST-generating system • parameterized distribution, connectivity, etc. • Identified “pseudo-density” connectivity surrogate in structured NESTs • covers realistic distributions, interesting properties • identified thresholds controlling connectivity • ‘Twisted tree’ disjoint spanning tree surrogate • demonstrated on various, non-idealized, distributions • Discovered group-theoretic structure exploitation • potential exponential compaction of discovery process Details in technical presentation, following.

9. Deliverables and Publications Generalizing Boolean Satisfiability I: Background and Existing Work. Heidi E. Dixon, Matthew L. Ginsberg, and Andrew J. Parkes: to appear in JAIR, 2004. Generalizing Boolean Satisfiability II: Theory. Heidi E. Dixon, Matthew L. Ginsberg, and Andrew J. Parkes: in preparation, to be submitted to JAIR. Generalizing Boolean Satisfiability III: Implementation. Heidi E. Dixon, Matthew L. Ginsberg, and Andrew J. Parkes: in preparation, to be submitted to JAIR. Scaling Properties of Pure Random Walk on Random 3-SAT. Andrew J. Parkes. Proceedings of the Eighth International Conference on Principles and Practice of Constraint Programming (CP2002). Published in Lecture Notes in Computer Science, LNCS 2470. Pages 708--713. Easy Predictions for the Easy-Hard-Easy Transition. Andrew J. Parkes. Eighteenth Nat’l Conference on Artificial Intelligence (AAAI-02) Likely Near-Term Advances in SAT Solvers. Heidi E. Dixon, Matthew L. Ginsberg, Andrew J. Parkes,at MTV-02. Inference methods for a pseudo-Boolean satisfiability solver. Heidi E. Dixon and Matthew L. Ginsberg. AAAI-02. Six further papers are currently in preparation for submission to AAAI-03, titles will be added after the blind review period has expired.

10. Success Criteria • Metrics • predictive accuracy of discovered surrogates • utility of the control surrogates that are discovered • design process simplification through surrogates • generality/reusability of methodology • Decision points • ability to generate predictive/control models for static problems of interest • ability to demonstrate transitions implying existence of, and to identify, surrogates for those problems

11. Project Plans: Collaboration • Collaborate in extreme scaling effort • help identify hard, make-or-break, issues • model problematic aspects of “ideal” approach • develop surrogates that enable minimal functionality Metric: impact of models, problems, surrogates found • Assist in exploitation of existing surrogates • twisted-path implementation, etc. Metric: degree of successful exploitation • Develop new surrogates • exfiltration, sentry/relay/sleep issues, scalability, etc. Metric: utility for design or modeling

12. Project Plans: Modeling • Study problem-distribution’s influence on surrogates • Payoff: help find useful, reliable, surrogates • Risk: open ended; difficult to determine right distributions • Strengthen/weaken constraint-set to identify surrogates • Payoff: provide systematic methods • Risk: unproven; may not provide effective mechanism • Off-line search (characterize search space, and identify useful gradient indicators that aid convergence) • Payoff: improved ability to find surrogates automatically • Risk: required understanding of the underlying search space may be slow in coming Metrics: in all of these, the metrics will be predictive accuracy and the ability to find useful surrogates

13. Project Schedule and Milestones Temporally structured NESTs Structured NESTs Random NESTs • Upcoming milestones: • demonstrate use of local, static surrogates in structured NESTs • discover local, static surrogates in structured NESTs • determine local and global static thresholds • elaborate network generator to produce more sophisticated NESTs

14. Random NESTs/synthetic properties develop parameterized network generator Identify static properties of interest Discover surrogates Identify globally observable surrogates Identify local surrogates Determine thresholds for global surrogates for local surrogates Build demonstration systems use global surrogates to predict behavior use of local surrogates Legend: On schedule Partially completed Delayed Structured NESTs/synthetic properties Develop network generator based on community specifications/needs Identify functional properties Discover surrogates Discover global surrogates Notes: As described previously, efforts on 1.3.b, 1.4.b, and 1.5.b were suspended in order to more quickly move the project to domains of practical interest to the NEST community. Continuing work is expected to be done to make the system developed in 2.1 more widely useful to the community. Specific Milestones

15. Technology Transition/Transfer • N/A

16. Program Issues • N/A

17. Technical Progress Summary • Modeling of realistic sensor deployments • developed simulator for experiments • identified transition behavior in big-brother problem • Surrogates for connectivity • pseudo-density controls many measures of interest • Fairly robust routing • twisted trees: minimal disjoint spanning trees surrogate • Group-theoretic structure exploitation (ZAP) • exponential simplification of certain search problems • potential for automatic generation of surrogates • by simplification of generators for relevant groups

18. The Overall Plan • Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly

19. The Overall Plan • Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly • Example: satisfiability in random 3-SAT • Must be possible to evaluate surrogate quickly

20. The Overall Plan • Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly • Example: satisfiability in random 3-SAT • Must be possible to evaluate surrogate quickly • Quality/feature being modeled must have real operational use

21. The Overall Plan • Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly • Example: satisfiability in random 3-SAT • Must be possible to evaluate surrogate quickly • Quality/feature being modeled must have real operational use • Feature being modeled must be beyond the reach of existing capabilities

22. The Overall Plan • Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly • Example: satisfiability in random 3-SAT • Must be possible to evaluate surrogate quickly • Quality/feature being modeled must have real operational use • Feature being modeled must be beyond the reach of existing capabilities

23. The Overall Plan • Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly • Example: satisfiability in random 3-SAT • Surrogate is clause/variable ratio • Quality/feature being modeled must have real operational use • Feature being modeled must be beyond the reach of existing capabilities

24. The Overall Plan • Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly • Example: satisfiability in random 3-SAT • Surrogate is clause/variable ratio • Satisfiability is the coin of the realm in this domain ffff • Feature being modeled must be beyond the reach of existing capabilities

25. The Overall Plan • Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly • Example: satisfiability in random 3-SAT • Surrogate is clause/variable ratio • Satisfiability is the coin of the realm in this domain ffff • 3-SAT is NP-complete

26. Multipath Routing: July • Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly • Example: satisfiability in random 3-SAT • Must be possible to evaluate surrogate quickly • Quality/feature being modeled must have real operational use • Feature being modeled must be beyond the reach of existing capabilities

27. Multipath Routing: July • Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly • Example: biconnectedness • Must be possible to evaluate surrogate quickly • Quality/feature being modeled must have real operational use • Feature being modeled must be beyond the reach of existing capabilities

28. Multipath Routing: July • Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly • Example: biconnectedness • Density as a surrogate (V2/N) • Quality/feature being modeled must have real operational use • Feature being modeled must be beyond the reach of existing capabilities

29. Multipath Routing: July • Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly • Example: biconnectedness • Density as a surrogate (V2/N) • Quality/feature being modeled must have real operational use • Not hard! Vijay: linear-time algorithms exist

30. Multipath Routing: July • Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly • Example: biconnectedness • Density as a surrogate (V2/N) • Not useful, either! • Not hard! Vijay: linear-time algorithms exist

31. Multipath Routing: July • Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly • Example: biconnectedness • Density as a surrogate (V2/N) • Not useful, either! Long paths • Not hard! Vijay: linear-time algorithms exist

32. Multipath Routing: July • Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly • Example: biconnectedness • Density as a surrogate (V2/N) • Not useful, either! Long paths, impossible to find • Not hard! Vijay: linear-time algorithms exist

33. Multipath Routing: July • Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly • Example: biconnectedness • Density as a surrogate (V2/N) • Not useful, either! Long paths, impossible to find and who cares about disjointedness anyway? • Not hard! Vijay: linear-time algorithms exist

34. Multipath Routing: December • Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly • Example: ability to find multiple, short, nearly disjoint routings

35. Multipath Routing: December • Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly • Example: ability to find multiple, short, nearly disjoint routings • computationally viable surrogate

36. Multipath Routing: December • Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly • Example: ability to find multiple, short, nearly disjoint routings • density (V2/N) still works

37. Multipath Routing: December • Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly • Example: ability to find multiple, short, nearly disjoint routings • density (V2/N) still works • problem has to be hard

38. Multipath Routing: December • Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly • Example: ability to find multiple, short, nearly disjoint routings • density (V2/N) still works • problem is known to be NP-hard (disjoint case)

39. Multipath Routing: December • Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly • Example: ability to find multiple, short, nearly disjoint routings • density (V2/N) still works • problem is known to be NP-hard (disjoint case) • has to be useful

40. Multipath Routing: December • Surrogates provide computationally effective alternates to operational features that are impractical to evaluate directly • Example: ability to find multiple, short, nearly disjoint routings • density (V2/N) still works • problem is known to be NP-hard (disjoint case) • clear impact on both robustness and power consumption

41. Multipath Routing: December • Biconnectedness is too easy • Robust routing is too hard • Fairly robust routing is just right • And curiously enough, V2/N is a surrogate for all of them

42. Fairly Robust Routing • Goal: “approximately disjoint” spanning trees • low cost to construct (time/communication) • short • well dispersed • Approach: exploit weak localization information • augment standard flood fill with directional bias • produce a loose “spiral” in toward root • Results: near-shortest-path trees • good spatial dispersion • low cost to construct • no spanning tree needed (extreme scalability)

43. Twisted Trees • Angular bias applied to flood fill algorithm • based on angle between arcs to neighbor and root • for a 30º left bias, choose neighbor bearing closest to 30º left of the bearing to the root • Works in high surrogate density (V2/N) regions • Serial complexity is O(V+E) • makes efficient use of distribution of sensors • conjecture: O(log(V)) parallel complexity • Experimentally validated, proofs in progress • good spatial separation (not perfect!) • short paths (small multiplier of optimal)

44. Gauss network graph

45. Shortest path routing tree

46. Boundary edges for router

47. Twisted routing tree

48. Multipath route

49. Crop-duster network graph

50. Shortest-path routing tree