1 / 27

Trust-Sensitive Scheduling on the Open Grid

Trust-Sensitive Scheduling on the Open Grid. Jon B. Weissman with help from Jason Sonnek and Abhishek Chandra Department of Computer Science University of Minnesota Trends in HPDC Workshop Amsterdam 2006. Background. Public donation-based infrastructures are attractive

sook
Télécharger la présentation

Trust-Sensitive Scheduling on the Open Grid

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Trust-Sensitive Scheduling on the Open Grid Jon B. Weissmanwith help from Jason Sonnek and Abhishek Chandra Department of Computer Science University of Minnesota Trends in HPDC Workshop Amsterdam 2006

  2. Background • Public donation-based infrastructures are attractive • positives: cheap, scalable, fault tolerant (UW-Condor, *@home, ...) • negatives: “hostile” - uncertain resource availability/connectivity, node behavior, end-user demand => best effort service

  3. Background • Such infrastructures have been used for throughput-based applications • just make progress, all tasks equal • Service applications are more challenging • all tasks not equal • explicit boundaries between user requests • may even have SLAs, QoS, etc.

  4. Service Model • Distributed Service • request -> set of independent tasks • each task mapped to a donated node • makespan • E.g. BLAST service • user request (input sequence) + chunk of DB form a task

  5. BOINC + BLAST workunit = input_sequence + chunk of DB generated when a request arrives

  6. The Challenge • Nodes are unreliable • timeliness: heterogeneity, bottlenecks, … • cheating: hacked, malicious (> 1% of SETi nodes), misconfigured • failure • churn • For a service, this matters

  7. Some data- timeliness Computation Heterogeneity - both across and within nodes PlanetLab – lower bound Communication Heterogeneity - both across and within nodes

  8. The Problem for Today • Deal with node misbehavior • Result verification • application-specific verifiers – not general • redundancy + voting • Most approaches assume ad-hoc replication • under-replicate: task re-execution (^ latency) • over-replicate: wasted resources (v throughput) • Using information about the pastbehavior of a node, we can intelligently size the amount of redundancy

  9. System Model

  10. Problems with ad-hoc replication Unreliable node Task x sent to group A Reliable node Task y sent to group B

  11. Smart Replication • Reputation • ratings based on past interactions with clients • simple sample-based prob. (ri) over window t • extend to worker group (assuming no collusion) => likelihood of correctness (LOC) • Smarter Redundancy • variable-sized worker groups • intuition: higher reliability clients => smaller groups

  12. Terms • LOC (Likelihood of Correctness), lg • computes the ‘actual’ probability of getting a correct answer from a group of clients (group g) • Target LOC (ltarget) • the task success-rate that the system tries to ensure while forming client groups • related to the statistics of the underlying distribution

  13. Trust Sensitive Scheduling • Guiding metrics • throughput r: is the number of successfully completed tasks in an interval • success rate s: ratio of throughput to number of tasks attempted

  14. Scheduling Algorithms • First-Fit • attempt to form the first group that satisfies ltarget • Best-Fit • attempt to form a group that best satisfies ltarget • Random-Fit • attempt to form a random group that satisfies ltarget • Fixed-size • randomly form fixed sized groups. Ignore client ratings. • Random and Fixed are our baselines • Min group size = 3

  15. Scheduling Algorithms

  16. Scheduling Algorithms (cont’d)

  17. Different Groupings ltarget = .5

  18. Evaluation • Simulated a wide-variety of node reliability distributions • Set ltarget to be the success rate of Fixed • goal: match success rate of fixed (which over-replicates) yet achieve higher throughput • if desired, can drive tput even higher (but success rate would suffer)

  19. Comparison gain: 25-250% open question: how much better could we have done?

  20. Non-stationarity • Nodes may suddenly shift gears • deliberately malicious, virus, detach/rejoin • underlying reliability distribution changes • Solution • window-based rating (reduce t = 20 from infinite) • Experiment: “blackout” at round 300 (30% effected)

  21. Role of ltarget • Key parameter • Too large • groups will be too large (low throughput) • Too small • groups will be too small (low success rate) • Adaptively learn it (parameterless) • maximizing r * s :“goodput” • or could bias toward r or s

  22. Adaptive algorithm • Multi-objective optimization • choose target LOC to simultaneously maximize throughput r and success rate s • a1 r + a2 s • use weighted combination to reduce multiple objectives to a single objective • employ hill-climbing and feedback techniques to control dynamic parameter adjustment

  23. Adapting ltarget • Blackout example

  24. Throughput (a1=1, a2=0)

  25. Current/Future Work • Implementation of reputation-based scheduling framework (BOINC and PL) • Mechanisms to retain node identities (hence ri) under node churn • “node signatures” that capture the characteristics of the node

  26. Current/Future Work (cont’d) • Timeliness • extending reliability to encompass time • a node whose performance is highly variable is less reliable • Client collusion • detection: group signatures • prevention: • combine quiz-based tasks with reputation systems • form random-groupings

  27. Thank you.

More Related