Presentation Transcript

1. Computer Ethics & Social Issues

   Software Development
2. Some Ethical Issues Quality Does the software meet the needs of the customer? How do you know? What is the statement of work? How do you measure success? What are the deliverables? Is the software free from defects? QA Testing Code security review
3. Some Ethical Issues Liability When are you responsible for the poor quality of a product? Recklessness Negligence What steps do you take to limit liability? What is reasonable?
4. Quality Rigorous quality testing = less defects = higher standardsof safety = greater chance of customer acceptance != inexpensive SDLC != less overhead != small investment, large return
5. Corrupted Blood Incident World of Warcraft 2005 release of the Zul’Gurub raid Hakkar the Soulflayer Infects players with “Corrupted Blood” which damages hit points and is contagious. Originally intended to be limited to the self-contained raid encounter, but developers forgot that players could teleport out of the area and enter in-game cities where the plague could spread. Many WoW customers complained as their characters died continuously due to constant spread and re-infection of the plague.
6. Hakkar the Soulflayer
7. Not a bug… a feature, right?
8. Corrupted Blood Incident Reaction Many players, upset that they could not enjoy the game, decided to suspend their subscriptions Many players found enjoyment in deliberately spreading the plague, leading some to compare this malicious activity with terrorism Blizzard attempted a voluntary quarantine, which failed due to malicious player circumvention
9. Therac-25 A machine which administered radiation therapy for cancer patients Physical fail-safes to prevent high-energy electron beams from destroying a patient’s body present in previous versions were replaced with software-based fail-safes An overlooked bug in the system allowed an overflow error, causing the failsafe to fail.
10. Therac-25
11. Therac-25 Reaction Patient death Institution of IEC 62304 international standard for DLC for medical devices Cause Lack of code review Overconfidence in the system by the designers Lack of unit testing with various hardware and software components
12. Knight Capital Group August 1st, 2012 Global financial firm Largest trader in in U.S. equities Accidentally deployed development code into production Software error allowed automatic purchase and sale of millions of shares with no human oversight $500,000,000 lost in half an hour due to faulty code producing unwise transactions
13. Ethics & Software Dev
14. Recklessness v Negligence Recklessness Not showing proper concern about the possible bad results of your actions Negligence Failure to take the care a responsible person usually takes : lack of normal care or attention www.Merriam-Webster.com
15. Reckless Driving
16. Reckless Decision Example: Developing software solely to cause harm in violation of the law Distributing an exploit kit to the world knowing that it will be used to attack financial systems Ignoring the possibility of a well-documented vulnerability in an medical computing system because it is too much work to try to fix it if it exists
17. Negligent Homicide
18. Negligent Decision Example: Developing software which could be used for criminal purposes if misused AND you decide to not apply a mechanism to prevent this type of use because there is no explicit expectation to do so Downloading an exploit kit to your company workstation as part of your job to analyze malware for improved defense but do via your normal desktop and not a VM by accident, exposing your IP address and introducing a backdoor Ignoring the possible vulnerability in a medical computing system and not applying the patch because the system has been offline for greater than 180 days and thus you assume it to be decommissioned per normal operating procedure
19. Recklessness v Negligence Recklessness How do you test for recklessness? Knowingly unsafe or unlawful performance of an otherwise legal act, more or less obvious DUI Criminal charges result Reasonable doubt Negligence How do you test for negligence? Accidental behavior without reckless, careless or expressly-unlawful behavior Civil charges result Preponderance of the evidence
20. Negligence Five elements are required to satisfy a legal burden of proof: The existence of a legal duty to exercise reasonable care A failure to exercise reasonable care Cause in fact of physical harm by the negligent conduct Physical harm in the form of actual damages Proximate cause http://www.law.cornell.edu/wex/negligence
21. Reasonable Care The degree of caution and concern for the safety of one’s self and others a reasonable personwould use under the circumstances. This test is subjective.
22. Reasonable Care Do the parents of a child have a legal duty to feed, clothe, and care for their child? Official Code of Georgia Annotated “Neglect or exploitation of a child by a parent or caretaker if said neglect or exploitation consists of a lack of supervision, abandonment, or intentional or unintentional disregard by a parent or caretaker of a child’s basic needs for food, shelter, medical care, or education as evidenced by repeated incidents or a single incident which places the child at substantial risk of harm.” O.C.G.A. § 49-5-180(5)(B).
23. Reasonable Care Does someone with not familial or custodial connection to a given child have a legal duty to feed, clothe, and care for said child? Would such a legal duty be reasonable?
24. Actual Causation How do you know if something is the actual cause of an event? But-For Test “But for the existence of x, would y have occurred? If the answer is yes, x is the actual cause of y. Example “But for the open laceration in the cat’s neck, the cat would not have exsanguinated. Therefore, the open laceration in the cat’s neck is the actual cause of its death.”
25. Poor Kitty
26. Proximate Cause The But-For Test shows causation, but does not assign culpability – the actual moral or legal blame. Proximate Cause is a test which determines if an action can be the legally culpable cause of the harm. “Bob stabbed the cat in the neck with a box cutter. This caused a laceration which in turn caused the cat’s death by exsanguination. Therefore, Bob is culpable for the cat’s death because he stabbed the cat in the neck.”
27. Proximate Cause Proximate Cause limits the responsibility for negligent behavior. “The cat was Bob’s wife’s most prized possession. Bob stabbed the cat in retaliation for learning that his wife was having an affair. The man she was having an affair with left the box cutter in Bob’s house during their latest tryst.” Is Bob’s wife’s lover culpable for the cat’s death since Bob would never have had access to the box cutter if it was not left behind by the lover?
28. Palsgraf v Long Island Railroad Co. 1928 At a railroad station, a passenger holding a package hurried to catch a moving train. Two RR employees believed that the passenger was going to fall off the rain, so they physically helped the person onto the train. During the course of this action, the passenger dropped their package and it fell onto the rails. The package was a wrapped parcel of fireworks, which exploded when it hit the rails.
29. Palsgraf v Long Island Railroad Co. The shock from the explosion knocked down scales located upon a ledge at the other end of the platform. These scales injured Palsgraf, the plaintiff. Palsgraf sued the railroad for injury caused by the negligent behavior of the railroad employees.
30. Palsgraf v Long Island Railroad Co. The courts originally found in favor of Palsgraf, but the NY Court of Appeals reversed upon appeal. “There was nothing in the situation to suggest to the most cautious mind that the parcel wrapped in newspaper would spread wreckage through the station. If the guard had thrown it down knowingly and willfully, he would not have threatened the plaintiff's safety, so far as appearances could warn him.” This illustrated the need for proximate cause and introduced the concept of foreseeability.
31. Foreseeability Limits liability to the consequences of an act that could reasonably be foreseen. “Bob stabbed the cat in the neck with a box cutter. There is no reasonable expectation that the cat could have survived as a result; therefore, Bob has no defense under a lack of foreseeability and is subject to proximate cause.” “Bob’s wife’s lover accidentally left behind his box cutter in Bob’s house. There is no reasonable expectation that he could have foreseen the misuse of said box cutter by Bob upon the cat. Therefore, Bob’s wife’s lover cannot be held liable under proximate cause.”
32. Foreseeability?
33. Hakkar, Revisited
34. Hakkar, Revisited Sheila is a subscriber to World of Warcraft. $60 down payment, $15 per month to play. For a little over a month, she finds playing the game undesirable due to the Corrupted Blood plague and believes that she is no longer receiving the kind of game play and service that she normally expects. Sheila then decides to file a civil claim against Blizzard to seek remediation for her damages. What result?
35. What is the situation? A consumer of a software product has experienced a reduction in the quality of the product and service. This negative experience is a direct result of a software defect which made the “Corrupted Blood Plague” possible. Despite efforts taken by Blizzard satisfy the problem, the negative situation is perpetuated by other players of the game who delight in sacrificing their normal game play in order to keep the plague spreading.
36. What is the complaint? Sheila believes that she has suffered damages in the form of a $60 software fee and a $15 per month subscription to play an online game which is unplayable due to a software bug. Sheila argues that Blizzard is liable for the damages because they are liable for the defects in their product, World of Warcraft.
37. Product Liability The liability of manufacturers, sellers, lessors, and others for injuries caused by defective products Injury can be physical, death, loss of revenue or increase in expense due to the software failure.
38. Product Liability Software product liability claims are typically based on one or more of the following: Strict liability Negligence Breach of warranty Misrepresentation
39. Sheila v Hakkar the Soulflayer Is Blizzard liable for Sheila’s damages? Strict liability – “a defect exists, therefore liable for damages caused” Sheila: “Blizzard released a defective product, the defect caused me injury, and therefore Blizzard is liable.” Blizzard: “The Corrupted Blood debuff was intentional and not a software bug. Any attempt to seek remediation for perceived damages is unreasonable, as this was not a software defect.”
40. Sheila v Hakkar the Soulflayer Is Blizzard liable for Sheila’s damages? Negligence – “failure to be reasonable” Sheila: “The Corrupted Blood Plague is the result of a lack of reasonable code testing and quality control, which caused me to suffer damages. Blizzard is therefore liable.” Blizzard: “Upon the first outbreak of the plague, we immediately released hotfixes and took every reasonable course of action to prevent the plague from spreading. The players contributed to any perceived negligence, as if it were not for the malicious users the plague condition would have been eliminated within a day, after realm server resets. As it was, only the major cities in game were affected by the plague, so it would have been possible to play outside of the infected area with no loss to the player’s gaming experience.”
41. Sheila v Hakkar the Soulflayer Is Blizzard liable for Sheila’s damages? Breach of warranty – “standard of quality” Sheila: “Blizzard breached their warranty when World of Warcraft became unplayable due to a reduced quality of the product.” Blizzard: “World of Warcraft is a game played with other players. Our warranty does not cover the actions of other players, as they are just as free to act within the game as Sheila is.”
42. Sheila v Hakkar the Soulflayer Is Blizzard liable for Sheila’s damages? Misrepresentation – “you lied to me!” Sheila: “Blizzard misrepresented their software product when they stated that it would be available to play. The plague incident made it impossible for me to play the game as intended, therefore Blizzard owes me damages.” Blizzard: “We never explicitly stated that there would never be an outbreak of a virtual virus in the virtual world of Azeroth. It is unreasonable to expect that such a claim can be read from our product advertisements and Sheila’s damages result in her choosing to not play the game, not from a denial of service.”
43. Sorry Sheila
44. Software Development
45. Software Development Life Cycle
46. Systems Analysis & Design @ UNG CSCI 3300 This course goes in-depth on this issue
47. Software Testing Dynamic Testing Actually running the program, interacting with the system in a test environment to simulate actual conditions to discover bugs Black Box The actual source code or system design is unknown. All tests are performed “in the dark” White Box Internal design is known. Allows for more robust testing, but could discover bugs that would not otherwise be found
48. Software Testing Static Automated software analyzes source code, looking for specific patterns which indicate defect Integration Software or system is integrated into a subsystem to test for defects in the integration System All subsystems are combined, focusing testing on the overall system performance User Acceptance Independent testing performed by the end user to see if expected quality is met
49. Success? Have you designed the product that your customer expects? Have you cut corners or gold plated anything? Has the product met with all quality and safety requirements? Standards? Laws? Warranties? Are you prepared to defend against liability? Even with 100% ethical behavior, lawsuits are a likely possibility. Is your due diligence documented? Have you taken reasonable care?