COSC4607 - Computer Security

1. COSC4607 - Computer Security • Dr. Haibin Zhu • Assistant Professor • Department of CS and Math, Nipissing University • Room: A124A, Ext.: 4434 • Email: haibinz@nipissingu.ca • URL: http://www.nipissingu.ca/faculty/haibinz • Office Hour: Mon. & Thurs. 2:30pm-4:30pm; or by appointment

2. Learning Outcomes • On completion of this course you will be able to: • Understand a number of basic design principles in computer security. • Demonstrate an understanding of the importance of security models with reference to the security of computer systems. • Describe the features and security mechanisms which are generally used to implement security policies. • Provide examples of the implementation of such features and mechanisms within a number of particular operating systems. • Display a breadth of knowledge of the security vulnerabilities affecting computer systems. • Demonstrate an understanding of the main issues relating to Web security in the context of computer systems.

3. Schedule • Introduction to Computer Security • Access Control • Security Models • Security Mechanisms • Linux / Unix Security • Windows 2000 Security • Computer Virus • Software Security • Security of Distributed Systems • Network Security • Cryptography

4. Resources Notes Lecturers Books The Internet … and YOU and YOUR FELLOW STUDENTS

5. Books • Textbooks: • D. Gollmann, Computer Security, John Wiley & Sons, 1999. • References: • C.P. Pfleeger, Security in Computing, Prentice-Hall, 1997 (2nd Ed). • R. Anderson, Security Engineering, Wiley, 2001. • B. Schneier, Secrets and Lies, Wiley, 2000. • John Viega & Gary McGraw: Building Secure Software, Addison Wesley, 2001

6. http://www.nipissingu.ca/faculty/haibinz Course Information

7. Questions ?

8. Review: Operating Systems (1) • It is useful to have some basic understanding of what an Operating System is and what it does. • An Operating System is an application that “sits” between users and applications and the computer hardware. Essentially, it provides a moderately friendly interface to the hardware. • Because computers “think binary” then they are extremely user-unfriendly! The Operating System removes some of the pain!

9. User Application Program Operating System Hardware Review: Operating Systems (2)

10. User Graphical User Interface (GUI) Application Program Command Interpreter System Services Operating System Review: Operating Systems (3) A more common version of the previous diagram is given below:

11. Review: Operating Systems (4) • The “System Services” layer is the real interface to the Operating System. It consists of many functions designed to translate an application request into something the Operating System can handle. • POSIX is becoming the standard for System Services, mainly driven by the need to make it easier to port an application from one Operating System to another. • So, what does an Operating System actually do? • The primary functions of an Operating System are given on the next slide. • Note that an Operating System must perform all its tasks efficiently and economically. Resource use by the system, in terms of CPU time, memory and disk usage must all be acceptable for the users.

12. Review: Operating Systems (5) • Starting and stopping programs and sharing the CPU between them • Managing memory: • Memory allocation; keeping track of memory usage • Input and Output: • Device drivers; “concurrent” handling of I/O devices • File management • Protection (preventing different programs from interfering with each other – “firewalling”) • Networking (“seamless” communication with other devices) • Error handling (detection, recovery, warning)

13. System Service Interface File Manager Process Manager Memory Manager I/O Manager Network Manager Hardware Interface Review: Operating Systems (6)

14. Security Problems • Security and reliability • Buffer overflows • Arrays and integers • Canonical representations • Race conditions • Precautions & defences • Dangers of abstraction

15. Security & Reliability • On a PC, you are in control of the software components sending inputs to each other • On the Internet, hostile parties can provide input • To make software more reliable, it is tested against typical usage patterns: ‘it does not matter how many bugs there are, it matters how often they are triggered’ • To make software more secure, it has to be tested against ‘untypical’ usage patterns (but there are typical attack patterns)

16. Secure Software • Software is secure if it can handle intentionally malformed input; the attacker picks (the probability distribution of) the inputs • Secure software: protect the integrity of the runtime system • Secure software ≠ software with security features • Networking software is a popular target • intended to receive external input • involves low level manipulations of buffers

17. Preliminaries • In code written in a typical programming language values are stored in variables, arrays, etc. • To execute a program, memory sections (‘buffers’) have to be allocated to variables, etc. • In programming languages like C or C++ the programmer allocates and de-allocates memory • Type-safe languages like Java guarantee that memory management is ‘error-free’

18. Buffer overflows • If the value assigned to a variable exceeds the size of the buffer allocated to this variable, memory locations not allocated to this variable are overwritten • If the memory location overwritten had been allocated to some other variable, the value of that other variable can be changed • Depending on circumstances, an attacker could change the value of a protected variable A by assigning a deliberately malformed value to some other variable B

19. Buffer overflows • Unintentional buffer overflows can make software crash • Intentional buffer overflows are a concern if an attacker can modify security relevant data • Attractive targets are return address (specifies the next piece of code to be executed) and security settings • In safe languages such errors cannot occur

20. stack FFFF memory 0000 heap Stack & heap • Stack: contains return address, local variables and function arguments; it is quite predictable where a particular buffer will be placed on the stack • Heap: dynamically allocated memory; it is more difficult but by no means impossible to predict where a particular buffer will be placed on the heap

21. return address my_address write to A: value1| value2| my_address value2 value1 buffer for variable A Stack overflows • Find a buffer on the runtime stack of a privileged program that can overflow the return address • Overwrite the return address with the start address of the code you want to execute • Your code is now privileged too

22. Precautions • Be sparing with privileges: if the code attacked runs with few privileges, the damage is limited • If feasible, use a type-safe language • In C and C++ avoid dangerous instructions like gets() and use instructions like fgets() where the length of the argument has to be specified; extensive lists of ‘good’ and ‘bad’ instructions exist • Non-executable stack: stack cannot be used to store attack code

23. Defences • Canaries: (random) check values placed before the return address (StackGuard) • Before returning, check that the canary still has the correct value return address my_address write to A: value1| value2| my_address to A check value value2 ≠ check value canary value1 buffer for variable A attack detected

24. Heap overflows • More difficult to determine how to overwrite a specific buffer • More difficult to determine which other buffers will be overwritten in the process; if you are an attacker, you may not want to crash the system before you have taken over • Attacks that do not succeed all the time are a threat • Heap overflow attacks have started to occur

25. Integers • Mathematics: integers form an infinite set • Programming languages: signed 4-byte integers, unsigned 4-byte integers, long integers,… • Truncation (Unix example): input UID as signed integer, value ≠ 0?, truncate to unsigned short integer: 0x10000  0x0000 (root!) • Different interpretation of signed and unsigned integers: (signed) -1 = 0xFFFF = 216-1 (unsigned)

26. Integers • Addition: 0xFF00 + 0x0100 = 0x0000: base + offset < base • Of course, the runtime may raise an exception when an overflow occurs on addition • Ashcraft & Engler [IEEE S&P 2002]: “Many programmers appear to view integers as having arbitrary precision, rather than being fixed-sized quantities operated on with modulo arithmetic.”

27. allocate 4 elements assign 5 elements base base base Arrays • Buffer overflow: the length of an array is not checked when elements are written to the buffer • Wrap-around to lower addresses: when arithmetic operations do not have the result expected by the programmer

28. Scripting • In scripting languages, executables can be passed as arguments • Escape characters indicate that an argument is an executable • ‘Unescaping’ (making input non-executable) can protect code from malicious input • Filters for escape characters have to know the escape characters and the character set in use [CA-2000-2]

29. Canonical Representations • File names, URLs, IP addresses, … can be written in more than one way • Directory traversal: c:\x\data = c:\y\z\..\..\x\data = c:\y\z\%2e%2e\%2e%2e\x\data • Dotless IP: a.b.c.d  a224 + b216 + c28 + d • Symbolic link: file name pointing to another file • Canonicalization computes the standard representation • When access right depends on location, you better get the location right; do not rely on the names received as input

30. Race conditions • Multiple computations access shared data in a way that their results depend on the sequence of accesses • Multiple processes accessing the same variable • Multiple threads in multi-threaded processes (as in Java servlets) • An attacker can try to change a value after it has been checked but before it is being used • TOCTTOU (time-to-check-to-time-of-use) is a well-known security issue

31. Example – CTSS (1960s) • Once the message of the day was the password file • Every user had a unique home directory; when a user invoked the editor, a scratch file with fixed name SCRATCH was created in this directory • Several users working as system manager: • system manager one starts to edit message of the day: SCRATCH  MESS • system manager two starts to edit the password file: SCRATCH  PWD • system manager two stores the edited file: MESS  PWD

32. Defences – Code inspection • Code inspection is tedious: we need automation • K. Ashcroft & D. Engler: Using Programmer-Written Compiler Extensions to Catch Security Holes, IEEE S&P 2002 • Meta-compilation for C source code; ‘expert system’ incorporating rules for known issues: untrustworthy sources  sanitizing checks  trust sinks; raises alarm if untrustworthy input gets to sink without proper checks • Code analysis to learn new design rules: Where is the sink that belongs to the check we see? • Applied to Linux and OpenBSD kernels

33. Defences – Black-box testing • Black-box testing when source code is not available • You do not need the source code to observe how memory is used or to test whether inputs are properly checked • Oulu University Secure Programming Group: PROTOS project (Juha Röning, http://www.ee.oulu.fi/research/ouspg/) • Syntax testing of protocols based on formal interface specification, valid cases, anomalies • Applied to SNMP implementations

34. Defences – Type-safety • Type safety: guarantees absence of untrapped errors • Cardelli: Practitioners who invented type safety often meant just “memory integrity”, while theoreticians always meant “execution integrity”, and it’s the latter that seems more relevant now. • A language does not have to be typed to be safe: LISP • Safety guaranteed by static checks and by runtime checks

35. Type-safety • Marketing ploy: “We are type safe, therefore we are secure” • Type safety is difficult to prove completely • Proofs are conducted in an abstract model, problems may hide in the actual implementation (e.g. SUN Security Bulletin #00218) • PROTOS: Also software in Java were shown to have buffer overflows in native code sections • Type safety is a useful property to have for security but type safety does not imply security

36. Keeping up-to-date • Sources of information: CERT advisories, BugTraq at www.securityfocus.com, security bulletins from software vendors • Hacking tools have attack scripts that automatically search for and exploit known type of vulnerabilities • Analysis tools following the same ideas will cover most real attacks • Patching vulnerable systems in not easy: you have to get the patches to the users and avoid introducing new vulnerabilities through the patches

37. number of intrusions Time disclosure attack scripts released patch released Intrusion patterns W. Arbaugh, B. Fithen, J. McHugh: Windows of Vulnerability: A Case Study Analysis, IEEE Computer, 12/2000

38. The wider picture • We could treat all these problems individually and look for specific solutions, often limited to a given programming language or runtime system (penetrate-and-patch at a meta-level) • Overall, a general pattern: familiar programming abstractions • variable, array, integer, data & program, address (resource locator), atomic transaction, … • are implemented in a way that can break the abstraction

39. Summary of Security Problems • Many of the problems listed may look trivial • There is no silver bullet • Code-inspection: better at catching known problems, may raise false alarms • Black-box testing: better at catching known problems • Type safety: guarantees from an abstract (partial) model need not carry over to the real system • Experience in high-level programming languages may be a disadvantage when writing low level network routines

40. Introduction to Computer Security • Security objectives • Security strategies • Distributed systems security = computer & communications security • Aspects of computer security • Fundamental design principles for • computer security

41. Security objectives • Confidentiality: prevent unauthorized disclosure of information • Integrity: prevent unauthorized modification of Information • Availability: prevent unauthorized withholding of information or resources • Other aspects: accountability, authenticity

42. Confidentiality • Historically, security and secrecy were closely related; sometimes, security and confidentiality are used as synonyms • Prevent unauthorized disclosure of information (prevent unauthorized reading) • Privacy: protection of personal data • Secrecy: protection of data belonging to an organization

43. Integrity • ITSEC Definition: The property that prevents unauthorized modification of information (prevent unauthorized writing) • Orange Book (US Trusted Computer Systems Evaluation Criteria): • Data Integrity - The state that exists when computerized data is the same as that in the source document and has not been exposed to accidental or malicious alteration or destruction (integrity synonymous for external consistency) • In communications: detection and correction of intentional and accidental modifications of transmitted data

44. Availability • IS 7498-2 ( Basic Reference Model for Open Systems Interconnection (OSI) Part 2: Security Architecture): The property of being accessible and usable upon demand by an authorized entity • Denial of Service (DoS): The prevention of authorized access of resources or the delaying of time-critical operations • Distributed denial of service (DDoS) is receiving a lot of attention and systems are now designed to be more resilient against these attacks

45. Accountability • Audit information must be selectively kept and protected so that actions affecting security can be traced to the responsible party.

46. Dependability • The property of a computer system such that reliance can justifiably be placed on the service it delivers. Here, the service delivered by a system is its behavior as it is perceived by its users; a user is another system which interacts with the former.

47. A Remark on Terminology • Definition: Computer security deals with the prevention and detection of unauthorized actions by users of a computer system. • There is no single definition of security • When reading a document, be careful not to confuse your own notion of security with that used in the document • A lot of time is being spent - and wasted – trying to define an unambiguous notation for security • Resources: • http://www.radium.ncsc.mil/tpep/process/faq.html • http://www.cesg.gov.uk/assurance/iacs/itsec/index.htm • ftp://ftp.cse-cst.gc.ca/pub/criteria/CTCPEC

48. Security strategies • Prevention: take measures that prevent your assets from being damaged • Detection: take measures so that you can detect when, how, and by whom an asset has been damaged • Reaction: take measures so that you can recover your assets or to recover from a damage to your assets

49. Example 1– Private Property • Prevention: locks at doors, window bars, walls round the property • Detection: stolen items are missing, burglar alarms, closed circuit (cable) TV • Reaction: call the police, replace stolen items, make an insurance claim … • Footnote: Parallels to the physical world can illustrate aspects of computer security but they can also be misleading

50. Example 2 – E-Commerce • Prevention: encrypt your orders, rely on the merchant to perform checks on the caller, don’t use the Internet (?) … • Detection: an unauthorized transaction appears on your credit card statement • Reaction: complain, request new card number, etc.