Integer Overflows

1. Integer Overflows James Walden Northern Kentucky University

2. Topics • Computer Integers • Integers in C and Java • Overflow Examples • Checking for Overflows CSC 666: Secure Software Engineering

3. Integer Overflow December 25, 2004 Flight crew scheduling software stopped. Cancelled all 1100 flights that day. What happened? Winter weather led to many crew changes. Number of changes > 32,767 CSC 666: Secure Software Engineering

4. Integers Computer integers are not the same set of numbers as mathematical integers. • Finite set, not infinite. • What happens when integer calculations result in a number outside that set? CSC 666: Secure Software Engineering

5. Unsigned Integers 0 1 7 000 111 001 2 110 010 6 011 101 100 3 5 4 CSC 666: Secure Software Engineering

6. Two’s Complement Two’s complement = One’s complement + 1. Sign is represented by most significant bit. Range: -2n-1..2n-1-1, only one representation of 0. +75 0 1 0 0 1 0 1 1 Comp 1 0 1 1 0 1 0 0 +10 0 0 0 0 0 0 1 -75 1 0 1 1 0 1 0 1 CSC 666: Secure Software Engineering

7. Two’s Complement 0 1 -1 000 111 001 2 -2 110 010 011 101 100 3 -3 -4 CSC 666: Secure Software Engineering

8. CSC 666: Secure Software Engineering

9. C Integers CSC 666: Secure Software Engineering

10. Java Integers CSC 666: Secure Software Engineering

11. Java Factorial Program public static void main(String args[]) { long product = 1; for(int i = 1; i <= 21; i++) { System.out.print(i); System.out.print("! = "); product *= i; System.out.println(product); } } CSC 666: Secure Software Engineering

12. Output 1! = 1 2! = 2 3! = 6 …. 20! = 2432902008176640000 21! = -4249290049419214848 CSC 666: Secure Software Engineering

13. Java BigInteger Class import java.math.BigInteger; public class BigFactorials { public static void main(String args[]) { BigInteger product = BigInteger.ONE; BigInteger index = BigInteger.ONE; for(int i = 1; i <= 21; i++) { System.out.print(i); System.out.print("! = "); product = product.multiply(index); System.out.println(product); index = index.add(BigInteger.ONE); } } } CSC 666: Secure Software Engineering

14. Output 1! = 1 2! = 2 3! = 6 …. 20! = 2432902008176640000 21! = 51090942171709440000 CSC 666: Secure Software Engineering

15. Problems of Integer Overflows Difficult to detect after they’ve happened. • Compilers generally ignore them. • Assembly code can check carry flag, but high level languages can’t without calling assembly code. Difficult to avoid. • Subtle bugs can result in integer overflows. CSC 666: Secure Software Engineering

16. Integer Overflows in Voting Broward County 2004 election Amendment 4 vote was reported as tied. Software from ES&S Systems reported a large negative number of votes. Discovery revealed that Amendment 4 had passed by a margin of over 60,000 votes. CSC 666: Secure Software Engineering

17. TKADV2009-002 Integer overflows in Amarok media player. • Reads input size + input from file. • Allocates input size + 1 bytes, which can be very small. • Reads file data into very small buffer, leading to a buffer overflow. CSC 666: Secure Software Engineering

18. Strip Extension void StripExtension(char * filename) { unsigned short int newSize = strlen(filename) - 4; char * buffer = (char *)malloc(newSize + 1); strncpy(buffer, filename, newSize); buffer[newSize] = ‘\0’; printf(“%s”, buffer); free(buffer); } CSC 666: Secure Software Engineering

19. Valid Use What would happen if StripExtension were called as follows?StripExtension(“a.txt”); CSC 666: Secure Software Engineering

20. Invalid Use What would happen if StripExtension were called as follows?// User failed to include the extension. StripExtension(“a”); CSC 666: Secure Software Engineering

21. Answer • newSize = 0xffffd = (1 minus 4) = -3 • newSize is an unsigned short integer • This value is 65533. • The function creates a 65534-byte buffer. CSC 666: Secure Software Engineering

22. Unsigned Addition An unsigned addition unsigned int x, y, sum; sum = x + y; Precondition if( x > UINT_MAX – y) /* error */ Postcondition if( (x >= 0 && sum < y) || (x < 0 && sum > y) ) /* error */ CSC 666: Secure Software Engineering

23. Signed Addition Preconditions CSC 666: Secure Software Engineering

24. Integer Multiplication Overflow CESA-2004-001: libpng info_ptr->row_pointers = (png_bytepp)png_malloc(png_ptr, info_ptr->height * sizeof(png_bytep)); If height > INT_MAX / sizeof(png_bytep) Size of new buffer will be a small integer. User data in image file can overflow buffer. CSC 666: Secure Software Engineering

25. Widening Conversions A conversion from a type with a smaller range of values to type with a larger range of values. Examples: byte -> short, short -> long Sign extension Propagates signed bit from source type to all unused bits in destination type. Magnitude and sign are preserved. CSC 666: Secure Software Engineering

26. Widening Conversion Example Source type: byte Value: -7 1 1 1 1 1 0 0 1 Destination type: short Value: -7 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 1 CSC 666: Secure Software Engineering

27. Narrowing Conversions Conversions from a wider type to a narrower type. Examples: long -> byte, int -> short Truncation Bits from source type that don’t fit into narrower destination type are discarded. Magnitude and sign may change. CSC 666: Secure Software Engineering

28. Narrowing Conversion Example Source Type: short Value: 257 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 Destination Type: byte Value:1 0 0 0 0 0 0 0 1 CSC 666: Secure Software Engineering

29. Sign Extension Vulnerability CERT CA-1996-22: bash yy_string_get() reads user data as chars. Each char converted to an int when parsed. A char value of 255 sign extended to int -1. Integer -1 means command separator. Example exploit bash -c 'ls\377who' CSC 666: Secure Software Engineering

30. Range Checking Check that integer ranges are valid. Be more specific than INT_MIN, INT_MAX. Liquid water temperatures range 0..100. Use type system to check. Some languages allow integer ranges. Create abstract data types in languages that don’t provide integer range types. CSC 666: Secure Software Engineering

31. Proposal: Ranged Integers in C All integer types can be ranged. Static: range determined at compile time. Dynamic: range determined at run time. Semantics Saturation: values beyond range = max. Wrap: values wrap to bottom of range. Examples Saturation: int 0|..|100 temperature = 0 Wrap: long min<..>max circular; CSC 666: Secure Software Engineering

32. Compiler Checks Microsoft VS 2005 CL • Runtime integer error checks: /RTCc • Use highest warning level /W4 • Check for #pragma warning(disable, C####) GCC • Runtime integer error checks: -ftrapv • Use integer-relevant warnings: -Wconversion –Wsign-compare • Check for #pragma GCC diagnostic ignored option CSC 666: Secure Software Engineering

33. Secure Integer Libraries IntegerLib • Designed for C, but usable in C++. • Available from CERT. IntSafe • C library written by Michael Howard. • Uses architecture specific inline assembly. SafeInt • C++ template class from David LeBlanc. CSC 666: Secure Software Engineering

34. SafeInt<T> C++ Class int main(int argc, char *const *argv) { try { SafeInt<unsigned long> s1(strlen(argv[1])); SafeInt<unsigned long> s2(strlen(argv[2])); char *buff = (char *) malloc(s1 + s2 + 1); strcpy(buff, argv[1]); strcat(buff, argv[2]); } catch(SafeIntException err) { abort(); } } CSC 666: Secure Software Engineering

35. When to use Secure Int Libraries? Use Secure Integer libraries when Integers come from untrusted sources. Don’t use Secure Integer libraries when Integers not influenced by external sources. Tight loops: check int values before loop. CSC 666: Secure Software Engineering

36. Integer Overflow: Key Points Integer arithmetic. • Two’s complement format signed ints. • Know your language’s integer conversions. Impact of integer overflows • Can be used to defeat bounds checks. • Influence important data, like vote counts. Mitigating integer overflows. • Precondition or postcondition testing. • Use safe integer libraries where possible. CSC 666: Secure Software Engineering

37. References • Brian Chess and Jacob West, Secure Programming with Static Analysis, Addison-Wesley, 2007. • Jeff Gennari et. al., Ranged Integers for the C Programming Language. CMU/SEI-2007-TN-027, 2007. • Michael Howard and David LeBlanc, Writing Secure Code, 2nd edition, Microsoft Press, 2003. • Robert C. Seacord, Secure Coding in C and C++, Addison-Wesley, 2006. • Robert C. Seacord, CERT Secure Coding Standards: Integers, https://www.securecoding.cert.org/confluence/display/seccode/04.+Integers+(INT), 2009. • John Viega and Gary McGraw, Building Secure Software, Addison-Wesley, 2002. • David Wheeler, Secure Programming for UNIX and Linux HOWTO, http://www.dwheeler.com/secure-programs/Secure-Programs-HOWTO/index.html, 2003.

38. James Walden Northern Kentucky University Error Handling

39. Topics • Error Handling • Return Codes • Exceptions • Logging • Survivability

40. Security Impact of Error Handling Information leakage • Stack traces • Database errors Resource leakage • Return on error without de-allocation • Exceptions bypass de-allocation CSC 666: Secure Software Engineering

41. Error Handling Techniques Return a neutral value: return a value that’s known to be harmless, i.e. 0 or “”. Substitute the next piece of data: continue reading from hardware or file until a valid record is found. Return same answer as last time: don’t keep reading; instead return the last valid answer. Substitute closest legal value: if velocity has a range of 0..100, show a 0 when backing up. Log a warning message: Write a warning to a log, then continue on, perhaps using one of the other techniques. Terminate program: Terminate program execution. Return an error code: Report error by • Setting the value of a status variable (errno) • Return status as the function’s return value • Throw an exception CSC 666: Secure Software Engineering

42. Return Codes Use function return code to indicate error. • Easy to ignore. Simply ignore return code. • Error handling logic is mixed with logic processing normal return codes. • No universal convention for error codes. Common return code patterns. • Negative values when nonnegative expected. • NULL values for pointer return codes. CSC 666: Secure Software Engineering

43. Example: character get functions fgetc(), getc(), getchar() read char, return int Use int to represent EOF error code. Incorrect example: return value is declared as a char char buf[BUFSIZ]; char c; int i = 0; while ( (c = getchar()) != '\n' && c != EOF ) { if (i < BUFSIZ-1) { buf[i++] = c; } } buf[i] = '\0'; /* terminate NTBS */ Correct example char buf[BUFSIZ]; int c; int i = 0; while ( ((c = getchar()) != '\n') && !feof(stdin) && !ferror(stdin)) { if (i < BUFSIZ-1) { buf[i++] = c; } } buf[i] = '\0'; /* terminate NTBS */ CSC 666: Secure Software Engineering

44. Example: sprintf() Incorrect example: sprintf returns -1 on error, count can be out of bounds int i; ssize_t count = 0; for (i = 0; i < 9; ++i) { count += sprintf( buf + count, "%02x ", ((u8 *)&slreg_num)[i] ); } count += sprintf(buf + count, "\n"); Correct example uses sprintf_m function f/ CERT managed string library int i; rsize_t count = 0; errno_t err; for (i = 0; i < 9; ++i) { err = sprintf_m( buf + count, "%02x ", &count, ((u8 *)&slreg_num)[i] ); if (err != 0) { /* Handle print error */ } } err = sprintf_m( buf + count, "%02x ", &count, ((u8 *)&slreg_num)[i] ); if (err != 0) { /* Handle print error */ } CSC 666: Secure Software Engineering

45. Resource Leaks Resources leak due to early returns • Memory • Filehandles Example char *getblock(int fd) { char *buf = (char *)malloc(1024); if (!buf) { return NULL; } if (read(fd, buf, 1024) != 1024) { return NULL; } return buf } CSC 666: Secure Software Engineering

46. Using goto for error handling Problem: de-allocate resources on return • Each return is different since • Different resources allocated at each point. Solution: single de-allocation point • Check if resource is allocated, then • De-allocate if it is, and • Return with appropriate error code. Why goto? • Avoids deep nesting. • Improves code readability. • Commonly used technique in kernel. CSC 666: Secure Software Engineering

47. Fixed version with goto char *getblock(int fd) { char *buf = (char *)malloc(1024); if (!buf) { goto ERROR; } if (read(fd, buf, 1024) != 1024) { goto ERROR; } return buf; ERROR: if (buf) { free(buf); } return NULL; } CSC 666: Secure Software Engineering

48. Exceptions Advantages of exceptions • Cannot be ignored by not checking for errors. • Separate main code from error code. Disadvantages of exceptions • Difficult to avoid resource leaks, as exceptions create many implicit control flow paths. • Can still ignore exceptions try { // code that can throw an exception } catch (AnException e) { // empty catch block } CSC 666: Secure Software Engineering

49. Checked Exceptions Checked exceptions: Exceptions that the language requires client code to handle. • C++, C#: no checked exceptions • Java: exceptions that inherit from Exception Unchecked exceptions: Exceptions that can be ignored by client code. • C++, C#: all exceptions are unchecked • Java: exceptions that inherit from RuntimeException. CSC 666: Secure Software Engineering

50. Exception Guarantees Levels of exception safety for a class. Basic Guarantee • No resources are leaked. Strong Guarantee • Exceptions leave state exactly as it was before the operation started. No Throw Guarantee • Component will handle all exceptions itself. No Exception Safety • Component may leak resources and leave object in an inconsistent unusable state. CSC 666: Secure Software Engineering