Streams and Input/Output

1. Streams and Input/Output References: Horstmann: 11.1, Chapter 19 Horstmann& Cornell: Core Java Volume I – Fundamentals Chapter 12

2. Basic principles Program needs I/O for Source/Destination • primitive types (int, float, double, …) - console • Strings - files • objects - internet • bytes - byte arrays -other • Program ideal • Create an object do to the reading, e.g., TypeOfInDevice myInDevice = new TypeOfInDevice( ….); • The type of the object will reflect • where it will be read from • (perhaps) what types need to be read • Use method names independent of the specific source/destination e.g., int i = myInDevice.readInt(); Object obj = myInDevice.readObject(); • How can this be done? • Define an interface/abstract class for the I/O operations with multiple implementations.

3. Two basic systems: • I/O of text data • I/O of binary data • Consider input of text data e.g., Scanner consoleIn = new Scanner(System.in); int i = consoleIn.nextInt(); e.g., Scanner fileIn = new Scanner(new File(fileName)); int i = fileIn.nextInt(); e.g., Scanner stringScan = new Scanner(“ 345 Smith 4.5 “); int i = stringScan.nextInt(); Note that if the String literal for a fileName contains a backslash (\), two backslashes must be used because backslash is used as an escape character within Java strings.

4. Technically, Scanner is not part of the I/O system • The Scanner obtains characters from some other object, often an I/O object, and processes them into the primitive types and Strings. • The other object is passed in as a parameter to the Scanner constructor. • Within the Scanner, the other object is stored in the field • private Readable source; • Readable is an interface with only one (abstract) method public int read(java.nio.CharBuffer cb) When the method is called, the CharBuffer must be non-null, and the implementation of Readable must enter characters into the buffer. It returns the number of characters entered into the buffer. • Hence, Scanner uses the Readable object to “read” characters into a character buffer, and then the Scanner processes these characters to construct the values returned. • Readable is an interface with many implementations dependent upon • from where the characters come • what processing is required for the characters

5. Types for the parameter of the Scanner constructor, and implementations of Readable for the “source” field of Scanner. parameter typeReadable implementations String StringReader File FileReader InputStream InputStreamReader Readable any of the above and others e.g., BufferedReader Note: System.in accesses the static field called “in” that has type InputStream. • The constructor of Scanner uses the type of the parameter to determine what type of Readable object to create to read the data. • Where do these Readable objects obtain the characters? StringReader from the String parameter the characters of the String FileReader from a FileInputStream converts the bytes in the file to characters InputStreamReader from an InputStream converts the bytes of the stream to characters BufferedReader from another Reader the characters of the Reader Note that Strings and the CharBuffer used by Scanner store Unicode characters. The sources of the characters might not use Unicode characters, e.g., text files usually store ASCII characters. The Readable class does the conversion when necessary.

6. Steps e.g., Scanner obtains Unicode characters from FileReader that obtains bytes from a FileInputStream e.g., Scanner obtains Unicode characters from InputStreamReader that obtains bytes from an InputStream (eg System.in) read() read() i.e., Scanner Reader InputStream char byte • A BufferedReader acts as an intermediary doing buffering. read() read() read() i.e., Scanner BufferedReader Reader InputStream char char byte Note that there are two types of Readers: Generator Reader Reader that gets characters/bytes from a non-Reader. For example: StringReader, FileInputReader, and InputStreamReader. Decorator/filter Reader Reader that gets characters from another Reader. For example: BufferedReader. They add something to the character stream. More than one decorator is possible.

8. Simplifying property: • Most of the time, the characters needed by the Scanner are originally bytes from some source. The source is the parameter supplied to the Scanner constructor. The Scanner automatically creates the correct type Reader to read from the source. If an InputStream is needed, the Scanner also creates it. e.g., Scanner consoleIn = new Scanner(System.in); // System.in is an InputStream int i = consoleIn.nextInt(); e.g., Scanner fileIn = new Scanner(new File(fileName)); // a file needs a FileInputStream int i = fileIn.nextInt(); e.g., Scanner stringScan = new Scanner(“ 345 Smith 4.5 “); int i = stringScan.nextInt();

9. Binary values • In memory, values of the primitive types are stored in binary • An int value is stored in a binary notation using 4 bytes • A float value is stored in a binary notation using 4 bytes • A double value is stored in a binary notation using 8 bytes • A char value is a 2 byte encoding of Unicode characters • A byte value is the integer value stored in 1 byte • A boolean value is stored as 1 bit of a byte • Often binary representation requires less space than the string representation • int value 3456 uses 4 bytes, while “3456” uses 8 bytes • Quicker to read a numeric value stored in binary than if it is stored in a string. • don’t need to do the conversion from string representation to binary • Hence, the values in large files are usually stored in binary notation, unless the data is all characters. Files storing values in binary are called binary files. • Disadvantage: binary files are not easily understood by humans. Note that if binary values are written into a file, they must be read as binary values. Similarly, if text (characters) are written into a file, they must be read as text.

10. Consider input of binary data • In Java, most objects that read data from an external source (console, file, internet, etc.) have type InputStream, or a descendant of it. • InputStream has • an operation to read 1 byte from the source and return it • an operation to read an array of bytes from the source Hence, as the class returns bytes, it is said to generate a byte stream. • Common InputStream generators for the byte stream • FileInputStream • the constructor has a File object or a fileName as its parameter • reads bytes from the file • InputStream • generic version of the class • an instance of this class is stored in System.in • returns a byte sequence that reflects the keys typed at the keyboard • an instance of this class can also be obtained from a URLConnection or a Socket to read from the internet • ByteArrayInputStream • the constructor has a byte array as a parameter • reads bytes from the byte array All the generators generate a sequence of bytes from some (usually external) source. The sequence can be fed into a Reader, to be converted into characters. Alternatively, the sequence can be fed into any byte stream consumer.

11. Byte stream consumers • The constructor has a InputStream as a parameter. From the InputStream, a sequence of bytes can be obtained. • The consumer has methods to structure the information in the bytes into more useful forms. • Common byte stream consumers: • DataInputStream • methods: readInt(), readFloat(), readChar(), readByte(), readLine(), … Reads binary values. • ObjectInputStream • Methods: readObject(), plus all the methods from DataInputStream Note that readObject() returns an instance of type Object. The user must know what type of object appears next in the byte stream and cast the object. Reads binary values. • Reader The Readers discussed before, FileReader and InputStreamReader, are consumers. They convert the byte sequence into a Unicode character sequence. Reads text values.

12. InputStream decorators (filters) • The constructor for a decorator has an InputStream as a parameter. From the InputStream, a sequence of bytes can be obtained. • The decorator transforms the byte sequence in some way to yield another byte sequence, i.e., it has a read method that a client can invoke. • Some InputStream decorators/filters • BufferedInputStream • CipherInputStream • InflatorInputStream • gzip, zip and jar versions • Extreme example: DataInputStream dataIn = new DataInputStream( new CipherInputStream( new ZipInputStream( new FileInputStream(fileName)), new Cipher( …))) float f = dataIn.readFloat(); int i = dataIn.readInt(); double d = dataIn.readDouble(); … • The user has a lot of flexibility in determining actions to do on the bytes.

13. Example from SpaceInvaders: Class HighScores publicstaticfinal String HIGH_SCORES_FILE_NAME = "HighScores.ser"; public LinkedList<GameResult> highScores; @SuppressWarnings("unchecked") private HighScores() { FileInputStream fileIn = null; ObjectInputStream objectIn = null; try { fileIn = new FileInputStream(HIGH_SCORES_FILE_NAME); objectIn = new ObjectInputStream(fileIn); highScores = (LinkedList<GameResult>) objectIn.readObject(); objectIn.close(); } catch (FileNotFoundException fnfe) { highScores = new LinkedList<GameResult>(); } catch (IOException ioe) { ioe.printStackTrace(); highScores = new LinkedList<GameResult>(); } catch (ClassNotFoundException cnfe) { cnfe.printStackTrace(); highScores = new LinkedList<GameResult>(); } }

14. Serialization • In order to convert an object into a byte sequence, all the data in the object and in any object referenced by the object (directly or indirectly) must be encoded into a form that can later be read. • Converting an object into a byte sequence is called serialization, and the reverse process is called deserialization. Method readObject() uses deserialization. • The algorithms to do these tasks are non-trivial, as there can be a mess of references involved. Further, a given object may be referenced more than once, so the serialization followed by deserialization must result in exactly one object corresponding to the given object, i.e., the algorithm must be able to recognize when it has reached an object previously handled, and not deserialize it as a new distinct object. • Fortunately, Java has built in algorithms to do the task. However, to enable the serialization of an object, the class of the object must implement the interface Serializable. No new methods need be implemented, or fields defined, but the “implements Serializable” must be included in the class header. • Most of the GUI components and all of the data structures in Collections are serializable. However, if an instance of a user-defined class might need to be serialized, the class must implement Serializable.

15. publicclass GameResult implements Serializable { publicstaticfinallongserialVersionUID = 1l; …. } • Consider the following sequence of tasks: • Some Serializable class A exists. • An instance of A is written into a file. • The class A is changed to add a field. • The object is read from the file, cast to type A, and assigned to a variable of type A. • The result is the variable has a type that includes a field, but the object referenced by the variable does not have the field. Bad situation. • Where is the problem and how can it be solved? • Note that from the information stored in the file, the system knows that it read an instance of class A, so it believes that the cast and assignment should be OK. • We must tell the Java system that a new version of the class was created. This is done by the serialVersionUID variable. When a new version is created, the value of the serialVersionUID variable should be changed. By using this variable, the Java system can recognize that a new version of A is being used and not allow the cast.

16. Idealized structure of Reader (Decorator Design Pattern) and the character stream.

17. Idealized structure of InputStream (Decorator Design Pattern) and the byte stream.

18. Output for a binary file is similar except that • the user provides the output that is turned into a byte stream • the output device (file, internet) is a consumer of the bytes in the stream • still have decorators/filters • OutputStream generators • They generate a byte sequence. • Classes: • DataOutputStream • Constructor parameter is an OutputStream where the byte stream is sent • methods: writeInt(int i), writeFloat(float f), writeChar(char c), writeChars(String s), write(byte b), write(byte[ ] ba) • ObjectOutputStream • Constructor parameter is an OutputStream where the byte stream is sent • methods: writeObject(object obj), and all the methods of DataOutputStream • PrintStream • Constructor parameter is an OutputStream where the byte stream is sent • The type of the static variable “out” in class System • methods: print( … ), println( … ) • Instead of PrintStream, use PrintWriter (see later) to write text into a file • Writer (not a OutputStream, but generates a byte sequence) • Constructor parameter is an OutputStream where the byte stream is sent • Converts a Unicode character sequence into a byte sequence • Used to write into a text file

20. OutputStream consumers • They consumer a byte sequence. • Classes: • FileOutputStream • Constructor has a File or String parameter • Writes the bytes into the file • OutputStream • generic version • used to send output to the screen • an instance of this class can be obtained from a Socket to send bytes out to the internet • ByteArrayOutputStream • Constructor has a byte array parameter • Writes the bytes into the array • OutputStream decorators/filters • They modify the byte sequence in some way. • Some OutputStream decorators/filters • BufferedOutputStream • CipherOutputStream • DeflatorOutputStream • gzip, zip and jar versions

21. Extreme example: DataOutputStream dataOut = new DataOutStream( new CipherOutputStream( new ZipOutputStream( new FileOutputStream(fileName)), new Cipher( …))) dataIn.writeFloat(f); dataIn.writeInt(i); dataIn.writedouble(d); …

22. Example from SpaceInvaders: Class HighScores publicstaticfinal String HIGH_SCORES_FILE_NAME = "HighScores.ser"; public LinkedList<GameResult> highScores; privatevoid save() { try { FileOutputStream fileOut = new FileOutputStream(HIGH_SCORES_FILE_NAME); ObjectOutputStream objectOut = new ObjectOutputStream(fileOut); objectOut.writeObject(highScores); objectOut.close(); } catch (FileNotFoundException fnfe) { fnfe.printStackTrace(); } catch (IOException ioe) { ioe.printStackTrace(); } }

23. Output to a text file • General approach • Generate a character sequence • Possibly decorate/filter it • Convert the sequence to a byte sequence (by a Writer consumer) • Possibility decorate/filter it • Write the byte sequence into the file by a FileOutputStream • Writer generators • Generate a character sequence • One common Writer generator • PrintWriter • Constructor has another Writer (decorator or consumer) as a parameter • Same methods a PrintStream: print, println • Characters go to the Writer given as the constructor parameter • One Writer decorator/filter • Receives a character sequence and outputs a character sequence • BufferedWriter • Characters written to it are buffered. • The constructor has a Writer parameter to receive the characters generated • When the buffer is full, the characters are passed along to a Writer consumer

24. Writer consumers • They receive a character sequence and do something with it. • Some Writer consumers • FileWriter • The constructor has File or a fileName for a parameter • The character sequence is converted to a byte sequence and passed along to a FileOutputStream for the file specified. • OutputStreamWriter • The constructor has an OutputStream for a parameter • The character sequence is converted to a byte sequence and passed along to the OutputStream. • CharArrayWriter • The class has an internal char array • The character sequence is saved in the char array • The characters can be retrieved via toCharArray() and toString() • StringWriter • The class has an internal StringBuffer • The character sequence is saved in the buffer • The characters can be retrieved via getBuffer() and toString()

25. Convenient shortcuts for text files PrintWriter fileOut = new PrintWriter(fileName); // implicitly creates the FileWriter and FileOutputStream // implicitly buffers the characters fileOut.println( … ); … fileOut.close(); Scanner fileIn = new Scanner(new File(fileName)); // implicitly creates the FileReader and FileInputStream int i = fileIn.nextInt(); … fileIn.close();

26. Special file type: RandomAccessFile • Constructor parameters: File and either “r” or “rw” or fileName and either “r” or “rw” • Not associated with any streams. • Views a file as byte array of a flexible size • Methods related to the location in the file • long size() • return the number of bytes in the file • long getFilePointer() • return the present position in the file – number of bytes from the start • seek(long adr) • adr >= 0 and adr <= size() • move to the adr byte in the file • Output methods • writeInt(i), writeFloat(f), etc (the same methods as in DataOutputStream) • write(byte[ ] ba); the byte array can be obtained from an OutputStream • start writing at the current location (overwriting whatever was there) • Input methods • readInt(), readFloat(), etc (the same methods as in DataInputStream) • read(byte[ ] ba): the byte array can be returned to an InputStream • start reading from the current location • it is the programmer’s responsibility to ensure that valid data is at this location.

27. Access by index in a RandomAccessFile • Select a fixed size, SIZE, for records/objects to be written to the file. • The ith record is written at location (i-1)*SIZE. • Therefore, the jth record can be read from location (j-1)*SIZE. • Access by key in a RandomAccessFile • When the record with a specific key is written into the file, record its location in a table in the form of a (key, file location) pair. • This table will be much smaller than the actual file, as it only stores the pairs (not all the data for each key). Hence, often it can be stored in memory, even if the whole file cannot. • To access a record by key, • Look up the key in the table to obtain the file location. • Read the record from the file starting at the file location obtained.

28. Two types of file access • Random access • Use class RandomAccessFile(String fileName) • Can move to an arbitrary location • Read or write binary values at the current location • writeInt(i), writeFloat(f), write(byte[ ] ba) … ; readInt(), readFloat(), read(byte[ ] ba), … Example: RandomAccessFile file = new RandomAccessFile(name, “rw”); FileOutputStreamfileOut = new FileOutputStream(file.getFD()); ObjectOutputStreamobjOut = new ObjectOutputStream(fileOut); FileInputStreamfileIn = new FileInputStream(file.getFD()); ObjectInputStreamobjIn = new ObjectInputStream(fileIn); … file.seek(pos); // where pos is a valid position in the file objOut.writeObject(x); // where the type of x is a reference type … file.seek(pos); Object obj = objIn.readObject();

29. Stream access • Data is read or written sequentially • At any given time, a file can be open for reading or writing, but not both. • Two kinds of files • binary • The primitive values are stored in internal representation and objects serialized. • Output: DataOutputStream(new FileOutputStream(fileName)) // no shortcut writeInt(i), writeFloat(f), write(byte[ ] ba), … or ObjectOutputStream(new FileOutputStream(fileName)) // no shortcut writeObject(obj), writeInt(i), writeFloat(f), write(byte[ ] ba), … • Input: DataInputStream(new FileInputStream(fileName)) // no shortcut readInt( ), readFloat( ), read(byte[ ] ba), … or ObjectInputStream(new FileInputStream(fileName)) // no shortcut readObject(obj), readInt( ), readFloat( ), read(byte[ ] ba), … • text • Unicode characters are converted to/from the locale-dependent character encoding used in the file • Output: PrintWriter(String fileName) print( … ), println( … ) • Input: Scanner(new File(String fileName)) nextInt(), nextFloat(), nextLine(), … , hasNext(), hasNextInt(), …