TELNET Protocol & HyperText Transfer Protocol

1. TELNET Protocol & HyperText Transfer Protocol

2. TELNET vs. telnet • TELNET is a protocol that provides “a general, bi-directional, eight-bit byte oriented communications facility”. • telnet is a program that supports the TELNET protocol over TCP. • Many application protocols are built upon the TELNET protocol. TELNET

3. The TELNET Protocol • Reference: RFC 854 • TCP connection • data and control over the same connection. • Network Virtual Terminal • intermediate representation of a generic terminal. • provides a standard language for communication of terminal control functions. TELNET

4. Network Virtual Terminal Server Process User Terminal NVT NVT TCP TCP TELNET

5. Negotiated Options • All NVTs support a minimal set of capabilities. • Some terminals have more capabilities than the minimal set. • The set of options is not part of the TELNET protocol, • so that new terminal features can be incorporated without changing the TELNET protocol. • Two endpoints negotiate a set of mutually acceptable options • Line mode vs. character mode • echo modes • character set (EBCDIC vs. ASCII) TELNET

6. Control Functions • TELNET includes support for a series of control functions commonly supported by servers. • This provides a uniform mechanism for communication of (the supported) control functions. TELNET

7. Control Functions • Interrupt Process (IP) • suspend/abort process. • Abort Output (AO) • send no more output to user’s terminal. • Are You There (AYT) • check to see if system is still running. • Erase Character (EC) • delete last character sent • Erase Line (EL) • delete all input in current line. TELNET

8. Command Structure • All TELNET commands and data flow through the same TCP connection. • Commands start with a special character called the Interpret as Command escape character • The IAC code is 255. • If a 255 is sent as data - it must be followed by another 255. • If IAC is found and the next byte is IAC • a single byte is presented to application/terminal • If IAC is followed by any other code • the TELNET layer interprets this as a command. TELNET

9. Playing with TELNET • You can use the telnet program to play with the TELNET protocol. • telnet is a generic TCP client. • Sends whatever you type to the TCP socket. • Prints whatever comes back through the TCP socket • Useful for testing TCP servers (ASCII based protocols). • Many Unix systems have these servers running (by default): • echo port 7 discard port 9 • daytime port 13 chargen port 19 TELNET

10. telnet hostname port > telnet amele-2.cse.unr.edu 7 Trying 134.197.40.246... Connected to amele-2.cse.unr.edu (134.197.40.246). Escape character is '^]'. Hi mehmet Hi mehmet stop it stop it ^] telnet> quit Connection closed. TELNET

11. telnet vs. TCP • Not all TCP servers talk TELNET (most don't) • You can use the telnet program to play with these servers, but the fancy commands won't do anything. • type ^], then "help" for a list of fancy TELNET stuff you can do in telnet. TELNET

12. HyperText Transfer Protocol (HTTP) • HTTP is the protocol that supports communication between web browsers and web servers. • A “Web Server” is a HTTP server • Most clients/servers today speak version 1.1, but 1.0 is also in use. • RFC 1945 (HTTP 1.0) • RFC 2616 (HTTP 1.1) HTTP

13. From the RFC • “HTTP is an application-level protocol with the lightness and speed necessary for distributed, hypermedia information systems.” • Transport Independence • The HTTP protocol generally takes place over a TCP connection, • but the protocol itself is not dependent on a specific transport layer. HTTP

14. Request - Response • HTTP has a simple structure: • client sends a request • server returns a reply. • HTTP can support multiple request-reply exchanges over a single TCP connection. • The “well known” TCP port for HTTP servers is port 80. • Other ports can be used as well... HTTP

15. HTTP 1.0+ Request Request-Line Headers . . . • Lines of text (ASCII). • Lines end with CRLF “\r\n” • First line is called “Request-Line” blank line Content... HTTP

16. Request Line MethodURIHTTP-Version\r\n • The request line contains 3 tokens (words). • space characters “ “ separate the tokens. • Newline (\n) seems to work by itself • but the protocol requires CRLF HTTP

17. Request Method • The Request Method can be: GET HEAD DELETE PUT POST TRACE OPTIONS future expansion is supported • GET, HEAD and POST are supported everywhere (including Lab 2!). • HTTP 1.1 servers often support PUT, DELETE, OPTIONS & TRACE. HTTP

18. Methods • GET: retrieve information identified by the URI. • Typically used to retrieve an HTML document • HEAD: retrieve meta-information about the URI. • used to find out if a document has changed • POST: send information to a URI and retrieve result. • used to submit a form HTTP

19. More Methods • PUT: Store information in location named by URI. • DELETE: remove entity identified by URI. • TRACE: used to trace HTTP forwarding through proxies, tunnels, etc. • OPTIONS: used to determine the capabilities of the server, or characteristics of a named resource. HTTP

20. URI: Universal Resource Identifier • URIs defined in RFC 2396. • Absolute URI: • scheme://hostname[:port]/path • http://www.cse.unr.edu:80/~mgunes/cpe401 • Relative URI: • /path • /blah/foo No server mentioned HTTP

21. URI Usage • When dealing with a HTTP 1.1 server, only a path is used (no scheme or hostname). • HTTP 1.1 servers are required to be capable of handling an absolute URI, but there are still some out there that won’t… • When dealing with a proxy HTTP server, an absolute URI is used. • client has to tell the proxy where to get the document! • more on proxy servers in a bit…. HTTP

22. HTTP Version Number “HTTP/1.0” or “HTTP/1.1” • Starting with HTTP 1.0 the version number is part of every request. • Client tells the server what version it can talk (what options are supported, etc). • HTTP 0.9 did not include a version number in a request line. • If a server gets a request line with no HTTP version number, it assumes 0.9 • HTTP 0.9 was used for many years. HTTP

23. The Header Lines • Request Headers provide information to the server about the client • what kind of client • what kind of content will be accepted • who is making the request • Each header line contains • an attribute name followed by a “:” followed by a space and the attribute value. • There can be 0 headers (HTTP 1.0) • HTTP 1.1 requires a Host: header HTTP

24. Example HTTP Headers Accept: text/html Host: www.cse.unr.edu From: mgunes@cse.unr.edu User-Agent: Mozilla/4.0 Referer: http://www.unr.edu/ HTTP

25. End of the Headers • Each header ends with a CRLF ( \r\n ) • The end of the header section is marked with a blank line. • just CRLF • For GET and HEAD requests, the end of the headers is the end of the request! HTTP

26. POST • A POST request includes some content (some data) after the headers (after the blank line). • There is no format for the data (just raw bytes). • A POST request must include a Content-Length line in the headers: Content-length: 267 HTTP

27. Example POST Request POST /~mgunes/cpe401/grades.cgi HTTP/1.1 Accept: */* Host: www.cse.unr.edu User-Agent: SecretAgent V2.3 Content-Length: 35 Referer: http://www.unr.edu/ stuid=6660182722&item=test1&grade=99 HTTP

28. Example GET Request GET /~mgunes/cpe401/lab1.htm HTTP/1.1 Accept: */* Host: www.cse.unr.edu User-Agent: Internet Explorer From: mgunes@cse.unr.edu Referer: http://www.unr.edu/ There is a blank line here! HTTP

29. HTTP Response Status-Line Headers . . . • ASCII Status Line • Headers Section • Content can be anything (not just text) • typically an HTML document or some kind of image. blank line Content... HTTP

30. Response Status Line HTTP-Version Status-Code Message • Status Code is 3 digit number (for computers) • 1xx Informational • 2xx Success • 3xx Redirection • 4xx Client Error • 5xx Server Error • Message is text (for humans) HTTP

31. Example Status Lines HTTP/1.0 200 OK HTTP/1.0 301 Moved Permanently HTTP/1.0 400 Bad Request HTTP/1.0 500 Internal Server Error HTTP

32. Response Headers • Provide the client with information about the returned entity (document). • what kind of document • how big the document is • how the document is encoded • when the document was last modified • Response headers end with blank line HTTP

33. Response Header Examples Date: Wed, 30 Jan 2002 12:48:17 EST Server: Apache/1.17 Content-Type: text/html Content-Length: 1756 Content-Encoding: gzip HTTP

34. Content • Content can be anything (sequence of raw bytes). • Content-Length header is required for any response that includes content. • Content-Type header also required. HTTP

35. Single Request/Reply • The client sends a complete request. • The server sends back the entire reply. • The server closes it’s socket. • If the client needs another document it must open a new connection. This was the default for HTTP 1.0 HTTP

36. Persistent Connections • HTTP 1.1 supports persistent connections (this is the default). • Multiple requests can be handled over a single TCP connection. • The Connection: header is used to exchange information about persistence (HTTP/1.1) • 1.0 Clients used a Keep-alive: header HTTP

37. Try it with telnet > telnet www.cse.unr.edu 80 GET / HTTP/1.0 HTTP/1.0 200 OK Server: Apache ... Request-line Blank Line (end of headers) Response HTTP

38. Try it with telnet (persistent) > telnet www.cse.unr.edu 80 GET / HTTP/1.1 Host: www.cse.unr.edu HTTP/1.0 200 OK Server: Apache ... Required! HTTP

39. HTTP Proxy Server Browser HTTP Server Proxy HTTP

40. Network Lab #2 HTTP Proxy • You need to write a proxy server. • Must be able to handle GET, HEAD and POST requests. • Filtering: Your proxy will be given a list of domain names on the command line, you should refuse to forward requests to any server whose name is within a specified domain. • send back status line: 403 Forbidden. Lab #2

41. The code you need • Proxy is both a client and a server • Parsing the HTTP request is needed. • You need to understand HTTP • You will need to parse headers. • need to look at Content-length, Connection, etc. Lab #2

42. Testing • Tell your browser to use a proxy • Edit preferences/options. • Interrupt a long transfer (press stop). • Fill out a form (probably uses POST). • Test it with a browser. • Test it with telnet • Write an abusive client and a rude server! Lab #2

43. What is expected • We should be able to surf through your proxy! • Proxy should print some info about each request (print the request line). • No memory leaks! • Check every system call for errors! • We should not be able to kill your proxy by • sending a bad request. • using a server that sends bad replies. • No crashes, no matter what kind of nonsense we send your proxy. Lab #2

44. HTTP V1.1 Details • The RFC is 114 pages! • we don’t expect you to read it all or to support every nitty-gritty detail. • work on creating a working proxy (one you can use through a browser). • performance is not a big deal (but it shouldn’t be horribly worse than without your proxy). • Don’t worry about persistence, pipelining, chunking, etc. • you need to turn off persistence if you don't want to handle it. Lab #2

45. HTTP Headers • You will need to look at the Content-Length header in a POST. • you need to know how many bytes to read after the end of the headers. • You will need to either look at Connection (Proxy-Connection) headers or (at a minimum) to force Connection: close as a request header. Lab #2

46. Stuff you might need to know(that we have not covered) • Converting hostnames to IP addresses. • Handling signals (SIGPIPE) • Check out section 5.13 in the text • Providing Concurrency (not required, but not hard either). • just fork the server after calling accept. • MAKE SURE YOU TAKE CARE OF ZOMBIES! Lab #2