Project: Http Bomber

1. Project:Http Bomber Final Presentation Liat Ben-Ami Yonathan Perez Instructor: Roy Mitrany

2. Presentation Overview • Goal • Market Survey – where does HTTP Bomber fit • HTTP Background • How does HTTP Bomber work? • Technologies • Design Overview • Important Modules • Concurrency • GUI • Summary & Gained Knowledge

3. Goals • Generate Http traffic load in order to test server performance • Analyze server responses and provide analysis results with a clear graphic display

4. Market survey: existing products • Several traffic generators: generate traffic according to user-defined workload. • HP httperf , curl-loader project,.. • Http Analyzers / Monitors : monitor all http traffic from and to a user-agent, analyze it and provide graphic display of the results. • Http Analyzer V5 by IE Inspector, Http Debugger Pro by Softx,… • HTTP Bomber is an HTTP traffic generator that also provides analysis and graphic display of the traffic to and from the tested server.

5. HTTP Protocol • Most significant application-layer protocol in the World Wide Web. • Operates over reliable transport layer protocol (TCP) • Goal: Transfer data of various types (Resources) over the Internet. Resources are identified by URIs. • A stateless request-response protocol

6. HTTP Messages <initial line> Header 1: value 1 Header 2: value2 … Header N: value N <Optional Message body>

7. HTTP Messages – cont. • Request initial line: Method + URI + protocol version • Methods supported by HTTP Bomber: GET & POST • Response initial line: protocol version + status code + textual explanation • status codes: 1xx: informational 2xx: success 3xx: redirection 4xx: Client error 5xx: Server error

8. How does HTTP Bomber work? • User sets execution parameters (including tested server’s name) • User builds a list of requests to be sent to server • User starts simulation • Virtual users send requests to server and collect responses. Responses are analyzed and statistical info is gathered and displayed.

9. Technologies • Java • Eclipse • GUI: javax.swing, Jigloo • Jfreechart • Apache Http components

10. Design Overview HTTP Bomber GUI / API Response Recorder write read Generator Analyzer (including Graphic display) Request List Server

11. Design Overview – Cont. • Modular design • HttpBomber – Top hierarchy module. Its public methods are used as user interface – as API or through GUI. • Generator: generate workload and communicate with the server • Analyzer: gather statistical info, write report, display graphs • Generator and Analyzer interact through a common synchronized data structure

12. HttpBomber – Sequence Diagram

13. HttpGenerator - Purpose Send requests to server according to user-defined workload (execution parameters and request list) and produce records of server’s responses.

14. HttpGenerator - Implementation • Requests are sent by virtual users (threads) through a multi-threaded Apache HttpClient. • Number of virtual users is defined in execution parameters. • Each virtual user iterates over the request list in a cyclic order. For each request: send request, receive response, generate a record and update the response recorder. • Custom handling for IO and protocol errors

15. Generator – Sequence Diagram

16. Analyzer - Purpose Gather and produce statistical info from response records, produce report file and display graphs.

17. Analyzer - Implementation • getData thread copies new data from the synchronized response recorder, updates counters and local data structures and writes record content to the report file. • Gathered info: requestsSent, errorNum, dataReceived, dataSent, number of responses per each status code • Separate threads run graphs – one thread per graph. Updated according to gathered info. • at end of simulation – statistical info is computed and added to report and final graphs are displayed

18. Report File Example ******Start Report****** Start Time: Tue Apr 26 02:05:33 IDT 2011 Workload Parameters: Tested Server: www.google.co.il, port: 80, Number of virtual users: 1, max time cap: 9223372036854775807 msec max requests per user: 5, max timeout: 60000 msec, max allowed retrials per request: 2 Response Records: Request index: 0, Status Code: 404, Data Sent: 65 bytes, Data Received: 11983 bytes, Response Time: 921 msec, Time stamp: Tue Apr 26 02:05:34 IDT 2011 Request index: 1, Status Code: 405, Data Sent: 68 bytes, Data Received: 11999 bytes, Response Time: 115 msec, Time stamp: Tue Apr 26 02:05:34 IDT 2011 Request index: 2, Status Code: 200, Data Sent: 60 bytes, Data Received: 10597 bytes, Response Time: 332 msec, Time stamp: Tue Apr 26 02:05:34 IDT 2011 Request index: 0, Status Code: 404, Data Sent: 65 bytes, Data Received: 11983 bytes, Response Time: 125 msec, Time stamp: Tue Apr 26 02:05:34 IDT 2011 Request index: 1, Status Code: 405, Data Sent: 68 bytes, Data Received: 11999 bytes, Response Time: 131 msec, Time stamp: Tue Apr 26 02:05:35 IDT 2011 Total requests sent: 5 Total data received: 58.56 KB Total data sent: 0.33 KB Total errors: 0 Average traffic rate: 31.97 KB/sec Response Time Statistics: Minimal - 115 msec, Maximal - 921 msec, Median - 131 msec, Average - 324 msec Status Code Distribution (zero indicates IO or protocol errors): 200 : 20% 404 : 40% 405 : 40% Time: Tue Apr 26 02:05:35 IDT 2011 Simulation ended:simulation ended *********End of report********

19. N - Number of requests in request list 1 2 N Records of responses to request i Synchronized Response Recorder

20. HttpBomber Class Diagram

21. Graph Display – Online Graphs

22. Graph Display – Offline Graphs

23. Graph Display - Implementation

24. Graph Display – Sequence Diagram

25. Concurrency • HTTP Bomber is a highly-concurrent application • Significant need for Synchronization. Minimize access to mutable shared data. • Several “virtual users” threads send requests simultaneously to server and write to the Response Recorder • Analyzer: Analyzer main thread, getData thread and a thread for each of the graphs

26. Main thread Simulation start Analyzer thread start start start start Virtual Users Threads getData Thread Graph Threads …. join done done join join join

27. User Guide - GUI

28. User Guide - GUI

29. User Guide - GUI

30. User Guide – using API example public class MainTest{ public static void main(String[] args) { URI uri1 = null; URI uri2 = null; try { uri1 = new URI("http://www.google.co.il/"); uri2 = new URI("/jiadb"); } catch (URISyntaxException e) { return; } HttpBomber bomber = new HttpBomber(); /* set execution parameters */ bomber.setExecutionParameters("www.google.co.il", 10, 60000, 100, Long.MAX_VALUE, 2); /* build request list */ try{ bomber.addGetRequest(uri2); bomber.addStringPostRequest(uri1, "blabla", "text/plain; charset = UTF-8"); bomber.addGetRequest(uri1); bomber.addHeader(0, "User-Agent", "HTTP Bomber"); bomber.addHeader(1, "User-Agent", "HTTP Bomber"); bomber.addHeader(2, "User-Agent", "HTTP Bomber"); }catch(Exception e){ e.printStackTrace();} /* set graphic display */ bomber.setGraphDisplay(true, true, true, true); /* run simulation */ bomber.Init(); } }

31. Summary & Gained Knowledge What we did: • Created a reliable and user-friendly application to analyze performance of web servers What we learned: • Java • HTTP Protocol and Apache Http Components • GUI design • Graphic library API • Design and implementation of a complex multi-threaded software tool

32. ?