KPC-Toolbox Demonstration

1. KPC-ToolboxDemonstration Eddy Zheng Zhang, Giuliano Casale, Evgenia Smirni Computer Science Department College of William & Mary

2. What is KPC-Toolbox for? • KPC-Toolbox: MATLAB toolbox • Workload Traces  Markovian Arrival Process (MAP) • Why MAP? • Very versatile • High variability & temporal dependence in Time Series • Easily incorporated into queuing models • Friendly Interface • Departure from previous Markovian fitting tools • Fit the automatically (no manual tuning)

3. User Interface • Requirement: Matlab installed • Input • A trace of inter-event times • Or a file that already stores the statistics of the trace • E.g., a file stores the moments, autocorrelations and etc • Help Information • Type “help FunctionName”, • E.g., “help map_kpcfit” • Website Keeps Up-To-Date Tool version • http://www.cs.wm.edu/MAPQN/kpctoolbox.html

4. A Simple Example of MAP‏ • Two state jumps c -a-c Background Jumps D0 = Jumps With Arrivals d -b-d c 2 1 b a a 0 D1 = d b 0 Arrivals: Time: I1 I2 I3

5. Challenges • How large is the MAP? • MAP(n): determine n? • Which trace descriptors are important? • Literature: Moments of interval times, lag-1 autocorrelation • But, for long range dependent traces? • Need temporal dependence descriptors • MAP Parameterization • Construct MAP(n) with matrices D0 and D1 (2n2– n entries)

6. 1 2 Example: Important Trace Statistics • Seagate Web Server Trace Queue Prediction, 80% Utilization Fit With MAP(2) First, second, third moment and lag-1 autocorrelation accurately fit The queuing prediction ability is not satisfactory!

7. 1 13 ……… ……… 2 14 ……… 3 15 4 16 Example: Higher Order Statistics Matter A grid of joint moments and a sequence of autocorrelations fitted, E[XiXi+kXi+k+h] • Seagate Web Server Trace Queuing Prediction, 80% Utilization Fit with MAP(16) Much Better Results!

8. Fitting Guidelines • Higher Order Correlations V.S. Moments • Correlations capture sequence in the time series • Correlations are very important • Summary: • Matching up to the first three moments is sufficient • Matching higher order correlations with priority Ref: "KPC-Toolbox: Simple Yet Effective Trace Fitting Using Markovian Arrival Processes", G. Casale, E.Z. Zhang, E. Smirni, to appear in QEST’08

9. Challenge (1): Determine MAP Size • Definition: •  lag-k ACF coefficient • MAP(n) Property: • Linear Recursive Relationship of n consecutive ACF coeffs • BIC Size Selection: • Linear regression model on estimated ACF coeffs • BIC value assesses goodness of model size MAP(8) MAP(16) MAP(32)

10. Challenge (2): Trace Descriptor Matching • Kronecker Product Composition (KPC) • KPC Properties: Composition of Statistics • Moments are composed from moments of small MAPs • MAP Parameterization by KPC to Match • Mean and SCV Exactly • Higher order correlations as Close as Possible

11. KPC Tool Overview Moments ACF Correlations …… Size of MAP N Trace Size Selection Extract Statistics Optimization J = log2N MAP(2)s MAP(2) MAP(2) MAP(2) MAP(2) …… KPC MAP(N) This work is supported by NSF grants ITR-0428330 and CNS-0720699

12. Thank you! 

13. Appendix • What are higher order correlations? • Joint moments of a sequence of inter-arrival times in the time series • Which higher order correlations to fit in KPC? • E[XiXi+jXi+j+k], where i can be arbitrary without loss of generality, and [j,k] chose from a grid of values • E.g., [10 100 1000 10000] × [10 100 1000 10000] = {[10,10], [10,100], [10,10000], …}