Project: IEEE P802.15 Working Group for Wireless Personal Area Networks (WPANs)

1. Project: IEEE P802.15 Working Group for Wireless Personal Area Networks (WPANs) Submission Title:[System Simulation Metricsfor BAN] Date Submitted: [9thSeptember, 2008] Source: [Seung-Hoon Park / Noh-Gyoung Kang / Chihong Cho / Eun-Tae Won / Ranjeet Kumar Patro / Giriraj Goyal / Ashutosh Bhatia / Kiran Bynam / Arun Naniyat] Company [Samsung Electronics Co. Ltd.] Address [416, Maetan-3dong, Yeongtong-gu, Suwon-si, Gyeonggi-do, 443-742, Korea] Voice: [+82-31-279-4579], FAX: [+82-31-279-5130], E-Mail: [shannon.park@samsung.com] Re: [] Abstract: [This document describes the necessity of system simulation and related metrics for BAN] Purpose: [To promote discussion of performance evaluation issues for BAN] Notice: This document has been prepared to assist the IEEE P802.15. It is offered as a basis for discussion and is not binding on the contributing individual(s) or organization(s). The material in this document is subject to change in form and content after further study. The contributor(s) reserve(s) the right to add, amend or withdraw material contained herein. Release: The contributor acknowledges and accepts that this contribution becomes the property of IEEE and may be made publicly available by P802.15. Seung-Hoon Park et al., Samsung

2. System Simulation Metrics for BAN Seung-Hoon Park et al., Samsung

3. Evaluation Methodologies • Theoretical analysis • Simple scenario • Ideal assumption • Average result metrics • Test-bed • Static tuning parameters • Difficult monitoring • Much cost consuming • Real result metrics Seung-Hoon Park et al., Samsung

4. Evaluation Methodologies • Simulation • Various realistic scenarios • Scalability (network size, traffic, etc) • Cost effective • Time- & memory- consuming • Abstraction model affects the performance • Diverse performance metrics can be obtained easily Seung-Hoon Park et al., Samsung

5. The Necessity of System Simulation • To analyze the system-level performance of proposed BAN spec • To find pros & cons ofproposals in the same condition (link budget, channel model, etc) • To estimate easily the performance of merged draft spec Seung-Hoon Park et al., Samsung

6. Purpose of Presentation • To facilitate evaluation issues of BAN by introducing various system simulation metrics of general WSN • To discuss the way to define proper metrics to evaluate BAN Seung-Hoon Park et al., Samsung

7. Channel PHY MAC Network Application Battery Model The Range of System Simulation • There are wide range of system simulations regarding to the type of evaluation metric • Evaluation target • MAC layer protocol • Routing protocol • QoS • Source coding • Power consumption • etc System Simulation PHY Simulation Seung-Hoon Park et al., Samsung

8. Type of Metrics • Global metrics • Packet-level metrics • Scenario metrics • Joint metrics • Other metrics Seung-Hoon Park et al., Samsung

9. Global Metrics [1/3] • Energy consumption • Total energy consumed by the sensor nodes in a network • Network utilization • The effective time period of the data sink receiving valid data • Protocol overhead • e.g. : RTS/CTS overhead, resource management overhead, encryption overhead Seung-Hoon Park et al., Samsung

10. Global Metrics [2/3] • Network lifetime • The time since initial deployment that the WSN is able to • i) sustain a minimum number of surviving nodes • ii) maintain the number of disconnected nodes below a certain threshold • iii) provide a sensing coverage greater than the pre-defined threshold Seung-Hoon Park et al., Samsung

11. Global Metrics [3/3] • Percentage of time spent by nodes in different modes • Active( transmission, reception), idle, sleep and off • Network scalability • The number of nodes that the sensor network can scale to and reliably preserve the performance Seung-Hoon Park et al., Samsung

12. Packet-level Metrics [1/2] • Y-axis metrics • Throughput • Average data rate [one-hop, source-to-sink] • Delay • Average delivery delay [one-hop, source-to-sink] • This includes all possible delays caused by buffering during route discovery latency, queuing at the interface queue, retransmission delays at the MAC, propagation and transfer times • Packet Delivery Ratio [one-hop, source-to-sink] • The ratio of number of packets received to the number of packets sent • Packet Delivery Ratio = 1 – Packet Loss Ratio • Packet Collision Ratio [one-hop] • The ratio of number of collision to the number of transmission • Queue status [at the MAC-layer queue] • Average queue occupancy • Average queue drops Seung-Hoon Park et al., Samsung

13. Packet-level Metrics [2/2] • X-axis metrics • SNR (Signal to Noise Ratio) • SIR (Signal to Interference Ratio) • Distance between source to sink • # of hops between source to sink • etc Seung-Hoon Park et al., Samsung

14. Scenario Metrics • Network metric • Size : # of connected nodes • Density : # of nodes per unit area • Topology : random graph ~ structural graph • Traffic load • Average generation rate (external arrival rate) [pkts/sec] • Average transmission rate (internal arrival rate) [pkts/sec] • Packet size [bits] • Single traffic or mixed traffic • Node property • Speed • Maximum queue length • Battery size • Beacon rate Seung-Hoon Park et al., Samsung

15. Joint Metrics • A representative metric to describe the performance of multi-objective optimization involving more than two metrics • Example • Sum of log(R) • R: data rate • Considering both throughput and fairness • It is required for joint metric of BAN to represent both • Energy and throughput • Energy and delay • Energy and reliability • We can define new joint metric for BAN Seung-Hoon Park et al., Samsung

16. Other Metrics [1/2] • Node association • Connection setup time • Node association rate • Authentication related metrics • Authentication cost • The sum of cost of authentication at the several levels of transmission for a number of requests for different security classes • Authentication delay • The sum of time when a user sends an authentication request to the time it receives the approval for a number of requests for different security classes • Packet dropping probability for authentication Seung-Hoon Park et al., Samsung

17. Other Metrics [2/2] • Routing related metrics • Network settling time • The time required for a collection of mobile wireless nodes to automatically organize itself and transmit the first message reliably • Network join time • The time required for an entering node or group of nodes to become integrated into the sensor network • Network depart time • The time required for the network to recognize the loss of one or more nodes, and reorganize itself to route around the departed nodes • Network recovery time • The time required for a collapsed portion of the network, due to traffic overload or node failures, to become functional again once the load is reduced or the nodes become operational Seung-Hoon Park et al., Samsung

18. Conclusion • There are many kinds of system simulation metrics for analyzing general WSN • We should prescribe principal metrics to evaluate BAN proposals for less simulation effort • Evaluation methodology by system simulation can contribute to excellent BAN specification Seung-Hoon Park et al., Samsung

19. References • [1] “Distributed sensor processing over an ad hoc wireless network: simulation framework and performance criteria”, Van Dyck, R.E.; Miller, L.E.; Military Communications Conference, 2001. MILCOM 2001. Communications for Network-Centric Operations: Creating the Information Force. IEEE Volume 2, 28-31 Oct. 2001 Page(s):894 - 898 vol.2; Digital Object Identifier 10.1109/MILCOM.2001.985967 • [2] “Performance Evaluation of IEEE 802.15.4 Ad Hoc Wireless Sensor Networks: Simulation Approach”, Woon, W.T.H.; Wan, T.C.; Systems, Man and Cybernetics, 2006. SMC '06. IEEE International Conference on Volume 2, 8-11 Oct. 2006 Page(s):1443 – 1448; Digital Object Identifier 10.1109/ICSMC.2006.384920 • [3] “Modeling the performance of wireless sensor networks”, Chiasserini, C.-F.; Garetto, M.; INFOCOM 2004. Twenty-third AnnualJoint Conference of the IEEE Computer and Communications Societies Volume 1, 7-11 March 2004 Page(s):; Digital Object Identifier 10.1109/INFCOM.2004.1354496 • [4] “A framework for empirical evaluation of nature inspired routing protocols for wireless sensor networks”, Saleem, M.; Farooq, M.; Evolutionary Computation, 2007. CEC 2007. IEEE Congress on 25-28 Sept. 2007 Page(s):751 – 758; Digital Object Identifier 10.1109/CEC.2007.4424546 Seung-Hoon Park et al., Samsung

20. Thank You! Seung-Hoon Park et al., Samsung