Performance Analysis for VoIP System Members R94922009 周宜穎 R94922020 吳鴻鑫 R94922064 張嘉輔
Outline • What is Performance • Performance Bound • How to analyze Performance • Some Performance Analysis Exmaple
What is Performance ?  • There are numerous factors that affect the performance assessments. • Human factors • Device factors • Network factors
Human factors -Audiovisual Quality Assessment Metrics • Subjective quality assessment - MOS • Objective quality assessment • Signal-to-Noise Ratio (SNR) • Mean Square Error (MSE) • Perceptual Analysis Measurement System (PAMS) • Perceptual Evaluation of Speech Quality (PESQ) • E -model • E-model • R-scale ( 0 to 100 ) <=> MOS rankings and User Satisfaction
Human factors Voice Quality Classes
Device factors • Essential devices such as • VoIP endpoints • Gateways • MCUs (Multipoint Control Units ) • Routers • Firewalls • NATs (Network Address Translators ) • Modems • Operating System • Processor • memory
Network factors • Network congestion • Link failures • Routing instabilities • Competing traffic • General Measruement : • Delay • Jitter • Packet loss
The Performance Standard • Delay • Good (0ms-150ms) • Acceptable (150ms-300ms) • Poor (> 300ms) • Jitter • Good (0ms-20ms) • Acceptable (20ms-50ms) • Poor (> 50ms). • Loss • Good (0%-0.5%) • Acceptable (0.5%-1.5%) • Poor (> 1:5%)
E-model • ITU-T recommendation • Well established computational model • Using Transmission parameters to predict the quality • We can get the basic Performance Standard by through the model
Basic formula for the E-model • R-value = Ro - Is - Id - Ie + A • Ro • the basic signal-to-noise ratio based on sender and receiver loudness ratings and the circuit and room noise • Is • the sum of real-time or simultaneous speech transmission impairments,e.g. loudness levels, sidetone and PCM quantizing distortion • Id • the sum of delay impairments relative to the speech signal, e.g., talker echo, listener echo and absolute delay • Ie • the equipment impairment factor for special equipment, e.g., low bit-rate coding (determined subjectively for each codec and for each % packet loss and documented in ITU-T Recommendation G.113) • A • the advantage factor adds to the total and improves the R-value for new services.
Estimating the R • R = (Ro − Is) − Id − Ie + A • Ro , Is • do not depend on network environment • Id • This the Argument of Delay • Ie • It mostly affect by codec and packet loss • A • Additional adjust argument ,not considered in general
Estimating Id and Ie • Id = Idte + Idle + Idd • Idte -Talker echo delay • Idle - Listener echo delay • Idd - Long delay • Ie • It base on codec, but packet loss affect can be emulated as a function
Estimating Id and Ie • The distortion as a function of packet loss also depends on whether or not PLC (Packet Loss Concealment) • increases 4 units for codecs with PLC (in the R scale per 1% packet loss) • 10 units for codecs without PLC
Test Setup • Using 9 scenarios to test 27 possibilities • Using NISTnet network emulator • (http://snad.ncsl.nist.gov/itg/nistnet/) • create the various network health scenarios
Normalized Each unit in the normalized scale corresponds to delay : 150ms jitter : 20ms loss : 0.5%.
The Conclusion about Performance bounds • We show that end-user perception of audiovisual quality is more sensitive to the variations in end-to-end jitter than to variations in delay or loss • We get a simple standard about the Performance to estimate Performance
How to Analyze Performance • Thinking about two topic • Measurement • Network Condition • Measurement mean the analysis model that estimate key parameters • Of course, it is the way to compute delay, jitter ,packet loss
Two Measurement ,, • There are two methods in performance measurement • passive measurement • records and analyzes existing traffic. • active measurement • Inject sample packets into the network.
Introduce a simple Measure • Measurement Method in LAN • sends sequences of UDP packets to unlikely values of destination port numbers (larger than 30,000) • This causes the destination host’s UDP module to generate an ICMP port unreachable error when the datagram arrives
ICMP • TCP/UDP/IP 協定若有錯誤情形發生時，會利用 Internet Control Message Protocol（ICMP）協定來送錯誤訊息 。 • 在 ICMP 的 type 中，目前約有 15 種 • The ICMP echo mechanism should be installed in host in the measurement
How to Compute? • Ti = Bi / v + Di /v + CL + C • Ti − CL =(Bi + Di) / v + C.
Keep estimating • one-way delay (T i ) • T i = (Ri − Si) −Di / v −CL/2− C/2 • This calculation assumes that all delay happens on the sending path. • J i,i+1 = (T i+1 − T i ) • Packet loss = packetslost / packetssent
How about more complicated? • Precision timestamping • Queuing Model • Special Model for Protocol or device • Seem to Traffic Analysis!?
Ex: SIP Traffic Model  • A model for SIP Traffic • Two Sub Model • IP Path Model • SIP Finite State Machine
FSH Notation • Q = State set • M = fixed number of sessions • C = the bottleneck transmission rate( bit/s) • R = total capacity of IP Path measure in packets of D bits • rtt = round trip time measured in seconds • p = probability of 3xx Response • ps = successful probability of packet transmission
Sample Computation • Call Dropping rate pcd
Enviroment condition for VoIP performance  ,  • The aspects about VoIP Performance Analysis • Protocols • H.323 v.s. SIP • Network • Ethernet network v.s. wireless LAN (WLAN) network • Security for VoIP Communication • VPN protocols : PPTP v.s. IPSec
Delay in Ethernet Network • Both SIP and H.323 incurred higher delays in secure network-to-network environment. SIP H.323
Jitter in Ethernet Network • IPSec produced the highest jitter values for both H.323 and SIP communications.
Jitter in Wireless-LAN • IPSec-based VoIP communications generally incurred the highest jitter values.
Packet Loss Rates • IPSec and PPTP increased the packet loss rate in both Ethernet and WLAN. SIP H.323
Performance in Satellite Network  • Also provides IP-base data services • For remote region • As backup links
The purpose • The performance under • Delay • Random errors , burst errors • Link loading • Two codecs • 8 kb/s G.729 • 6.3/5.3 kb/s G.723.1
Baseline Tests • Bandwidth and bandwidth efficiency • Environment • No background traffic • No error • Link delay set 270ms • Run 15min with all 24 channel
Bandwidth • A single channel
Link Errors Tests • Random Error Tests and burst Error Tests • BERs (bit error rates) = BD/(B+GC) • Burst length (B) • Burst density (D) • Gap length (G) • Link capacity kb/s (C)
Link Loading Tests • Environment • With different link loading levels • Link errors or not • Packet loss • Packet delay
Tests with Errors • Combine effect of both link loading and link errors. • Error ↑,background traffic↓ link loading level↓ link loading level can’t be pre- determined