1 / 24

Automated SIP Interoperability Testing

Automated SIP Interoperability Testing. Archana Rao and Henning Schulzrinne Department of Computer Science Columbia University http://www.cs.columbia.edu/IRT/voip-testbed/. SIP 2009 (Paris, January 2009). Overview. The problem of SIP interoperability Far from perfect

gyala
Télécharger la présentation

Automated SIP Interoperability Testing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Automated SIP Interoperability Testing Archana Rao and Henning Schulzrinne Department of Computer Science Columbia University http://www.cs.columbia.edu/IRT/voip-testbed/ SIP 2009 (Paris, January 2009)

  2. Overview • The problem of SIP interoperability • Far from perfect • Damages “brand”, harms customers, encourages single-vendor deployments • Testbed Overview • Architecture • Components • Experiments • Interoperability study • End-client characteristics • Summary

  3. Interoperability Is it a real problem ? Hi Henning,You may remember me from IETFs past -- I haven't attended any in some time because I couldn't find any really interesting projects. I'm finallygetting into SIP. I've got Speakeasy VoIP service, two sipphone accounts, a Cisco 7960 and a copy of x-ten on my Mac. And I still can't make it work. Voice flows in one direction only. I'm not even behind a NAT or firewall -- both machines have global addresses, with no port translations or firewalls. I've been working with Internet protocols for over 20 years. I've implemented and contributed to them. And if *I* can't figure out how to make this stuff work, how is the average grandmother expected to do so? SIP is unbelievably complex, with extraordinarily confusing terms. There must be half a dozen different "names" -- Display Name, User Name, Authorization User Name, etc -- and a dozen "proxies". Even the word "domain" is overloaded a half dozen different ways. This is ridiculous!Sorry. I just had to get this off my chest. Regards, Phil Karn. Excerpts from an email posted on IETF RAI mailing list – Reference: http://www1.ietf.org/mail-archive/web/rai/current/msg00082.html

  4. Interoperability issues Why do we have interoperability problems? • SIP is complex • More than 150 RFCs and 500 active Internet Drafts • Efforts like – “draft-ieft-sip-hitchhikers-guide” help to a certain extent, but not sufficient • SIP has no proposed architecture/profile • SIP usage in variety of environments and domains make it impossible • Standardization from IETF is different from that of ISO/ITU • SIP is continually evolving • Non trivial for implementers to follow the life-cycle from IDs to RFCs • Walled gardens, non-standard environments • Vendors need to make products that work in their customer’s environment • Poor implementations, intentional non-interoperability etc.

  5. Improving interoperability Current efforts to combat SIP interoperability issues • SIP Interoperability (SIPit) events • Week long gatherings of SIP implementers to test interoperability • Coordinated by SIP Forum • SIP Forum • SIP Connect 1.0 (and 1.1)  outsourced enterprise VoIP • IETF Basic Level of Interoperability for SIP Services (BLISS) • Focus on resolving interpretability issues in SIP features • Line sharing, call parking, automated call handling and call queuing • University of New Hampshire – InterOperability Laboratory • Vendor-neutral lab dedicated to testing data networking technologies • 20 different testing programs, each costing about $20,000 per year

  6. Interoperability So, isn’t the problem solved? • These forums focus on details of advanced SIP features • Basic-level of interoperability between innumerable SIP devices in the market • Can I take any SIP phone and make a VoIP call, through any SIP service provider? How do we go about it? 1. Identify the basic real-world scenarios for SIP registration and session establishment 2. Use our VoIP testbed to configure a variety of SIP infrastructure 3. Study the behavior of SIP end-clients and servers for interoperability

  7. Studying interoperability • Define a minimum set of call flows constituting basic interoperability • RFC 3665, RFC 3666, RFC 4317 • Use a VoIP testbed to realize a variety of real-world SIP infrastructure setups • Columbia VoIP testbed • Study the behavior of SIP end-clients and servers in each of these scenarios • Interoperability matrices • Categorization of common issues  • Lack of clarity in the specification • Implementation of an older specification • Incomplete implementation of the specification • Incorrect implementation of the specification • Failure against robustness tests

  8. Columbia VoIP testbed What? • VoIP infrastructure for experimentation, analysis, testing, prototyping and deployment of SIP/VoIP components in a variety of environments. • Part of a multi-university research project supported by NSF • University of North Texas, Purdue University and University of California, Davis Why? • There’re no telecom testbeds available for research & experimentation • VoIP implementations & experiments have lacked scientific rigor • Continuously emerging standards • Need to develop repeatable, accurate test frameworks • Realistic VoIP experiments require a distributed testbed

  9. http://www.cs.columbia.edu/IRT/voip-testbed/ http://www.cs.columbia.edu/IRT/voip-testbed/ your equipment here :-)

  10. Columbia VoIP testbed: servers • 5 SIP servers • Asterisk, Pbxnsip, Sipd, SER, OpenSER • 3 Platforms • Microsoft Windows XP • Linux Fedora Core 6 • Sun Solaris X • 3-way Connectivity • the Internet • VPN • PSTN

  11. Columbia VoIP testbed: UAs • 20+ SIP hard-phones, soft-phones, WI-FI phones with different capabilities • More details – www.columbia.edu/~ahr2114/VoIPtestbed/end-clients

  12. PSTN Network UNIVERSITY-A UNIVERSITY-B UNIVERSITY-D The Internet UNIVERSITY-C NSF VoIP testbed Image courtesy: http://secnet.csci.unt.edu/7.15.funding_application.ppt

  13. Columbia VoIP testbed experiments Planned or ongoing • Interoperability study • Can any two SIP devices talk to each other? • SIP end-client characteristics • Signaling and codec features in SIP devices • Robustness • VoIP with/despite NAT • Performance issues • Signaling delay, voice mouth-ear delay, packet loss resilience • Servers – Scalability, maximum capacity • Security • Separate testbed for DOS and other attacks work in progress

  14. Interoperability study: call flows Registration (RFC 3665) • Successful new registration • Unsuccessful registration • Cancellation of registration • Update of contact list Session Establishment (RFC 3665) • Successful session establishment • Session establishment through two proxies • Session establishment with multiple proxy authentication • Successful session with proxy failure • Unsuccessful no answer • Unsuccessful busy • Unsuccessful no response from user agent • Unsuccessful temporarily unavailable Codec Negotiation (RFC 4317) • Audio and DTMF • Audio, video and DTMF • Audio and video codec reordering

  15. Interoperability study: matrix Interoperability Matrix Test case #1 - Successful new registration

  16. Interoperability study: auth name Issue 1 – Use of different formats for authentication name • Authorization: Digest username=“user1”,realm=“proxy01.sipphone.com”,nonce=“xxx”, uri=“sip:proxy01.sipphone.com”,response=“yyy”, algorithm=MD5 Authorization: Digest username=“sip:user1@sipphone.com”, realm=“proxy01.sipphone. com”,nonce=“xxx”,uri=“sip:proxy01.sipphone.com”,response=“yyy”, algorithm=MD5 • Implication • Registration and call setup failure, if both formats are not supported by the UAs • Instances • Polycom PVX/VSX, Wengophone don’t register with 3Com / sipd proxies • Workaround • Use Asterisk between Polycom PVX and 3Com proxy

  17. SIP server UAC INVITE (Dest port:5060) 180 Ringing (Src port:12345) 200 OK (Src port:12345) ACK (Dest port:12345) 200 OK (Src port:12345) ACK (Dest port:12345) Interoperability study: rport Issue 2 – Behavior in the absence of rport parameter • Implication • Incorrect/incomplete signaling resulting in multiple retransmissions and failed transaction • Instances • Calls to Cisco 7940 via Sipphone proxy are always unsuccessful • Calls from Polycom VSX via 3com or sipd proxies are always unsuccessful • Workaround • Nothing

  18. Interoperability Study: codecs Issue 3 – Codec Negotiation • Implication – Audio/Video failure (mostly unidirectional, but bidirectional sometimes) • Problem Instances • Audio failure when Cisco calls voicemail service of sipphone provider • Video failure when Polycom PVX calls Xlite via Asterisk • [Offer] v=0 o=Cisco-SIPUA 28422 0 IN IP4 128.59.17.67 s= SIP Call t=0 0 m=audio 31030 RTP/AVP 0 8 18 101 c=IN IP4 128.59.17.67 a=rtpmap:0 PCMU/8000 a=rtpmap:8 PCMA/8000 a=rtpmap:18 G729/0 a=fmtp:18 annexb=no a=rtpmap:101 telephone-event/8000 a=fmtp:101 0-15 a=sendrecv • [Answer] v=0 o=Cisco-SIPUA 28422 0 IN IP4 128.59.17.67 s= SIP Call t=0 0 m=audio 31030 RTP/AVP 18 0 8 101 c=IN IP4 128.59.17.67 a=rtpmap:18 G729/0 a=fmtp:18 annexb=no a=rtpmap:0 PCMU/8000 a=rtpmap:8 PCMA/8000 a=rtpmap:101 telephone-event/8000 a=fmtp:101 0-15 a=sendrecv

  19. UA characteristics: features • Basic • Configuration, number of lines, call hold, call forwarding, DND, NAT support • Advanced • Encryption, symmetric RTP, conferencing, audio/video codecs, PoE • Audio quality – silence suppression, echo cancellation, comfort noise • Transport, signaling and support protocols – TCP, TLS, HTTP, DHCP, DNS, TFTP, NTP • More details at – http://www.columbia.edu/~ahr2114/VoIPTestbed/EndClients.htm

  20. UA characteristics: robustness Robustness • SIP Torture Tests (RFC4475) • A set of SIP messages aimed at stressing the SIP parser, based on experiences at SIPit. • PROTOS • Test suite with 4500+ INVITE requests, developed by University of Oulu Example Results

  21. UA characteristics: NATs SIP and NAT • Problematic for both signaling (SIP) and media transfer (RTP) • Circumventing NAT effect – solutions exist for both end-clients and proxies • Quest to overcome this, could lead to techniques uglier than NAT itself! NAT test setup • NAT realization - Linux iptables, CLICK router • Studying the behavior in different scenarios • NAT support enabled in both the proxy server and end-client • NAT support enabled in only one of them etc • Aiming to analyze the existing practices – the good, the bad and the ugly!

  22. UA with NATs SIP devices in NAT environment

  23. What now? Summary • Tests indicate that basic-level interoperability is far from perfect • Instances of interoperability failures illustrated in our IETF workshop paper • Achieving Interoperability shouldn’t be left just to vendors • non-interoperability damages SIP brand, causes grief for third parties and harms customers Going ahead – some ideas • Establish designated interop liaison for each vendor • calling customer support unlikely to be useful • Encourage vendors to publish interoperability reports • in standard format • Provide self-certification tests, supported by a remotely accessible test rig

  24. Summary Current status • The VoIP testbed & VPN connectivity is operational • Systematic study of real-world interoperability issues • Basic-level interoperability is nowhere near 100% • IETF 70 – The 1st SIP Forum Workshop on SIP Interoperability • Evaluating SIP end client characteristics Testbed plans • Experiments on NAT, performance, security • Make the testbed accessible and useful for engineers & deployments • Create a knowledge repository about SIP devices & tools for interoperability testing

More Related