Lightweight Test Stubs and Moles for .NET

1. LightweightTest Stubs and Moles for .NET Peli de Halleux, Nikolai Tillmann Research in Software Engineering Microsoft Research, Redmond, USA

2. The Y2K bug DEMO • 1999… people packing groceries for the Y2k bug • How do you replace DateTime.Now? DateTime.Now if (DateTime.Now == new DateTime(2000,1,1)) throw Y2KBugException(); DateTime.Now = () => new DateTime(2000,1,1); MDateTime.NowGet = () => new DateTime(2000,1,1); Moles

3. (C# 3.0 Syntax) • Delegates naming convention • Lambda Expressions and Statements delegate void Action<T>(T t); // void f(inti); delegate R Func<T, R>(T t); // int f(string i); // C# 2.0 Func<string, int> f = delegate(string s) {return 0; } // C# 3.0 Func<string, int> f = (s) => 0 Func<string, int> f = (s) => { return 0; } Func<string, int> f = _ => 0

4. DEMO Y2K Bug

5. Motivation Isolation in Unit Testing

6. Unit Testing • A unit test is a small program with assertions • Tests a single (small) unit of code in isolation • Reality check: Real unit tests are not that simple! void ReadWrite() { var list = new List(); list.Add(3); Assert.AreEqual(1, list.Count); }

7. Unit Testing is not that easy • Components depend on other components • Hidden Integration Tests boolIsFileEmpty(string file) { var content = File.ReadAllText(file); return content.Length == 0; } File.ReadAllText(file); void FileExistsTest() { File.Write(“foo.txt”, “”); var result = IsFileEmpty(“foo.txt”) Assert.IsTrue(result); } File.Write(“foo.txt”, “”);

8. Isolation is critical • Slow, complicated setup, non-deterministic tests • Solution: Replace by Simpler Environment (“mocking”) • Testable Design: Abstraction layer + Dependency Injection + Mocks for testing • Simply uses virtual methods • Hard-coded Design: No abstraction layer, static methods, sealed types. • Runtime rewriting needed

9. Stubs and Moles Framework • Replace Any .NET method with A Delegate • Method can be overridden? Use Stubs • Interfaces, Abstract classes, • Virtual methods in non-sealed types • Method cannot be overridden? Use Moles • Static methods, • Sealed types, • Inline Constructor calls

10. Stubs for Unit Testing

11. Testable Design • Introduce abstraction for external components • Replace them with something simpler, i.e. a Mock IFileSystemfs boolIsFileEmpty(IFileSystemfs, string file) { var content = fs.ReadAllText(file); return content.Length == 0; } Mock, Stub, Double, Fake, … void FileExistsTest() { IFileSystemfs = ???; fs.Write(“foo.txt”, “”); var result = IsFileEmpty(fs,“foo.txt”) Assert.IsTrue(result); } IFileSystemfs = ???;

12. Stubs– Delegate Based Stubs • Replace Any .NET method with A Delegate interface IFileSystem { string ReadAllText(string file);} string ReadAllText(string file); class SIFileSystem : IFileSystem { Func<string,string> ReadAllTextString; string IFileSystem.ReadAllText(string file) { return this.ReadAllTextString(file); }} // </auto-generated> Func<string,string> ReadAllTextString; this.ReadAllTextString(file); varfs = new SIFileSystem() { ReadAllTextString = file => “”; }; file => “”;

13. DEMO IFileSystemDemo

14. Moles for Unit Testing

15. Hard-coded Design • Existing external components cannot be re-factored • SharePoint, Asp.NET, VSTO • Need mechanism to stubnon-virtual methods • Static methods, methods in sealed types, constructors • MSIL code rewriting required • Other Tools provide this functionality

16. Moles– Delegate Based Detours • Method redirected to user delegate, i.e. moled • Requires Code Instrumentation,e.g. via Profiler! • Pex provides [HostType(“Pex”)] • NUnit, xUnit, etc… also supported bool result = IsFileEmpty(“foo.txt”); Assert.IsTrue(result); MFile.ReadAllTextString = file => “”;

17. Moles under the Hood mscorlib.dll File.ReadAllText(string name) { } .NET Runtime Just In Time Compiler File.ReadAllText(string name) { var d = GetDetour(); if (d != null) return d(); } push ecx push edx push eax ExtendedReflection

18. DEMO File.ReadAllTextDemo

19. Stubs and Moles • Lightweight Framework • Type Safe • Refactorable • Testable and “Hard-coded” Code • Overridable methods -> Stubs • Any other -> Moles • Delegate Based – use the language!

20. Stubs, Moles and PexIsolated Parameterized Unit Testing

21. Parameterized Unit Testing • A Unit Test has Three essential ingredients: • Data • Method Sequence • Assertions // for all item, capacity... void Add(int item, int capacity) { void Add() { int item = 3; int capacity = 4; var list = new List(capacity); list.Add(item); var count = list.Count; void List.Add(T item) { if (this.count >= this.Capacity) this.ResizeArray(); this.items[this.count++] = item;} if (this.count >= this.Capacity) Capacity = 0 Test Case! Assert.AreEqual(1, count); }

22. Isolated Parameterized Unit Testing • Automated White box Analysis does not ‘understand’ the environment • Isolate Code using Stubs and Moles ??? if (DateTime.Now == new DateTime(2000,1,1)) throw new Y2KException(); DateTime.Now Void Y2k(DateTimedt) {MDateTime.NowGet = () => dt ... } MDateTime.NowGet = () => dt DateTime.Now == dt

23. DEMO Stubs, Moles and PexIsolated Parameterized Unit Testing

24. Pex Components Z3 Constraint Solver Pex Test Generation Automated White box Analysis Future standalone download Stubs Moles ExtendedReflection Runtime Code Instrumentation Source Code Generation

25. Stubs and Moles in Details

26. Stubs Naming Conventions • Types • Methods • Properties Bar.IFoo-> Bar.Stubs.SIFoo void Foo(string v) -> FooString String Value {get;} -> ValueGet

27. Moles Naming Conventions • Types • Methods • Properties Bar.Foo-> Bar.Stubs.MFoo void Foo(string v) -> FooString string Value {get;} -> ValueGet

28. Moles Type Structure class Foo { static intStaticMethod() {…} intInstanceMethod() {…} } class MFoo : MoleBase<Foo> { static Func<int> StaticMethod { set; } Func<int> InstanceMethod { set; } implicit operator Foo (MFoofoo); }

29. Side Effects for free • Compiler generates closures for us void Test(string content) { varfs = new SIFileSystem(); boolcalled = false; fs.ReadAllText = file => { called = true; return content; }; ... Assert.IsTrue(called); } boolcalled = false; called = true; called

30. Recursive Stubs • For free with Object Initializers interface IBar { IFooFoo {get;} } interface IFoo { string Value {get;} } IBar bar = … if(bar.Foo.Value == “hello”) ... new SIBar() .Foo .Value var bar = new SIBar { FooGet = () => new SIFoo { ValueGet = () => “hello” } };

31. Recursive Moles • For free with Object Initializers class Bar { public FooFoo {get;} } class Foo { public string Value {get;} } if(new Bar().Foo.Value == “hello”) ... new Bar() .Foo .Value MBar.Constructor = me => { new Mbar(me) => { FooGet = () => new MFoo { ValueGet = () => “hello” }}}

32. Recursive Moles And Stubs • It just works! class Bar { public FooFoo {get;} } interface IFoo {string Value {get;} } if(new Bar().Foo.Value == “hello”) ... new Bar() .Foo .Value MBar.Constructor = (me) => { new Mbar(me) => { FooGet = () => new SIFoo { ValueGet = () => “hello” }}}

33. Bind = Runtime Duck Typing • Bind all methods of an interface at once class Foo : IEnumerable<int> {...} int[] values = {1,2,3}; varfoo = new MFoo() .Bind(values); // bind all methods of // Ienumerable<int>

34. Trapping Environment • Set a trap to flag any call to a type • Iteratively build the mole sequence MFoo.FallbackToNotImplemented();

35. Per Instance Moles • Dispatching moles per instance class Foo { public int Value {get;}} varfoo = new MFoo { ValueGet = () => 1 }; var bar = new MFoo { ValueGet = () => 2 }; Assert.IsTrue(foo.Value != bar.Value);

36. Moles for New Objects • Attach Mole when Contructor is run class Foo { public Foo() {} public int Bar() {…} } MFoo.Constructor = me => { varfoo = new MFoo(me) { Bar = () => 10 }; MFoo.Constructor = null; // only 1 instance}; MFoo.Constructor = _ => new MFoo(_) { Bar = () => 10 };

37. Partial Stubs • Stubs inherited from class may call base implementation abstract class FooBase { public virtual string Value {get;}} varfoo = new SFooBase() { CallBase = true; } // call base class if no stub provided var value = foo.Value; CallBase = true;

38. Stubs Fallback Behavior • Defines behavior when stub not provided • Default throws exception interface IFoo { string Value {get;}} StubFallbackBehavior.Current = StubFallbackBehavior.Default; varfoo = new SIFoo(); var value = foo.Value; // returns null

39. Moles Fallback Behavior • Defines behavior when mole not provided • Default throws exception class Foo { string Value {get;}} varfoo = new MFoo() { InstanceFallbackBehavior = MoleFallbackBehavior.Default }.Instance; var value = foo.Value; //returns null

40. Pex Ready Stubs • Pex automatically detects and uses Stubs • Pex provides return values interface IFoo { string Value {get;}} [PexMethod] void Test(IFoofoo) { // pex uses SIFoo if (foo.Value == “foo”) throw ...; // pex chooses value

41. Give your feedback,Shape the future