WildHeart.HTTPListener an open source Web Server for Dyalog APL v11.0

1. WildHeart.HTTPListeneran open source Web Server for Dyalog APL v11.0 Stefano “WildHeart” Lanzavecchia

2. Agenda APL should be a popular language for developing web pages, because: APL has all you need to generating static and dynamic web content It is not used as much as it deserves, because: Even the best tools need good frameworks to level the learning curve a bit No frameworks are readily available Dyalog User Group Conference

3. Goal • Explore the problem which needs to be solved and demonstrate a proposed solution... (obviously ) • So, what’s the problem? • The easy part is generating HTML • Hopefully nobody will disagree that APL is an EXCELLENT language for generating character vectors Dyalog User Group Conference

4. A ”web” Some very simple HTML Dyalog User Group Conference

5. The hard part If we have a form like this and the user hits the submit button… Dyalog User Group Conference

6. Before we look at the response… <tr> <td>Maximum Interest Rate (%)</td> <td><select name="IntrMax" size="1"> <option>15.0</option> <option>14.5</option> ..etc down to… <option>1.0</option> </select></td> </tr> <tr> <td>Minimum Interest Rate (%)</td> <td><select name="IntrMin" size="1"> <option>15.0</option> <option>14.5</option> <option>14.0</option> <option>13.5</option> <option selected="selected">13.0</option> <option>12.5</option> <option>12.0</option> …etc… <option>1.5</option> <option>1.0</option> </select></td> </tr> </table> <p><input type="submit" value="Calculate Repayments"/></p> </div> </form> Some of you will want to knowhow the HTML looks for the form? <form action="RUN" method="post" name="FrontPage_Form1"> <div> <table border="0" cellpadding="3"> <tr> <td>Purchase Amount</td> <td><input type="text" size="9" name="LoanAmt"/></td> </tr> <tr> <td>Percent Down</td> <td><input type="text" size="6" maxlength="6" name="PercentDown"/></td> </tr> <tr> <td>Maximum Years in Loan</td> <td><input type="text" size="2" maxlength="2" name="LenMax"/></td> </tr> <tr> <td>Minimum Years in Loan</td> <td><input type="text" size="2" maxlength="2" name="LenMin"/></td> </tr>… continued on the right … Dyalog User Group Conference

7. When the user hits the button… POST /loan/RUN HTTP/1.1 Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/x-shockwave-flash, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, */* Referer: http://localhost:8080/loan/loan.htm Accept-Language: it Content-Type: application/x-www-form-urlencoded Accept-Encoding: gzip, deflate User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; WinFX RunTime 3.0.50727) Host: localhost:8080 Content-Length: 74 Connection: Keep-Alive Cache-Control: no-cache LoanAmt=10000&PercentDown=50&LenMax=10&LenMin=15& IntrMax=15.0&IntrMin=13.0 Dyalog User Group Conference

8. The hard part… • APL is not a bad tool for taking character vectors apart, either  • But: Some knowledge about the HTTP protocol is required to decipher it • A rule for calling the code to deal with the request (in this case #.loan.RUN) is needed • Some way to decode the body of the request • This is what Web Server frameworks like Microsoft IIS coupled with ASP.NET do… Dyalog User Group Conference

9. What is a web application? “A collection of functions that take an HTTP Request as input and produce an HTTP response as output” Dyalog User Group Conference

10. Solutions • Dyalog APL integrates quite well with Microsoft IIS+ASP.NET, and can be called from other web servers, but: • These solutions require a lot of heavy machinery from other vendors • The learning curve is still quite high • Dyalog provide a sample SERVER workspace, but: • It leaves almost all the work of deciphering HTTP to you • It’s based on old standards Dyalog User Group Conference

11. Solutions The author has: • A keen (geek) interest in OO, web and other software development methodologies • Written numerous tools on top of the Dyalog TCPSocket object Dyalog User Group Conference

12. Enter WildHeart.HTTPListener • The advent of Object Orientation in Dyalog 11.0 makes it attractive to start again • My goal is to create a framework which: • Requires a little knowledge of HTML and hardly any knowledge of HTTP • Allows ”ordinary” APL developers to easily write Web applications, yet: • Is built from reusable, replaceable components which can be extended by APL developers who learn a little more... • Is portable across all platforms which support TCP • Is run as an open source project, to be shared and extended by anyone who wishes to contribute Dyalog User Group Conference

13. To write or Not to write • Why NOT TO WRITE a web server in APL: • This is the third web server I develop: • I was paid to develop the first (and it was never used) • I was almost paid to the develop/steal the second (and it’s heavily used) • I was NOT paid to develop the third and I had to find the time to do it at night and during weekends • A wise man once said: “only a fool would do that” Dyalog User Group Conference

14. Not to write, seriously • HTTP is easy but tricky • Reliability • Security • Speed • So many web servers out there Dyalog User Group Conference

15. To write or not to write • Why write a web server in APL? (I wrote an editorial about it in Vector a few years ago) • Easily embeddable in an APL application • Easily understood by APLers • Can stay near the core of the application Dyalog User Group Conference

16. To write or not to write "One of the reasons to use Lisp in writing Web-based application is that you can use Lisp. When you're writing software that is only going to run on your own servers, you can use whatever language you want.“ Paul Graham Dyalog User Group Conference

17. To write! “Morten needs a demo. I need an HTTP/1.1 web server, written in APL. APL Italiana, the company I work for, also needs one, even if they haven’t realised this yet.” Stefano Lanzavecchia Dyalog User Group Conference

18. HTTPListener HTTPRequest Headers Cookies Request Variables HTTPResponse Cookies HTML Utils Event pipeline User Validation Sessions Static file server Namespace as filesystem… File Handlers: aplx files apls files … Components of… WildServer’06 Dyalog User Group Conference

19. Dyalog APL: an OO language • Web server: a good test bed for the OO extensions • Abstractions: divide et impera • Inheritance: this looks a bit like that • Keyed properties: syntax tricks • What is missing in the language? • Is the IDE good enough? Dyalog User Group Conference

20. HTTPListener fs←⎕NEW #.WildHeart.FileSystem('c:\aplserve') listener←⎕NEW web.HTTPListener(8080 fs) listener.Start Then point your web browser at http://localhost:8080/ “It’s a kind of magic” Dyalog User Group Conference

21. The trip of an HTTP request • TCPSocket gets TCPRecv events • HTTPRequest objects collects them, and parses the request according to the HTTP specs • When the request is complete, it gets shipped to a dispatcher… Dyalog User Group Conference

22. The dispatcher: FileSystem • To me a FileSystem looks like a namespace: • Its folders are sub-namespaces • Idea: we can inject virtual folders: • A virtual folder can be a folder on the HD coming from a different sub-folder • A virtual folder can be… an APL namespace! Its functions would be files… Dyalog User Group Conference

23. APLNamespace http://localhost:8080/loan/RUN executes #.loan.RUN but http://localhost:8080/loan/bullet.gif will transmit the file c:\aplserve\loan\bullet.gif fs←⎕NEW #.WildHeart.FileSystem('c:\aplserve') fs['loan']←⎕NEW web.APLNamespace(#.loan'c:\aplserve\loan') listener←⎕NEW web.HTTPListener(8080 fs) listener.Start Dyalog User Group Conference

24. ZipFileSystem • Now the folders and files come from a zip file. • Neat, uh? • In fact this presentation… fs←⎕NEW web.ZipFileSystem('c:\aplserve.zip?aplserve') fs['loan']←⎕NEW web.APLNamespace(#.loan'c:\aplserve\loan') listener←⎕NEW web.HTTPListener(8080 fs) listener.Start Dyalog User Group Conference

25. FileSystem • Basically one method: ProcessRequest ∇ ProcessRequest ctx;html;ns;raw;aplo :Access Public Instance Override ctx←#.WildHeart.HTTPContext.Current html←_NS⍎MapFileName ctx.URI r←ctx._Response r._IsBodyRaw←raw←82≠⎕DR html r._Body←html :If ''≡⊃r._Headers[⊂'Content-type'] r._Headers[⊂'Content-type']←u.MimeMappings['htm' 'bin'[1+raw]] :EndIf ctx.SendResponse ∇ Dyalog User Group Conference

26. Tricks: thread local storage […] ctx←#.WildHeart.HTTPContext.Current […] • Thread context: wouldn’t it be nice to have an object local to a thread? Dyalog User Group Conference

27. HTTPHandler • A FileSystem delegates the handling of the requested resource to the appropriate handler: • StaticFileHandler by default • AplxHandler • AplsHandler • … more to come fs._FileHandlers[⊂'aplx']←#.WildHeart.AplxFileHandler Dyalog User Group Conference

28. HTTPHandler • One method: ProcessRequest ∇ ProcessRequest ctx;fname;tn;ext;etag :Access Public Instance Overridable fname←ctx._FileSystem.MapFileName ctx.URI tn←fname ⎕NTIE 0 ctx._Response._IsBodyRaw←1 ctx._Response._Body←⎕NREAD tn,83,⎕NSIZE tn ⎕NUNTIE tn ctx.SendResponse ∇ Dyalog User Group Conference

29. AplxHandler “This is only a test” <?@ master="../newindex.maplx" execute="" ?> <apl:content id="title"> <title>Rain Graphics: Climate</title> </apl:content> <apl:content id="cssstyle"> <link rel="stylesheet" type="text/css" href="../styles/spiffy.css"/> </apl:content> <apl:content id="main" execute="#.rain.Climate"> </apl:content> Dyalog User Group Conference

30. <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns=“http://www.w3.org/1999/xhtml” xmlns:apl="http://wildheart.com/2006/10/" > <head> <apl:placeholder id="cssstyle"> <link rel="stylesheet" type="text/css" href="styles/spiffy.css"/> </apl:placeholder> <apl:placeholder id="title"> <title/> </apl:placeholder> </head> <body> <div id="header"><h1 class="banner">WildHeart.HTTPListener</h1></div> <div id="subheader"> <apl:placeholder id="bcrumb" execute="#.WildHeart.WebDemos.BreadCrumb" /> </div> <div id="toc"> <apl:placeholder id="apltoc"> <div class="toc"><h2>APL Web Server</h2> <ul> <li><a href="/intro.aplx">Introduction</a></li> <li><a href="/cv/Index.aplx">View Code</a></li> … A master page Dyalog User Group Conference

31. The event pipeline • As a request goes through the FileSystem hierarchy, events are raised: • OnEnter • OnAuthenticate • OnAuthorize • … Dyalog User Group Conference

32. :Class Event :field _Observers ∇ make name :Access Public Instance :Implements Constructor _Observers←⍬ ⎕DF'Event: ',name ∇ ∇ Raise(sender arg);obj;fnname :Access Public Instance :For obj fnname :In _Observers (obj⍎fnname)(sender arg) :EndFor ∇ ∇ Add(obj fnname) :Access Public Instance _Observers,←⊂obj fnname ∇ ∇ Del(obj fnname) :Access Public Instance _Observers← _Observers~⊂obj fnname ∇ :EndClass Tricks: the observer pattern Dyalog User Group Conference

33. The observed shows what it’s got • onAuthenticate←⎕NEW #.WildHeart.Event('Authenticate') auth←⎕NEW web.WebDemos.FormsAuth fs.onAuthenticate.Add auth'OnAuthenticate' • onAuthenticate.Raise ⎕THIS lurl Dyalog User Group Conference

34. Two simple authentication modules • SimpleAuth: based on HTTP challenge response: • Quick and dirty • FormsAuth: based on cookies • More flexible • More work Dyalog User Group Conference

35. Sessions • HTTP is a stateless protocol • People want to have conversations with peers who don’t forget instantly what they just said • Have a cookie! Dyalog User Group Conference

36. Sessions • As easy as: fs.EnableSession ⎕NULL fs.onSessionStart.Add #’DoSomething' fs.onSessionEnd.Add #’UndoSomething' • In the application: :If ⎕NULL≡ns←ctx._Session[⊂'footer'] ns←⎕NS'' ns.count←0 ctx._Session[⊂'footer']←ns :EndIf ns.count+←1 Dyalog User Group Conference

37. Extensibility • Some of the features (ZipFileSystem, session module, authentication, aplx handler) were hacked together to prove that the server could support them • Even before its completion, the WildServer has: • Had a file handler written (by Nic & Morten) • Been used to run a course in writing Web Applications using Dyalog APL Dyalog User Group Conference

38. Open Source • Open Source means: “free as in free beer” (except: is there such a thing as free beer?) • No strings attached: • You get the source code and whatever documentations exists with it at the time • You get the right to do whatever you want with it (even sell it! But only a bad boy would do that…) • You get from me all the support I can give you when I can if I can (alas, I am very busy and maybe one day I’ll get a life) Dyalog User Group Conference

39. Why Open Source? • Have you looked at Ruby on Rails? • Tons of contributors do a lot of work • Tons of clever contributors do a lot of good work! • OK, maybe that’s a good product. Dyalog User Group Conference

40. TODO list • It’s long, it’s scary, but… • APL Prototypes seem to acquire a life of their own • The “ducks” have started contributing Dyalog User Group Conference

41. Thanks • Peter Michael Haager: I have adapted his XML parser to produce trees of objects instead of nested arrays Dyalog User Group Conference

42. Conclusion • Initial feedback from users (Nic, Morten and Mondays course participants) is promising • The WildServer is still very much ”work in progress”, and the author wants to keep control of it until about Christmas • However, you are all welcome to try it now and suggest or code enhancements Dyalog User Group Conference

43. Questions? • If not… now I need some sleep. 輝く空と君の声を感じたい Dyalog User Group Conference

44. What is missing in the language • No public operators! C# has them! • No protected members! C# has them! • No object serialization! C# has it! • C# is a statically typed language. OK, substitute in the slide “C#” with “Ruby” Dyalog User Group Conference