A Little Language for Surveys: Constructing an Internal DSL in Ruby

1. A Little Language for Surveys: Constructing an Internal DSL in Ruby H. Conrad Cunningham Computer and Information ScienceUniversity of Mississippi

2. Domain-Specific Language (DSL) • “programming language or executable specification language that offers, through appropriate notations and abstractions, expressive power focused on, and usually restricted to, a particular problem domain” – van Deursen, Klint, and Visser • “little language” – J. Bentley

3. Typical DSL Characteristics • Not general-purpose (often not Turing-complete) • Small (at least initially) • Declarative • Semantically expressive of domain • Targeted at domain specialists, not necessarily programmers

4. External or Internal External: different from main programming language • make, yacc, pic, XML-based config files Internal (embedded): constrained use of the main programming language • use of Lisp macros, Haskell algebraic types, metaprogramming in Ruby, etc.

5. Defining Internal DSLs in Ruby (1 of 2) Flexible syntax enables convenient DSL statements • optional parentheses on method calls • variable number of arguments question “Male or female?” # has optional second arg

6. Defining Internal DSLs in Ruby (2 of 2) Blocks (closures) provide DSL structuring and delayed execution • anonymous function definitions • passed as arguments question “Male or female?”do response “Male” response “Female”do @fem = true end end

7. Implementing Internal DSLs in Ruby (1 of 2) Reflexive metaprogramming: writing programs that manipulate themselves as data • obj.instance_eval(str) executes string str as Ruby code in context of object obj • execute DSL statements dynamically • mod.class_eval(str) executes string str as Ruby code in context of module mod • declare new methods and classes dynamically

8. Implementing Internal DSLs in Ruby (2 of 2) • obj.method_missing(sym,*args) invoked when undefined method sym called with arguments args • take remedial action • obj.send(sym,*args) calls method sym on object obj with arguments args • dynamically “send a message”

9. Little Language for Surveys Domain: Specification and presentation of simple, multiple-choice surveys Tasks: • Analyze domain for language design • Design Ruby internal DSL • Implement DSL

10. Domain Analysis

11. Commonality/Variability Analysis • Determine domain boundaries (scope) • Define specialized terms and concepts (terminology) • Identify unchanging features among domain members (commonalities) • Identify features that may change among domain members (variabilities)

12. Scope • Support definition of simple, multiple-choice surveys • specification of survey • presentation of survey and collection of responses • Exclude tabulation of aggregate results for now

13. Terms and Commonalities • Survey has a title • Survey has a sequence of questions • Question has a sequence of responses • Use of conditional question depends upon previous responses • Response to silent question determined from previous responses • Survey execution presents appropriate questions to respondent and collects choices of responses

14. Variabilities • Texts for title, questions, and responses • Number and order of questions within survey • Number of responses expected for question • Number and order of responses in question • Condition under which question included • Method for answering silent question • Source of survey specification • Method for displaying questions and collecting responses during execution

15. Internal DSL Design

16. DSL Design Strategy • Adopt declarative approach – structure explicit but processing implicit • Use terminology and commonalities to suggest language statements • Represent variabilities as values that survey programmers supply

17. Survey DSL Example (1 of 3) # C1 survey has title (V1) title“ACMSE Conference Survey” # C2 survey has seq of questions (V2) question“Are you an author?”do # (V3) # C3 question has seq of responses (V4) response "Yes" do # execute if chosen @author = true end response“No” do # execute if chosen @author = false end end

18. Survey DSL Example (2 of 3) # C4 conditional question (V5) question "What type of paper?" do condition { @author } # when execute? response "Regular" do @p = :rg end response "Student" do @p = :st end response "Work-in-progress" do @p = :wp end # V5/V6 block on response & action sets # state for conditions & silent choices action {@t = 25; @t = 15 if @p == :wp} end

19. Survey DSL Example (3 of 3) # C5 silent question calculates choice (V6) result "How long is presentation?“do condition { @author } alternative "25" do # when choose? @p == :rp || @p == :sp end alternative“15” do # when choose? @p == :wp end end

20. Internal DSL Implementation

21. Two-Pass Implementation • Parse DSL and build abstract syntax tree (AST) • Execute survey by traversing AST AST DSL parse execute

22. First Pass: DSL Parsing • Use instance_eval to execute DSL input as Ruby code (V7) • Let Ruby interpreter do most of parsing • Add methods for each DSL statement – title, question, action, etc. • Check specialized syntax and build AST as 3-level tree (survey, question, response) • Defer conditions and actions by storing unevaluated blocks (i.e., closures)

23. Second Pass: DSL Interpretation • Traverse AST to display questions and collect responses • Execute deferred blocks needed for conditions and actions • Use missing_method and class_eval to create reader/writer methods for variables in blocks

24. Architecture First pass – use Object Scoping, Context Variable, and Memento patterns for safety and flexibility, Deferred Evaluation for actions Second pass – use Visitor pattern for flexibility (C6, V8)

25. Object Scoping (1 of 2) class SurveyDSL # METHOD BODIES OMITTED attr_accessor :context def title(text) def question(text,*args) # text, nsel, block def result(text,*args) # text, nsel, block def condition(&check) def action(&action) def response(resp_text,&action) def alternative(alt_text,&guard) end#SurveyDSL class SurveyBuilder < SurveyDSL

26. Object Scoping (2 of 2) class SurveyBuilder < SurveyDSL . . . def read_DSL(rb_dsl_file) # Omit checks for file existence rb_file = File.new(dsl_file) instance_eval(rb_file.read, dsl_file) # load/eval rb_file.close self end#read_DSL . . . end#SurveyBuilder

27. Parser Implementation (1 of 3) def title(text) @context.incr_ql_count @context.ql_error = false if @context.level == :survey_level && @context.survey.title == nil @context.survey.title = text.to_s else illegal_stmt_msg(”title") stmt_text_msg("Title",text) if @context.survey.title != nil at_most_one_msg("title","survey") end end self end#title

28. Parser Implementation (2 of 3) def question(text,*args) # text, nsel, block @context.incr_ql_count @context.ql_error = false if @context.level == :survey_level &&block_given? @context.level = :question_level @context.qtype = :question_type nsel = 1 nsel = args[0].to_i if args.size > 0 @context.question=QuestionNode.new(text.to_s,nsel) yield # execute the block on the question call nresp = @context.question.responses.size if nresp < @context.question.num_to_sel # error messages omitted end

29. Parser Implementation (3 of 3) if !@context.ql_error @context.survey.add_question(@context.question) else # error messages omitted end @context.level = :survey_level @context.qtype = :no_type else end @context.question = nil self end#question

30. Did Survey Language Follow Internal DSL Recommendations? JMock designers Freeman and Pryce • Separate syntax and interpretation into layers • Use, and abuse, the host language • Don't trap the user in the internal DSL • Map error reports to the syntax layer

31. Summary • Illustrated how commonality/variability analysis can be adapted for DSL design • Demonstrated how Ruby facilities can be used for internal DSL development • Explored how design patterns can help lead to safe and flexible DSL processors

32. Future Work • More systematic techniques to explore domain and discover needed constructs • Improved runtime error handling tied to DSL input • Better facilities for user extension • Investigation and comparison of other languages – Groovy, Scala, Haskell

33. Acknowledgements • H. C. Cunningham. A little language for surveys: Constructing an internal DSL in Ruby, In Proceedings of the ACM SouthEast Conference, 6 pages, March 2008. • Members of Fall 2006 graduate class on Ruby and Software Development • Suggestions on the paper by Chuck Jenkins, Yi Liu, Pallavi Tadepalli, and Jian Weng

34. Questions