Devops – The Last Mile

1. Devops – The Last Mile Workspace Management

2. jay.flowers@gmail.com http://jayflowers.com Jay Flowers

3. Workspace Management The act of defining, creating, and maintaining the software tool stack that comprises a developer’s or tester’s computer configuration.

4. Why?

5. Problem? Why do we need Workspace Management? Do we have a problem? As workspace configurations drift the more pervasive and sever “works on my box” syndrome is encountered. They drift from each other as well as away from production.

6. The Norm • Manually • Tribal knowledge • Some documentation, generally out of date • Lots of options, room for variation • Unreliable results • Difficult to reproduce same results

7. The Goal • Fully automated • Infrastructure as code • Institutionalized knowledge • Code as documentation • No options • Tested • Reliable, repeatable results

8. Needs • We need a way to unobtrusively manage all the developer/tester workspaces on a project • We need all the software and configuration that makes up the workspace to be centrally managed • We need all the workspaces to look and function identically • We prefer to use the same tool to manage workspaces as we use to manage servers

9. Chef • Automation for installing and configuring software • Open Source • Supports Windows, OSx, and Linux • Why Chef and not Puppet? • Internal DSL, more powerful than Puppet’s external DSL • 100% free, Puppet is only partially free • Can bootstrap to create setup application • System level CM • Testable with Vagrant or EC2 • Higher quality from agile testing

10. Demohttp://jayflowers.com/Misc%20Downloads/Workspace-Setup.exeDemohttp://jayflowers.com/Misc%20Downloads/Workspace-Setup.exe

11. Chef Modes • Server/Client • Secure communication, authentication, and authorization with public/private keys • Servers control execution of clients • Solo • Client without a server • Execution is initiated at the command line on the client

12. When to use Solo and when to use Server/Client • Solo for workspace management • Updates controlled by workspace owner, on demand • Cookbooks stored in workspace • Server/Client for everything else • Update controlled by server, pushed to clients • Cookbooks stored on server • We can use Chef to deploy the applications we build!

13. Chef Anatomy • Server/Client or Solo • Cookbooks (written to be idempotent) • Resources and Providers – used to perform typical actions (make user, create directory, install package, etc…) • Attributes – parameters specific to the cookbook • Nodes • The client list of cookbooks to apply • Roles • Lists of cookbooks to be applied • Helpful in organizing nodes into groups (e.g. frontend apache server) • Databags • Parameters specific to the target nodes and or organization

14. Bootstrap Solo Configuration require 'minitest-chef-handler' file_cache_path "c:/tools/chef/cache" cookbook_path [File.dirname(__FILE__) + "/Chef/cookbooks"] file_backup_path "c:/tools/chef/backup" role_path "c:/tools/chef/roles" json_attribsFile.dirname(__FILE__) + "/node.json" handler = MiniTest::Chef::Handler.new(:path => File.join(Chef::Config[:cookbook_path], "*", "test", "*test*.rb")) report_handlers << handler log_level :debug log_location STDOUT

15. Node.json { "run_list": [ "recipe[git]", "recipe[tortoisegit]", "recipe[get_git_source]", "recipe[springsource_sts]" , "recipe[java]", "recipe[sysinternals]", "recipe[virgo]", "recipe[gradle]", "recipe[notepadplusplus]", "recipe[maven]", "recipe[groovy]" ]}

16. Java Cookbook windows_package"java" do source node['java']['url'] checksum node['java']['checksum'] action :install installer_type :custom options "/s /v \"/qn INSTALLDIR=#{node['java']['java_home']}\"" not_if { File.directory? node['java']['java_home'] } end

17. Git Clone Cookbook dir = node['get_git_source']['dir'] directorydirdo inherits true recursive true action :create end

18. Git Clone Cookbook ruby_block"clone_git_repo" do block do url = node['get_git_source']['url'] Chef::Log.info("Cloning the repo #{url} to directory #{dir}, this could take a while...") git = Chef::ShellOut.new("git clone #{url} #{dir}") git.run_command puts git.stdout ifgit.stderr != '' puts "error messages: " + git.stderr end # Raise an exception if it didn't exit with 0 git.error! end action :create

19. Git Clone Cookbook not_ifdo Chef::Log.info("Checking if #{dir} is a working git repo") git = Chef::ShellOut.new("git status", :cwd => dir) git.run_command puts git.stdout ifgit.stderr != '' puts git.stderr end ifgit.exitstatus == 0 true else false end end end

20. Chef Foots the Bill • Written to easily manage large numbers of servers, translates to workspaces too • Installs and configures software and operating systems creating reliable and repeatable results • Creation of new servers and workspaces becomes so easy allowing us to treat them as disposable resources • Can be used to repair/heal broken workspaces and servers

21. Benefits • What works on my box works everywhere else… • Dramatically reduced time to create new workspace • Workspace updates are non-events • Heal broken workspace • Can be managed by unskilled team members • Pairs well with other scripting (e.g. automated deployments)

22. Cucumber • Test Tool • Open Source • Tests are written in plain text • Tests are implemented in Ruby, the same language as Chef is implemented in… • Tests use virtualization libraries to create, control, and destroy test instances/environments

23. Simple Example

24. Cucumber Example

25. Minitest Example

26. Test Driven • These tools enable a test driven approach • Tests can be written before Chef cookbooks are written • Tests can be executed in a local workspace as well as on a CI server

27. Test Resources • ChefSpec - https://github.com/acrmp/chefspec • MiniTest - https://github.com/calavera/minitest-chef-handler • Test Kitchen - https://github.com/opscode/test-kitchen • Examples • https://github.com/opscode-cookbooks/apache2/ • https://github.com/opscode-cookbooks/mysql/

28. A LookUnder the Hoodhttps://github.com/jflowers/virgo.sample-greenpages