TWiki at CERN Service Evolution

1. TWiki at CERNService Evolution Terje Andersen, Peter Jones for IT-PES-IS Jan 2014

2. Agenda • Introduction • Overview • Current use • Content maintenance • Issues identified with SNOW • New development (LDAP integration, search) • TWiki web server configuration (current and future) • TWiki Software future • TWiki at CERN roadmap • Questions TWiki at CERN

3. TWiki at CERN • TWiki CMS, an Open Source collaboration tool • Installed in 2003 (requested by Atlas software developers) • Required: apache, Perl and any browser. • Initially installed on AFS with kerberos. TWiki at CERN

4. Who is using TWiki • LHC experiments • Groups / Sections • Software projects • Personal websites • 13000+ registered users • 150,000+ topics • 300+ webs • 800,000+ attachments • The trend shows a need to keep improving the service TWiki at CERN

5. TWiki at CERN worldwide use GeoIP lookup DB provided by TWiki at CERN

6. What are they doing • Collaboration using • Blackboards • Dashboards • Blogs • Manuals or Workbook creation and PDF generation • Taking of minutes of meetings • Action lists • FAQs • Spreadsheets • Photo albums • Slideshows • etc… TWiki at CERN

7. TWiki use example TWiki at CERN

8. Content Maintenance • TWiki webs and topics need care and attention • wiki gardening • Organise pages appropriately (e.g. move of Atlas Computing pages) • Prune back the superfluous information • Reshape topics into its most useful form TWiki at CERN

9. TWiki Gardening • To aid performance • Deleting or archiving unused topics (being investigated) • Review backend store • Use %VARCACHE% for faster page rendering • To help with navigation • Group pages together (static or dynamic) • Provide better topic names • Use menus, tags or plugins To keep content current and correct it is important to undertake regular reviews TWiki at CERN

10. TWiki navigation tools %TREEVIEW{web=“MyWeb” topic=“MyTopic”}%) TWiki at CERN

11. TWiki navigation tools continued %WEBMAP{lr="1" excludesystem="1"}% TWiki at CERN

12. TWiki navigation tools continued %SEARCH{ "." scope="topic" type="regex" nonoise="on" order="modified" reverse="on" header="|*Topic*|*Last Edited By*|*Date*|" format="$percntCALC{$IF($EXACT($wikiname, %WIKINAME%), |$topic| $wikiusername|$date|, <nop>)}$percnt"}% TWiki at CERN

13. Agenda • Introduction  • Overview  • Current use  • Content maintenance  • Issues identified with SNOW • New development (LDAP integration, search) • TWiki web server configuration (current and future) • TWiki Software future • TWiki at CERN roadmap • Questions TWiki at CERN

14. TWiki and SNOW • Improve the service by analysing the SNOW tickets • Categorize the Incidents and Requests • Objectives: create modules to decrease tickets by category. Example: LDAP integration, search optimization Tickets up until Sep 2013 TWiki at CERN

15. TWiki development New development (LDAP integration, search) TWiki at CERN

16. Misc development • Implementing a two-step logout in TWiki. • Problem: The TWiki logout mechanism does not kill SSO session by default. • Consequence: CERN TWiki logout links points directly to SSO logout, not killing the TWiki session. • Solution: We created CernLogoutPlugin which intercepts a logout attempt, kills the local TWiki session before redirecting to SSO logout. Kill SSO Session Kill TWiki Session Go-back link TWiki at CERN

17. Misc development • Short URLs • Using apache rewrite rules • Can be extended to other types of actions: • /e/CMS/PixelOfflineSoftware for edit • /pdf/CMS/PixelOfflineSoftware for generating pdf TWiki at CERN

18. Misc development • 3-way search bar • Using the search bar in the top right, a CERN TWiki user can choose to search for content using TWiki search, CERN search and Google search. TWiki at CERN

19. TWiki development Incorporating LDAP in TWiki TWiki at CERN

20. Incorporating LDAP in TWiki • Get rid of the TWiki registration process. • Used to tie a TWikiName to a CERN identity. • Most often FirstnameLastname, like JohnDoe. • Solution: Ask LDAP and build it automatically. Typical entry-point; A first-time visitor to a protected topic. SSO Authentication (Automatic) LDAP-Generated TWikiName (Automatic) TWikiRegistration (Manual) The protected topic TWiki at CERN

21. Incorporating LDAP in TWiki • E-group availability on allTWiki scripts. • Each TWiki action have a different script. • TWiki is partly open to the public; some scripts not protected. • In a ACL protected topic, we must redirect to read e-groups. • Solution: Ask LDAP instead of redirecting the user. /view/Web/Topic /viewauth/Web/Topic LDAP TWiki at CERN

22. Incorporating LDAP in TWiki • We’re using LdapContrib, a TWiki community extension • Offers identity management, authentication and authorization using LDAP. • Ability to contribute code back to the TWiki developer community, as well as receiving free bug-testing and improvements. • We chose it for identity management and authorization, since authentication is handled by Shibboleth (SSO). • LdapContrib stores LDAP data in a local cache using a BerkeleyDB v1 to avoid querying the LDAP server unnecessary. TWiki at CERN

23. Incorporating LDAP in TWiki • Issues solved in LdapContrib • LdapContrib uses the Perl module DB_File for database operations which does not support locking. • With the use of module DB_File::Lock we are able to set read and write locks to the cache. TWiki at CERN

24. Incorporating LDAP in TWiki • Issues solved in LdapContrib • The default TWiki user mapping is not preserved. • In our case 3000 / 13000 registered users will in our case get a new WikiName if this is automatically calculated. • When solve this by implementing that all these rules are loaded immediately after the cache first is created, taking precedence. • Users removed from LDAP is not preserved in the LdapContrib cache • This makes it possible for TWikiNames changing hands. • Solution: Rewriting the package requiring users to be manually deleted once they are put in the cache. TWiki at CERN

25. Incorporating LDAP in TWiki • Better e-group support; TWikiGroups in topic ACLs will be deprecated • Still supported, but not reccommended. • Use TWikiUsers directly for restricting access to individuals. • For example for user topics. • Long-term goal: Eliminate TWikiGroups from ACLs. ACL examples: TWikiUsers TWikiGroups TWiki at CERN

26. Incorporating LDAP in TWiki • Additional plug-in: LdapContribAdminPlugin • Creating an admin panel for administering the LdapContrib database • Allows viewing, changing and deleting user and group records, viewing statistics. • Created with JQuery making AJAX calls over REST to a Perl backend, reading from and writing to the cache. • Target group: CERN TWiki administrators, and the second level support team for CERN TWiki. • Code will be contributed back to the TWiki developer community. TWiki at CERN

27. Incorporating LDAP in TWiki TWiki at CERN

28. Incorporating LDAP in TWiki • Additional plug-in: LdapUserInfoPlugin • Makes all TWiki identities in a topic dynamic • By hovering over them, you get a LDAP-generated summary about the user behind the TWikiName without having to do guesswork and go to the CERN phonebook. • No need to look at a TWikiName, guess the name of the user, go to phonebook and then do a search. • Information shown depends on the login status and access level. • Created with JQuery and AJAX • Code will be contributed back to the TWiki developer community. TWiki at CERN

29. Incorporating LDAP in TWiki TWiki at CERN

30. TWiki development • Deployment forecast: • 3-way search bar Already implemented • LdapContrib End of January • LdapContribAdmin End of January • LdapUserInfoPlugin End of January +~ 1 week. • Two-step logout End of January +~ 1 week. • Compact URLs End of January +~ 1 week. • After successful deployment, enhancements and new plugins will be contributed back to the TWiki developer community. TWiki at CERN

31. Agenda • Introduction  • Overview  • Current use  • Content maintenance  • Issues identified with SNOW  • New development (LDAP integration, search)  • TWiki web server configuration (current and future) • TWiki Software future • TWiki at CERN roadmap • Questions TWiki at CERN

32. TWiki server changes • Change of architecture • Soon to move off the Dell Blades to an OpenStack VM cluster . More flexible • TWiki Grizzly VM SLC6 machines created and managed by puppet. • Load balancing using ‘sticky’ sessions enabled. (mod_proxy) • Tests made with 2 front end and 3 backend m1.xlarge machines • Performance issues with NFS. Better with tuned-adm profile default • Current production load time less that 1 sec • VM load times between 1.2 -1.6 seconds and more tuning necessary TWikiUser Load-balancing Frontend Backend TWiki at CERN

33. TWiki Backend Store Changes • Backend = storage for topics and attachments • Moved. DB to DSS NetApp filers Aug 2013 TWikiservers NFS mounted NetApp Filer Storage for TWiki topics and attachments • Backup and snapshots available • Results are good with a better and more stable response than before Aug 2013. • For disaster recovery we a read-only offsite copy prepared. TWiki at CERN

34. Wikis – other future options • Checkout other wiki solutions for the future. • Many wikis solutions available • There are 143 listed in http://www.wikimatrix.org • Examples: • MediaWiki: GPL, php, Mysql, nonstructured (wikipedia) • DokuWiki: GPL, php, files DB (small companies) • TikiWiki: LGPL, php, Mysql (similar to Drupal wiki-centric) • Confluence: Atlassian, Java, SQL, linkup with JIRA and MS. • Foswiki: a TWiki Fork, Perl , files, RCS. • We have imported data from other wikis into TWiki TWiki at CERN

35. TWiki Fork • 2008 - dispute between TWiki developers. • FOSWIKI - a TWiki fork was created. • Timing coincided with LHC startup • CERN-IT stayed with TWiki for stability • Same backend for both TWIki and Foswiki • Further tests to be made on Foswiki • We monitor development on both versions Number of contributors who made changes to the project source code each month. http://www.ohloh.net TWiki at CERN

36. TWiki Future • Version 6.0.0 released late 2013 • 50,000 Small Businesses, many Fortune 500 Companies, and Millions of People use TWiki TWiki at CERN

37. TWiki Roadmap Schedule for changes • TWiki NetApp migration - August 2013  • TWiki engine upgrade to 5.1.4 – end Sep 2013  • Grizzly cluster preparations- Autumn 2013 • Migration to the VM cluster – 1Q 2014 ?? • Migration to LDAP – 1Q 2014 • Test and upgrade to TWiki 6 – 1H 2014 • Installation and tests of Foswiki and/or other wikis • Investigate and test archiving options TWiki at CERN

38. Agenda • Introduction  • Overview  • Current use  • Content maintenance  • Issues identified with SNOW  • New modules (LDAP integration, search)  • TWiki web server configuration (current and future)  • TWiki Software future  • TWiki at CERN roadmap  • Questions TWiki at CERN

39. Questions Thanks For Listening…. Terje Andersen, Peter Jones IT-PES-IS TWiki at CERN