Using Plone to devolve Web site management and maintenance

1. Using Plone to devolve Web site management and maintenance Vancouver Python Workshop 2004 Dominic Hiles Web Developer

2. Overview • Who am I ? • Project background • Devolving content maintenance • Business roles • Workflow • Editing content • Devolving user management • MembershipTool limitations • Notifications mechanism • Transfer of object ownership on account deletion

3. Who am I ? • Web developer within the Internet Development (ID) group of the Institute for Learning and Research Technology (ILRT) • ID group has 10+ staff: usability engineers, designers, developers, project managers • Started Zope/Python development April 2003 • A Plone "newbie" – started September 2003

4. Church of England Project • Proposal finalised Feb 2003 • Graphic design agreed Sep 2003 • Plone development started October 2003 • Development happened in tandem with Plone 2 release cycle

5. Background - content • 2000 static HTML pages • 350 images, 450 "text" files (e.g. PDFs, MS Word docs) • 4 ASP Web applications, serving data from around 20,000 database records, held in 4 different databases

6. Migration of static content • HTML pages passed through HTMLTidy to convert to XHTML • All pages then undergo manual editing whilst on the file system • Pages, folders and files (e.g. PDFs, images etc.) imported into ZODB via script and created as Plone objects

7. Migration of Web applications • 3 MS Access databases exported as CSV files and imported into new Archetypes objects • 1 remaining database: • Data must be maintained elsewhere • Previously exported as CSV, manually imported into MS Access, then displayed via ASP • Now, CSV file uploaded "Through The Plone" (TTP) and data are automatically imported into PostgreSQL

9. Devolving content maintenance • All content can now be maintained TTP but… • …up to 100 users with responsibility for content in different areas of the site

10. Talk focus • Providing the mechanisms for these users to easily maintain the content for which they’re responsible • Providing the tools to manage these users

11. Content maintenance • Create the business roles – what should people be able to do to the content? • Create the workflow – provides the mechanism to underpin these roles • Provide the GUI for maintaining the content

12. Content maintenance - roles • Broadly, 4 different business roles: • Editor • Creates and edits content in an area, submitting it for publishing • Reviewer • Reviews submitted content, altering, rejecting or publishing it as appropriate

13. Content maintenance – roles (2) • Administrator • Able to perform functions of Editor and Reviewer • Grants roles to other users in the area for which they’re responsible • Site Manager • Able to perform functions of Administrator • Manages users • Uses other custom tools (e.g. import of CSV data)

14. Content maintenance – roles (3) • 4 Business roles distilled to 3 Zope roles: Editor, Reviewer, Administrator • "Site Manager" = Administrator role granted at site root • Groups were created and named according to a folder-dependent role • e.g. info_editors (editors of the info folder) • Users were placed in group(s) according to their role(s) in a given content area

15. Workflow • Business roles implemented by designing appropriate workflow • Approx. 40% of Plone development time spent on aspects of workflow design and implementation

16. Submit Pending Publish Retract Reject Private Published Retract Publish

17. Workflow – key requirements • All actions should be collaborative • Editors and reviewers must be able to work together on content in folders for which they're responsible • No notion of ownership – content is owned by the "business" • Creator, workflow history and edit history should still exist • Not the traditional Plone ethos! • Solution: transition guards and state permissions don't reference Owner role

18. Workflow – key requirements • Revisioning (aka simple Staging) • The ability to work on a draft copy of a published object, before committing this copy to a new publishing cycle

19. Re-edit Published object: index.html Revision: ~~revision~~ Publish Workflow - revisioning • Re-edit transition available on objects in published state • Takes a copy of the published object, and attaches it to that published object as an attribute, with a fixed id

20. Workflow - revisioning

21. Workflow – key requirements • Versioning • Ability to view and revert to previously published versions of the same document

22. Publish Versions container Published object: index.html Version 1 Version 3 Version 2 Version 4 Revert Workflow - versioning • Publish transition creates a versions container on the published item • Old copy is placed in this container each time the object is published

23. Workflow - versioning • The versions container class HistoricalVersions (SimpleItem): """Manages historical versions of an object""" security = ClassSecurityInfo() meta_type = 'Historical Content Versions' def __init__( self, id ): self.id = id self._versions = Folder('_versions') self._delete_history=() • Note use of _versions so that old versions items aren't accessible via traversal

24. Workflow - versioning

25. Versioned Submit Pending Publish Retract Revert Reject Private Published Re-edit (copy) Retract Publish

26. Workflow – key requirements • Notifications • The closest, most appropriate person needs to be notified of actions on content • Consider a folder : /info/transport/trains • Bert is a Reviewer of /info/transport • Fred is a Reviewer of /info/transport/trains • If an editor submits content for publishing in /info/transport/trains, Fred should be notified but Bert should not • Solution: skin scripts, triggered by transitions, send mail appropriately

27. Workflow – under the bonnet • Uses DCWorkflow • DCWorkflow allows scripts to be triggered on state transition • By default, scripts or external methods must be added to ZODB • Subclassed DCWorkflowDefinition to allow • Use of FSPythonScripts, accessible via skin and maintainable in CVS • Use of (unprotected) methods held within Product module (to provide versioning and revisioning) • Added custom states, permissions & guard conditions

28. Workflow – key requirements • Email management - "Great workflow, but I don't want all of this Junk mail!" • Prioritise • Not all emails require action, e.g. "An item is awaiting your review" vs. "An item has been published" • Users should be able to subscribe only to emails requiring an action • Provide a digest option • Provide an unsubscribe option • Users may go on holiday or want to hide! • Other users need to be notified to take action instead

29. Workflow – email management • Solution: two user options • Email frequency • Immediate (default) • Digest • Disabled • Email categories • Info (e.g. "An item has been published in your area") • Warning (e.g. "An item is awaiting your review") • Critical (e.g. "Your account has been deleted") • Critical mails always sent immediately • Users can just subscribe to warning emails, or receive info emails as well

31. Workflow – email management • digest_mail tool subclasses MailHost • Possible workflow actions are set on the tool by the developer, grouped into the three categories • Workflow emails are sent with: sendTriggeredMail(self, messageText, mto_userid, mfrom=None, subject=None, encode=None, action=None) • sendTriggeredMail() looks up user preferences from mto_userid, cross-checks them with the action category and then handles email appropriately • Digest mails are created on the FS, the tool sends them when sendQueuedMail() is called (e.g. by a cron job)

32. Workflow – key requirements • Deletion must be managed • A workflow concept, but normally handled outside of DCWorkflow • Should hook into standard notifications machinery • Should be possible to restore deleted content • State dependant e.g. editors should be able to delete objects in their Private state but not when published

33. Workflow – managing deletion • Idea was to mimic Windows Recycle Bin • folder_delete() skin method provides a simple hook • Drop in Product - "PortalWastebasket" • Items moved to Wastebasket on deletion • Items restored to original location on demand • State dependency handled by controlling "Delete objects" permission with workflow

34. Workflow – managing deletion • Essentially just another folder but… • Must be collaborative – anyone should be able to restore content in areas for which they have responsibility • Original security context lost when item is moved to folder, so how do you set e.g. View permissions? • Objects must be moved to private state when restored • Must be possible to delete more than one object with the same id, or the same object more than once!

35. Workflow – managing deletion • Solution: • Move object to Wastebasket, holding its original id/path in a registry • Transition object to private state en route to Wastebasket (now invisible to Anonymous) • Grant users with Authenticated role View permission on wastebasket items • When restoring the object, use workflow definition to reinstate security in correct context

36. Workflow – managing deletion

37. Workflow - summary • 3 roles: Editor, Reviewer and Administrator • 3 states: Private, Pending and Published • Collaborative workflow (i.e. no "Owner") • Revisioning and versioning system • Email notifications • Email management • Wastebasket (cf. Windows Recycle Bin) …

38. Maintaining the content • WYSIWYG editor • Why not Epoz/Kupu? • Both were immature when project started • Inconsistent tidying from MS Word • No spell checker • Inconsistent XHTML quality • No support for Mac OS X in Kupu • Previous experience of edit-On Pro

39. edit-On Pro • Commercial WYSIWYG editor, produced by RealObjects • Java applet • Created Zope Product to integrate into Plone • Biggest problem was/is requirement for recent JRE to be installed locally • Kupu looking increasingly attractive!

40. edit-On Pro in action

41. Devolving content maintenance Summary • Decide on the business roles • Create the workflow to facilitate the business roles • Provide appropriate editing tools

42. Devolving user management • Good TTP interface, but there are some problems • MembershipTool permissions and local role methods make delegation hard, in effect, requiring Zope "Manager" role • No notification mechanism for changes in users’ status • No mechanism for transferring ownership upon user deletion

43. User management - solutions • MembershipTool • Subclassed tool and redefined permissions where necessary • Overloaded local role methods to handle custom roles

44. User management – solutions (2) • Notification • Hooked user management methods to fire emails when users are added, deleted or change their group memberships • Used digest_mail tool

45. User management – solutions (3) • Ownership Transfer • When a user is deleted: • Find objects owned by user • Transfer ownership to the closest appropriate person in context of each object • Notify any users to whom ownership has been transferred • Notify the user who has been deleted

46. User management – summary • Effective delegation of user management is impossible in "out-the-box" Plone • Plone can be extended to add this functionality • CMFMember looks promising for providing a more generic solution

47. Final review • Effectively devolving content maintenance was hard, principally because of the workflow design and development • Lots of open-source Products now appearing to tackle complex workflows, including versioning and revisioning (e.g. Occams) • Devolving user management was much easier – CMFMember may make it easier still • Extensible Zope/Plone architecture allowed functionality to be added as required

48. Questions? Slides available at: http://www.ilrt.bris.ac.uk/publications/conf/vanPy2004/slides.ppt