1 / 58

Sakai 2.1 Update

Sakai 2.1 Update. Charles Severance December 7, 2005. Sakai Babies. Rachel/Stanford (1.0 pre-alpha) Quingru/Stanford (1.0), Rashmi/ Tanush /Foothill (1.5) Mallika/ Saahil /Foothill (1.5.1) Joanne/UM (2.0) Seth/Zoe/Columbia (2.0.1) Rob Lowden/TBD/IU (2.1) Anthony Whyte/TBD/Sakai (2.1.1)

lara
Télécharger la présentation

Sakai 2.1 Update

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Sakai 2.1 Update Charles Severance December 7, 2005

  2. Sakai Babies • Rachel/Stanford (1.0 pre-alpha) • Quingru/Stanford (1.0), • Rashmi/Tanush/Foothill (1.5) • Mallika/Saahil/Foothill (1.5.1) • Joanne/UM (2.0) • Seth/Zoe/Columbia (2.0.1) • Rob Lowden/TBD/IU (2.1) • Anthony Whyte/TBD/Sakai (2.1.1) • John Ellis/TBD/rSmart (2.1.2)

  3. Rob Lowden

  4. Rob Lowden

  5. Carol Dippel We love you Carol, Oh yes we do We love you Carol, And we’ll be true If you don’t do QA, We’re blue! Oh Carol, we love you. We thank you Carol, Oh yes we do We thank you Carol, And we’ll be true Your work on Sakai, Has been cool! Oh Carol, we thank you.

  6. August 2005

  7. August 11, 2005

  8. August 12, 2005, 12:46 PM From: Charles Hedrick <hedrick@rutgers.edu> To: Bill Crosbie <bcrosbie@rci.rutgers.edu> Cc: Glenn Golden <ggolden@umich.edu>, ZE Joseph <hardin@umich.edu>, Charles Severance <csev@umich.edu>, Jon Andersen <janderse@umich.edu>, lance D Speelmon <lance@indiana.edu>, Andrew J Poland <ajpoland@iupui.edu>, John Leasia <jleasia@umich.edu>, Robert J Lowden <rlowden@iupui.edu>, David Haines <dlhaines@umich.edu>, "Bradley C. Wheeler" <bwheeler@indiana.edu> Date: Aug 12, 2005 12:46 PM Subject: Re: Urgent: Memory Leak in Sakai I agree with Bill. Some comments. 1) I'll tell you what we're seeing, but it's possible not all sites are seeing the same thing. 2) Your first priority needs to be setting up monitoring, so you can see what you are actually running out of. Our current theory is that the problem is in permanent space. This would never be visible from normal log files. The only way we see what's happening is because we're running Sun's memory monitoring tools, particularly VisualGC. 3) My current theory is that our problem is with dynamically loaded classes. When the system starts, we have about 10000 classes loaded, with permanent space taking about 34 MB. After 14 or so hours today, we have 15490 class loads, and permanent memory use of 60.8 MB. The default is 64 MB maximum, so we think it's likely that by the end of the day we will have exhausted the default permanent memory, unless a GC gets lots of space back. Yesterday it did not. It unloaded something like a dozen classes and reclaimed very little memory. I don't know what's causing the continuing loading of new classes. I don't know whether that is intended or not, and if it's intended, when the classes are supposed to be released. I've now get the maximum in permanent expanded to 128MB. That will let us see whether space eventually stabilizes, and if so at what level. My theory is that this problem has been intermittent because usage quickly expands to 60 out of 64. That's so close to the limit that depending upon details of the load sometimes it goes over and sometimes it doesn't. Whether there's an actual bug (i.e. memory usage expands for ever) I don't feel I can tell yet. I think by sometime next week the pattern will be clear enough. In the short run, we will try to add enough permanent memory to avoid the problem. If memory use grows forever, we'll try to add enough that we can survive by doing a nightly restart. Note on setting up VisualGC: Most of the tools come with Java 5.0. Visual GC itself is a separate download. You will need Java 5.0 installed, because the tools require it, but it doesn't need to be (and at Rutgers, isn't) your default version of Java. Setting it up is quite easy. I've used VisualGC running locally (displaying remotely using X11), and also the remote version (which needs a copy of jstatd running on the server). The commands for remote use are not entirely obvious from the man page, so here they are: On the server:    ./jstatd -p 25209 -J-Djava.security.policy=<FILE> where 25209 is a random TCP port number and <FILE> is a file containing a Java policy (which is given in the man page) On the client:   bin/visualgc rmi://26429@sakai2.rutgers.edu:25209 where 26429 is the PID of the JVM on the server, and 25209 is the random TCP port number In this case the server is Solaris 9 SPARC and the client OS X Tiger, but that shouldn't matter. The visualgc shell script requires significant repair on OS X, though the program runs fine.

  9. Sakai 2.1 - Dates • 2.0 - 06/15/2005 • 2.1 Integration Week - 10/24/2005 • 2.1.0_001 - 10/28/2005 • Release Engineering and QA begins • 2.1.0_004 - 11/17/2005 • Maintenance branch established • 2.1 Release - 12/01/05

  10. Code Trends Sakai 2.1.0 is just under one million lines of code

  11. Carol Dippel Glenn Golden Clay Fenalson Peter Knoop David Haines Lydia Li Josh Holzman Ray Davis Margaret Petit Crystal Hancock Zhen Qian Charles Severance Lance Speelmon Rob Lowden Mike Elledge Oliver Heyer Ed Smiley Daisy Flemming Gonzalo Silverio Jim Eng Seth Therauilt Stephen Marquad Sakai 2.1 Release Mgt

  12. Sakai 2.1 - Numbers • JIRA • 1752 Issues Updated • 684 Bugs Resolved • 233 Tasks Completed • SVN • Sakai 2.0 - 550 svn commits • Sakai 2.1 - 3793 svn commits (through r4344)

  13. Sakai 2.1: Process in Transition • Sakai 1.0, 1.5, and 2.0 were managed “Sakai Project Style” • Central control • Controlled communication • Kind of like a company • Sakai 2.1 needed to be managed “Sakai Foundation Style” to the extent possible • Expand commit lists (started in March 2005) • Manage the 2.1 release in the open • Kind of like a community

  14. Process Changes for 2.1 • My Goal: Just be more open on all fronts • Send regular status reports including good, bad, and the ugly straight to the dev list • Publish all tags, and target dates openly • Discuss bugs, JIRA, problems, and progress openly • When plans had to change - just admit and change the plans in public • When tradeoffs had to be made - discuss rationale in public • No “internal” mailing list - no internal announcements (i.e. tags, release candidates, etc) • Break down communication barriers between QA and Development teams • Invite more folks to the release management calls

  15. Contributors Sakai Committers ajpoland@iupui.edu andersjb@iupui.edu andrew@caret.cam.ac.uk angell@pdx.edu arwhyte@umich.edu benbr@mit.edu bfhawkins@mac.com bkirschn@umich.edu ccount@mit.edu chmaurer@iupui.edu chris.coppola@rsmart.com clayf@bu.edu csev@umich.edu cwen@iupui.edu daisyf@stanford.edu dlhaines@umich.edu esmiley@stanford.edu ggolden@umich.edu gql@mit.edu gsilver@umich.edu hedrick@rutgers.edu hquinn@stanford.edu htripath@indiana.edu ian@caret.cam.ac.uk janderse@umich.edu jeffkahn@mit.edu jholtzman@berkeley.edu jimeng@umich.edu jlannan@iupui.edu jleasia@umich.edu john.ellis@rsmart.com kevin.carpenter@rsmart.com lance@indiana.edu lydial@stanford.edu markjnorton@earthlink.net michael.desimone@rsmart.com natjohns@indiana.edu oliver@media.berkeley.edu ray@media.berkeley.edu rdiblasi@iupui.edu rshastri@iupui.edu rwellis@umich.edu s-githens@northwestern.edu slt@columbia.edu suiyy@umich.edu vgoenka@sungardsct.com zach.thomas@txstate.edu zqian@umich.edu zqingru@stanford.edu aaronz@vt.edu admin@makash.org.il alceu@unisinos.br alex@asic.udl.es andrew.petro@yale.edu anthony.atkins@vt.edu caseyd1@stanford.edu Carol Manchó David Barroso duffy@email.arizona.edu fribeiro@ufp.pt gonzalez.cynthia@itesm.mx Il-hwan Kim jim.doherty@gmail.com jmingsun@unisa.ac.za Jose Garcia kajita@nagoya-u.jp kasper@pagels.dk Kun Lv machrist@cs.indiana.edu mj_tsai@yahoo.com salmon@salmon.sk sdonovan@unisa.ac.za Tianhua Ding udcsweb@unisa.ac.za vdberj@unisa.ac.za yves.epelboin@upmc.fr

  16. Becoming A Committer • Figure out some chunk of Sakai • Become a contributor working through a current committer in that area • At some point you will be contributing so much that the committers will • Realize that you are a great asset to the group and quite sharp and • get tired of putting in your changes for you and make you a committer

  17. Framework - Glenn Golden Resources Tool - Jim Eng OSP - John Ellis Samigo - Lydia Li / Marc Brierley Gradebook - Ray Davis I18n - Beth Kirshner Web Services - Steven Githens Providers - Charles Severance Site Info - Zhen Qian Schedule - Joanne Sui WSRP - Vishal Goenka Legacy Tools - John Leasia Sections Tool - Josh Holzman Wiki - Ian Boston Roster - Lance Speelmon Syllabus - Chen Wen Profile - Lance Speelmon Admin Tools - Glenn Golden Skin/CSS - Gonzalo Silverio Portal Integration - Charles Severance / Andrew Petro TwinPeaks - Jeff Kahn lancs.uk - Adrian Fish Sakai Project Leaders

  18. How 2.0 Branched 2.1 2.1 Indiana Features 2.0 2.0.x UM 2.0.1 Jun 05 Aug 05 Oct 05 Dec 05

  19. How 2.0 Branched 2.1 Danger -> Indiana 2.0 2.0.x 2.1 UM 2.0.1 Jun 05 Aug 05 Oct 05 Dec 05

  20. Everyone Else in 2.0 2.1 Danger -> Indiana 2.0 2.0.x 2.1 UM 2.0.1 Jun 05 Aug 05 Oct 05 Dec 05

  21. 2.1 Branch Plan QA 2.2 QA 2.2_nnn Danger -> 2.1 QA 2.1.x QA 2.1.2 2.2 2.1.1 Dec 05

  22. User Visible Features in 2.1 • Improved Resources Tool with Meta-Objects • Section Tool • Group Support in Site Info • Announcements Tool Section-Aware • Samigo Section-Aware • Gradebook Section-Aware • Course Site Template Out of the Box • Localizations - completed and partial

  23. New Resources Tool

  24. Meta Objects

  25. Section Tool

  26. Group Support

  27. Gradebook Section-Aware

  28. Course Site Template

  29. Localization

  30. Localization Status • Substantially complete • Chinese (China) - Tianhua Ding, Kun Lv • Korean - Il-hwan Kim • Dutch - Jim Doherty • Partial • Japanese - Tatsuki Sugura / Shoji Kajita

  31. Danish - Kasper Pagels Hebrew - Dov Winer Brazilian Portugese - Alceu Fernandes Filho Portugese - Feliz Gouveia Slovokian - Michal Mosovic Catalan - Alex Balleste Chinese (Taiwan) French - Yves Epelboin Spanish - Cynthia Gonzalez Localiztions in Progress

  32. Sakai 2.1 Framework • AuthzGroups - Groups within Sites • SecurityAdvisor - Multi-context Authz • Provider Cleanup • CourseProvider getTab • UserDirectory providerOrder, alwaysCreate • Rutgers MySql Performance Changes • Entity Producer - Refactor

  33. Roster: CS1 dogle zhen csev lance marc In TA TA St St In: rwd TA: rw St: r Pre-AuthZGroup Structures Course/Realm Providers Site: CS1 ID: F05 CS1 A dogle csev lance In TA St ID: F05 CS1 B dogle zhen marc In TA St

  34. Roster: CS1 dogle zhen csev lance marc In TA TA St St In: rwd TA: r St: r Site: CS1 AuthZGroups in 2.1 Roster: CS1 A dogle zhen marc In TA St ID: F05 CS1 A dogle csev lance In TA St In: rwd TA: rwd St: r Roster: CS1 B ID: F05 CS1 B dogle csev lance In TA St dogle zhen marc In TA St In: rwd TA: rwd St: r

  35. Roster: CS1 dogle zhen csev lance marc In TA TA St St In: rwd TA: r St: r Site: CS1 SectionTool Roster: CS1 A dogle zhen marc In TA St In: rwd TA: rwd St: r Section Tool Roster: CS1 B dogle csev lance In TA St In: rwd TA: rwd St: r ID: F05 CS1

  36. Protecting a Resource Resources Service Access Servlet /content/eecs280 content.read

  37. Attaching a Resource Across Sites Resources Service Access Servlet Schedule Service Access is allowed for eecs280 content.read users but not for sakai-dev users so this fails in the sakai-dev schedule attachment case. /site/sakai-dev schedule.read /content/eecs280 content.read 01/02/05 blah

  38. Attaching a Resource (compromise) Resources Service Access Servlet Schedule Service /attachments/ FF987DEF /site/sakai-dev schedule.read /content/eecs280 content.read Copy 01/02/05 blah

  39. Using SecurityAdvisor Resources Service Access Servlet Schedule Service /site/sakai-dev schedule.read /content/eecs280 content.read 01/02/05 blah When schedule tool sends the user to Access, Schedule adds a SecurityAdvisor. This allows the bypass of the normal security protecting the content.

  40. EntityProducer • Re-factored the 2.0 Resources Interface and made much more rich • Allows cleaner “drop-in” of new capabilities by distributing brittle code in central locations into each of the Components • Function creation and management (melete.author) • AuthZGroup resolution (Reference.java) • Ends up adding constraints to Components which produce “entities” • Going forward will enable other capabilities like improved import/export, hierarchy, search, fine-grained entity reuse, etc..

  41. Provisional Tools • Came from the “bundle” debate we have had throughout the project • We needed something between “yes” and “no” • Re-thought the inclusion/exclusion decision from a “community or peers” perspective rather than top-down central approach • Came up with a set of criteria • Provisional tools are “almost in” Sakai • Part of the release • Available and tested during QA period • Hidden in the release with a simple way to unhide

  42. Provisional Criteria • Community Support • Must have commit list and be in SVN • Must run in production at >=2 sites • Must have proper license • Must be willing to answer questions • Need test plans and specifications • Needs to be tracked in JIRA

  43. Provisional Criteria (cont) • Technical • Support HSQL, MySql, Oracle • Use AutoDDL properly • Use sakai.properties • Do AUTHZ functions like the rest of Sakai • No patches to other elements • Must cluster • Use proper versions of Spring, Hibernate, etc.

  44. Provisional Criteria (cont.) • Interaction and Visual Design • Inherit skins properly • Look “like” the rest of Sakai tools (fit in) • Follow interaction designs in style guide • Use JSF UI Components (if applicable) • Include help - properly added to the Sakai Help system

  45. Provisional Tools (cont) • Desirable elements (required for full inclusion) • Internationalized • Accessible (including a review) • Separation of persistence and business logic into a properly factored Sakai Component • Event generation as appropriate • Fit logically scope-wise with the rest of Sakai • Process • Notify community 30 days before code freeze to allow comment • If problems arise during QA that are not easily fixed, provisional tools are dropped

  46. *Sample* Attribute Matrix

  47. Provisional Tools For 2.1 • SakaiScript - Web Services • Steven Githens - Lead • WSRP - Web Services For Remote Portals • Vishal Goenka - Lead • Become User Tool • Zach Thomas - Lead • Roster Tool • Lance Speelmon - Lead • Wiki - Based on Radeox • Ian Boston - Lead • TwinPeaks - DR OSID • Jeff Kahn - Lead

  48. SU Tool

  49. TwinPeaks

  50. Wiki

More Related