CPAN6

1. CPAN6 Mark Overmeer & Sam Vilain YAPC::EU 2006, Aug 31 Birmingham UK

2. “CPAN is the best thing that ever happened to Perl” CPAN is..

3. “CPAN is the best thing that ever happened to Perl” CPAN is..

4. “CPAN is the best thing that ever happened to Perl5” CPAN is..

5. CPAN is... • Just over 10 years youngyounger than • Perl5 • WWW • UNIX

6. CPAN is... • Just over 10 years youngyounger than • Perl5 • WWW • UNIX • Just over 10 years olddeveloped before break-through of • XML/SOAP/WSDL • CVS/SVN/SVK/GIT • spam, viruses • unicode

7. ...concerns... • 10k modules, a lot is bad quality orpoorly maintained • 3500 authors, who is who? • who is making the rules of CPAN?

8. ...concerns... • 10k modules, a lot is bad quality orpoorly maintained • 3500 authors, who is who? • who is making the rules of CPAN? GBD#2What will happen when infected code is detected on CPAN? Will Perl lose itsbest thing?

9. ...future... • Where will we store • Perl6 modules • pir, pbc, pil, pod,… modulesname-space battles! • parTcl? Brainf*ck?

10. ...future... • Where will we store • Perl6 modules • pir, pbc, pil, pod,… modulesname-space battles! • parTcl? Brainf*ck? • Can we improve • security = trust • professionalism = regulation/deregulation • deployment = less content independent • platform integration = YaST, apt, ... • capacity = name-space management

11. Releases ≠ Revisions • Revisions • a logical sequence of development steps of a file • each step may solve some bugs • each step extends the code and documentation • steps are created by a single person • Releases • are created to be distributed • a defined `frozen' state of the software(some combination of file versions) • published by someone who is responsible

12. CPAN ≠ YA VCS • CPAN is a collection of releases • distributing information • name-spaces and trust • searching in the collection • Version Control Systems manage (related) (development) revisions of files • collecting pieces of information • traceable changes • tracing and solving bugs

13. Needs • improved security: alternatives to Pause-ID

14. Needs • improved security: alternatives to Pause-ID • professional approach: described rules, especially how to resolve disputes/problems.

15. Needs • improved security: alternatives to Pause-ID • professional approach: described rules, especially how to resolve disputes/problems. • extended deployment: support for other programming languages (and other data)

16. Needs • improved security: alternatives to Pause-ID • professional approach: described rules, especially how to resolve disputes/problems. • extended deployment: support for other programming languages (and other data) • platform integration: environment friendly

17. Needs • improved security: alternatives to Pause-ID • professional approach: described rules, especially how to resolve disputes/problems. • extended deployment: support for other programming languages (and other data) • platform integration: environment friendly • capacity growth: more name-spaces

18. NO QUICK FIX! but a good alternative

19. Split CPAN • CPAN6: • idea of having a collection of releases,combined into archives • uploading releases • downloading releases • distributing the archives like mirrors • combining archives like ftp sites • filtering archives sub-setsCPAN6 is the DISTRIBUTION

20. Split CPAN • CPAN6 is the distribution • Pause6: • one archiveis one name-spaceis set of releases (with some relation) • a group of releases relate to a project(name) • projects are allocated to authors • authors have an defined identity • an archive has well-defined rulesPause6 is ADMINISTRATION

21. Split CPAN • CPAN6 is the distribution (network) • Pause6 is the administration (archives) • CPAN6.pm • installs Perl5, Perl6 etc distributions • blends into the platform • collects meta-data to publish releases • programming language specific componentCPAN6.pm for INSTALLATION

22. Split CPAN • CPAN6 is the distribution • Pause6 is the administration • CPAN6.pm installs releases locally Of course perl -MCPAN6 'install Test::More' will work!

23. CPAN

24. CPAN6 concept

25. CPAN6 structure • One system can host multiple archives, as commissioner and/or deployer; combined in one service (daemon) • Multiple repositories are combined into a store • Stores (data) are not protected, and can therefore be kept on other systems than the related daemons.

26. Pause6 Implementation

27. Pause6 implementation • Allocates the name-space • project names are taken by a few authors • entrusting board members • limits to project names and version labels

28. Pause6 implementation • Allocates the name-space • project names are taken by a few authors • entrusting board members • limits to project names and version labels • User identities • PKI based signing • publisher, authors, board, processes proof right to make changes with signatures. • keeps track on trust

29. Pause6 projects • Release has project-name, version and state • Each state change requires signatures • The release states are • uploading, initiated by publisher • published, distributable to deployers • embargo, download blocked • released, by the authors • deprecated, by the authors • expired, by the authors • rejected, by the board/authors • installed, by the end-user

30. Pause6 projects • Projects are sets of releases • Each release is a set of files • The archive collects per release • checksums of each file (SHA-256 or better) • location of each file in the store • state of the release • user provided meta-data, like description for searching • transport traces etc

31. Pause6 projects • Any set of files can be a release of a project, not only Perl5 tar-ball distributions. • Other “project” types are:

32. Pause6 projects • Any set of files can be a release of a project, not only Perl5 tar-ball distributions. • Other “project” types are: • constitution • user and daemon identities • archive references • license description • ...

33. Pause6 projects • Any set of files can be a release of a project, not only Perl5 tar-ball distributions. • Other “project” types are: • constitution • user and daemon identities • archive references • license description, etc • therefore • everything follows same release protocol • one transport implementation covers all aspects

34. Release trails

35. Release trails

36. CPAN6 archives

37. Hierarchy • Scribes can be configured to create archive hierarchies, for example: • global cpan-perl5 • business level sub-set • department sub-set • system local • personal installed • personal development

38. CPAN6 service daemon • Manage a set of archives • as commissioner • as deployer • trigger scribe processes • local or remote stores • Configuration • an archive with archive-references • archive board are the system administrators

39. CPAN6 Scribes • Copy published releases from commissioner to deployers • Copy releases between archives, allowing • selection rules, like “only last”, “license GPL” • auditing, content checks • trust calculation, signature checks • transport protocol conversions • Used to build virtual hierarchies of archives

40. Example: project perl -MCPAN6 'install DBD::Oracle' is something like LOC=$(pause6 project get http://cpan.org?project=DBD::Oracle) cd $LOC tar xzf *.tar.gz perl Makefile.PL make test install

41. Example: project pause6 init pause6 archive use http://cpan6.net/perl5 pause6 archive use http://pause-id.cpan.org as pauseid pause6 identity default pauseid/MARKOV pause6 archive create ~/perl/devel as myperl pause6 project create perl5/DBD::Oracle mkdir ~/perl/dbd-oracle; cd $_ vi <anything> make dist pause6 release create DBD::Oracle 0.01 as ora pause6 release add ora DBD-Oracle-0.01.tar.gz pause6 release edit ora pause6 release publish ora to perl5 # or myperl DIR=$(pause6 release get perl5/DBD::Oracle) Syntax subject to change

42. Example: service CPANTS pause6 archive create http://archives.cpan6.net/cpants # Triggered daily by cron pause6 archive query perl5 -c name,version release=last \ | while read NAME VERSION do LOC=$(pause6 release get “pause6/$NAME;$VERSION”) Q=$(calculate_kwalitee $LOC) pause6 project create cpants/$NAME 2>/dev/null pause6 release create $NAME $VERSION as new pause6 release add new $Q pause6 release publish new to cpants done

43. Status • CPAN6/Pause6 global design document • CPAN6 design and implementation doc • Pause6 design and implementation doc • platform integrated install tools

44. Needed... • ... discussion and attention • extensions to the design, use-cases • contacts with other potential user groups • signing, licensing, trust algorithms • funding to create base implementation

45. Plans... • implement local archives (daemon-less) • implement network archives (daemon) • implement scribes (cron tasks)

46. Plans... • implement local archives (daemon-less) • implement network archives (daemon) • implement scribes (cron tasks) • create CPAN mirror in CPAN6/Pause6 • create CPAN6.pm, based on CPAN.pm

47. Plans... • implement local archives (daemon-less) • implement network archives (daemon) • implement scribes (cron tasks) • create CPAN mirror in CPAN6/Pause6 • create CPAN6.pm, based on CPAN.pm • start pir, pasm, partcl,... archives • start javascript, python, php, ... archives • create ftp-server wrappers (get public)