NGS Status

1. http://www.ngs.ac.uk http://www.grid-support.ac.uk NGS Status Stephen Pickles Technical Director, GOSC (with thanks to Andy Richards, NGS Co-ordinator) CERN, 17/1/2005

2. Outline • Users • registrations, usage, helpdesk queries • Services & Middleware • what we offer today • managing change • Expansion • “joining” the NGS

3. Users

4. NGS core nodes data nodes at RAL and Manchester compute nodes at Oxford and Leeds free at point of use apply through NGS web site accept terms and conditions of use light-weight peer review 1-2 weeks to do: project or VO-based application and registration all access is through digital X.509 certificates from UK e-Science CA or recognized peer National HPC services HPCx CSAR Must apply separately to research councils Digital certificate and Conventional (username/ password) access supported Gaining Access

5. User registrations so far

6. Helpdesk http://www.grid-support.ac.uk support@grid-support.ac.uk UKGSC: Queue Totals For Month Prior To (10:11 AM, 26/11/2004) • Notes • Certification still dominant query type • General contains GGUS related queries from EGEE/GGUS helpdesk • (A separate project for this has now been created) • LCG savannah generated queries are increasing through this helpdesk

7. NGS Core Services Globus, SRB, Oracle, OGSA-DAI, and others

8. NGS Core Services: Globus • Globus Toolkit version 2 • GT 2.4.3 from VDT 1.2 • Job submission (GRAM) • File transfer (GridFTP) • Shell (GSI-SSH) • Information Services (MDS/GIIS/GRIS) • Information providers from GLUE schema • Use BDII implementation of MDS2 (as does EGEE)

9. NGS Core Services: SRB • Storage Resource Broker from SDSC • Location transparent access to storage • Metadata catalog • Replica management • Clients on compute nodes • Servers on data nodes • Issues/to do: • licensing • MCAT replication and failover

10. NGS Core Services: Oracle • Oracle 9i database • Only on data nodes • Populated by users/data providers • Infrastructure maintained by NGS database administrators • Accessed directly or via OGSA-DAI

11. NGS Services: OGSA-DAI • Developed by UK e-Science projects OGSA-DAI and DAIT • OGSA-DQP (Distributed Query Processing) • Experimental service based on OGSI/GT3 on Manchester data node only • containment: 2 cluster nodes reserved for development and production • will consider WS-I and WSRF flavours when in final release • Uses Oracle underneath • User-provided Grid Data Services validated on test system, then transferred to production during scheduled maintenance • Early users from e-Social Science (ConvertGrid) • Established liaisons with OGSA-DAI team

12. NGS Core Services: other Operated by GOSC for NGS and UK e-Science programme In production: • Certificate Authority • Information Services (MDS/GIIS) • MyProxy server • Integration tests and database • Cluster monitoring • LCG-VO In testing: • VOMS • EDG Resource Broker In development: • Accounting • using GGF Usage Record standard for interchange, software from MCS project and TeraGrid • Portal

13. UK Campus and other Grids EGEE… ETF NGS Other software sources Software with proven capability & realistic deployment experience Prototypes & specifications Operations OMII ‘Gold’ services Feedback & future requirements Deployment/testing/advice Managing middleware evolution • Important to coordinate and integrate this with deployment and operations work in EGEE, LCG and similar projects. • Focus on deployment and operations, NOT development.

14. Maintaining Compatibility • Operating a production grid means valuing robustness and reliability over fashion. • NGS cares about: • alignment/compatibility with leading international Grid efforts • special requirements of UK e-Science community • easy migration/upgrade paths • proven robustness/reliability • based on standards or standards-track specifications • NGS cannot support everything • Everyone wants service-oriented grids • but still settling out: WS-I, WS-I+, OGSI, WSRF, GT3, GT4, gLite • Caution over OGSI/WSRF has led to wide convergence on GT2 for production grids and hence some inter-Grid compatibility • but there are potentially divergent forces at work • Significant changes to NGS Minimum Software Stack will require approval by NGS Management Board on conservative time scales

15. Expansion Adding new resource providers

16. Expansion Resource providers join the NGS by • Defining level of service commitments through SLDs • Adopting NGS acceptable use and security policies • Run compatible middleware • as defined by NGS Minimum Software Stack • and verified by compliance test suite • Support monitoring and accounting Two levels of membership • Affiliation • a.k.a. connect to NGS • Partnership

17. Affiliation Affiliates commit to: • running NGS-compatible middleware • as defined in NGS Minimum Software Stack • this means users of affiliate’s resources can access these using same client tools they use to access NGS • a well-defined level of service and problem referral mechanisms • SLD approved by NGS Management Board and published on NGS web-site • providing technical, administrative, and security (CERT) contacts • providing an account and mapping for daily compliance tests (GITS++) • accepting UK e-Science certificates • maintaining baseline of logs to assist problem resolution

18. Partnership Partners commit to same as affiliates, plus: • making “significant” resources available to NGS users • creation of accounts/mappings • in future, VO support, pool accounts, etc • recognise additional CAs with which UK e-Science programme has reciprocal agreements • publish/provide additional information to support resource discovery, brokering • ability to compile code for computational resources Bristol and Cardiff going through certification process now. • supported by “buddies” and NGS-Rollout list • useful feedback on viability of NGS Minimum Software Stack

19. Benefits Affiliation: • NGS brand • certified NGS-compatible • better integrated support for local users who also access NGS facilities • assistance/advice in maintaining NGS-compatibility over time Partnership: • higher brand quality • representation on NGS management board • either direct, or through regional or functional consortia NGS brand must be valuable to make this work.

20. Recent Developments and Summary

21. Recent Developments • NGS newsletter • OMII helpdesk • Resource Broker (LCG) available for early adopters • TeraGrid • UK certificates accepted on TeraGrid • Looking at INCA monitoring with Jenny Schopf • Trial of accounting software from MCS project • using GGF Usage Record draft standard (as EGEE) • Meeting with Open Science Grid’s iGOC • Collecting user exemplars • Agreed form of acknowledgement • Workshop towards an NGS user induction course. Need: • Easier access (e.g. user interface machines and portals) • T-infrastructure

22. Summary TODAY • 4 core nodes operational • 150 Users registered (50 since 1 September ’04) • Grid enabled – Globus v2 (VDT distribution v1.2) at present • BDII information service (GLUE + MDS Schemas) • Data Services – SRB, Oracle, OGSA-DAI • Growing base of user applications • MyProxy and CA services • VO Management Software – LCG-VO • User support: Helpdesk Next… • Other Middleware [gLite/OMII etc…] • NGS Portal • Resource Broker • SRB production service • Accounting • Continued expansion