1 / 29

CDS 2013 Accomplishments and 2014 Directions

CDS 2013 Accomplishments and 2014 Directions. John Allison (for Greg Stossmeister ), Steve Williams, Jody Williams, Charlie Martin, Mike Daniels. Collaborative Tools and Metadata. CTM Accomplishments 2013. Field Catalog 2.0 Deployed – May 2013 - Catalog Maps with A/C Cameras & Replay

walter
Télécharger la présentation

CDS 2013 Accomplishments and 2014 Directions

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CDS 2013 Accomplishments and 2014 Directions John Allison (for Greg Stossmeister), Steve Williams, Jody Williams, Charlie Martin, Mike Daniels

  2. Collaborative Tools and Metadata

  3. CTM Accomplishments 2013 • Field Catalog 2.0 Deployed – May 2013 • - Catalog Maps with A/C Cameras & Replay • - Catalog Theming with Responsive Design • - Nagios for monitoring FC servers and product streams • - Catalog Ingest implemented with metadata vetting • - New workflows for Catalog Admins • First Campaigns: MPEX • SAS • User Survey implemented following the Field Campaigns • - Guided redesign of field catalog navigation

  4. CTM Accomplishments 2013 • Field Catalog 2.1 Deployed – November 2013 • - Re-designed Navigation • - Functional Mockups User Feedback • - Reports (Db based) • - Multi-author facility status • - Catalog Ingest debugged – John & Tom • - Incorporating DOW radar data – Catalog Maps • - Dedicated MySQL server • - Remote Field Catalog Server • Field Campaigns:OWLeS • CONTRAST & HIWC (in January)

  5. CTM Accomplishments 2013 • Field Projects: • - HIWC Forecasting Exercise • - MPEX • - SAS • - DEEPWAVE Dry Run • - OWLeS • Presentations • - Erik: SSG – Browser-side Field Catalog 2.0 • - Erik: ESIP – NCAR EOL’s Experience using Twitter Bootstrap and Responsive Web Design for Field Catalog 2.0 • Storage Appliance Task Force - Erik

  6. CTM Accomplishments 2013 Data archive - Continued metadata maintenance and user support - Delivered 3.7 TB online data to 1458 unique users Codiac improvements - inline video previews - station-based static image previews - updated EOL branding for data.eol.ucar.edu Continued work with DMG on ACADIS and metadata issues

  7. CTM 2014 Plans Deploy Field Catalog 2.1.1, 2.2 - Refactoring from OWLeS, CONTRAST, HIWC - Model products in Catalog Maps - IRC Bot replacement – Chris B. - Web Mapping Service (WMS) - Flight simulation - Dry Runs/ISF products – Charlie - Rapid updates of aircraft info – Catalog Maps - Metrics - kml text Merging of Master Lists with EMDAC DB

  8. Data Management Group

  9. HIGHLIGHTS OF GENERAL DMG ACCOMPLISHMENTS - 2013 • Provided feasibility and data management planning support to 10+ future Projects • Worked on 18+ projects during 2013 in various stages of completion and support • (data collection, processing, quality assurance, archival, and dissemination) • Major Projects in 2013 included SAS/NOMADSS/SOAS, MPEX, DC3, and DYNAMO • Major Upper Air “Composite” Dataset created for DYNAMO ( ~ 24,000 Soundings) • Completed extensive HIPPO archive (EOL HIPPO team won the UCAR Scientific and • Technical Advancement Award) • - Completed the Plone to Drupal web content transition • - Completed processing of archived NWS high vertical resolution Soundings (> 750,000) • Provided Support for both EOL and NCAR Strategic Plans • Continued membership on both internal and external committees (EOL Web advisory, • EOL Internal Advisory, EOL Diversity Committee, NCAR Strategic Planning, SEA, • NCAR Allocation Review Panel, GRUAN … ) • Mentored and supported 4 UCAR Interns (DCERC, TIP, High School) and students • Attended Training Classes and Professional Conferences (presentations/posters) • Participation in UCAR/NCAR Data Citation and IT Strategic Planning Groups • Collaboration with CISL on EOL data off-site backup and future on-line data • JTECH journal article accepted for publication on DYNAMO Sounding quality control

  10. EOL DATA ARCHIVE STATUS EOL has identified over 400 Field Projects in the past 45 years that we have participated in or have data for in our archives. The Line Islands Experiment (LIE) was the first major NCAR Field Project in 1967 EOL Data Legacy Archives (Inventory as of Nov 2013) EOL Digital Archives Data housed on NCAR HPSS and EOL Servers: ~ 6000 datasets > 17 million files > 150 TB data Since 1995: 11,553 users placed > 90K orders, receiving 15 TB data online 516 users requested tapes totaling 4,492 GB

  11. EOL DATA LEGACY AND STEWARDSHIP ACTIVITIES - 2013 • Continued Inventory of EOL Archives (Tape, HPSS, Hardcopy, other misc. media ….) • VHS Aircraft Tape digitization nearing completion ( ~ 900 tapes digitized over 4 years) • Development of Legacy Project web pages (provide access to descriptions, data, • publications, documents, photography ….. ) • Migration of Project web pages to Drupal from older html format • Creating EMDAC metadata for existing digital data as located • Collaboration with NCAR Library/Archives on scanning documentation • Performing EOL-wide Instrumentation and Data Audit • Revising EOL Data Policy in coordination with NSF • Evaluation and testing of additional media migration • Article on EOL Legacy/Stewardship Data Activities (UCAR AtmosNews, Oct 2013) • Project relies on strong involvement of students and DCERC interns • 20 Countries • 40 research ships • 12 research aircraft • > 1000 GATE papers

  12. HIGHLIGHTS OF DMG ARCTIC ACCOMPLISHMENTS - 2013 • Participated in ACADIS Project (Project Management, Working SubGroups, Special • Requirements Data, ADAC, NSF Panel Review) • Cleaning up of Arctic metadata (2194 Data Sets) and developing data citations • Providing review of ACADIS organization and management structure • Completed USCGC Icebreaker Polar Star Mapserver/Ice Observations Project • Completed PacMARSMapserver/Archive (includes ~ 150 data layers and village profiles) • Provided Planning Support to the Distributed Biological Observatory (DBO) Project • Participated in bi-annual Arctic Observing Summit (Vancouver) • Attended Project meetings and Professional Conferences (presentations/posters)

  13. DMG 2014 PLANS • DOI generation for all datasets • Investigate the use of another DCERC student • Convert the Master List tool to the EMDAC database • Expansion of arctic metadata entry tool for EOL-wide holdings • Data Legacy Report for EOL Management • Data Management support for HIWC, CONTRAST and DEEPWAVE • Pursue additional funding for Polar projects

  14. Systems Infrastructure Group

  15. 2013 Accomplishments:Systems Infrastructure • Security – no security incidents; EOL led UCAR’s diminishing phishing attempts. Webmail was put back outside the perimeter. • Reliability: no major server outages except the SCALE issues • Drupal web server • Hyper-V cluster – virtualized many aging servers (eol-rt-data, eol-epicac, wiki, weather,…)

  16. SIG Field Work ops-in-a-box, MPEX, SAS (SOAS, NOMADDS) working with CTM for OWLES (setting up a new server, ftp areas)

  17. Systems Infrastructure • Storage issues – SCALE issues, worked a lot on debugging, working with Contracts, the reseller, and VP of SCALE. Tier 1 storage task force – purchase NetApp • Windows XP to Windows 7, RHEL to Scientific Linux • New Servers: eol-doppel, eol-doppel2, barolo, aeolus (CTM), farskol (CTM), wombat (CTM), eol-apar (RSF), vdras (RSF), NetApp • Decommissioned Servers: bora, m5, malbec, eve, SCALE • Committees: CSAC, NCAB, ACCISS, CFM, SINAP, EWAG, IAC, ITC, ITC Summit, Strategic plan work for UCAR/NCAR • Special Projects – RSF and Front, Earthcube real-time data workshop support, RAF remodel to include on-site Ops Center.

  18. SIG jira requests

  19. Systems Infrastructure – Future Plans • New “Fire Drill” procedures • NetApp integration • SINAP Survey Round II • Google Apps • Equipment move to Mesa Lab computer room • Field Programs: HIWC, CONTRAST, DEEPWAVE

  20. Software Systems Group

  21. 2013 Accomplishments: Software Systems Field Programs

  22. 2013 Accomplishments: Software Systems Hardware Systems • HIAPER Cloud Radar (HCR) • SPOL • 3V-CPI • GV AVAPS • 449 Profiler • Microwave Temperature Profiler (MTP) • GV Satcom • CSWR Doppler on Wheels (DOW) • Laser Air Motion Systems (LAMS) • GV & C-130 Data Systems • ISFS

  23. 2013 Accomplishments: Software Systems Software Package Developments

  24. 2013 Accomplishments: Software Systems Other Activities • EOL Development Conference • EOL Software Summit • Summer Intern: Mike Patterson (HTML5) • SEA Conference • EOL Software Center (Drupal) • EarthCube Real-Time Workshop • AMS Radar Conference • RIC security audit • A-10 “Ice Breaker”

  25. Looking Forward to 2014: Software Systems • IFRACS • CONTRAST • DEEPWAVE • ISF: The “Year of the Sounding” • APAR • SSG Student Assistant • Field Catalog cross-dev w/ CTM

  26. Final Remarks

  27. Overall CDS Funding (FY06-FY14) NOTE: For FY14, NSF Spec and Non-NSF amounts are projections, only.

  28. Overall CDS Spending (FY06-FY14) NOTE: For FY14, NSF Spec and Non-NSF/Other amounts are projections, only.

  29. CDS Future Directions in 2014 • Pursue outside funding(!): EarthCube, Polar, Data Legacy, Software Infrastructure, other EOL-related • Creation of an Open Source software environment for our community • Stronger lab-wide integration of software developments and data services activities • Plans for longer-term enhancements of software and data services • Emphasis on Project Management

More Related