1 / 38

澳洲工作介绍 2013.5.20-2013.11.10

澳洲工作介绍 2013.5.20-2013.11.10. China-VO and Astroinformatics 2013 2013.11.13-17 雅安. 报告人：张海龙 单 位：中国科学院新疆天文台 二零一三年十一月. Outline: ICRAR,MRO,MWA,IVEC NGAS GAVO Our DC. ICRAR

midori
Télécharger la présentation

澳洲工作介绍 2013.5.20-2013.11.10

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 澳洲工作介绍2013.5.20-2013.11.10 China-VO and Astroinformatics 2013 2013.11.13-17 雅安 • 报告人：张海龙 • 单 位：中国科学院新疆天文台 • 二零一三年十一月

  2. Outline: ICRAR,MRO,MWA,IVEC NGAS GAVO Our DC

  3. ICRAR The International Centre for Radio Astronomy Research is a collaborative centre based in Perth, Western Australia.  Already achieving research excellence in astronomical science and engineering, ICRAR has grown steadily since its launch in September 2009. MRO The Murchison Radio-astronomy Observatory (MRO) is the home of the Australian SKA Pathfinder (ASKAP) and the Murchison Widefield Array (MWA). It will shortly also be the home of a significant part of the Square Kilometre Array (SKA) itself (South Africa will also host part of the SKA). It is located in Western Australia’s Mid West, approximately 300km north east of the coastal city of Geraldton, within the Shire of Murchison. This outback region is roughly the size of the Netherlands with a resident population of around 140 people and a population density of around one person for every 350 sq.km. The centre of the 70 km radius circle defining the MRO is at S26o42’15”, E 116o39’32”. The MRO and the surrounding landscape are protected through federal legislation from any activities that might interfere with the operation of radio telescopes within 150km of the core site. The establishment of this ‘Radio Quiet Zone’ (RQZ) combined with a very low and sparse local population means the MRO is the perfect place for constructing and operating the most sensitive radio telescopes the world has ever seen.

  4. MWA The MWA consists of 2048 dual-polarization dipole antennas optimized for the 80-300 MHz frequency range, arranged as 128 "tiles", each a 4x4 array of dipoles. The array has no moving parts, and all telescope functions including pointing are performed by electronic manipulation of dipole signals, each of which contains information from nearly four steradians of sky centered on the zenith. Each tile performs an analog beamforming operation, narrowing the field of view to a fully steerable 25 degrees at 150 MHz. The majority of the tiles (112) are scattered across a roughly 1.5 km core region, forming an array with very high imaging quality, and a field of view of several hundred square degrees at a resolution of several arcminutes. The remaining 16 tiles are placed at locations outside the core, yielding baseline distances of about 3 km to allow higher angular resolution for solar burst measurements.

  5. ASKAP ASKAP is designed to be a high speed survey instrument with high dynamic range. This requires an extremely wide field-of-view interferometer with Phased Array Feeds (PAFs). The PAF has a total of 188 elements, 94 in each of two polarizations. The specification for ASKAP is: A total collecting area of approximately 4,000 square metres, from 36 antennas each 12 metres in diameter System temperature less than 50 K Frequency range from 700 MHz to 1.8 GHz 300 MHz instantaneous bandwidth 36 independent beams, each of about 1 square degree, yield overlapping to a 30 square degree field-of-view at 1.4 GHz Maximum baseline of approximately 6 km Full cross-correlation of all antennas Possible remote array station capability located in NSW, approximately 3,000 km from the core site.

  6. IVEC iVEC is an unincorporated joint venture between CSIRO, Curtin University, Edith Cowan University, Murdoch University and the University of Western Australia and is supported by the Western Australian Government. Fornax The SGI supercomputer comprises 96 nodes, each containing two 6-core Intel Xeon X5650 CPUs, an NVIDIA Tesla C2075 GPU and 72 GB RAM, resulting in a system containing 1152 cores and 96 GPUs. Storage Data Stores @ iVEC also includes the Petabyte Data Store (PB Store) which is a scalable storage solution based on Sun’s SAM-QFS (Storage and Archive Manager File System & Quick File System) software, and consists of a 70TB of disk cache and a second 100TB file system. The SAM-QFS software automates the migration and retrieval of files across multiple tiers of storage from the online disk cache, to near-line tape storage, to offline tape archival. This system allows researchers to see their complete directory hierarchy and have seamless access to their files within a single file system. Each file that is placed on the system has a copy written to two separate tapes.  LTO4 and StorageTek SL8500

  7. NGAS (Next Generation Archiving System) ESO’s answer to the data explosion challenge. It is an integrated system consisting of hardware and software. Andreas wicenec andreas.wicenec@uwa.edu.au Research Winthrop Professor UWA data intensive research expert. Specialisation: Data flow and storage design and HPC integration

  8. NGAS, NG/AMS, NGAST???? • Next Generation Archive System: ESO’s answer to the data explosion challenge. It is an integrated system consisting of hardware and software. • NG/AMS: Next Generation Archive Management System; this is the NGAS software. • NGAST: The name of the ESO project leading to the NGAS. Started in 2000. First prototype system installed on La Silla in 2001. Now there are >100 NGAS servers in the ESO Archive. • Never call something NG? True, but the intention is to keep this at the forefront of technology. 14

  9. Scope of NGAS • Secure, scalable, distributed and affordable file store. • Secure: Do not lose files. • Scalable: Be prepared for peta-bytes of data. • Distributed: Operate transparently across physically and logically separated locations. • Affordable: TCO at least comparable to DVDs in juke-boxes. • Lowest component of the archive. 15

  10. NGAS concepts • Distributed storage, but not global file system. • Almost complete separation of storage and business logic: • NGAS stores many files on many media, which may be distributed and moved around the globe. • Knowledge and functionality to extract the contents, do groupings and associations is somewhere else. • NGAS deals only with file names, IDs, sizes, locations, media IDs, capacity…. And that information is stored in a data base. 16

  11. NGAS concepts • No user knowledge required about the physical location of a given file. In an operational environment there is one central access point (master), which passes requests to the correct NGAS server using HTTP proxy or HTTP redirect. Any server can be used as a master. • Ensure that there are always at least two copies on two different media of any given file in the system. • Special commands for file and media removal. • Active health check of media and files. • Active consistency checks of media contents vs. DB. • NG/AMS uses modular plug-in architecture. • Very flexible and easy adjustments to hardware and environment changes. 17

  12. Advanced NGAS features • Subscription service: • One NGAS server can subscribe to another. • Parallel cluster mirroring. • Disk and file cloning. • Processing plug-in. • Power saving mode: • Automatic sleep and wakeup system built in. Data flow Archiving NGAS NGAS Subscription NGAS NGAS NGAS 18

  13. NG/AMS Interfaces • All user I/F and APIs are based on HTTP. • Management information is kept in a DB. • Interfaces for five DBs. • Interface to syslog • E-mail notification services. • Message and config-file format is XML. 19

  14. NGAS as an Embedded System • NGAS is a tightly integrated system in order to be ready for operations: • Minimal boot time • Automated boot and shutdown procedures • Hardware monitoring • Integration of raid controller • Media verification using access to on-disk BIOS • Cluster configuration VirtualEnv 20

  15. Implementation • 100% Python • about 25,000 lines + 15,000 lines of in-line documentation • advanced system integration tests • two person team, plus some help from system admin people. • additional code added for mirroring • Advanced fully automatic installation procedure.

  16. Data Intensive Research Pathfinder Hierarchical Storage Manager

  17. What’s Unique? • Transparent global distribution of archival data. • Global mirroring service over slow networks • Fully traceable disk based data transfer • Very robust and flexible (plugins) • Full control of code • Used to store VLT, ALMA, MWA, VLA, eVLA and VLBI data. • Supports processing inside archive

  18. Where Does it Fit? 40T MRO MWA IVEC-DC Raw Visibilities 1Gb/s 80T Raw Visibilities 10Gb/s MRO MWA IVEC-DC

  19. GAVO (The German Astrophysical Virtual Observatory (GAVO) Dr. Markus Demleitner Astronomisches Rechen-Institut in Heidelberg. Woring on the GAVO project, an effort to contribute to Virtual Observatory, e.g., through the operation of the GAVO data center and developing the DaCHS multiprotocol server suite. http://www.tfiu.de/homepage/ gavo@ari.uni-heidelberg.de

  20. GAVO (The German Astrophysical Virtual Observatory (GAVO) GAVO is the German contribution to the IVOA (International Virtual Observatory Association), the international effort to create and expand the Virtual Observatory (VO). ) The goals of the Virtual Observatory are to allow or improve access to astronomical data of all kinds (astrometry, photometry, spectroscopy, time series,...), from everywhere, in well-defined protocols, let astronomers easily discover, access and use data relevant to them, ensure that data does not simply disappear, that it is properly described and can be accessed and understood in the future, provide software to help astronomers to use all this.

  21. Tier 0 Nan Shan 25M Online processing Online archive Correlator PFB Receiver DataCapture NGAS Client 10Gb/s 10Gb/s Tier 2 Tier 1 1G/10Gbps Mirrored Archive (All Data) Publish Data 1Gbps Dedicated Network Qi Tai Site Nan Shan Other Places MS Backup QA Raw Data Catalog VO Data Query VO Based WEB Interface Long-term archive MS RS Processed Data Web local storage Computing resources RV XAO Headquarters (Storage Arrays) Scripting Image, Time sequence UI HPC XAO’s Future Data Archiving, Processing and Query System Zhanghailong

  22. ICRAR MRO MWA NGAS GAVO OUR DC Thank you Question?

More Related