1 / 45

SCARI e : e VLBI Software Correlation Over Dynamic Lambda Grids

SCARIe is a project to develop a software correlator for Very Long Baseline Interferometry (VLBI) on top of a grid middleware and infrastructure. It explores two modes of operation: batch and real-time correlation. Networking challenges are also addressed through experiments using DAS-3 and StarPlane.

loisc
Télécharger la présentation

SCARI e : e VLBI Software Correlation Over Dynamic Lambda Grids

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Damien Marchal, on behalf of the SCARIe team. University of Amsterdam The Netherlands SCARIe: eVLBI Software Correlation Over Dynamic Lambda Grids

  2. SCARIe, is a project to make a software correlator for VLBI on top of a grid middleware and infrastructures.

  3. Software Correlation Over Dynamic Lambda Grids Outline • Introduction: • Radio astronomy • VLBI and eVLBI • Correlation • The SCARIe project • What is a software correlator • Two mode of operation: Batch or Real-time • Networking challenges • Experiment using DAS-3 and StarPlane • Conclusions

  4. Software Correlation Over Dynamic Lambda Grids Outline • Introduction: • Radio astronomy • VLBI and eVLBI • Correlation • SCARIe project • What is a software correlator • Two mode of operation: Batch or Real-time • Networking challenges • Experiment using DAS-3 and StarPlane • Conclusions

  5. Software Correlation Over Dynamic Lambda Grids Radio and optical 408 Mhz optical 1.4 Ghz

  6. Software Correlation Over Dynamic Lambda Grids Larger dish: higher resolution Resolution: λ / b Sensitivity: b2 λ = wavelength b = diameter telescope Arecibo: D = 305 m

  7. Software Correlation Over Dynamic Lambda Grids Aperture Synthesis Imaging Image Credits: Avruch and Pogrebenko • A technique that uses a number of telescopes to simulate a much larger one. A larger dish, real or simulated, improves image clarity and brightness. This requires coordination between the telescopes and a supercomputer. Consider the examples displaying aperture size, aperture distribution and image quality.

  8. Software Correlation Over Dynamic Lambda Grids Westerbork / Very Large Array

  9. Software Correlation Over Dynamic Lambda Grids Even higher resolution: VLBI

  10. Software Correlation Over Dynamic Lambda Grids Radio-Telescopes participating…

  11. Software Correlation Over Dynamic Lambda Grids Data Flow Past Today Results Today Soon Today

  12. Software Correlation Over Dynamic Lambda Grids Data Flow Past Today Results Today Soon correlation Today

  13. Software Correlation Over Dynamic Lambda Grids Outline • Introduction: • Radio astronomy • VLBI and eVLBI • Correlation • SCARIe project • What is a software correlator ? • Two mode of operation: Batch or Real-time • Networking challenges • Experiment using DAS-3 and StarPlane • Conclusions

  14. Software Correlation Over Dynamic Lambda Grids SCARIe: the project The starting point: The increase of size/performance of a modern grids/clusters and networks make of grid a “to investigate” plate-form for software correlation. Today Results Software correlator (on a grid) Today

  15. Software Correlation Over Dynamic Lambda Grids SCARIe: the project The starting point: The increase of size/performance of a modern grid/cluster and networks make of grid a “to investigate” plate-form for software correlation. Software Correlator • High flexibility: • Tracking of spacecrafts • Research on masers • Share the resource/grid with others to lower the cost • Run everywhere Hardware correlator • Low flexibility • Superior computing power • More efficient use of computing resources(Lower energy consumption)

  16. Software Correlation Over Dynamic Lambda Grids SCARIe: distributed correlation Telescopes Correlator

  17. Software Correlation Over Dynamic Lambda Grids SCARIe: distributed correlation Telescopes Correlator Input nodes

  18. Software Correlation Over Dynamic Lambda Grids SCARIe: distributed correlation Telescopes Correlator Input nodes Correlation nodes

  19. Software Correlation Over Dynamic Lambda Grids SCARIe: distributed correlation Telescopes Correlator Input flow = sum(output flows) Input nodes Correlation nodes Output node

  20. Software Correlation Over Dynamic Lambda Grids SCARIe: distributed correlation Telescopes Correlator Input flow = sum(output flows)‏ Input nodes Current experiment at: 4 x 256Mbps Full size experiment: 16x1Gbps Future eVLBI: 32x4Gbps Correlation nodes Output node

  21. Software Correlation Over Dynamic Lambda Grids SCARIe: distributed correlation We can identify two kinds of data flows. Input flows, link to the RT. Correlator In-grid flows.

  22. Software Correlation Over Dynamic Lambda Grids SCARIe: distributed correlation We can identify two kinds of data flows. Input flows, link to the Radio-Tel. (point to point, long distance, several Gbps) Correlator In-grid flows. (one to all, short distance, low latency).

  23. Software Correlation Over Dynamic Lambda Grids SCARIe: operational mode Two kinds of operational mode Batch correlation: • the data are saved on hard-drives and replayed; • the network performance and number of nodesimpact the processing speed. Real-time correlation: • the radio-telescopes are streaming the data directly into the software correlator; • the network performance and number of nodes impact the success of the observation.

  24. Software Correlation Over Dynamic Lambda Grids SCARIe: operational mode We can identify two kind of operational mode… Batch correlation: • the data are saved on hard-drives and replayed; • the network performance and number of nodesimpact the processing speed. Fastness ! Real-time correlation: • the radio-telescope are streaming the data directly into the software correlator; • The network performance and number of nodes impact the success of the observation. Reliability!

  25. SCARIe: workflow Software Correlation Over Dynamic Lambda Grids

  26. SCARIe: workflow Software Correlation Over Dynamic Lambda Grids

  27. SCARIe: workflow Software Correlation Over Dynamic Lambda Grids

  28. Software Correlation Over Dynamic Lambda Grids Outline • Introduction: • Radio astronomy • VLBI and e-VLBI • Correlation • SCARIe project • What is a software correlator • Two mode of operation: Batch or Real-time • Networking challenges • Experiment using DAS-3 and StarPlane • Conclusions

  29. Experiment on DAS-3 • Goals: • evaluate the scalability of a software correlator; • evaluate the DAS-3 network capabilities; • evaluate Bandwidth on Demand service. Software Correlation Over Dynamic Lambda Grids

  30. What is DAS-3 ? • DAS-3 is composed of: • 5 cluster sites, • 1GE Network, • 10G Myrinet network, • Interconnexion photonic network called StarPlane. • StarPlane manages: • several 10 Gbps lighpaths • lightpaths have to reserved Software Correlation Over Dynamic Lambda Grids

  31. Software Correlation Over Dynamic Lambda Grids What is StarPlane ? Key principle: applications have to request to StarPlane the right to use a certain amount of lightpaths of the fast interconnexion network. Two ways to acquire a path with StarPlane: procedurally by direct calling the webservice function:getPath(“source”, “dst”, “time”, “duration”) declaratively through a query language like: query( path(X, “leiden”, L1), path(X, “delft”, L2), available([L1, L2]), reserve([L1, L2], ResTicket) )‏ X L1 L2 leiden delft

  32. Software Correlation Over Dynamic Lambda Grids DAS-3 + StarPlane Demonstration: Two applications are transferring data between two clusters. Initially they start using the default Internet interconnexion network. The middleware requests path to StarPlane. When path is acquired the middleware redirect the traffic to make use of it.

  33. Software Correlation Over Dynamic Lambda Grids DAS-3 + StarPlane Demonstration: Two applications are transferring data between two clusters. Initially they start using the default Internet interconnexion network. The middleware requests path to StarPlane. When path is acquired the middleware redirect the traffic to make use of it. 'a' is started and uses the shared 1GE network; 'b' is started and and uses the shared 1GE network; 'a' has acquired the right to use one of the lighpath.

  34. Software Correlation Over Dynamic Lambda Grids SCARIe experiment on DAS-3 Correlator

  35. Software Correlation Over Dynamic Lambda Grids SCARIe experiment on DAS-3 Mapping the application to the resource/service set: Correlator

  36. Software Correlation Over Dynamic Lambda Grids SCARIe experiment on DAS-3 Mapping the application to the resource/service set: Correlator

  37. Software Correlation Over Dynamic Lambda Grids SCARIe experiment on DAS-3 Mapping the application to the resource/service set: Correlator

  38. Software Correlation Over Dynamic Lambda Grids SCARIe experiment on DAS-3 Node and network resource co-allocation query: query( cluster(“UvA”), cluster(C2), cluster(C3), C2 \= C3, NC1 + NC2 + NC3 == 100, nodes(“UvA”, NC1, N1), nodes(C2, NC2, N2), nodes(C3, NC3, N3), path(“UvA”, C2, L1), path(“UvA”, C3, L2), reserve([N1, N2, N3, L1, L2], ResTicket) )‏

  39. Software Correlation Over Dynamic Lambda Grids Results

  40. Software Correlation Over Dynamic Lambda Grids Results • Input nodes (quad core 2Ghz CPU) are close to handle 1Gbps data flow. • We need around 100 correlation nodes to compute a 4x256Mbps observation in real-time. This is much more than our “theoretical estimation”. • Networking speed is not (yet ?) a bottleneck. • StarPlane permits a simple, per application QoS between the clusters by controlling the access to the medium at a large grain.

  41. Software Correlation Over Dynamic Lambda Grids International Bandwidth on Demand Service (eg. Gn2 AutoBahn) Correlator Last 200m Future plan • We need to use a BoD service (like GN2 Autobahn) to establish QoS paths between the radio-telescopes and the grid.

  42. Software Correlation Over Dynamic Lambda Grids Future plan • Integrate the BoD service into the “default” StarPlane network service to submit complex, integrated, jobs to DAS-3. query( path(CINPUT, “RadioTelescope@Arecibo”, L1), path(CINPUT,“RadioTelescope@Dwingeloo”, L2), cluster(CINPUT), cluster(C2), cluster(C3), alldifferent( [CINPUT, C2, C3] ), NC1 + NC2 + NC3 == 100, NC1 >= NumRT, nodes(CINPUT, NC1, N1), nodes(C2, NC2, N2), nodes(C3, NC3, N3), path(CINPUT, C2, L1), path(CINPUT, C3, L2), reserve([N1, N2, N3, L1, L2], ResTicket) )‏

  43. Software Correlation Over Dynamic Lambda Grids Conclusion Software correlation is challenging : lots of input flows to bring the data; lots of computations that need to be distributed; over multi-domain; for large size (Gbps speed); over several scales: LAN, nation, world; with a “per experiment” dynamicity. Software correlation is possible but it is unclear at what cost ! In term of computing power: CPU is slow for a systematic computation like FFT. => using GPU/FPGA for co-processing. In term of networking: fat pipes have a cost; dynamicity (connectivity) has a cost; QoS has a cost. => Selecting the adequate layer to implement the needed service.

  44. Software Correlation Over Dynamic Lambda Grids Questions/Answers Contact information Nico Kruithof Joint Institute for VLBI in Europe (JIVE) Kruithof@jive.nl Damien Marchal Universiteit van Amsterdam (UvA) dmarchal@science.uva.nl SCARIe is made possible through the support of the Netherlands Organisation for Scientific Research (NWO).

  45. Software Correlation Over Dynamic Lambda Grids Middleware for StarPlane • Based on linux dlopen to hijack the socket API. Application Application Starplane Hijacker Hijacker Socket API Normal interconnexion Socket API Lighpath if available

More Related