1 / 47

Backward Chaining (Cont.)

Process. On Demand. Insert terminates with an insert event notification to Subscribe,. Subscribe. which in turn triggers initiation of ETS algorithm. Register Subscription. Notification. Store. Archive Data. Catalogue Data. Locate Data. Retrieve Data. Backward Chaining (Cont.).

davida
Télécharger la présentation

Backward Chaining (Cont.)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Process On Demand Insert terminates with an insert event notification to Subscribe, Subscribe which in turn triggers initiation of ETS algorithm Register Subscription Notification Store Archive Data Catalogue Data Locate Data Retrieve Data Backward Chaining (Cont.) PRODUCTION

  2. 2. Notify (AST_09T:Insert, direct) Communications Subsystem SBSRV Interoper- ability ADSRV Ingest Subsystem Ingest 1. Trigger AST_09T:Insert event Data Mgmt Subsystem DDICT Planning Subsystem abc def ghi 3. Data Processing Request (ETS) jkl mno Planning Data Server Subsystem Data Processing Subsystem 0110001011 0010110010 1101011101 0110001011 Data Proc Collection DDIST SDSRV Inven- tory DB Client STMGT ASTER: Notification and Subscription Triggering Process Notify all AST_09T:Insert subscribers. This includes notification of the Planning Subsystem, for chained processing.

  3. Data Management EcDpPrDM Begins ETS staging and processing activities Execution Management EcDpPrEM EcDpPrSMFLink PGE Management EcDpPrRunPGE EcDpPrRusage Data Distribution EcDsDistributionServer Science Data Server EcDsScienceDataServer Archive Server EcDsStArchiveServer (Separate software libraries for SSI&T) Staging Monitor Process EcDsStStagingMonitorServer EcDsStStagingDiskServer Data Distribution GUI EcDsDdistGui Virtual Terminal VT File Transfer Protocol FTP/KFTP Deletion Server EcDpPrDeletion DCE Services Name Service Security Service Time Service Subscription Server EcSbSubServer Job Management EcDpPrJobMgmt EcDpPrJobMgmtClient EcDpPrViewJobStates HDF EOS Server EcDsHdfEosServer AutoSys COTS Product SDP ToolKit (SDPTK) CSCI Receives event notice from SDSRV; queries database; notifies Planning Resource Manager Process EcDsSt8MMServer EcDsStD3Server EcDsStIngestFtpServer EcDsStFtpDisServer EcDsStCDROMServer EcDsStPrintServer Crypto- graphic Manage- ment Interface CMI Data Base Sybase Server (COTS) FTP Notification CsFtpNotify Planning Subsystem (PLS) Resource Planning Workbench EcPlRpRe EcPlRpSi EcPlRpTl Data Store Sybase Database (COTS) Landsat7 Gateway EcCsLandsat7Gateway Data Preprocessing EcDpPrAm1AncillaryDPREP EcDpPrAm1EphemerisGapFillDPREP EcDpPrAm1FddAttitudeDPREP EcDpPrAm1FddReplaceEphemerisDPREP EcDpPrAm1ToolkitToHdfDPREP EcDpPrDumpAttitudeDPREP EcDpPrDumpEphemerisDPREP abc Domain Name Service DNS def ghi QA Monitor EcDpPrQaMonitorGUI jkl Ground Event Process EcDpPrGE mno Science Data Server GUI EcDsSdsrvGui Releases ETS job Planning Triggers event; retrieves metadata for requested input granules from the Sybase/SQS database Data Distribution (DDIST) CSCI Bulk Data Server BDS Adds UR for new granule (AST_09T) to data context; matches metadata against job input needs; notifies Job Management to release ETS E-Mail Parser Gateway EcCsEmailParser E-Mail Services Email Server Resource Management DpPrRM Data Processing Subsystem (DPS) 0110001011 Data Store Sybase Database (COTS) 0010110010 1101011101 Production Planning Workbench EcPlWb EcPlTl Sybase/SQS Sybase Server (COTS) 0110001011 Production Request Editor EcPlPrEditor Data Proc Network File System NFS Processing (PRONG) CSCI Infra- structure Library PF, SRF, Urs, Message Passing, Event Logging, Service Locator, Failure Recovery Framework Pull Monitor Process EcDsStPullMonitorServer DAR Comm Gateway EcGwDARServer Subscription Service EcSbGui Data Server Subsystem (DSS) Filecopy EcUtFileCopy EcUtFileExec Science Data Server (SDSRV) CSCI Data Base Sybase Server (COTS) Collection Insert/Acquire Tools EcDpAtINSTALL EcDpAtInsertStaticFile EcDpAtInsertExeTarFile EcDpAtInsertTestFile EcDpAtStageDAP EcDpAtGetMCF SSAP GUI EcDpAtSSAPGui Check Software Tools EcDpAtCheckPCF EcDpAtBadFuncGui EcDpAtBadFunc EcDpAtPrologs EcDpAtReplaceString View/Compare Data Tools EOSView, IDL EcDpAtBinDiffGui EcDpAtBinDiffPrepareFiles EcDpAtCheckHdfFile hdiff, xdiff DDIST SDSRV Planning Subscription Editor EcPlSubsEditor ProductionStrategies GUI EcPlProdStrat SubscriptionManager EcPlSubMgr MOJO Gateway EcCsMojoGateway Storage Management (STMGT) CSCI Inven- tory DB Distributed Computing Configuration Item (DCCI) STMGT Define PGE EcDpAtCheckODL EcDpAtCreateODLTemplate EcDpAtRegisterPGE EcDpAtOpDbGui Database; contains metadata for search and retrieval Production Planning (PLANG) CSCI SSIT Manager EcDpAtMgr EcDpAtMgrLogDump Algorithm Integration and Test Tools (AITTL) CSCI ASTER: CSCI/Component Role in Notification/Subscription Triggering

  4. Process On Demand Subscribe Register Subscription Notification Store Archive Data Catalogue Data Locate Data Retrieve Data Backward Chaining (Cont.) PRODUCTION Retrieve AST_ANC (ASTER ancillary data set), and ETS PGE granules as input to ETS production job; PGE execution begins

  5. Communications Subsystem SBSRV 1. Acquire (AST_ANC & ETS, ftpPush) Interoper- ability ADSRV Ingest Subsystem Ingest Data Mgmt Subsystem DDICT Planning Subsystem abc def ghi jkl mno Planning 2. Distribution Request (AST_ANC & ETS files, ftpPush) Data Server Subsystem Data Processing Subsystem 0110001011 0010110010 1101011101 0110001011 Data Proc Collection DDIST SDSRV Inven- tory DB Client STMGT 4. ftpPush (AST_ANC & ETS files) 3. Copy (AST_ANC & ETS files, ftpPush) ASTER: Job (ETS) Staging Process ETS production job retrieves required ETS PGE and AST_ANC (ASTER ancillary data set) input data granules (Note: AST_09T is already available on DPS resources).

  6. Data Management EcDpPrDM Execution Management EcDpPrEM EcDpPrSMFLink PGE Management EcDpPrRunPGE EcDpPrRusage Data Distribution EcDsDistributionServer Handles requests to the Archive for retrieving and inserting granules Controls staging and processing activities Science Data Server EcDsScienceDataServer Archive Server EcDsStArchiveServer (Separate software libraries for SSI&T) Staging Monitor Process EcDsStStagingMonitorServer EcDsStStagingDiskServer Data Distribution GUI EcDsDdistGui Creates staging disks for granule files; requests granule file retrieval by STMGT Deletion Server EcDpPrDeletion Job Management EcDpPrJobMgmt EcDpPrJobMgmtClient EcDpPrViewJobStates HDF EOS Server EcDsHdfEosServer AutoSys COTS Product SDP ToolKit (SDPTK) CSCI Releases and sequences DPRs (Jobs) Resource Manager Process EcDsSt8MMServer EcDsStD3Server EcDsStIngestFtpServer EcDsStFtpDisServer EcDsStCDROMServer EcDsStPrintServer Data Base Sybase Server (COTS) Stages granule files to read-only cache disks Data Preprocessing EcDpPrAm1AncillaryDPREP EcDpPrAm1EphemerisGapFillDPREP EcDpPrAm1FddAttitudeDPREP EcDpPrAm1FddReplaceEphemerisDPREP EcDpPrAm1ToolkitToHdfDPREP EcDpPrDumpAttitudeDPREP EcDpPrDumpEphemerisDPREP QA Monitor EcDpPrQaMonitorGUI Ground Event Process EcDpPrGE Science Data Server GUI EcDsSdsrvGui Data Distribution (DDIST) CSCI Retrieves metadata; creates metadata Staging Disks; creates and passes metadata files to DDIST Resource Management DpPrRM Data Processing Subsystem (DPS) 0110001011 Data Store Sybase Database (COTS) 0010110010 1101011101 Sybase/SQS Sybase Server (COTS) 0110001011 Data Proc Processing (PRONG) CSCI Pull Monitor Process EcDsStPullMonitorServer Data Server Subsystem (DSS) Allocates space in response to Distribution Requests Science Data Server (SDSRV) CSCI Data Base Sybase Server (COTS) Collection Insert/Acquire Tools EcDpAtINSTALL EcDpAtInsertStaticFile EcDpAtInsertExeTarFile EcDpAtInsertTestFile EcDpAtStageDAP EcDpAtGetMCF SSAP GUI EcDpAtSSAPGui Check Software Tools EcDpAtCheckPCF EcDpAtBadFuncGui EcDpAtBadFunc EcDpAtPrologs EcDpAtReplaceString View/Compare Data Tools EOSView, IDL EcDpAtBinDiffGui EcDpAtBinDiffPrepareFiles EcDpAtCheckHdfFile hdiff, xdiff DDIST SDSRV Storage Management (STMGT) CSCI Inven- tory DB Database; contains metadata for search and retrieval STMGT Define PGE EcDpAtCheckODL EcDpAtCreateODLTemplate EcDpAtRegisterPGE EcDpAtOpDbGui Provides Resource Manager for ftpPushes SSIT Manager EcDpAtMgr EcDpAtMgrLogDump Algorithm Integration and Test Tools (AITTL) CSCI ASTER: CSCI/Component Rolein Job (ETS) Staging

  7. Archive newly created AST_08 (L2 Surface Temperature) and AST_05 (L2 Surface Emissivity) granules after completion of ETS PGE; Process On Demand update catalogue with references to AST_08 and AST_05 Subscribe Register Subscription Notification Store Archive Data Catalogue Data Locate Data Retrieve Data Backward Chaining (Cont.) PRODUCTION

  8. Communications Subsystem SBSRV 1. Get DSS UR Interoper- ability ADSRV Ingest Subsystem Ingest 2. Insert (AST_08 & AST_05) Data Mgmt Subsystem DDICT Planning Subsystem abc def ghi jkl mno Planning Data Server Subsystem Data Processing Subsystem 0110001011 0010110010 4. Add (AST_08 & AST_05 metadata) 1101011101 0110001011 Data Proc Collection DDIST SDSRV 3. Store (AST_08 & AST_05 files) Inven- tory DB Client STMGT ASTER: PGE (ETS) Execution and Output Insertion Process ETS PGE is successfully executed and newly created AST_08 (L2 Surface Temperature) and AST_05 (L2 Surface Emissivity) granules are archived; inventory is updated.

  9. Interoper- ability (IOS) Data Management EcDpPrDM Execution Management EcDpPrEM EcDpPrSMFLink PGE Management EcDpPrRunPGE EcDpPrRusage Data Distribution EcDsDistributionServer Science Data Server EcDsScienceDataServer Archive Server EcDsStArchiveServer (Separate software libraries for SSI&T) Staging Monitor Process EcDsStStagingMonitorServer EcDsStStagingDiskServer Advertising Service Replication Server Sybase Replication Server (COTS) Advertising Service SQL Server Sybase Server (COTS) Advertising Server EcIoAdServer Data Distribution GUI EcDsDdistGui Deletion Server EcDpPrDeletion Gets DSS UR from Advertising Server; handles requests to the Archive for retrieving and inserting granules Executes PGE; places output files in output directory Job Management EcDpPrJobMgmt EcDpPrJobMgmtClient EcDpPrViewJobStates HDF EOS Server EcDsHdfEosServer AutoSys COTS Product SDP ToolKit (SDPTK) CSCI Advertising Server Component Provides DSS Universal Reference (UR) Resource Manager Process EcDsSt8MMServer EcDsStD3Server EcDsStIngestFtpServer EcDsStFtpDisServer EcDsStCDROMServer EcDsStPrintServer Data Base Sybase Server (COTS) Data Preprocessing EcDpPrAm1AncillaryDPREP EcDpPrAm1EphemerisGapFillDPREP EcDpPrAm1FddAttitudeDPREP EcDpPrAm1FddReplaceEphemerisDPREP EcDpPrAm1ToolkitToHdfDPREP EcDpPrDumpAttitudeDPREP EcDpPrDumpEphemerisDPREP QA Monitor EcDpPrQaMonitorGUI Ground Event Process EcDpPrGE Science Data Server GUI EcDsSdsrvGui Data Distribution (DDIST) CSCI Advertising Service (ADSRV) CSCI Provides Metadata Configuration File (MCF); requests STMGT to insert output granule files; adds metadata to inventory Resource Management DpPrRM Data Processing Subsystem (DPS) 0110001011 Data Store Sybase Database (COTS) 0010110010 1101011101 Sybase/SQS Sybase Server (COTS) 0110001011 Data Proc Processing (PRONG) CSCI Pull Monitor Process EcDsStPullMonitorServer Data Server Subsystem (DSS) Reads files from DPS disks and stores files in Archive Science Data Server (SDSRV) CSCI Data Base Sybase Server (COTS) Collection Insert/Acquire Tools EcDpAtINSTALL EcDpAtInsertStaticFile EcDpAtInsertExeTarFile EcDpAtInsertTestFile EcDpAtStageDAP EcDpAtGetMCF SSAP GUI EcDpAtSSAPGui Check Software Tools EcDpAtCheckPCF EcDpAtBadFuncGui EcDpAtBadFunc EcDpAtPrologs EcDpAtReplaceString View/Compare Data Tools EOSView, IDL EcDpAtBinDiffGui EcDpAtBinDiffPrepareFiles EcDpAtCheckHdfFile hdiff, xdiff DDIST SDSRV Storage Management (STMGT) CSCI Inven- tory DB STMGT Define PGE EcDpAtCheckODL EcDpAtCreateODLTemplate EcDpAtRegisterPGE EcDpAtOpDbGui Database; receives and stores metadata SSIT Manager EcDpAtMgr EcDpAtMgrLogDump Algorithm Integration and Test Tools (AITTL) CSCI ASTER: CSCI/Component Role in PGE (ETS) Execution and Output Insertion

  10. RETRIEVAL Deliver Electronic Search & Order Order Products Subscribe Standing Orders Notification Store Archive Data Catalogue Data Locate Data Retrieve Data Backward Chaining (Cont.) Insert terminates with an insert event notification to Subscribe, which triggers e-mail notification to the Science User that the AST_08 granule has been inserted; standing order processing for new AST_08 granule can begin

  11. 2. Notify (AST_08:Insert, e-mail) Communications Subsystem SBSRV Interoper- ability ADSRV Ingest Subsystem Ingest 1. Trigger AST_08:Insert event Data Mgmt Subsystem DDICT Planning Subsystem abc def ghi jkl mno Planning Data Server Subsystem Data Processing Subsystem 0110001011 0010110010 1101011101 0110001011 Data Proc Collection DDIST SDSRV Inven- tory DB Client STMGT ASTER: Notification and Subscription Triggering (AST_08) Process With insertion of the AST_08 (L2 Surface Temperature) granule, the ASTER Scientist is notified by e-mail; processing can begin for the standing order

  12. Data Distribution EcDsDistributionServer Science Data Server EcDsScienceDataServer Archive Server EcDsStArchiveServer Receives event notice from SDSRV; queries database; executes e-mail notification Staging Monitor Process EcDsStStagingMonitorServer EcDsStStagingDiskServer Data Distribution GUI EcDsDdistGui Virtual Terminal VT File Transfer Protocol FTP/KFTP DCE Services Name Service Security Service Time Service Subscription Server EcSbSubServer HDF EOS Server EcDsHdfEosServer Sendmail delivers e-mail message to mail host Resource Manager Process EcDsSt8MMServer EcDsStD3Server EcDsStIngestFtpServer EcDsStFtpDisServer EcDsStCDROMServer EcDsStPrintServer Crypto- graphic Manage- ment Interface CMI Data Base Sybase Server (COTS) FTP Notification CsFtpNotify Landsat7 Gateway EcCsLandsat7Gateway Domain Name Service DNS Science Data Server GUI EcDsSdsrvGui Data Distribution (DDIST) CSCI Bulk Data Server BDS Triggers event (AST_08:Insert) E-Mail Parser Gateway EcCsEmailParser E-Mail Services Email Server Sybase/SQS Sybase Server (COTS) Network File System NFS Infra- structure Library PF, SRF, Urs, Message Passing, Event Logging, Service Locator, Failure Recovery Framework Pull Monitor Process EcDsStPullMonitorServer DAR Comm Gateway EcGwDARServer Subscription Service EcSbGui Data Server Subsystem (DSS) Filecopy EcUtFileCopy EcUtFileExec Science Data Server (SDSRV) CSCI Data Base Sybase Server (COTS) Collection DDIST SDSRV MOJO Gateway EcCsMojoGateway Storage Management (STMGT) CSCI Inven- tory DB Distributed Computing Configuration Item (DCCI) STMGT ASTER: CSCI/Component Role in Noti-fication/Subscription (AST_08) Trigger

  13. RETRIEVAL Deliver Electronic Search & Order Order Products Subscribe Standing Orders Notification Store Archive Data Catalogue Data Locate Data Retrieve Data Backward Chaining (Cont.) Submit order for new AST_08 on behalf of scientist

  14. Communications Subsystem SBSRV Interoper- ability ADSRV Ingest Subsystem Ingest 1. Acquire (AST_08: ftpPush) Data Mgmt Subsystem DDICT Planning Subsystem abc def ghi jkl mno Planning Data Server Subsystem Data Processing Subsystem 0110001011 0010110010 1101011101 0110001011 Data Proc Collection DDIST SDSRV Inven- tory DB Client STMGT ASTER: Standing Order, Acquire Submission Process Subscription Server submits order for AST_08 (L2 Surface Temperature) data, via ftpPush, on behalf of the Science User

  15. Data Distribution EcDsDistributionServer Science Data Server EcDsScienceDataServer Archive Server EcDsStArchiveServer Connects to SDSRV; establishes data context; submits Acquire request for ftpPush of granule and request for e-mail notice of distribution Staging Monitor Process EcDsStStagingMonitorServer EcDsStStagingDiskServer Data Distribution GUI EcDsDdistGui Virtual Terminal VT File Transfer Protocol FTP/KFTP DCE Services Name Service Security Service Time Service Subscription Server EcSbSubServer HDF EOS Server EcDsHdfEosServer Resource Manager Process EcDsSt8MMServer EcDsStD3Server EcDsStIngestFtpServer EcDsStFtpDisServer EcDsStCDROMServer EcDsStPrintServer Crypto- graphic Manage- ment Interface CMI Data Base Sybase Server (COTS) FTP Notification CsFtpNotify Landsat7 Gateway EcCsLandsat7Gateway Domain Name Service DNS Science Data Server GUI EcDsSdsrvGui Data Distribution (DDIST) CSCI Bulk Data Server BDS E-Mail Parser Gateway EcCsEmailParser E-Mail Services Email Server Sybase/SQS Sybase Server (COTS) Retrieves metadata Network File System NFS Infra- structure Library PF, SRF, Urs, Message Passing, Event Logging, Service Locator, Failure Recovery Framework Pull Monitor Process EcDsStPullMonitorServer DAR Comm Gateway EcGwDARServer Subscription Service EcSbGui Data Server Subsystem (DSS) Filecopy EcUtFileCopy EcUtFileExec Science Data Server (SDSRV) CSCI Data Base Sybase Server (COTS) Collection DDIST SDSRV MOJO Gateway EcCsMojoGateway Storage Management (STMGT) CSCI Inven- tory DB Distributed Computing Configuration Item (DCCI) Database; stores metadata for search and retrieval STMGT ASTER: CSCI/Component Role in Standing Order, Acquire Submission

  16. RETRIEVAL Deliver Electronic Search & Order Order Products Subscribe Standing Orders Notification Store Archive Data Catalogue Data Locate Data Retrieve Data Backward Chaining (Cont.) Retrieve newly created AST_08 granule

  17. Communications Subsystem SBSRV Interoper- ability ADSRV Ingest Subsystem Ingest 1. Distribution Request (AST_08:ftpPush) Data Mgmt Subsystem DDICT Planning Subsystem abc def ghi jkl mno Planning Data Server Subsystem Data Processing Subsystem 0110001011 0010110010 1101011101 0110001011 Data Proc Collection DDIST SDSRV 2. Copy (AST_08, ftpPush) Inven- tory DB Client STMGT ASTER: Retrieval of Data for Distribution Process Retrieve newly created AST_08 (L2 Surface Temperature) granule from Archive.

  18. Creates staging disks for granule files; requests granule file retrieval by STMGT Data Distribution EcDsDistributionServer Science Data Server EcDsScienceDataServer Archive Server EcDsStArchiveServer Staging Monitor Process EcDsStStagingMonitorServer EcDsStStagingDiskServer Stages granule files to read-only cache disks Data Distribution GUI EcDsDdistGui HDF EOS Server EcDsHdfEosServer Resource Manager Process EcDsSt8MMServer EcDsStD3Server EcDsStIngestFtpServer EcDsStFtpDisServer EcDsStCDROMServer EcDsStPrintServer Data Base Sybase Server (COTS) Science Data Server GUI EcDsSdsrvGui Creates metadata Staging Disks; creates and passes metadata files to DDIST Data Distribution (DDIST) CSCI Allocates space in response to Distribution Requests Sybase/SQS Sybase Server (COTS) Pull Monitor Process EcDsStPullMonitorServer Data Server Subsystem (DSS) Database; contains metadata for search and retrieval Science Data Server (SDSRV) CSCI Data Base Sybase Server (COTS) Collection DDIST SDSRV Provides Resource Manager for ftpPushes Storage Management (STMGT) CSCI Inven- tory DB STMGT ASTER: CSCI/Component Role in Retrieval of Data for Distribution

  19. RETRIEVAL Deliver Electronic Search & Order Order Products Subscribe Standing Orders Notification Store Archive Data Catalogue Data Locate Data Retrieve Data Backward Chaining (Cont.) Ftp newly created AST_08 granule to scientist’s workstation together with an e-mail notification of the distribution

  20. 3. Send e-mail notification of AST_08 distribution Communications Subsystem 2. Build e-mail notification Interoper- ability ADSRV Ingest Subsystem Ingest Data Mgmt Subsystem DDICT Planning Subsystem abc def ghi jkl mno Planning Data Server Subsystem Data Processing Subsystem 0110001011 0010110010 1101011101 0110001011 Data Proc Collection DDIST SDSRV Inven- tory DB Client STMGT 1. Ftp AST_08 data 4. Forward notification ASTER: Electronic Data Push Distribution Process Ftp AST_08 (L2 Surface Temperature) granule to ASTER Scientist’s workstation.

  21. Data Distribution EcDsDistributionServer Science Data Server EcDsScienceDataServer Archive Server EcDsStArchiveServer Executes ftp Staging Monitor Process EcDsStStagingMonitorServer EcDsStStagingDiskServer Builds e-mail notifica-tion that user’s order has been filled; sends it to DAAC Distribution Tech-nician for forwarding Data Distribution GUI EcDsDdistGui Virtual Terminal VT File Transfer Protocol FTP/KFTP DCE Services Name Service Security Service Time Service Subscription Server EcSbSubServer HDF EOS Server EcDsHdfEosServer Sendmail delivers e-mail message to mail host Resource Manager Process EcDsSt8MMServer EcDsStD3Server EcDsStIngestFtpServer EcDsStFtpDisServer EcDsStCDROMServer EcDsStPrintServer Crypto- graphic Manage- ment Interface CMI Data Base Sybase Server (COTS) FTP Notification CsFtpNotify Landsat7 Gateway EcCsLandsat7Gateway Domain Name Service DNS Science Data Server GUI EcDsSdsrvGui Data Distribution (DDIST) CSCI Bulk Data Server BDS E-Mail Parser Gateway EcCsEmailParser E-Mail Services Email Server Sybase/SQS Sybase Server (COTS) Network File System NFS Infra- structure Library PF, SRF, Urs, Message Passing, Event Logging, Service Locator, Failure Recovery Framework Pull Monitor Process EcDsStPullMonitorServer Client for CSS execution of actual ftp DAR Comm Gateway EcGwDARServer Subscription Service EcSbGui Data Server Subsystem (DSS) Filecopy EcUtFileCopy EcUtFileExec Science Data Server (SDSRV) CSCI Data Base Sybase Server (COTS) Collection DDIST SDSRV MOJO Gateway EcCsMojoGateway Storage Management (STMGT) CSCI Inven- tory DB Distributed Computing Configuration Item (DCCI) STMGT ASTER: CSCI/Component Role in Electronic Data Push Distribution

  22. Scientist uses EOSView to perform Quality Assurance (QA) check on the AST_08 product and e-mails DAAC Operations; DAAC Operations updates AST_08 inventory metadata RETRIEVAL Deliver Electronic Search & Order Order Products Subscribe Standing Orders Notification Store Archive Data Catalogue Data Locate Data Retrieve Data Backward Chaining (Cont.)

  23. Communications Subsystem SBSRV Interoper- ability ADSRV Ingest Subsystem Ingest 4. Request update of QA metadata Data Mgmt Subsystem DDICT Planning Subsystem abc def ghi jkl mno Planning 5. Update metadata inventory, AST_08 Data Server Subsystem Data Processing Subsystem 0110001011 0010110010 1101011101 0110001011 Data Proc 3. Invoke QA Monitor Collection 1. Invoke EOSView to view AST_08 DDIST SDSRV Inven- tory DB Client STMGT 2. Scientist e-mails DAAC with request to update the Science QA metadata for the granule ASTER: QA Metadata Update Process Science User uses EOSView tool to review AST_08 (L2 Surface Temperature) product and sends e-mail request to DAAC Operations for update of the Science QA Metadata for the granule; DAAC Operations uses QA Monitor tool for the update.

  24. Data Management EcDpPrDM Execution Management EcDpPrEM EcDpPrSMFLink PGE Management EcDpPrRunPGE EcDpPrRusage Data Distribution EcDsDistributionServer Science Data Server EcDsScienceDataServer Archive Server EcDsStArchiveServer (Separate software libraries for SSI&T) User Registration Tool (URT) Scripts EcClWbUr EcClWbUrConfirm EcClWbUrSubmit Submits request to SDSRV for update of granule metadata Staging Monitor Process EcDsStStagingMonitorServer EcDsStStagingDiskServer Data Distribution GUI EcDsDdistGui Deletion Server EcDpPrDeletion Job Management EcDpPrJobMgmt EcDpPrJobMgmtClient EcDpPrViewJobStates HDF EOS Server EcDsHdfEosServer AutoSys COTS Product SDP ToolKit (SDPTK) CSCI Client (CLS) External/ Internal Users User Profile Gateway EcClDtUserProfileGateway Resource Manager Process EcDsSt8MMServer EcDsStD3Server EcDsStIngestFtpServer EcDsStFtpDisServer EcDsStCDROMServer EcDsStPrintServer Data Base Sybase Server (COTS) Data Preprocessing EcDpPrAm1AncillaryDPREP EcDpPrAm1EphemerisGapFillDPREP EcDpPrAm1FddAttitudeDPREP EcDpPrAm1FddReplaceEphemerisDPREP EcDpPrAm1ToolkitToHdfDPREP EcDpPrDumpAttitudeDPREP EcDpPrDumpEphemerisDPREP QA Monitor EcDpPrQaMonitorGUI Ground Event Process EcDpPrGE Science Data Server GUI EcDsSdsrvGui Data Distribution (DDIST) CSCI HDF Product Viewing Tool EOSView Resource Management DpPrRM Data Processing Subsystem (DPS) Generates query calls to inventory database; packages and returns results; updates metadata 0110001011 Data Store Sybase Database (COTS) Displays HDF files for visualization 0010110010 1101011101 Sybase/SQS Sybase Server (COTS) 0110001011 Data Proc Processing (PRONG) CSCI Pull Monitor Process EcDsStPullMonitorServer Java DAR Tool EcClWbJdt Data Server Subsystem (DSS) Science Data Server (SDSRV) CSCI Data Base Sybase Server (COTS) Collection Insert/Acquire Tools EcDpAtINSTALL EcDpAtInsertStaticFile EcDpAtInsertExeTarFile EcDpAtInsertTestFile EcDpAtStageDAP EcDpAtGetMCF SSAP GUI EcDpAtSSAPGui Check Software Tools EcDpAtCheckPCF EcDpAtBadFuncGui EcDpAtBadFunc EcDpAtPrologs EcDpAtReplaceString View/Compare Data Tools EOSView, IDL EcDpAtBinDiffGui EcDpAtBinDiffPrepareFiles EcDpAtCheckHdfFile hdiff, xdiff DDIST Workbench (WKBCH) CSCI SDSRV Storage Management (STMGT) CSCI Inven- tory DB STMGT Define PGE EcDpAtCheckODL EcDpAtCreateODLTemplate EcDpAtRegisterPGE EcDpAtOpDbGui Database; contains metadata SSIT Manager EcDpAtMgr EcDpAtMgrLogDump Algorithm Integration and Test Tools (AITTL) CSCI ASTER: CSCI/Component Role inQA Metadata Update

  25. Generate Plan INSERTION RETRIEVAL Deliver Ingest Search & Order Subscribe Store ASTER Scenario: Expedited Data PRODUCTION External Data Provider User Data Subscription Data Insertion Data Notification

  26. INSERTION Ingest Polling Subscribe Register Subscription Notification Store Archive Data Catalogue Data Locate Data Retrieve Data Expedited Data DAAC submits subscription, on behalf of scientist for notification of new ASTER Expedited data

  27. 1. Subscribe (AST_EXP, e-mail) Communications Subsystem SBSRV Interoper- ability ADSRV Ingest Subsystem Ingest Data Mgmt Subsystem DDICT Planning Subsystem abc def ghi jkl mno Planning Data Server Subsystem Data Processing Subsystem 0110001011 0010110010 1101011101 0110001011 Data Proc Collection DDIST SDSRV Inven- tory DB Client STMGT ASTER: User Subscription Registration for AST_EXP Process DAAC enters subscription, on behalf of scientist, for e-mail notification of newly inserted ASTER Expedited Data product.

  28. Virtual Terminal VT File Transfer Protocol FTP/KFTP DCE Services Name Service Security Service Time Service Subscription Server EcSbSubServer Stores subscription in the Sybase Database Crypto- graphic Manage- ment Interface CMI FTP Notification CsFtpNotify Landsat7 Gateway EcCsLandsat7Gateway Domain Name Service DNS Bulk Data Server BDS E-Mail Parser Gateway EcCsEmailParser E-Mail Services Email Server Network File System NFS Infra- structure Library PF, SRF, Urs, Message Passing, Event Logging, Service Locator, Failure Recovery Framework DAR Comm Gateway EcGwDARServer Subscription Service (a service of CSS, not a CSCI) EcSbGui GUI used by User Services Representative to create subscription for Science User Filecopy EcUtFileCopy EcUtFileExec MOJO Gateway EcCsMojoGateway Distributed Computing Configuration Item (DCCI) ASTER: CSCI/Component Role in AST_EXP Subscription Registration

  29. EDOS INSERTION Ingest Polling Subscribe Register Subscription Notification Store Archive Data Catalogue Data Locate Data Retrieve Data Expedited Data (Cont.) EDOS initiates AST_EXP (Expedited Data) ingestion via EDOS PDR

  30. 2. Move EDR file to directory Communications Subsystem SBSRV 4. Getmcf (AST_EXP) Interoper- ability ADSRV Ingest Subsystem Ingest Data Mgmt Subsystem DDICT Planning Subsystem abc def ghi jkl mno Planning Data Server Subsystem Data Processing Subsystem 0110001011 0010110010 1101011101 0110001011 Data Proc Collection DDIST SDSRV Inven- tory DB Client STMGT ASTER: Polling Ingest for AST_EXP Process Ingest begins polling specified location (directory), looking for an Expedited Data Record (EDR) file. EDOS EDOS transfers the file, via ftp, to the predetermined directory. [Location, directory, username, and password are as specified in the ASTER-ECS Interface Control Document (ICD).] 1. Ingest Subsystem polling for files 3. Read EDR file 5. Read AST_EXP files

  31. Data Distribution EcDsDistributionServer Science Data Server EcDsScienceDataServer Archive Server EcDsStArchiveServer Ingest reads delivery record file and data files from provider site via ftp Staging Monitor Process EcDsStStagingMonitorServer EcDsStStagingDiskServer Data Distribution GUI EcDsDdistGui Provides metadata configuration file (mcf) HDF EOS Server EcDsHdfEosServer Staging Disk Server receives request from Ingest to create staging disk and reserves space Resource Manager Process EcDsSt8MMServer EcDsStD3Server EcDsStIngestFtpServer EcDsStFtpDisServer EcDsStCDROMServer EcDsStPrintServer Data Base Sybase Server (COTS) Polls remote location for files named *.EDR.XFR; packages requests Science Data Server GUI EcDsSdsrvGui Data Distribution (DDIST) CSCI Sybase/SQS Sybase Server (COTS) Media Ingest Interface Process EcInGUI Automated Network Ingest Interface Process EcInAuto Pull Monitor Process EcDsStPullMonitorServer Data Server Subsystem (DSS) Packages Ingest Request into Granules Polling Ingest Client Interface Process EcInPolling Ingest Request Manager Process EcInReqMgr Science Data Server (SDSRV) CSCI Data Base Sybase Server (COTS) Collection DDIST SDSRV Storage Management (STMGT) CSCI Inven- tory DB Ingest Granule Server Process EcInGran STMGT Cross-Mode Ingest Interface EcInEmailGWServer Requests meta- data configuration file (mcf) from SDSRV Ingest Database Sybase Server (COTS) Ingest (INGST) CSCI Ingest Subsystem (INS) Ingest ASTER: CSCI/Component Role in Polling Ingest for AST_EXP

  32. INSERTION Ingest Polling Subscribe Register Subscription Notification Store Archive Data Catalogue Data Locate Data Retrieve Data Expedited Data (Cont.) Archive ingested AST_EXP data; update catalogue with reference to AST_EXP

  33. Communications Subsystem SBSRV 1. Insert (AST_EXP) Interoper- ability ADSRV Ingest Subsystem Ingest Data Mgmt Subsystem DDICT Planning Subsystem abc def ghi jkl mno Planning Data Server Subsystem Data Processing Subsystem 0110001011 0010110010 3. Add (AST_EXP metadata) 1101011101 0110001011 Data Proc Collection DDIST SDSRV 2. Store (AST_EXP files) Inven- tory DB Client STMGT ASTER: AST_EXP Insertion Process Newly ingested AST_EXP granule is archived; inventory is updated.

  34. Data Distribution EcDsDistributionServer Science Data Server EcDsScienceDataServer Archive Server EcDsStArchiveServer Validates metadata; requests archiving of granule files; adds metadata to inventory Staging Monitor Process EcDsStStagingMonitorServer EcDsStStagingDiskServer Reads inserted files from Ingest staging disks Data Distribution GUI EcDsDdistGui HDF EOS Server EcDsHdfEosServer Resource Manager Process EcDsSt8MMServer EcDsStD3Server EcDsStIngestFtpServer EcDsStFtpDisServer EcDsStCDROMServer EcDsStPrintServer Data Base Sybase Server (COTS) Builds metadata file; requests SDSRV to validate metadata; requests SDSRV to insert received files Science Data Server GUI EcDsSdsrvGui Data Distribution (DDIST) CSCI Stores validated metadata in the inventory Sybase/SQS Sybase Server (COTS) Media Ingest Interface Process EcInGUI Automated Network Ingest Interface Process EcInAuto Pull Monitor Process EcDsStPullMonitorServer Data Server Subsystem (DSS) Polling Ingest Client Interface Process EcInPolling Ingest Request Manager Process EcInReqMgr Science Data Server (SDSRV) CSCI Data Base Sybase Server (COTS) Collection DDIST SDSRV Storage Management (STMGT) CSCI Inven- tory DB Ingest Granule Server Process EcInGran STMGT Cross-Mode Ingest Interface EcInEmailGWServer Ingest Database Sybase Server (COTS) Ingest (INGST) CSCI Ingest Subsystem (INS) Ingest ASTER: CSCI/Component Rolein AST_EXP Insertion

  35. INSERTION Ingest Polling Subscribe Register Subscription Notification Store Archive Data Catalogue Data Locate Data Retrieve Data Expedited Data (Cont.) Insert terminates with an insert event notification to Subscribe. Subscribe e-mails ASTER Scientist notice of a completed AST_EXP granule insert

  36. Communications Subsystem SBSRV Interoper- ability ADSRV Ingest Subsystem Ingest 2. Notify (AST_EXP:Insert, e-mail) 1. Trigger (AST_EXP:Insert) Data Mgmt Subsystem DDICT Planning Subsystem abc def ghi jkl mno Planning Data Server Subsystem Data Processing Subsystem 0110001011 0010110010 1101011101 0110001011 Data Proc Collection DDIST SDSRV Inven- tory DB Client STMGT ASTER: AST_EXP Event Notification Process Notify all AST_EXP:Insert event subscribers.

  37. Data Distribution EcDsDistributionServer Science Data Server EcDsScienceDataServer Archive Server EcDsStArchiveServer Triggers AST_EXP:Insert event; notifies the correct subscription server Staging Monitor Process EcDsStStagingMonitorServer EcDsStStagingDiskServer Data Distribution GUI EcDsDdistGui Virtual Terminal VT File Transfer Protocol FTP/KFTP DCE Services Name Service Security Service Time Service Subscription Server EcSbSubServer HDF EOS Server EcDsHdfEosServer Resource Manager Process EcDsSt8MMServer EcDsStD3Server EcDsStIngestFtpServer EcDsStFtpDisServer EcDsStCDROMServer EcDsStPrintServer Crypto- graphic Manage- ment Interface CMI Data Base Sybase Server (COTS) FTP Notification CsFtpNotify Landsat7 Gateway EcCsLandsat7Gateway Domain Name Service DNS Science Data Server GUI EcDsSdsrvGui Data Distribution (DDIST) CSCI Bulk Data Server BDS E-Mail Parser Gateway EcCsEmailParser E-Mail Services Email Server Sybase/SQS Sybase Server (COTS) Network File System NFS Fires subscriptions; sends e-mail notification of the AST_EXP:Insert event Infra- structure Library PF, SRF, Urs, Message Passing, Event Logging, Service Locator, Failure Recovery Framework Pull Monitor Process EcDsStPullMonitorServer DAR Comm Gateway EcGwDARServer Subscription Service EcSbGui Data Server Subsystem (DSS) Filecopy EcUtFileCopy EcUtFileExec Science Data Server (SDSRV) CSCI Data Base Sybase Server (COTS) Collection DDIST Sendmail delivers e-mail message to mail host SDSRV MOJO Gateway EcCsMojoGateway Storage Management (STMGT) CSCI Inven- tory DB Distributed Computing Configuration Item (DCCI) STMGT ASTER: CSCI/Component Rolein AST_EXP Event Notification

  38. Note: Landsat Scenario Landsat Goals - Directory Search - User Registration - Data Ingest - Standing Orders (media) - Access to WRS Scenes (fixed scene subsetting) -Viewing Order Status Landsat Preconditions Landsat ESDTs Inserted into ECS -L70R, L70RF1, L70RF2, L70RWRS, L70RWRS1, L70WRS2, L7CPF, Browse Subscription for: - L70R data notification - Standing Order for CPF production on tape

  39. RETRIEVAL Deliver User Access User Registration Subscribe Manage System Monitor & Coordinate Manage Applications Logistics User Registration Science User starts User Registration Tool from link on EOSDIS Home Page or on page for EOS Gateway Web Client and submits registration information.

  40. Communications Subsystem Interoper- ability ADSRV Mgmt Support Subsystem MSS Ingest Subsystem Ingest 2. Submit Registration Request Data Mgmt Subsystem DDICT Planning Subsystem abc def ghi jkl mno Planning Data Server Subsystem Data Processing Subsystem 0110001011 0010110010 1101011101 0110001011 Data Proc Collection 1. Request User Registration DDIST SDSRV Inven- tory DB Client STMGT User Registration Request Process Scientist clicks on link for user registration to invoke User Registration Tool and then enters and submits registration data.

  41. Provides User Registration applications (HTML) User Registration Tool (URT) Scripts EcClWbUr EcClWbUrConfirm EcClWbUrSubmit Client (CLS) External/ Internal Users Submits request to User Registration Server User Profile Gateway EcClDtUserProfileGateway HDF Product Viewing Tool EOSView Baseline Mgr XRP-II (COTS) Master Agent EcMsAgAgent SubAgent EcMsAgSubAgent Inventory/Logistics/ Maintenance Mgr XRP-II (COTS) Proxy Agent EcMsAgGenProxy Management Agent CSCI (MACI) Java DAR Tool EcClWbJdt Mgmt Support Subsystem MSS Deputy Agent EcMsAgDeputy Software Change Mgr ClearCase (COTS) Network and Enterprise Management HP OpenView NNM Tivoli Sentry Mode Mgmt Trouble Ticket aruser GUI (COTS) aradmin GUI (COTS) MsTfHTML web interface notifier GUI (COTS) notifier daemon (COTS) Submit, Query, Notify (COTS) armail daemon (COTS) UNIX sendmail Security anlpasswd (COTS) TCP Wrappers (COTS) Tripwire (COTS) SATAN (COTS) Crack (COTS) Change Request Mgr DDTS (COTS) Saves request as a “pending” User Registration Request awaiting approval Workbench (WKBCH) CSCI Software Distribution Mgr Tivoli Courier (COTS) Accountability EcMsAcRegUserSrvr EcMsAcRegUserGUI EcMsAcOrderSrvr EcMsAcOrderGUI Network Backup/Restore Networker Server (COTS) Networker Client (COTS) ASTER E-mail Header Handler EcMsAsAddHeader.pl EcMsRemoveHeader.pl Software License Mgr FLEXlm (COTS) iFOR/LS (COTS) MsLiiFORLSMkDayLog Management CSCI (MCI) Management Logistics CSCI (MLCI) CSCI/Component Role inUser Registration Request

  42. RETRIEVAL Deliver User Access User Registration Subscribe Manage System Monitor & Coordinate Manage Applications Logistics User Registration (Cont.) DAAC Operations (User Services) retrieves pending User Registration request and creates User Profile to register the user

  43. Communications Subsystem Interoper- ability ADSRV Mgmt Support Subsystem MSS Ingest Subsystem Ingest 1. Retrieve and approve User Registration; create and save User Profile Data Mgmt Subsystem DDICT Planning Subsystem abc def ghi jkl mno Planning Data Server Subsystem Data Processing Subsystem 0110001011 0010110010 1101011101 0110001011 Data Proc Collection DDIST SDSRV Inven- tory DB Client STMGT User Registration Approval Process DAAC User Services Representative uses User Registration Server Graphical User Interface (GUI) to retrieve pending User Registration request and complete a User Profile, which is saved to register the user.

  44. Baseline Mgr XRP-II (COTS) Master Agent EcMsAgAgent SubAgent EcMsAgSubAgent Inventory/Logistics/ Maintenance Mgr XRP-II (COTS) Proxy Agent EcMsAgGenProxy Management Agent CSCI (MACI) Mgmt Support Subsystem MSS Deputy Agent EcMsAgDeputy Software Change Mgr ClearCase (COTS) Network and Enterprise Management HP OpenView NNM Tivoli Sentry Mode Mgmt Trouble Ticket aruser GUI (COTS) aradmin GUI (COTS) MsTfHTML web interface notifier GUI (COTS) notifier daemon (COTS) Submit, Query, Notify (COTS) armail daemon (COTS) UNIX sendmail Security anlpasswd (COTS) TCP Wrappers (COTS) Tripwire (COTS) SATAN (COTS) Crack (COTS) Change Request Mgr DDTS (COTS) GUI retrieves pending requests; operator enters User Profile information; server stores the User Profile Software Distribution Mgr Tivoli Courier (COTS) Accountability EcMsAcRegUserSrvr EcMsAcRegUserGUI EcMsAcOrderSrvr EcMsAcOrderGUI Network Backup/Restore Networker Server (COTS) Networker Client (COTS) ASTER E-mail Header Handler EcMsAsAddHeader.pl EcMsRemoveHeader.pl Software License Mgr FLEXlm (COTS) iFOR/LS (COTS) MsLiiFORLSMkDayLog Management CSCI (MCI) Management Logistics CSCI (MLCI) CSCI/Component Role inUser Registration Approval

  45. RETRIEVAL Deliver User Access User Registration Subscribe Manage System Monitor & Coordinate Manage Applications Logistics User Registration (Cont.) System Management Support Services (MSS) sends e-mail to the Science User confirming registration

  46. Communications Subsystem 1. Build new user confirmation message Interoper- ability ADSRV Mgmt Support Subsystem MSS Ingest Subsystem 2. E-mail notification Ingest Data Mgmt Subsystem DDICT Planning Subsystem abc def ghi jkl mno Planning Data Server Subsystem Data Processing Subsystem 0110001011 0010110010 1101011101 0110001011 Data Proc Collection DDIST SDSRV Inven- tory DB Client STMGT User Registration Confirmation Process System Management Support Services (MSS) User Registration Server creates a new user confirmation message and sends it via e-mail to the Science User.

  47. Virtual Terminal VT File Transfer Protocol FTP/KFTP DCE Services Name Service Security Service Time Service Subscription Server EcSbSubServer Sends e-mail notification confirming new user registration Crypto- graphic Manage- ment Interface CMI FTP Notification CsFtpNotify Landsat7 Gateway EcCsLandsat7Gateway Domain Name Service DNS Bulk Data Server BDS Baseline Mgr XRP-II (COTS) E-Mail Parser Gateway EcCsEmailParser Master Agent EcMsAgAgent SubAgent EcMsAgSubAgent E-Mail Services Email Server Inventory/Logistics/ Maintenance Mgr XRP-II (COTS) Proxy Agent EcMsAgGenProxy Network File System NFS Management Agent CSCI (MACI) Infra- structure Library PF, SRF, Urs, Message Passing, Event Logging, Service Locator, Failure Recovery Framework Mgmt Support Subsystem MSS Deputy Agent EcMsAgDeputy Software Change Mgr ClearCase (COTS) DAR Comm Gateway EcGwDARServer Subscription Service EcSbGui Network and Enterprise Management HP OpenView NNM Tivoli Sentry Mode Mgmt Trouble Ticket aruser GUI (COTS) aradmin GUI (COTS) MsTfHTML web interface notifier GUI (COTS) notifier daemon (COTS) Submit, Query, Notify (COTS) armail daemon (COTS) UNIX sendmail Security anlpasswd (COTS) TCP Wrappers (COTS) Tripwire (COTS) SATAN (COTS) Crack (COTS) Filecopy EcUtFileCopy EcUtFileExec Change Request Mgr DDTS (COTS) User Registration Server builds new user confirmation message MOJO Gateway EcCsMojoGateway Software Distribution Mgr Tivoli Courier (COTS) Distributed Computing Configuration Item (DCCI) Accountability EcMsAcRegUserSrvr EcMsAcRegUserGUI EcMsAcOrderSrvr EcMsAcOrderGUI Network Backup/Restore Networker Server (COTS) Networker Client (COTS) ASTER E-mail Header Handler EcMsAsAddHeader.pl EcMsRemoveHeader.pl Software License Mgr FLEXlm (COTS) iFOR/LS (COTS) MsLiiFORLSMkDayLog Management CSCI (MCI) Management Logistics CSCI (MLCI) CSCI/Component Role inUser Registration Confirmation

More Related