1 / 14

Census Bureau

Over the past three months, significant advancements have been made in the Decennial Response Integration System (DRIS) modernization. Key activities include the migration from DE101 to BE101, extensive script creation for load performance, and unit testing to ensure successful full data loading. Data retrieval criteria were updated to LCO standards, improving speed and efficiency, with substantial performance tuning and table partitioning. We achieved data loads in 1.5 hours compared to 30 hours on older machines, along with successful load testing and performance comparisons to enhance retrieval processes.

lamar
Télécharger la présentation

Census Bureau

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Census Bureau System IPT Decennial Systems Database Modernization Support DRIS Decennial Response Integration System Date: 07/30/2007

  2. Index • What have we done in past 3 months • dscmotd1 and DE101 compare • Moved to DE101 and then to BE101 • And many more … • Work done on DE101 • Scripts creation for load • Studying application for index creation • UECT creation from UNM • Writing script based on state retrieval • Unit testing for UECT

  3. Full load • Full test for UECT load- successful • Compare times with old machine • Performance and tuning • Partitioning tables • Comparing results for 2000 • Change in retrieval criteria • Retrieval criteria to LCO basis • Changed scripts • Partitioned tables based on LCO • Unit testing and full testing • Taking advantage based on CPUs • Running scripts in parallel

  4. Move DE101 to BE101 • Move scripts from DE101 to BE101 • Test all scripts for performance • Data modeling • Version difference • Production m/c • Work around/Solution • Reverse Engineering effort

  5. What have we done in past 3 months • dscmotd1 and DE101 compare • Moved to DE101 and then to BE101 • And many more … dscmotd1 BE101 DE101 • Got acceptable and best results on DE101 • Moved schema to DE101

  6. Work done on DE101 • Scripts created for • Data load • Indexes • analyzing table objects • Study application to create proper indexes • Fast data retrieval • Fast data update • Test data load and compare with old machine • Acceptable results • Data load in 1.5 hrs for whole country compare to 30 hrs on old machine

  7. UECT creation from UNM • Scripts based on state retrieval – 2000 • Unit testing • Full load testing • Full test for UECT load- successful • Compare times with old machine UNM table UECT process UNM table Retrieval based on state

  8. Performance & Tuning • Results not up to the mark • P & T was needed • Data size, too high • table partitioning UECT process Retrieval based on state

  9. Change in retrieval criteria • Data retrieval based on LCO and NOT on state based • Unit and full testing • Advantages of CPU bound operations • Scripts run in parallel UECT process Retrieval based on LCO

  10. Moving from DE101 to BE101 • Got new host • DE101 - Production • BE101 – Beta/test • Same configuration • Whole testing to be done

  11. Data Modeling Problems • Data repository was not compatible with Database version • Could not do data modeling using Oracle designer • Work around using third party tool. No stoppage of work Oracle db 10g Desktop Data modeling X Designer repository 6.x Versions not compatible DE101

  12. Work Around • Work around using third party tool. No stoppage of work Desktop Data modeling Third party tool Oracle db 10g Versions compatible Scripts creation/ execution DE101

  13. Reverse Engineering effort – not done Oracle db Desktop Data modeling DE101 Designer repository Versions compatible BE104

  14. THE END

More Related