Census Data Processing

1. Census Data Processing In BPS – Statistics Indonesia 1

2. About BPS – Statistics Indonesia • Vision: BPS is the provider of quality statistics. • A non-departmental government institution. • Directly responsible to the President of the Republic of Indonesia. • Headed by a director general. • Provide data to the government, and the public. • Assist statistics divisions of government ministries and other institutions, in developing the statistical system they need for setting up work programs and periodic reporting. • -Develop and promote standards to be incorporated in the implementation of statistical techniques and methods. • - Provide necessary services in the field of education and training in statistics. • -Establish cooperation with international institutions and other countries for the benefit of Indonesia’s statistical development.

3. Organizational Structure Director General Principal Secretary Institute of Statistics Education and Training Center Bureau of Program Management Bureau of Finance Bureau of Personnel & Legal Affairsl Bureau of General Affairs Main Inspectorate Deputy DG for Production Statistics Deputy DG for Methodo logy and Statistical Information Deputy DG for Social Statistics Deputy DG for Distribution and Services Statistics Deputy DG for Natio- nal Account and Statistical Analysis Directorate of Food Crops, Horticultural, and Estate Crops Statistics Directorate for Dev of Census & Survey Methodology Directorate of Population Statistics Directorate of Distribution Statistics Inspectorate Area I Directorate of Production Account Directorate of Social Welfare Statistics Directorate of Price Statistics Inspectorate Area II Directorate of Consumption Account Directorate of Animal Husbandry, Fishery, And Forestry Statistics Directorate of Statistical Dissemination Inspectorate Area III Directorate of Social Resilience Statistics Directorate of Financial, IT, and Tourism Statistics Directorate of Statistical Analysis Directorate of Statistical Information System Directorate of Indutrial Statistics BPS Regional Statistics Offices

4. Provincial and Regional Offices • There are • 33 BPS provincial offices • 344 BPS regency offices and • 96 BPS city offices. • they are the vertical representative offices of BPS. • At sub-district level, BPS employs a statistical staff (sub-district level Statistical Coordinator). • Technically and administratively, BPS regional offices are directly under the control of BPS-Statistics Indonesia. • The number of BPS staffs is 13,465 (as July 2008) • 1,466 persons in BPS Head Quarter (11%) • 11,999 persons in BPS-Regional Offices (89 %). 

5. Provincial Level Organization CHIEF OF BPS PROVINCIAL OFFICE ADMINISTRATION DIVISION ASSETS MANAGEMENT PROGRAM PERSONNEL INTERNAL AFFAIRS FINANCE MANAGEMENT AND LEGAL AFFAIRS SUB DIVISION SUB DIVISION SUB DIVISION SUB DIVISION SUB DIVISION DATA PROCESSING SOCIAL PRODUCTION DISTRIBUTION REGIONAL ACCOUNT AND STATISTICS STATISTICS STATISTICS AND STATISTICAL DISSEMINATION DIVISION DIVISION DIVISION ANALYSIS DIVISION DIVISION DEMOGRAPHIC CONSUMER NETWORKING AGRICULTURAL PRODUCTION AND HOUSEHOLD AND WHOLESALE AND STATISTICAL STATISTICS ACCOUNTS STATISTICS PRICE STATISTICS REFERENCE SUB DIVISION SUB DIVISION SUB DIVISION SUB DIVICION SUB DIVISION FINANCIAL DATA PROCESSING INTEGRATION MAN POWER INDUSTRIAL CONSUMPTION AND PRODUCER STATISTICS STATISTICS ACCOUNTS PRICE STATISTICS SUB DIVISION SUB DIVISION SUB DIVISION SUB DIVICION SUB DIVISION MINING, ENERGY COMMERCE STATISTICAL DISSEMINATION SOCIAL WELFARE & CONSTRUCTION & SERVICES ANALYSIS & STATISTICAL STATISTICS STATISTICS STATISTICS AND DEVELOPMENT SERVICE SUB DIVISION SUB DIVISION SUB DIVICION SUB DIVISION SUB DIVISION

6. Census in Indonesia • Population census: • in the years ending with 0. • previous population census: in 1961, 1971, 1980, 1990, 2000. • The next Population Census would be in 2010 • Agricultural census: • in the years ending with 3. • previous agricultural census: in 1983, 1993, 2003. • Economic census: • in the years ending with 6. • previous economic census: in 1986, 1996, 2006.

7. Data Processing Support for Population Census 1980, 1990 • Using Mainframe: • (NEC ACOS 500 in 1980) • (NEC ACOS 1520 in 1990) • Batch Processing (not interactive)

8. Data Processing Policy(after elimination of Mainframe) • Decentralized & Distributed Data Processing: • Encourage End-users Computing: subject matter units are entitled to process their own survey under coordination of data processing division. • Develop application in-house by HQ & decentralize operation to Regional offices. • Implement Distributed / Client-Server Data Processing Model. • IT Department (Directorate of SIS) focuses on IT infrastructures & complex DP Systems such as Censuses. 8

9. Data Processing Support for Population Census 2000 • PC based Client-Server Data Processing • Using Scanner: Kodak DS3500 • Speed: 3500 forms / hour • Use of OMR, ICR: Nestor Reader • ICR Ascent-Capture: started in 2003 (Census of Agriculture).

10. PC 2000 Data Capture Stages • Scan documents  (produce image file) • Recognize image  (produce ascii characters) • Verification & Correction  (operators decide characters ambiguous to recognize engine) • Completion  completeness check • Release  save into database

11. Experience from PC 2000 • Unexpected low quality of enumerator’s handwriting to recognize by ICR • Unable to recognize characters correctly • Rejected too many values (too high confidence level) • Accepted incorrect values (too low confidence level)  realized from tabulation output • Should check-digit help to increase accuracy ? • Ineffective learning engine • In a LAN environment using MS-SQl Server, 4 concurrent scanning processes slow down the whole DP activities consisting of 3 recognize stations, 10 correction stations,and 25 computer editing stations.

12. Experience from PC 2000 • Extra work to erase & rewrite Census Form • Powder from rubber eraser stick on Census forms & affected Scanner performance • Costly maintenance effort of Scanners which spread over all provinces • Relatively high cost (investment) for Data Capture H/W& S/W • However, Data Capture using scanner was able to shorten Data Processing Completion Time. • It was not easy to find out well motivated enumerators & Data Capture operators.

13. PC2010 Data Processing Plan Main Documents: • Short Form : L for Listing & basis for 10% sample, using data entry (DE), : Cusing scanner, complete enumeration (100%) of households& members • Long Form : S using data entry / scanner, 10% sample. Estimation: • No of Districts / Regency = 456 • No of Sub-districts = 5,901 • No of villages = 72,084 • No of Census Blocks = 658,308 (in year 2000) • No of smallest Administrative Areas (SLS) = 900,000 • No of households = 64 million • No of population = 236 million

14. Document: Batch Control

15. Document C

16. PC2010 Data Processing Plan (cont) Issues on existing Scanners: • Kodak DS3500 procured for PC 2000: 79 units • About 60 % of them need spare-parts replacement • Spare-parts are discontinued. (Dicom Singapore eager to reserve all available spare-parts to BPS) • Running out of Ascent Capture Counter (dongle) limit • Operable uncertainty for PC 2010

17. PC 2010 Data Processing Strategy • Utilize scanners with character recognition technology (OMR, OCR, ICR), focused on OMR. • Expect ICR with better Learning Engine capability. • Procurement of Scanning H/W and S/W in 2009. • Formation of 8 Data Processing Centers. • Utilize SMS facility for transmitting and accessing data, as well as monitoring data collection activities. • Utilize web-based application for monitoring data processing activities. • In 2008, all of 33 province level offices and some (34) district / city level offices are connected using Virtual Private Network (VPN). 17

18. PC2010 Data Processing Plan (cont) Formation of 8 Processing Centers

19. 8 Data Processing Centers

20. SMS / Web Based Progress Reporting & Monitoring Progress Reports Regional Offices Monitoring Headquarters Cellular Operator SMS Application Server Database Server GSM modem Internet / VPN Regional Offices Progress Reports Monitoring Headquarters

21. Thank you for your attention September 2008