Speaker: Viviana De Giorgi (Istat)

1. NTTS 2009 New Techniques and Technologies for Statistics 18-20 February 2009 Check, edit and imputation of the variable turnover of the Italian Business Register Speaker: Viviana De Giorgi (Istat)

2. Check, edit and imputation of the variable turnover of the Italian Business Register • Contents • Keywords • Background and motivations • The procedure • Conclusions Viviana De Giorgi – NTTS 2009 – Brussels, 18-20 February 2009

3. Check, edit and imputation of the variable turnover of the Italian Business Register Keywords administrative sources outliers detection data editing Viviana De Giorgi – NTTS 2009 – Brussels, 18-20 February 2009

4. Check, edit and imputation of the variable turnover of the Italian Business Register Keywords • ASIA • (Statistical Archive of Active Enterprises) Italian Business Register. • Vat returns • Administrative source of annual statements of VAT payments, provided by the Italian Tax Authority. • Sector Studies • Administrative source of the annual survey on business economic sectors carried out by the Tax Authority. • Istat Business Surveys • Annual Survey on Large Enterprises Accounts System (total) • Annual Survey on Small and Medium-sized Enterprises (sample) Viviana De Giorgi – NTTS 2009 – Brussels, 18-20 February 2009

5. Check, edit and imputation of the variable turnover of the Italian Business Register Background • A new need to improve the quality of the business register • In 2005, working on the data of the taxation year 2003, a new need arises: making the variable turnover available for dissemination. • In fact, until then, data from the VAT returns had been used for the ASIA units, without any processing, only for internal use. • The study implies: • analysis of the available sources and data, • analysis of the available category of enterprises, including missing values analyses • analysis of the method for imputing missing and non-correct values • Missing, zero and non-correct values are to be checked and imputed in order to obtain comparable and consistent data. Viviana De Giorgi – NTTS 2009 – Brussels, 18-20 February 2009

6. Check, edit and imputation of the variable turnover of the Italian Business Register Background Available procedures for editing and imputation of numerical data Automatic procedures for identifying invalid data, such as the Hidiroglou-Berthelot algorithm, which had not given good results in the case of the ASIA turnover. Software for editing and imputation of numerical data, such as the BANFF Sas application. Its implementation requires edits on the basis of numerical variables and in the case of ASIA we have only few variables to build effective rules for the turnover. An ad hoc procedure, which can better fit data and is the solution here proposed. Viviana De Giorgi – NTTS 2009 – Brussels, 18-20 February 2009

7. Check, edit and imputation of the variable turnover of the Italian Business Register • Available sources in Istat • Sector Studies • Annual VAT returns • Istat Business Surveys Background Viviana De Giorgi – NTTS 2009 – Brussels, 18-20 February 2009

8. Check, edit and imputation of the variable turnover of the Italian Business Register • Sector Studies • Instituted by the Tax Authority in 1996 in order to evaluate the capacity of the enterprises to produce income and knowing whether they pay taxes, they are a source of great interest as they gather a lot information on about 3.5 million enterprises. The subjects liable to Sector Studies are industrial commercial and manufacturing income owners and VAT number proprietors, with some exclusion principles (e.g. the turnover threshold and the activity type and time). • Financial information retrievable from the Sector Studies: • Volume of business • Proceeds • Agios and Income from the Sales of Goods Subject to Fixed Revenue • Other Proceeds • Other Positive Income Components • Other Operations that Produce Income Background Viviana De Giorgi – NTTS 2009 – Brussels, 18-20 February 2009

9. Check, edit and imputation of the variable turnover of the Italian Business Register Annual VAT returns Only the variable called Volume of Business is available. The definition is the same as the Sector Studies one. The VAT statement is declared by (almost) all the Italian enterprises (excluding some activities that are not subject to declaration). Istat Business Surveys These surveys provide for the variable named Proceeds, whose definition is the same as the Sector Studies variable. A part from the survey on large enterpriseswhose values are integrated in the imputation phase, Istat has only sample data on enterprise turnover, and they are used here mainly in the outcomes assessment phase. Other Istat turnover sample surveys are non taken into consideration. Background Viviana De Giorgi – NTTS 2009 – Brussels, 18-20 February 2009

10. Check, edit and imputation of the variable turnover of the Italian Business Register • Which source and variable? • Notwithstanding both the VAT returns and the Sector Studies provide for turnover time series we have chosen the variable called Volume of Business from the VAT returns to be used for imputing values and the same variable from the Sector Studies to be used as a cross check variable. • Why? • The VAT statements are declared by (almost) all Italian Enterprises whereas the Sector Studies are a partial survey. • The same variable is also in the Sector Studies source and it can be used for cross checks. • The VAT returns are less subject to laws and directions than Sector Studies. • From a statistical point of view the VAT returns missing data can be considered MAR. Background Viviana De Giorgi – NTTS 2009 – Brussels, 18-20 February 2009

11. Check, edit and imputation of the variable turnover of the Italian Business Register • The ASIA turnover imputation procedure • Characteristics • It is an ad hoc procedure. • It has been running since the taxation period 2004 after an experimental period of one year. • It combines administrative and statistical sources information, to provide non zero values for (almost) all the ASIA units. • It consists of 5 steps, strictly in sequence. • The ASIA turnover, available for all the ASIA units, is being disseminated by classes Viviana De Giorgi – NTTS 2009 – Brussels, 18-20 February 2009

12. Check, edit and imputation of the variable turnover of the Italian Business Register • The procedure • identification of the enterprises set to be checked, imputed and validated (reference set) • identification of the existence set of turnover permitted values (turnover range) • data editing through 3-year time series consistency (outliers detection) • missing zero and outliers imputation • data validation using external indicators (outcomes assessment) Viviana De Giorgi – NTTS 2009 – Brussels, 18-20 February 2009

13. Check, edit and imputation of the variable turnover of the Italian Business Register • Step 1: Reference set • All the following steps work on the enterprises active for at least six months of all economic activity sectors in the Istat business surveys with the exception of: • divisions 65* (Financial intermediation) and 66* (Insurance and pension funding): not in Istat surveys • divisions 85* ( Health and Social Work) and 67* (Activities auxiliary to financial intermediation): not liable to VAT • divisions with very few enterprises (Mining): non effective characteristics values • enterprises that have suffered a merger/acquisition event whit a non-zero turnover value • Enterprises excluded will have a value of turnover (including zero and missing values) as declared in the Volume of Business of the VAT statement. • *NACE rev. 1.1 The procedure Viviana De Giorgi – NTTS 2009 – Brussels, 18-20 February 2009

14. Check, edit and imputation of the variable turnover of the Italian Business Register • Step 2: Turnover range • ASIA previous year turnover bounds are used: the minimum and maximum values by economic activity division of the previous year data serve as a band-pass filter, determining the following sets: • set P (permitted values): contains all the values within the band-pass filter • set NP (non permitted values): consists of all the values outsides the bounds • Previous year data (checked and imputed) • are considered correct • are capable of managing the turnover short term variability The procedure Viviana De Giorgi – NTTS 2009 – Brussels, 18-20 February 2009

15. Check, edit and imputation of the variable turnover of the Italian Business Register • Step 3: Data editing for outliers detection • Use of the Sector Studies current year data as a cross check variable • Use of changes in value in the last three years of • turnover • turnover per capita • Different rules (bounds) for set P and set NP to calculate consistent and non-consistent changes in value by economic activity groups (3 digits) • Use of a three-element binary vector to identify • current year non-correct values (to be imputed) • previous year non-correct values (to be excluded in the imputation step) • Missing and zero values are beforehand considered non-correct (outlier) • definition from now on: • outlier is a zero, missing or non correct value The procedure Viviana De Giorgi – NTTS 2009 – Brussels, 18-20 February 2009

16. Check, edit and imputation of the variable turnover of the Italian Business Register • Step 4: Outliers imputation • (The large enterprises values are automatically replaced by the Istat enterprise survey ones, without any checks. Survey data are supposed to be correct. They are already checked and revised in the survey.) • The enterprises with a non correct current year value are divided into: • enterprises whose previous year data is non correct: their values are imputed with a mean value imputation • enterprises whose previous year is correct: their values are imputed with a previous year imputation The procedure Viviana De Giorgi – NTTS 2009 – Brussels, 18-20 February 2009

17. Check, edit and imputation of the variable turnover of the Italian Business Register Mean value imputation: multiplication of the number of employed persons by the average turnover per capita of the group the enterprise belongs (all data refer to the current year). The procedure Previous year imputation: multiplication of the number of employed persons by the sum of the previous year turnover per capita and the previous year turnover per capita multiplied by the change in value between the last two years. • itkis the imputed turnover in the current year • tpcxyk is the current year turnoverper capita of the activity groups (3-digit) or division (2-digits) x and the class of employed persons y (group xy) • ek is the number of persons an enterprise employs in the year k • tkis the turnover an enterprise obtained in the previous year • ∆tpcxyis the percentages turnover change in value that occurs in the group xybetween the previous and the current years Viviana De Giorgi – NTTS 2009 – Brussels, 18-20 February 2009

18. Check, edit and imputation of the variable turnover of the Italian Business Register The procedure • Step 5: Outcomes assessment • Comparing 1) Istat surveys and ASIA data • 2) ASIA previous and current year data • by economic activity and class of persons employed through: • by group absolute differences and/or ratios of the number of enterprises • by group absolute differences and/or ratios of the number of persons employed • by group differences of the amount of turnover between ASIA and the Istat surveys, before and after the imputation process • by group amount of imputed turnover and percentages of imputed enterprises, for both all the values and only missing and zero values • by group average values, before and after data imputation • Index 1 and 2 could be quite different from zero as the enterprise survey universe is an expansion by mean of sample coefficients; index 3, 4 and 5 are useful to understand the data quality improvement. Viviana De Giorgi – NTTS 2009 – Brussels, 18-20 February 2009

19. Check, edit and imputation of the variable turnover of the Italian Business Register • Conclusions • Every year: • about 3% of data are missing or zero values (the percentage decreased drastically in 1999) • about 2% of data are non correct values • only very few enterprises register a previous year non correct value (to not use in the previous year imputation) • few ad hoc studies are necessary, and it depends on the available data and the reference year (e.g. Aeolian power enterprises in the year 2004) • The procedure preserves time series consistency and comparisons, punctual data are available for internal uses, e.g. National Accounts and surveys, macro-data dissemination is possible by classes of turnover. Viviana De Giorgi – NTTS 2009 – Brussels, 18-20 February 2009

20. Check, edit and imputation of the variable turnover of the Italian Business Register • Future developments • As the activities liable to the Sector Studies has been increasing year by year, above all due to the inclusion of more activities and the rising of turnover thresholds, we arethinking about rebuilding the process, substituting the Volume of the business with the Proceeds coming from the Sector Studies, in order to: • make the use of Sector Studies information more effective • Improve integration of the VAT returns and the Sector Studies • make the ASIA turnover more comparable with Istat surveys data • build up an effective imputation for economic activities non liable to VAT Viviana De Giorgi – NTTS 2009 – Brussels, 18-20 February 2009

21. Check, edit and imputation of the variable turnover of the Italian Business Register REFERENCES Bernardi A., Cerroni F., De Giorgi V. (2008). The Tax Authority Source as an example of the use of an administrative source as a statistical one. Contributed Paper of IAOS Conference on Reshaping Official Statistics. Shanghai, 14-16 October 2008. Hidiroglou, M.A., Berthelot, J.-M. (1986). Statistical Editing and Imputation for Periodic Business Surveys. Survey Methodology, vol. 12, n. 1, 73-83. Italian DPR n. 633/72 on VAT returns (Italian). Sector Studies filling in instructions. Document retrievable on the Italian Tax Authority Internet site: http://www.agenziaentrate.gov.it (Italian). Statistics Canada - BANFF Support Team (2005). Functional Description of the Banff System for Edit and Imputation System. Quality Assurance and Generalized Systems Section Technical Report. TUIR Italian consolidated act on income taxes (Italian). VAT returns filling in instructions. Document retrievable on the Italian Tax Authority Internet site: http://www.agenziaentrate.gov.it (Italian). Viviana De Giorgi – NTTS 2009 – Brussels, 18-20 February 2009

22. Thank you for your attention! for further information: viviana.degiorgi@istat.it