As per the requirements of large-scale data processing, the total data processing work is structured over a number of functional steps. As the data processing is decentralized and a number of people are engaged in the data processing, each step of it is planned beforehand, documented in detail, concepts and definitions are explained, training workshops are organised both centrally and then locally, and mid-course discussions are also held to sort out unforeseen data problems. Hence, it is a formal system of data processing. The stages of data processing are:
i) Checking of identification and monitoring receipt position.
ii) Hot scrutiny by officers to identify errors, which are recurring in nature, committed by field
iii) Pre-data-entry scrutiny of schedules for manual checking of important fields.
iv) Data entry and 100% verification
v) Phase-I validation (Content Check): This includes preparation of error lists, checking of error list from schedules, updation of data files and insertion of records. Number of such checks for each type of schedule varies from 60 to 150. At this stage :
vi) Phase-II validation (Coverage Check) : Under this stage,
vii) Phase -III validation (Extreme value checking) : Here abnormal or suspected values or derived index are searched out and referred to for checking from the filled-in schedule.
viii) Special data checking are done by officers who are involved in tabulation. A list of doubtful cases are generated and checked by DPCs with filled-in schedules and necessary updation are in the data files.
ix) Computer-editing or Auto-correction : Under this stage,
x) Preparation of multiplier files, i.e. calculation of weighting factors for each Ultimate stage Units as per sample design.
xi) Preparation of work files, which are extracts from the data to facilitate table generation. Related tables are usually grouped together, and all the data fields required to generate those related tables are extracted into a single workfile.
xii) Tabulation of data: Typically the number of tables(As per the approved Tabulation Plan) to be prepared for each schedule varies from 70 to 200. The tables are usually generated sector x state x sex x other socio-economic category-wise. Based on the tables generated by DPD, the SDRD prepares subject wise reports which after due approval are subsequently published.
xiii) Release of unit level data with multiplier posted along with meta data for dissemination through Computer Centre immediately after the release of Key Indicators reports of the survey by SDRD.