14  INDIAN STATISTICAL SYSTEM

14.1                     Historical Perspective

14.1.1                The foundation of the statistical system in India was laid down by the British administration. The Provincial Governments were required to publish the relevant statistics in their annual administration reports. They, in turn, depended upon the district offices. These statistics covered a wide range of subjects. The forms for this information were later made uniform, and the first Statistical Abstract of British India (1840-1865) was based on such information provided by the Provinces. One of the notable contributions was the publication, in the first half of the nineteenth century, of District Gazetteers. Several Commissions and Committees appointed by the Government of India for studies of specific fields also recommended that the Provinces should create institutions to collect statistics in the relevant fields. Thus, the Indian Industrial Commission (1916-1918) recommended that a Department of Industries should be created in the Provinces, with representatives throughout the Province to collect information on industries. Later the Royal Commission on Agriculture in India (1924-1925) pointed out that not only should the Provinces be self-sufficient in the field of statistics, but also that there should be a large Central Organisation. Later, the Famine Enquiry Commission (1945) suggested the appointment of qualified Statistical Officers at Provincial Headquarters to assist the Director of Agriculture.

14.1.2                  Statistics in India thus developed in the Provinces expanding in scope to cover the fields of agriculture, industries, civil supplies (during the World War II), education, forestry, labour, cooperation, health and vital statistics. The organisational arrangement for collecting and publishing statistics, logically consistent with the organisation of Government administration and with the consequent delegated responsibilities and functions, has remained the same up to the present.

14.1.3                The first significant development in the pre-independence era was the constitution of a Statistical Committee (1862) for the preparation of forms to collect statistical information on different subject areas. This led to the publication entitled Statistical Abstract of British India in 1868. This publication was based on the returns of the local administrations and contained useful statistical information for all the British Provinces, and became an annual feature till 1923.

14.1.4                   Following the recommendations of the Indian Famine Commission, Agriculture Departments were opened in 1881 in various provinces inter alia for collection of Agricultural Statistics, while the work of coordination in the collection of Agricultural Statistics by the Provinces was vested in the Department of Agriculture. The first publication on the subject, Agricultural Statistics of British India, was brought out in 1886.

14.1.5                  A Statistical Branch was established in 1862 in the then Finance Department of the Government of India. In 1895, the Statistical Branch was converted into a full-fledged Statistical Bureau embracing subsequently, within its function the task of dissemination of commercial intelligence in 1905. Functions and activities of the Bureau were carried out through two well-defined wings namely, Commercial Intelligence and Statistics putting both under an organisation entitled Department of Commercial Intelligence and Statistics headed by the Director General. The Director General of Commercial Intelligence and Statistics until 1914 was responsible for the compilation and publication of almost all the principal statistical information on demography, crop production and prices, rainfall, industrial production, education, health and hygiene, mining, roads and communications, and other subject matters. In April 1914, a separate Directorate of Statistics came into being. Subsequently, the Directorate of Statistics and the Commercial Intelligence Department were merged into a single organisation, which was renamed as the Directorate of Commercial Intelligence and Statistics in January 1925.

14.1.6                  The first complete Population Census was conducted in 1881 on a uniform basis throughout the country. Since then the census is being conducted regularly after every ten years.  For this purpose, a Census Commissioner was appointed by the Government before each census assisted by Provincial Superintendents and District Census Officers.  Only in 1948 following a Census Act, a permanent Office of the Registrar General and Census Commissioner was created.

14.1.7                  In 1925, the Economic Enquiry Committee was set-up to enquire into ‘the question of adequacy of the statistical data available and the desirability and possibility of supplementing it, and of undertaking an economic enquiry’. The Committee recommended that the Central and Provincial Governments should come under the supervision of one central authority that would act as the adviser to the Government in all statistical matters. The Committee supported the placing of the entire statistical organisation on a statutory basis by enacting a Census and Statistics Act.

14.1.8                The development of statistics as an essential part of Government administration compartmentalised the content of statistics in many sectors and fields according to the various Government departments, which dealt with them individually. However, the administrators were fully aware that if all these statistics were viewed as a single body of information, they would create greater and better knowledge about the conditions of life of the Indian people than what they could convey when viewed in isolation. The Indian Economic Enquiry Committee (1925) recommended the establishment of a Central Statistical Bureau, along with similar Provincial Statistical Bureaux, whose “aim was to provide a common purpose and a central thinking office on the subject of Statistics”. It also recommended legislation to place the whole Statistical Organisation on a legal basis and thus to ensure or facilitate the collection of current economic data from individuals and firms. The Government of India did not accept these recommendations.

14.1.9                 The Committee appointed by the Government of India in 1934 under Messrs. Bowley and Robertson, for facilitating a further study of economic problems in India, was required, by one of its terms of reference, to make recommendation about the organisation of a Central Statistical Department. The two experts were clear that there should be in each major Province a whole time Statistician who would cooperate with the Central Director of Statistics and who would be as nearly independent of departmental control as administrative requirements permitted. The creation of the Central Economic Intelligence Organisation, under the Economic Adviser, and the emergence of the Department of Commercial Intelligence and Statistics were the results of these recommendations. Similar developments followed in the Provinces and the United Provinces Government was the first to set up a Department of Economics and Statistics in 1942. The Government of Bombay followed by the establishment of its Bureau of Economics and Statistics in 1946. Only after India became independent did the Government of India establish a Central Statistical Unit (1949), which was later (1951) converted into the Central Statistical Organisation (CSO) and the Department of Statistics, which constitute presently the Statistics Wing of the Ministry of Statistics and Programme Implementation.

14.1.10               The outbreak of the War in 1939 gave a fillip to the development of statistics to meet the requirements of the Government. In 1945, the Government of India set up an Inter-Departmental Committee with the Economic Adviser to the Government of India as Chairman to consider the statistical material available and to make recommendations for filling up of the gaps, and for improvement in the existing organisations. Among the organisational recommendations was a scheme coupled with the formation of a Central Statistical Office for coordination, the institution of a statistical cadre, establishment of Statistical Bureaus at the Headquarters of State Governments and the preparation of overall statistics for the entire country.

14.1.11               Professor P.C. Mahalanobis, who is regarded as a pioneer in both theoretical and professional statistics, was appointed as the first statistical adviser to the Cabinet, Government of India in January 1949. He was the architect of the statistical system of independent India. Professor P. V. Sukhatme, as Statistical Adviser to the Ministry of Agriculture, was responsible for the development of Agricultural Statistics.

14.1.12               The coming of the era of developmental planning in India, gave significant impetus to the development of statistics. Important phases of this development are enumerated below:

(a)                A nucleus statistical unit was set up at the Centre in the Cabinet Secretariat in 1949. This unit was developed later on in 1951 into the Central Statistical Organisation (CSO). The main responsibility assigned to the CSO was to bring about coordination of statistical activities among various statistical agencies in the Central Government and of Statistical Bureaus of State Governments, which was set up for similar coordination of activities of statistical agencies at the State level.

(b)                A National Income Committee was appointed in 1949 to work out a system for reliable estimation of national income.

(c)                The National Sample Survey (NSS) came into being in 1950 to collect information through sample surveys on a variety of socio-economic aspects.

(d)                In 1954, the National Income Unit was transferred from the Ministry of Finance to the CSO and a new Unit for Planning Statistics was set up.

(e)                In 1957, the subject of Industrial Statistics was transferred from the Ministry of Commerce and Industry to the CSO.

(f)                 In April 1961, the Department of Statistics was set up in the Cabinet Secretariat and the CSO became a part of it.

(g)                In 1972, a Computer Centre in the then Department of Statistics was set up.

(h)                In 1973, the Department of Statistics became a part of the Ministry of Planning.

(i)                  In February 1999, the Department of Statistics and the Department of Programme Implementation were merged and named as the Department of Statistics and Programme Implementation under Ministry of Planning and Programme Implementation.

(j)                  In October 1999, the Department of Statistics and Programme Implementation was declared as the Ministry of Statistics and Programme Implementation (MoS&PI).

14.1.13               The Indian Statistical Institute (ISI) was registered on 28 April 1932 at Calcutta as a non-profit-distributing learned society under the Societies Registration Act, 1860, with Professor P.C. Mahalanobis as its founder Director. This was set up to carry out research, teaching, training and project activities, and it gradually became an important part of the statistical system of India, through its pioneering work on large-scale sample surveys, design of agricultural experiments, statistical quality control, planning for national development and use of electronic computers in statistical work. By an Act of Parliament, the Institute was declared as an "Institute of National Importance" in 1959 and the right to hold examinations and award degrees and diplomas in Statistics was conferred on it. 

14.2                     Present Indian Statistical System: Organisation

14.2.1                  The Indian Statistical System presently functions within the overall administrative framework of the country. The Indian federal structure has influenced the organisation of the statistical system as well. The division of administrative functions between the Government of India and the State Governments is on the basis of the subject classifications under the Union, State and Concurrent Lists as detailed in the Constitution of India. At the Centre, the responsibilities are further divided amongst the various ministries and departments, according to the Allocation of Business Rules, 1961 that are amended from time to time. The collection of statistics on any subject generally vests in the authority (Central Ministry or Department or State Government Department) that is responsible for that subject according to its status in the Union, State or Concurrent Lists. By and large, the flow of statistical information emanates from the States to the Centre except in cases where the State-level operations are an integral part of Centrally- sponsored schemes or data are collected through national sample surveys.

Statistical System at the Centre

14.2.2                  The collection of statistics for different subject-specific areas, like agriculture, labour, commerce, industry, etc. vests with the corresponding administrative ministries. More often than not, the statistical information is collected as a by-product of administration or for monitoring the progress of specific programmes. Some of the ministries, like Agriculture, Water Resources, Health, etc. have full-fledged statistical divisions, while most others have only a nucleus cell. Large-scale statistical operations like the Population Census, Annual Survey of Industries, Economic Census, etc. are generally centralised, and these cater to the needs of other ministries and departments, as well as State Governments. In important ministries, officers of the Indian Statistical Service (ISS) and subordinate statistical staff perform the statistical functions. The Central Statistical Organisation (CSO) in the Ministry of Statistics and Programme Implementation (MoS&PI) is the nodal agency for a planned development of the statistical system in the country and for bringing about coordination in statistical activities among statistical agencies in the Government of India and State Directorates of Economics and Statistics.  Further details about the coordinating role of CSO along with its other activities have been given in the Report elsewhere.

Statistical System in the States

14.2.3                  The Statistical System in the States is similar to that at the Centre. It is generally decentralised laterally over the Departments of the State Government, with major Departments, such as, agriculture or health, having large statistical divisions for the work of departmental statistics. At the apex level is the Directorate (formerly Bureau) of Economics and Statistics (DES), which is formally responsible for the coordination of statistical activities in the State. The DESs have large organisations at the headquarters, with statistical offices in the districts and, in some cases, in the regions of the State. The statistical activity of the DESs is more or less uniform. They publish statistical abstracts and handbooks of the States, annual economic reviews or surveys, district statistical abstracts, and State budget analysis; work out the estimates of the State Domestic Product and Retail Price Index Numbers and engage in such other statistical activities as is relevant to the State. Most of them participate at least on a matching sample basis in the national Sample Survey Programme, and some of them carry out an Annual Survey of Industries for factories not covered by the ASI of the NSSO. Generally, the States do not have a common statistical cadre.

System Flow

14.2.4                  The flow chart given below attempts to depict a simplified version of the present Indian Statistical System, the flow of Administrative Statistics, and the links between different statistical offices and the strength of those links. The unshaded part of the chart shows the system that existed long before the State Directorates of Economics and Statistics and the Central Statistical Organisation (CSO) at the Centre were created, and which exists even now. It is a system built upwards from district offices of formerly Provincial and now State Government departments, to the level of these departments, and from there to the corresponding ministries at the Centre. This is its true representation: a collection of State-level systems forming a National system. In the current context, making the National system the starting point, from the perspective of an inverted view the system can be described as laterally decentralised among the Ministries of the Government of India (GOI), and in every one of them, vertically decentralised, between the Centre and the States. The bond between the State departments and the Central Ministries in the field of statistics has always been strong.

14.2.5                  The main features of the Indian Statistical System can be thus summarised as:

(a)                The Administrative Statistics System is its major component;

(b)                It is laterally and vertically decentralised;

(c)                In it, not only data collection but also compilation, processing and preparation of results are carried out by the States for most of the sectors; and

(d)                It is the State-wise results, which flow to the Centre, and statistics at the all-India level are obtained as the aggregates of State-level statistics.

Flow Chart 14.1:  Indian Statistical System 

                 
  Central Statistical Organisation (CSO)       Statistical Units (SUs) of GOI Ministries
       
       
  < < <
                           
    ^   ^         ^ ^

SU of GOI Ministry

I

 

 

SU of GOI Ministry

II

 

   

^

  ^         ^ ^
   

^

  ^         ^ ^
   

^

  ^         ^ ^
   

^

  ^         ^ ^ ^ ^ ^ ^
    ^   ^         ^ ^ ^ ^ ^ ^
   

^

  ^         ^ ^ ^ ^ ^ ^
   

^

  ^         ^ ^ SU of State Govt. Dept. I SU of State Govt. Dept. II
    ^   ^         ^ ^
   

^

  ^         ^ ^
   

^

  ^        

^

^

  State Directorates of Economics & Statistics(DES)       Statistical Units (SUs) of State Govt. Departments  

^

        ^      
        ^ ^ ^
        ^ ^ ^
  < < < ^ ^ ^
        ^ ^ ^
                  District offices of Dept. I District offices of Dept. II
> > > > Primary Flow of Administrative Statistics
> > > > Subsequent flow of Administrative Statistics
> > > > Strong Coordination
        Weak Coordination

14.2.6                  The question of lateral coordination, among Ministries at the Centre and among the Departments in the States, was not much of an issue when statistics were compartmentalised. It is a later-day need when, for treating all Government statistics as one subject, Central statistical offices were created at the Centre (the CSO) and in the States (the State Directorates of Economics and Statistics). When a single office was charged with the responsibility of bringing together all statistics relating to the country, and of examining them from the perspectives of quality, timeliness, accuracy and other desirable merits which statistics should possess, a new situation was created which, although it did not make the Ministry strictly answerable or accountable to the CSO for its statistics, required it at least to share information on the methodology and procedures of collecting those statistics with an office outside of itself. This created a new requirement, that of effective lateral coordination between the CSO on the one hand and the ministries on the other, for the system to function satisfactorily. The same situation was created in most States where, as the chart assumes, the system is fully laterally decentralised.

Functions of the Statistics Wing, Ministry of Statistics and Programme Implementation

14.2.7                  The Statistics Wing of Ministry of Statistics and Programme Implementation (MoS&PI) consists of the Central Statistical Organisation (CSO), National Sample Survey Organisation (NSSO), and Computer Centre. This is the apex body in the official statistical system of the country. Its responsibilities, as per Allocation of Business Rules, are the following: 

(a)                Act as the nodal agency for planning an integrated development of the statistical system in the country.

(b)                Coordination of statistical work with a view to identifying gaps in data availability or duplication of statistical work in respect of the departments of the Government of India and State Statistical Bureaus (SSBs) and to suggest necessary remedial measures.

(c)                Lay down and ensure maintenance of norms and standards, in the field of statistics, involving concepts and definitions, methodology of data collection, processing of data and dissemination of results.

(d)                Advise the Departments of the Government of India on statistical analysis of data.

(e)                Prepare national and regional accounts as well as publication of annual estimates of national product, Government and private final consumption expenditure, capital formation, saving, estimates of capital stock and consumption of fixed capital, as also State-level gross capital formation of supra-regional sectors and to prepare comparable estimates of State Domestic Product (SDP) at current prices.

(f)                 Compile and release the Index of Industrial Production (IIP) every month in the form of `quick estimates'; conduct an Annual Survey of Industries (ASI); and provide statistical information to assess and evaluate the changes in the growth, composition and structure of the organised manufacturing (factories) sector.

(g)                Organise and conduct periodic all-India Economic Census and Follow-up Enterprise Surveys.

(h)                Conduct large scale all-India sample surveys for creating database needed for studying the impact of specific problems for the benefit of different population groups in diverse socio-economic areas such as employment, consumer expenditure, housing conditions and environment, literacy levels, health, nutrition, family welfare, etc.

(i)                  Examine the survey reports from a technical angle and to evaluate appropriate sampling design including survey feasibility studies in respect of surveys conducted by the National Sample Survey Organisation (NSSO) and other Central Ministries and Departments.

(j)                  Provide an in-house facility to process data collected through various socio-economic surveys and Follow-up Enterprise Surveys of the Economic Census conducted by the National Sample Survey Organisation and the Central Statistical Organisation.

(k)                Disseminate statistical information on various aspects through a number of regular or ad hoc publications to Government, semi-Government, or private data user and agencies; and dissemination of data, on request, to United Nations Agencies like the United Nations Statistical Organisation, Economic and Social Commission for Asia and the Pacific and International Labour Organisation; and other international agencies.

(l)                  Give grants-in-aid to registered Non-Governmental Organisations and research institutions of repute for undertaking special studies or surveys, printing of statistical reports, and finance seminars, workshops, or conferences relating to different subject areas of official statistics.

(m)              Function as the Cadre Controlling Authority and deal with the centralised aspects of managing the Indian Statistical Service including all matters pertaining to training, career planning and manpower planning.

(n)                Ensure the functioning of the Indian Statistical Institute in accordance with the provisions of the Indian Statistical Institute Act, 1959 (57 of 1959).

Central Statistical Organisation

14.2.8                With a view to coordinating statistical activities of the different ministries of the Government of India and the State Governments and the evolving of statistical standards, the CSO was established in May 1951. The responsibilities of CSO include coordinating statistical activities and liaison with the Central Government Departments, State Governments and International Agencies; preparation of national accounts; conducting Annual Survey of Industries, Economic Censuses and their Follow-up Enterprise Surveys; constructing IIP and consumer price indices for urban non-manual employees; compiling Social Sector Statistics; imparting training in official statistics; formulating a Five Year Plan programme relating to development of statistics in the States and Union Territories; disseminating various statistical information including those relating to social and environment statistics; undertaking periodic revision of National Industrial Classification, etc. The CSO is also responsible for periodically conducting the Conference of Central and State Statistical Organisations.

14.2.9                The Director General (post lying vacant since September 1997) heads the CSO, a post that has recently been upgraded to the level of Special Secretary to Government of India. Three Additional Director Generals, four Deputy Director Generals and a team of other officers and supporting staff assist him in his work. The CSO is organised into the following major Divisions: National Accounts Division, Industrial Statistics Division, Economic Census Division, Social and Miscellaneous Statistics Division and Training Division.

14.2.10               Most of the activities of the CSO have been critically discussed and recommendations for improvement given elsewhere. The remaining activities – Coordinating role, Economic Census and Follow-up Enterprise Surveys of Economic Census – are discussed below.

Coordinating role of the CSO

14.2.11            One of the major responsibilities of the CSO is to act as the nodal agency for planned development of the statistical system of the country. The CSO is entrusted with the responsibility not only to coordinate the statistical activities of the Government of India and State Directorates of Economics and Statistics (DESs) but also to lay down and maintain norms and standards in the field of statistics. Though the CSO has no legal authority to enforce standards and coordination, the work is done through institutional arrangements like inter-departmental meetings of Working Groups, Technical Advisory Committee on various subjects, Standing Committee, etc. in the case of Central Ministries. Coordination with States was through appointed liaison officers, meetings with high-level statistical coordination committees of the State Governments and also through a Conference of Central and State Statistical Organisations (COCSSO). 

14.2.12              At the suggestion of the Standing Committee of Departmental Statisticians, the first Joint Conference of Central and State Statisticians was held in December 1951. During the period 1951-60, nine such sessions were held. In the year 1961, the Government of India constituted the Central Technical Advisory Council on Statistics to serve as an advisory body for the Department of Statistics constituted during the same year. It was also decided that the Annual Conference of Central and State Statisticians should be designated as the Central Technical Advisory Council on Statistics from its tenth session held in December 1961. During the period 1961-71, this Council had met only twice. In the year 1971, the above Council was renamed as the Conference of Central and State Statistical Organisations (COCSSO) with an objective to hold the COCSSO once in two years. The first meeting of the COCSSO was held in September 1971. During the period September 1971 – January 1992, ten such conferences were held.  After a long period of inactivity, during October 2000, the COCSSO was organised with a limited purpose to mostly facilitate the members of the present Commission to interact with the representatives of the States, Union Territories and Central Ministries and Departments for assessing their views on various issues identified by the Commission.

14.2.13            As per the recommendations of the Kripa Narain Committee as early as 1980, a National Advisory Board on Statistics (NABS) was established in 1982. However, this mechanism could not succeed primarily because of official apathy and a lack of legal or constitutional backing, as the decision taken in this forum has no binding on any agency (see Para 14.3.13 for further details).

14.2.14                The COCSSO provided a forum for exchange of views and experiences concerning the development of statistical activities in the country. However, this forum was not used; meetings were not held during February 1992 to September 2000. The other coordination mechanisms had also withered away.

Recommendations on the Coordinating Role of the CSO

14.2.15            The Commission recommends that:

(i)                  The post of Director General (DG), Central Statistical Organisation (CSO), should be immediately filled up.

(ii)                The practice of organising the Conference of Central and State Statistical Organisations once in two years should be revived.

(iii)               A unit in the CSO should be created for proper documentation of various papers connected with the meetings of the various technical committees. The CSO should publish periodically technical monographs on different subjects piecing together recommendations relating to various issues. 

(iv)              The CSO, as a coordinating agency, should maintain a pool of eminent experts in different subject areas. This would be useful for getting comments on various statistical matters quickly and also for constituting various committees and working groups on technical matters.

(v)                The CSO should acquire membership of the important national as well as international statistical associations and institutions and participate in the conferences arranged by them so as to develop expertise in the field of official statistics.

(vi)              In order to gain exposure to the methodologies adopted by other countries for compiling official statistics, arrangement should be made to get publications, journals and related documents brought out by the US Bureau of Census, Statistics Canada, Australian Bureau of Statistics, etc.

Economic Census and Follow-up Enterprise Surveys

14.2.16               From the early fifties, information about unorganised economic activities is being collected through sample surveys. To improve the frame for such surveys, a periodic census of all enterprises (including those in the organised sector) was considered necessary.  Accordingly, a scheme on Economic Census (EC) and Follow-up Enterprise Surveys was launched by the CSO in 1976 with two main objectives namely,

(a)                To provide a frame (list) from which samples could be drawn for collecting detailed information;

(b)                To provide at regular intervals some basic information on location, type of activity, nature of operation, etc. about the enterprises in the country and number of persons employed by them at the level of villages, and census enumeration blocks, i.e. EBs in case of large villages and urban areas (except in the recent EC in which Urban Frame Survey blocks of NSSO were used).

14.2.17             So far four ECs have been conducted during 1977, 1980, 1990 and 1998, respectively. While the First (1977) and the Fourth (1998) ECs were independently conducted by the CSO, the Second (1980) and the Third (1990) were integrated with the house listing operations of the 1981 and 1991 Population Censuses, respectively.

14.2.18             The EC, 1977 covered only non-agricultural establishments (i.e. enterprises employing at least one hired worker on a fairly regular basis). On the other hand, each of the other three subsequent ECs covered both agricultural (other than crop production and plantation) and non-agricultural activities and they covered establishments as well as own account enterprises (i.e. enterprises employing no hired worker on a fairly regular basis – i.e. run with the help of household members only).

14.2.19               The EC is financed, planned and technically supervised by the MoS&PI but the fieldwork, data entry and preparation of State-wise results are done through the States.

14.2.20                Each of the ECs collected certain basic items of information about the enterprises namely, location of enterprise, nature of operation, description of activity, type of ownership, social group of owner, whether power or fuel used and number of workers in the enterprise. However, there were inclusions of certain additional items in some of these ECs, for example, agency of registration in the First and Fourth ECs and value of annual output or turnover or receipt in the First EC.

14.2.21               Using the EC frame, Follow-up Enterprise Surveys of all non-agricultural enterprises under the following sectors have been conducted at periodic intervals

(a)                Manufacturing,

(b)                Mining and quarrying,

(c)                Trade,

(d)                Hotels and restaurants, transport,

(e)                Storage and warehousing,

(f)                  Other services.

14.2.22               Until recently, the fieldwork for all the Follow-up Enterprise Surveys was carried out by the Field Operations Division of the NSSO under the technical supervision of the CSO (Economic Census Division) except in the case of manufacturing and trade (own account enterprises and non-directory establishments), which had been the responsibility of the NSSO. In 1998-99, a combined survey of all non-agricultural activities barring a few was taken up by the Department of Statistics without the approval of the Governing Council of the NSSO. After 1998-99, the entire survey work for all non-agricultural sectors has been spread over different years and been conducted by the NSSO, with the approval of the Governing Council.

14.2.23               Processing of the data from the Follow-up Enterprise Surveys was the sole responsibility of the Computer Centre till 1993-94. Afterwards, the Data Processing Division of NSSO took over the data processing work of surveys planned by the NSSO. 

Deficiencies

14.2.24               The difficulties involved in conducting the Fourth EC have been enumerated in detail in the Report of its Monitoring Group. The difficulties clubbed under four major heads are:

Pre-Field Operations: Lack of statutory backing; failure to launch the scheme in time; inadequate time allowed for pre-field operations; delayed finalisation of rural and urban directories; absence of skeleton staff for undertaking the preliminaries; delay in creation and filling up of the posts sanctioned at the State Directorates of Economics and Statistics (DESs); lengthy schedules and instructions; delayed printing of schedules and instructions in local languages; difficulty in engaging primary enumerators; inadequate honorarium to enumerators and supervisors; inadequate training of enumerators and supervisors; and lack of public awareness and participation.

Field Operations: Non-completion of fieldwork within stipulated time period of one month; inadequate supervision; and difficulty in identification of Urban Frame Survey blocks.

Post-Field Operations: Delay in preparation of provisional results by most of the States; problems faced in the use of uniform software developed by the Computer Centre for data entry and processing; delay in identification and finalisation of data entry agency in some of the States; delay in finalisation of detailed results due to discrepancies in the data files and tabulated results received from the States; and difficulties in comparison of lower geographical level results over different Economic Censuses on account of re-alignment of boundaries.

General Administrative Problems: Delay in release of funds to the DESs by the respective State Governments; and lack of facility for quick communication with States and Union Territories.

Recommendations on the Economic Census and Follow up Enterprise Surveys

14.2.25               The Commission recommends that:

(i)                  In future the operation of EC should be part of house listing operations of the decennial Population Census. For this purpose, the census EBs and UFS blocks have to be linked in a manner such that each UFS block is made up of a number of complete census EBs. Since this involves close cooperation between the MoS&PI and the Office of the Registrar General and Census Commissioner, the issue should be taken up immediately. If for this purpose, changes are required in the Census Act, they should be made.

(ii)                Proper remedial measures should be taken to improve the quality of data in the Follow-up Enterprise Surveys as the existing sampling design and method of data collection have failed to provide satisfactory data. The responsibility of designing all the Follow-up Enterprise Surveys should rest with the NSSO.

Business Register: Recommendations

14.2.26               The Commission recognised the need for conducting a Survey of Non-Manufacturing Industries (SNMI) covering “bigger” units, other than those in the public sector. The “smaller” units are to be covered through Follow-up Enterprise Surveys of the Economic Census. To facilitate the SNMI, it is necessary to develop a list of such units for being used as the frame for sampling. The Commission feels that a beginning in this direction could be made by combining together available lists of such units – like the frame of the Fourth Economic Census, ASI frame, lists maintained by the Department of Company Affairs, Development Commissioner of Small Scale Industries, Municipalities and Sales Tax Departments of State and Union Territory Governments, various Associations or Chambers of Commerce, etc. The list so developed should contain in addition to identification details of the units, other relevant items of information to make it a so-called Business Register.

14.2.27             The methodology for combining the lists, criterion for “bigness”, items of information to be included, procedure for updating the list are all complex issues, which should be left to a technical group for examination. To what extent this work can be done with the involvement of State Governments and local agencies should also be examined in consultation with the State DESs.

14.2.28               An approach indicated in the paper (Annexe 14.1), prepared by the Secretariat of the National Statistical Commission, could be used as a starting point by the technical group. The Commission would however like to emphasise that after the project has been formulated, it should be implemented in phases starting with a pilot phase.  

National Sample Survey Organisation

14.2.29            The National Sample Survey (NSS), initiated in the year 1950, is a nation-wide, large-scale, continuous survey operation conducted in the form of successive rounds. It was established on the basis of a proposal from Professor P.C. Mahalanobis to fill up data gaps for socio-economic planning and policy-making through sample surveys.

14.2.30               Initially, all aspects relating to the designing of surveys, processing of data and preparation of reports were entrusted to the Indian Statistical Institute (ISI). The then Directorate of NSS in the Government of India had been responsible for carrying out the fieldwork in all areas except the State of West Bengal and Bombay City, where the fieldwork was carried out by the ISI. To get rid of inordinate delay in release of survey results, all aspects of survey work were brought under a single umbrella by setting up the National Sample Survey Organisation (NSSO) under the resolution dated 5th March 1970. However, the delays persisted till the nineties, when with effective use of modern electronic computers the problem was finally resolved. Since its creation, the NSSO has been functioning under the overall direction of a Governing Council with autonomy in the matter of collection, processing and publication of survey data, thus ensuring freedom from political and bureaucratic interference.

Current Status

14.2.31            The NSSO carries out Household and Enterprise Surveys, undertakes the fieldwork for the Annual Survey of Industries, provides technical guidance to the States in respect of the Crop Estimation Surveys besides assessing the quality of primary work done by the State Agencies in area enumeration and yield estimation, prepares the urban frames useful for selection of urban blocks for the surveys and collects price data for rural retail prices as well as selected items consumed by the urban non-manual employees required for the preparation of consumer price indices for agricultural labourers and urban non-manual employees, respectively.

14.2.32            The Director General and Chief Executive Officer (DG&CEO) heads the NSSO. He is responsible for implementing all activities of the organisation. The NSSO has four Divisions namely, the Survey Design and Research Division (SDRD), Field Operations Division (FOD), Data Processing Division (DPD), and Coordination and Publication Division (CPD), with each Division headed by an Additional or Deputy Director General. The headquarters of both the SDRD and the DPD are located at Kolkata. The DPD has Data Processing Centres at Ahmedabad, Bangalore, Delhi, Giridih, Kolkata and Nagpur. While the headquarters for the CPD are located at Delhi, the headquarters for the FOD are located at Delhi and Faridabad with a network of Zonal Offices, Regional Offices and Sub-Regional Offices spread over the country.

14.2.33             The NSS is carried out in the form of successive rounds. A unique feature of the NSS is that all the State and Union Territory Governments except the Union Territories of Andaman and Nicobar Islands, Dadra and Nagar Haveli, and Lakshadweep participate in the programme at least on an equal matching sample basis. Since its inception in 1950, the NSS has collected data on a large number of subjects of interest. Annexe 14.2 indicates the subjects covered in the NSS during last 10 years.

14.2.34             The NSS has completed its 56th Round of survey in June 2001. The subjects of enquiry were unregistered manufacture, household consumer expenditure and employment-unemployment. The fieldwork of the NSS 57th Round (covering household consumer expenditure, employment-unemployment and most of the non-agricultural economic activities other than manufacturing and trade) is in progress and is likely to be completed by the end of June 2002.

14.2.35               The activities of the NSS have been reviewed by Expert Committees from time to time. The latest one set up in September 1998 was a two-man Expert Committee comprising Professor J. Roy, Research Professor Emeritus, ISI, Kolkata and Shri S. Ramanatha Iyer, formerly Director, FOD, NSSO. The Expert Committee, in its report submitted in March 1999, made a number of recommendations on the functioning of the organisation. The Commission considered the Report of the Expert Committee. Major recommendations of this Committee along with the important views on these recommendations from some experts are summarised in Annexe 14.3. The Commission also forwarded a query to the major data users and producers on the adequacy of the subject coverage and the periodicity of data collection on different subjects in the NSS. The queries raised along with a summary of responses received are placed in Annexe 14.4.   

Strength of the NSS

14.2.36            The NSS is the largest repetitive survey operation in the world. It provides important data on various socio-economic characteristics, to meet the needs of planners, policy makers, researchers and other users.

14.2.37               The sample surveys are all scientifically designed so that great attention is paid to control of sampling and non-sampling errors. To account for seasonal variations, the survey operations (for yearlong surveys) are spread out by conducting the survey over quarterly seasons in the form of four sub-rounds.

14.2.38               The greatest strength of the NSS lies in its complete freedom from administrative and political influence, which is ensured through its autonomous Governing Council (GC) comprising academicians, professional statisticians and users.

14.2.39               Some of the landmark findings of the NSS which greatly influenced administrative decisions are:

(a)                Alarming growth of population during early fifties,

(b)                Self-sufficiency in food production in late sixties,

(c)                Pattern of landholdings,

(d)                Number of persons with different kinds of physical handicaps,

(e)                Changes in consumption pattern, etc.

14.2.40               Permanent and well-trained investigators carry out fieldwork in the NSS. This ensures maintenance of uniformity in concepts and definitions adopted for data collection. On the other hand, this may also cause occupational boredom, lead to too much familiarity with the subjects and breed complacency. 

14.2.41            The NSS provides alternative information to act as a crosscheck on the data generated on population, enterprises, agriculture and livestock through various censuses conducted by the respective agencies.

14.2.42             The Commission is happy to note that through dedicated teamwork, the NSSO released as many as thirty-one reports covering results of seven NSS Rounds, 48th (January – December 1992) to 54th (January – June 1998) during 1998 and 1999. With a clearing of the backlog of all the earlier survey results, the NSSO could start data processing for the NSS 55th Round (July 1999 – June 2000) almost concurrently with the collection of data and release key results of the NSS 55th Round in the form of four reports by December 2000, i.e. within six months after completion of the fieldwork (see Annexe 14.5 for details). A remarkable achievement indeed! 

14.2.43               The reports are now made available to any user in the form of printed publications or on floppy disks. Unit-level data are also available on floppy disks. Both are priced on a marginal cost basis. The demand for the reports is on the rise: the proceeds from sale of reports increased from about Rs. 1.64 lakh in 1998-99 to Rs. 2.19 lakh during 2000-01.  

Deficiencies

14.2.44            The scientific approach of problem solving through analytical studies and pilot experiments, for which the NSS was well known in its early days, have been given up under the pressure of day-to-day work. Survey sampling activities at the NSSO have almost been routinised. 

14.2.45            Almost from its inception, for most of its sample surveys the NSS has been using a standard sampling design: inter-penetrating, multi-stage, stratified, probability-proportional-to-size, circular systematic. There is no system of carrying out methodological studies for assessing the suitability of the sampling design and bringing about improvements. 

14.2.46               Though circular systematic sampling was an excellent innovation in the early fifties, the method has serious disadvantages. Although it allows estimation of the sampling error through inter-penetrating sub-samples, it has no way of relating the error to the sample size. It does not provide any rational way of determining, for example, the number of households, villages and urban blocks to be sampled.

14.2.47            There is no regular mechanism in the NSSO to analyse the quality of the survey results. Very seldom does the NSSO provide the users an indication of the margin of error of the statistics published by it.

14.2.48               The Governing Council (GC) of the NSSO, as an autonomous body, provides technical guidance on the conduct of the NSS. However, the Commission has noted that the GC has no role in certain activities of the NSSO like the collection of data in the Annual Survey of Industries, price collection and the work relating to compilation of Agricultural Statistics. The Commission also noted that in 1998-99, a survey of major non-agricultural activities was conducted by the Department of Statistics using all the resources of the NSSO but without approval from the GC.

14.2.49               All the resources of the NSSO are at present tied up with the regular survey work. No separate resources are available for quickly conducting an enquiry, the need for which might suddenly arise. There is no resource available to take up methodological studies including trying out innovations in survey practice.

14.2.50            Though the States collect a matching sample, the results from the State and Central samples have seldom been pooled together to provide more reliable estimates at the State level.

14.2.51            The NSSO has not been able to inform and educate the laymen about its activities and explain how these are beneficial to the public and the Government. The result has been the general indifference of the public towards the NSSO activities and consequent lack of cooperation.

14.2.52               Recently, large divergences between NSS estimates and those based on other sources have drawn critical attention of the media. Very seldom have efforts been made by the NSSO to examine the nature of such divergences and offer an explanation. Of late, an Expert Committee to review the divergences in the alternative data sets looked into some of these issues and made useful recommendations.

14.2.53            There are occasions in the recent past when the sampling design or method of data collection followed for a long time has been changed without adequate experimentation and sometimes even in the middle of a round. This has affected the quality and comparability of the survey results. A study carried out by Professor J. Roy on the consumer expenditure data collected by the NSSO in its 51st (July 1994 – June 1995) to 54th (January – June, 1998) Rounds, showed that there was no pattern in the change over time in respect of over 45 per cent of parameters while it is very reasonable to expect an approximately linear growth. This is a serious defect. Ad hoc arbitrary procedural changes in these rounds seem to be the reason.

14.2.54            For several rounds of the survey during the nineties, on the plea of economy, the existing practice of constituting technical working groups for the purpose was arbitrarily given up. This brought down the quality of the surveys as the study by Professor J. Roy has shown. 

14.2.55            Generally, determination of sample size for various States and Union Territories is solely based on the number of field investigators available. The practice followed has no statistical justification.

14.2.56            Most of the schedules canvassed by the NSS are unduly long causing respondents’ fatigue and thereby affecting the quality of the data.

14.2.57            The Commission has observed that, at times, the planning of a round of the survey has been considerably delayed. This affects the overall efficiency of the surveys and should be avoided. 

14.2.58               In the past, the NSSO’s journal, Sarvekshana used to contain many important articles and papers related to NSS. But over the years, the practice has almost stopped. At present, the journal contains mostly the survey reports prepared by the NSSO.

14.2.59               The Commission has also observed that there is an acute stagnation of a large number of staff in the organisation.     

Recommendations relating to the functioning of NSSO

14.2.60            The Commission has taken note of the pioneering role played by the NSSO in the field of conducting large-scale sample surveys on various socio-economic characteristics. It has also considered the Report of the Roy-Iyer Review Committee and the comments on it, as also the views of data users on subject coverage and periodicity of data collection in the NSS. The Commission makes recommendations under the following heads: (a) Organisational, (b) Methodological, (c) Survey Programme, and (d) Publicity and Dissemination.

14.2.61               Regarding organisational aspects, the Commission makes the following recommendations:

(i)                  All activities of the NSSO, presently excluded from the scope of the NSSO Governing Council, such as Annual Survey of Industries, Price Collection, or Agricultural Statistics, should be brought within the scope of the Governing Council.

(ii)                Highest priority should be given to develop specialised skills in the methodology of large-scale sample surveys.

(iii)               The NSSO through the Government of India should become an institutional member of the International Association of Survey Statisticians.

(iv)              The nature of the departmental journal, Sarvekshana should be changed to make it a medium of publication of technical papers based on NSS data or related to survey practice.

(v)                E-mail facilities should be made available to all offices of the NSSO and video discussion facilities between pairs of important offices.

(vi)              Steps should be taken to re-distribute total available resources, particularly investigators, among the States and Union Territories on a rational basis.

(vii)             The practice of field visits by officers of the SDRD, FOD and DPD followed up by interactive feedback sessions should be restored.

(viii)           Planning for any survey should start well in advance so that necessary requirements could be completed in time.

14.2.62               On methodological issues, the Commission recommends that a Methodological Study Unit should be set up in the NSSO to regularly undertake studies for bringing in improvements in the survey methodologies. The unit should be equipped with library and computer facilities. The requisite number of field and data processing staff should be transferred to this unit. Some topics which can be taken up urgently are listed below:

(a)                Pilot studies on the effectiveness of intensive stratification and one-by-one sampling in place of systematic sampling.

(b)                Theoretical and empirical studies on the use of time series data and information from sources outside the current survey, and to borrow strength from them to improve the precision of estimates through the technique of small area estimation.

(c)                Exploring use of the rotational sampling design for selection of first stage units for repetitive surveys to make comparisons over time more precise and cut down the cost of listing of first stage units.

(d)                Developing suitable procedures for data collection pertaining to the Trade and Services Sectors to get better estimates of value added.

(e)                Reconciliation of divergence between data from different sources and assessment of their quality.

(f)                 Ascertaining reasons as to why aggregates are usually underestimated in NSS while there is no such problem with averages and ratios.

(g)                Examining the feasibility of reducing the size of schedules canvassed by the NSS without affecting quality.

(h)                Extracting useful information from the  listing schedule.

14.2.63               On survey programme, the Commission makes the following recommendations:

(i)                  The present practice of covering various subjects with the existing periodicity should be continued. All the Follow-up Enterprise Surveys of Economic Census currently undertaken should be the responsibility of the NSSO. A ten-year programme indicating the subject coverage should be prepared and released in advance for the benefit of the users.

(ii)                There should be flexible arrangements for the inclusion of a few selected items of topical interest to the Government as an additional feature in the normal NSS programme.

(iii)               The NSSO may undertake, along with its normal programme, very short duration (monthly) surveys, each devoted to one topic of interest to any GOI Ministry, and produce results in a short period. After trying this on a modest basis, if the experiment shows that the users appreciate the scheme of such supplementary surveys, the scheme should be incorporated on a permanent basis and the necessary additional resources provided to the NSSO.

14.2.64               The Commission makes the following recommendations regarding publicity campaigns and dissemination of data:

(i)                  The commencement of a new survey and the main results of a concluded survey should be widely publicised through newspapers, magazines, radio and TV. Regular publicity campaigns should be arranged by the Government to apprise the respondents of the role played by the NSS in meeting the data requirements.

(ii)                The NSSO should identify two or three important characteristics in every round for quick release.

(iii)               All publications of survey results of the NSSO should contain an assessment of the associated errors.

(iv)              The NSSO should play an important role in apprising the Government, as early as feasible, of the changes taking place in the structure of the society, with respect to some of the important variables for which data are collected every year, like literacy level, employment-unemployment situation, sex ratio, consumption pattern, etc.

(v)                For providing an idea of the soundness of the estimates, the marginal totals of the number of reporting units should be indicated in the tables of the NSS reports for important survey characteristics.

Computer Centre

14.2.65            The Computer Centre was set up in 1967 as an office attached to the then Department of Statistics, Cabinet Secretariat, to cater to the data processing needs of the Department and other Departments in the Government of India. Since then it has effectively performed the task of building up a database for many organisations of the Government of India, and has also played an important role in imparting intensive training in systems analysis and data processing. The Computer Centre is headed by a Deputy Director General. 

14.2.66            The Computer Centre has undertaken the responsibility of processing and tabulation of NSS data right from the 27th Round (1972-73) onwards to the 50th Round (1993-94). Further, the Computer Centre undertook the summary and detailed tabulation of data for ASI 1974-75 to ASI 1992-93. The software for processing and tabulation of data of the Economic Census (EC) of 1977, 1980, 1990 and 1998 was also developed by the Computer Centre. The software developed for the Economic Census 1998 was also made available to the States for data entry, validation, processing and tabulation of State-level data. The data of different Follow-up Enterprise Surveys of the CSO have been processed at the Computer Centre. The processing of monthly price data, right from data entry to production of final consumer price index for urban non-manual employees CPI (UNME), is also being carried out at the Computer Centre. The Centre also completed the data processing work of the Time Use Survey, conducted by the CSO in collaboration with six participating States.

14.2.67            As per the ‘National Policy on Dissemination of Data’, approved by the Cabinet in January 1999, the Computer Centre has been entrusted with the responsibility for the creation and maintenance of the National Data Warehouse of Official Statistics. The Computer Centre has initiated action in this regard. Under the project, the Computer Centre will preserve data generated by various Central Ministries, State Governments and Public Sector Undertakings on electronic media, organise the data in the form of databases and provide remote access facilities to users through a network. A large volume of data, including those pertaining to various NSS rounds and Annual Surveys of Industries (ASI), has already been preserved by the Computer Centre on electronic media. These are now being disseminated to a large number of users. The users would be further benefited in the coming years if the Computer Centre starts functioning as a data-warehousing unit not only of official statistics generated by the MoS&PI but also of those generated by various other Central Ministries, State Governments and Public Sector Undertakings.

14.2.68            With effect from September 1999, the Computer Centre has also been given the responsibility for the creation, design and update of the MoS&PI’s website hosted by the National Informatics Centre. Efforts are underway to make the website bi-lingual. The Computer Centre regularly updates the material available in the website. To the extent possible, press releases are also put on the website on the same day.

14.2.69               Apart from processing monthly price data for compilation of CPI (UNME), the Computer Centre is responsible for the generation of all-India tables based on data collected through the Economic Censuses. The Computer Centre also conducts various training courses on IT for officials of Junior Certificate Courses in Statistics, software packages for middle-level Indian Statistical Service officers besides in-house training courses on new topics of IT for its officers and staff.  

Deficiencies

14.2.70            For several years in the past, one of the major and regular activities of the Computer Centre had been to tabulate the NSS data, ASI data and the data of periodic Enterprise Surveys planned by the Economic Census Division (ECD) of the CSO. But with the taking over of the tabulation work of the NSS and ASI data by the respective organisations and with the Centre now assigned the task of storing and dissemination of data, the Centre’s future course of activities needs to be properly assessed for effective utilisation of its resources.

14.2.71               Only the post of the chief of the Computer Centre (Deputy Director General) has been encadred in the Indian Statistical Service. Therefore, there is a lack of mobility and promotional avenues for other officers working in the Centre.

Recommendations relating to the Computer Centre

14.2.72               The Government has already decided to reorganise the Computer Centre as the Data Storage and Dissemination Office (DSDO) to act as the central repository of various data collected by the Government. It should be built by making use of modern data warehousing technology. A high-level technical group should be constituted immediately to work out the plan and a budget for setting up and maintaining the DSDO. As this is likely to take some time to be fully operational, the present Computer Centre should continue to take up data processing jobs for the  proposed NSO.

14.2.73               Recommendations relating to other uses of Information Technology in general are given in the section on Information Technology.

Role of the Private Sector in Statistics

14.2.74               Arising out of the trend of downsizing the Government, and spurred by considerations of economy in long-term costs and of obviating the problems of staff management, and to avoid possible bias and lack of objectivity in data collected by Government departments a view seems to be gaining ground that Governments may engage the NGOs and other reputed “private” sector organisations as official data collection agencies.

14.2.75               The other side of the argument is that the Government cannot abjure its primary duty to collect data on as many aspects of people’s life as possible, there is no guarantee that data collection by private agencies would not be subject to its own biases, that monitoring the quality of such data would be difficult, and finally, that the competence of such agencies may be doubtful.

Recommendations

14.2.76               Since the issue relating to data collection by private agencies is complex and important, the Commission recommends that the proposed NCS would formulate the necessary guidelines in this respect from time to time. Till then a committee of Central and State statisticians and experts outside the Government should go into all aspects of the question, before any data collection work is outsourced by any Government agency in India. 

14.3                     Administrative Statistical System

14.3.1                  The main sources of statistics in India as elsewhere are:

(a)                Administrative Statistics – generally collected by State Governments; consisting of statutory administrative returns and data derived as a by-product of general administration; and

(b)                Other important sources namely, censuses and sample surveys.

14.3.2                  Administrative Statistics are very much needed for effective planning of censuses and sample surveys. The state of the Indian Statistical System thus depends largely on the state of functioning of the Administrative Statistical System. In case of the system of direct data collection through sample surveys, the main failure had been in timely processing of data and release of results. But, with effective computerisation, the problem of delays in publication of results of sample surveys has, by and large, been resolved.

14.3.3                  The major failure is in the Administrative Statistical System. This system came into being as administrative information system whose essential purpose was to aid the Government Departments in the execution of their functions of implementation of different Acts, Rules and Regulations of Governments. Even when such Acts were passed by the Central Government, their implementation was decentralised through the State Government Departments and their district or other sub-offices. The statistics thus had a direct purpose of being not only of interest to but also necessary for the working of the departments. The regularity, quality, and completeness in the collection of these statistics, interwoven with the working of these departments, were thus indirectly ensured. The quality of this system is thus directly related to the interest the administrative departments take in it and the effective use they make of it. It is however a fact that strictness in the administrative functions of several departments of most State Governments is waning, resulting in a virtual neglect of the information system. The main reason for the near collapse of the Administrative Statistical System is this near total failure of the administrative machinery of the Governments. This results in (a) incomplete coverage, (b) delays in availability of information, and (c) unsatisfactory quality of Administrative Statistics. The other reason is the lack of effective coordination between different statistical agencies, especially at the Centre. Though charged with the responsibility of coordination as a nodal agency, the CSO could not effectively carry out this function and influence the other Central Ministries and State Governments to remedy the ills of their Administrative Statistical Systems.

Centralised and Decentralised Systems of Collection of Administrative Statistics

14.3.4                  Subjects such as money and finance, international trade, balance of payments, incorporated businesses, have meaning only at the all-India level. There are sectors, which straddle across more than one State and for which statistics are collected directly by Central agencies such as railways, postal services and telecommunications. Statistics on both types of subjects are collected by the Central Administrative Statistical System. All other Administrative Statistics are collected by the State Statistical System.  

14.3.5                  Administrative statistics provide information that is relevant to the working of the Departments. They serve the major purpose of aiding the Departments in the execution of their administrative functions of implementation and execution of different Acts, Rules and Regulations with which the Departments are charged. Consequently, the concerned Departments have a vital interest in the proper collection of the administrative statistics.

14.3.6                  There are other advantages too in the system of collection of statistics through the administrative set up. The collection of data by departmental agencies does not involve special costs. The collection is oriented to definite purposes, and the record and verification of information is part of administration. Departmental agencies and officials have not only good knowledge of the subject, but also of local language and local conditions, especially rural. Information collected is relevant and direct, and the respondents do not have to make calculations before answering a query. It is handled by agencies that have special knowledge of the subject Finally, there is an identifiable purpose in their data collection and they are in the best position to interpret the data. All this has lent a solid foundation to the decentralised administrative statistical system, and in turn, to the Indian Statistical System. An impression is carried by many that data collected by substantive Government departments are likely to suffer from bias. Therefore, they suggest that an independent agency should collect data to ensure objectivity. But, ignorance should not pass off as objectivity, making the solution worse than the problem. While the impression might be true for certain departments at certain times, it is easy to overstress the point as a justification for the solution suggested.

14.3.7                  As against these advantages of the decentralised Administrative Statistical System, it will be apposite to point out the disadvantages of a centralised system of sample surveys. First, it is simply an unmanageable system for a large country such as India, except when it is a sample survey system of reasonable size. Second, on the very same counts on which a decentralised system possesses advantages, the centralised system fares poorly. Third, the very process of centralisation has a natural tendency to gather momentum.

14.3.8                  A variant of this system is the one where a central agency collects data directly from the district offices of the State Government departments. When the State departments have to process the data and produce results, before they transmit them to the Centre, they are per force required to pay attention to timeliness in collection of data and their quality, and take corrective actions. But when the reporting units are to send the data directly to the Centre, which is a far distant agency, timeliness and quality of data will be affected adversely. For the States will have no responsibility for either. Worse, the State-level results, in this scheme of things, will become disaggregations of the National totals, and the States will be dependent on the Centre for State-level statistics, which are in fact in their domain.

14.3.9                  Centralisation will thus inevitably raise questions about the Centre-State relationship in the field of statistics, about their authority for collection and use of data, and the authenticity and acceptability of such data.

Failure of Administrative Statistical System

14.3.10               But over the years, the Administrative Statistical System has been deteriorating and has now almost collapsed in certain sectors. The deterioration had taken place at its very roots namely, at the very first stage of collection and recording of data, and has been reported so far in four sectors: agriculture, labour, industry and commerce. The foundation on which the entire edifice of Administrative Statistical System was built appears to be crumbling, pulling down the whole system and paralysing a large part of the Indian Statistical System. This indisputably is the major problem facing the Indian Statistical System today.

Weak Lateral Coordination

14.3.11               A similar criticism is that of the weakness of lateral coordination, which has come to be viewed as another major problem. First an explanation of its nature is required because the very perception of this as a problem depends upon the view one takes of the Indian Statistical System. It also needs an analysis mainly because the solution to this is implicitly considered also as a solution to the first major problem. A brief history of the failure of the lateral coordination is presented below.

14.3.12            As stated above, the creation of the CSO brought in the question of lateral coordination in statistics between this and other ministries. The CSO carried out its function of coordination mainly by means of the technical committees or working groups, either appointed by it, generally under the Chairmanship of the CSO Director, or by the Ministries in which case the CSO was generally represented on them. The other mode was the bi-annual Conference of the Central and State Statistical Organisations (COCSSO) organised by the CSO. The COCSSO provided a forum for exchange of views and experiences concerning development of statistical activities in the country.

14.3.13            The success of the CSO in its role as a co-ordinator depended, on the one hand, upon the degree of its initiative and ability to persuade, and on the other hand, on the co-operation of the ministries, and their willingness to participate in this process as a team and to be persuaded to accept the conclusions of the team about their statistical work. However, given the historical background, the statisticians from the ministry had a less flexible mode of thinking, being generally averse to change, and “outside” influence. The unsatisfactory experience in coordinating with the ministries led the CSO to search for an institution outside of itself and the ministries from which it could derive authority. The idea was concretised by the Committee to Review the National Statistical System (1980), referred to as Kripa Narain Committee hereafter, in its two recommendations. The first required the Government of India to formally declare by Executive Order that the Department of Statistics, to which the CSO belonged, was the “Nodal” department (for statistics) for undertaking integration of data required for Government’s decision-making, for setting and maintaining standards, and for improvement and development of statistics in all respect. The second was to create a National Advisory Board on Statistics (NABS) with the Deputy Chairman or Member in charge for Statistics of the Planning Commission as the Chairman and the Director General of CSO as the Vice-Chairman. The Government had constituted NABS in 1982 with Member in charge of Statistics and Surveys Division, Planning Commission as Chairman. Later on from 1992, the Minister in charge of Statistics was appointed as the Chairman.

14.3.14            The NABS covered Governmental statistical programmes and systems both at the Centre and in the States and was meant for providing technical guidance for policy issues (regarding statistics), for ensuring effective and better coordination in all matters concerned with statistics. However, the NABS was not effective, primarily because of lack of official, legal or constitutional support.

14.3.15               The cumulative result of different actions by the Department of Statistics was a considerable weakening of the CSO. The available institutional arrangements for coordination in the shape of Technical Working Groups on various subjects or Committees were either not continued or were terminated by the Department of Statistics. The Conference of Central and State Statistical Organisations was not convened for many years till 2000 after the NSC started working. And for long periods the post of the Director General of the CSO was not filled up. A vacuum at the top was thus created. The separate identities of the Department of Statistics, a purely administrative office, and of the CSO, a professional institution, was thus erased.  So far, the post of DG, CSO is still vacant, and during the work of the Commission, the views of the DG, CSO have not been represented before the Commission.

Modernisation Project

14.3.16               In the late nineties, the Department of Statistics was designated as the nodal agency for Real Sector data categories of International Monetary Fund’s Special Data Dissemination Standards (SDDS). Faced with this responsibility, and the perceived weakness of the statistical system, the DOS came up with a proposal for creating a Statistical Authority to have centralised control over all official statistical agencies. It also put up a project for modernisation of the Indian Statistical System, to be financed through a loan from the World Bank. The main thrust of the project is on three measures: (a) Conduct of additional surveys to cater to the need of SDDS requiring expansion of NSSO to replace the failing Administrative Statistical System, (b) Expanded use of Information Technology, and (c) Creating a Central Training Institute for all levels of statistical personnel. The project has not considered adequately the problem of strengthening of the statistical system in the States.

14.3.17            The first measure, the massive expansion of National Sample Surveys, as a quick means for data collection for GDP estimation needs a re-examination. Also, the employment of these surveys as an alternative system to the failing Administrative Statistical System will divert attention from the solution of the real systemic problem of the decentralised Indian Statistical System.

14.3.18               Without any real effort to improve collection of data or their quality, the second measure of computerisation and application of Information Technology is likely to result in quick processing of data deficient in quality, coverage, timeliness, accuracy or precision. The real apprehension is that the desirable flow of information via the route: Reporting unit Þ State Statistical System Þ Indian Statistical System will be replaced for the sake of management efficiency by a “fly-over” approach: Reporting unit Þ Indian Statistical System.

14.3.19               That there is a great need for training of statistical personnel goes without saying. However, one must distinguish here between two different types of training. The first type of training is at the operational level - on standard or routinised methods of data collection, processing and summarisation. This kind of training is best given at the work site. A large majority of statistical operatives in India are university graduates, and short in-service type of training is all that is needed. It is the second type of training – training on statistical methods to improve the practice of statistics, is much more important and necessary, particularly in the context of almost total lack of use of so called “applicable theoretical techniques” in official statistical work in India today. Centralised training is essential here, but this has to wait till a body of competent trainers is available. The immediate necessity is for training of trainers.      

Recommendations relating to the Modernisation Project

14.3.20               It is understood that the project is under review by the MoS&PI. While reviewing and reformulating it, the MoS&PI should consider the recommendations made by this Commission on the various subjects and the components of the project may be modified accordingly, if necessary.  For this, the project will have to shift its focus from expansion of sample surveys to improvement of the systemic issues of the Administrative Statistical System. Modernisation may be considered as a means for that purpose. It should also keep in view the essentially decentralised character of the Indian Statistical System and ensure that the States’ Statistical Systems are interwoven in the project architecture.

Bringing all technical-cum-supervisory positions under ISS

14.3.21               Statistical units of Central Ministries and Departments can be grouped into three main types:

(a)                Those headed by ISS officers;

(b)                Those headed by officers from other organised services;

(c)                Those headed by officers not belonging to any organised service.

14.3.22                With an organised Indian Statistical Service (ISS) in place, the continued existence of ministries and departments of types (b) and (c) above is a serious anomaly. It appears to be a historical legacy of the birth of the ISS from voluntary offering of posts by different ministries and departments. To bring about uniformity in management and strengthen coordination, the Commission recommends that all technical-cum-supervisory statistical positions in the Central Government should be brought under the umbrella of the ISS. However, this does not preclude Government from appointing in a few senior positions, professional statisticians with proven capability on an appropriate arrangement.

Filling up of Vacancies

14.3.23               The Commission notes that a number of positions at the highest and middle levels of the hierarchy in the ISS have remained vacant for long periods. For the sake of proper functioning of the statistical system, these positions must be filled up immediately.

Implementation of Above Recommendations

14.3.24               The recommendations made so far apply to problems currently faced by different statistical units of the MoS&PI under its present structure. But they do not address systemic issues. The Commission is of the view that for correcting the systemic problems of the Indian Statistical System, it is necessary to revamp and restructure the statistical system, starting from the top, by laying down a firm and lasting foundation for the Indian Statistical System. This is taken up later in this report (see paragraphs 14.5.1 to 14.5.27), where the constitution of National Commission on Statistics (NCS) is recommended. However, implementation of the above recommendations should proceed immediately and independently of the creation of the NCS. The existing institutions of National Advisory Board on Statistics (NABS), Governing Council of NSSO, Advisory committee on National Accounts Statistics and other technical Committees and institutional arrangements should also continue till the NCS is created.

Systemic Deficiencies

14.3.25               The Commission has noted that in recent years certain deficiencies of the statistical system of India have attracted serious media attention. These are:

(a)                Existence of gaps in availability of needed information;

(b)                Delays in publication of results;

(c)                Large and frequent revisions of published results;

(d)                Gross discrepancies between official statistics from different sources;

(e)                Occasional disagreement between tabulated summary results and publicly available basic data from which the summary has been produced; and

(f)                 Lack of transparency in statistical operations.

14.3.26               These deficiencies have led to a serious loss of credibility of official statistics. These have been largely due to the following reasons:

(a)                There is no policy-making and coordination body with legal authority, independent of the producers of statistics and free from covert or overt political and bureaucratic pressures that can serve as a link between the producers and users of statistics.

(b)                There is a lack of a system of assurance of the quality of the statistics that are disseminated by the system.

(c)                Over the years, the system of statutory administrative returns, which are the major sources of official statistics, have been seriously weakened.

(d)                Time tested means of coordination with various agencies of the decentralised statistical system – technical working groups, advisory committees, conference of State and Central statistical officers, etc. have withered away. The biennial Conference of Central and State statistical organisations was not held from the year 1992 till 2000.

(e)                Effectiveness of the Central Statistical Organisation, the nodal agency for coordination and standards, has been seriously affected by not filling up the position of Director General, CSO since 1997.

(f)                 There is a lack of appropriate legislation to provide a legal basis for collection of statistics, and to penalise official failures.

(g)                Absence of a human resource development policy has been responsible for a marked lack of motivation of official statisticians.

(h)                There is no spirit of innovation and research, to the extent that sample survey methodology developed in the fifties are still considered the only method to use whenever there is a demand for more information. Failure to adopt methods to “borrow strength from auxiliary information” has made dependence on larger and larger sizes of sample the only method available to cope with ever increasing demands for data. Time Series analysis, Use of statistical models, Classificatory Techniques, Hot deck methods of imputation, and other “applicable theoretical methods” have never been made use of.

(i)                  Use of Information Technology has been restricted mainly to processing survey data. It has not been used in any significant way too improve the efficiency of the statistical system at large.

(j)                  Absence of an explicit Citizen’s Charter or Mission Statement without which the expectations of the citizens from the statistical system cannot be formalised.

14.3.27               Before suggesting institutional changes necessary to solve the problems identified above, the Commission reviewed the international guidelines and the creation of apex institutions and supporting systemic arrangements by some statistically developed countries. A brief summary of this review is given below.

14.4                     International Developments in Reorganisation of National Statistical Systems

UN Fundamental Principles of Official Statistics

14.4.1                Official statistics are produced by Government for informed debate, decision-making and research both within Government and by the wider community. Objective, reliable, timely, trustworthy and accessible official statistics give people confidence in the integrity of Government and public decision-making. Hence, in producing official statistics it is important that the relevance, reliability and integrity of official statistics is maintained, and is perceived as such. For this purpose, the cooperation from respondents and providers of information has to be maintained, and information given on a confidential basis must remain so. When the Economist, a UK magazine, came out with a ranking of the National Statistical Offices (NSOs), there has been intense debate over the role and responsibility of the NSOs in improving the quality and relevance of the statistical information generated by them and in making the system more responsive to the needs of its varied users.

14.4.2                The debate led to the adoption of the Fundamental Principles of Official Statistics by the United Nations Economic Commission for Europe (UNECE) in its 47th Session at Geneva on 15 April 1992. This was later endorsed in 1994 by the United Nations Statistical Commission with some minor amendments. These principles are now a widely agreed framework for the mission of NSOs and thus also for official statistics.

14.4.3                  These Principles are as follows:

(a)                Official statistics provide an indispensable element in the information system of a democratic society, serving the Government, the economy and the public with data about the economic, demographic, social and environmental situation. To this end, official statistics that meet the test of practical utility are to be compiled and made available on an impartial basis by official statistical agencies to honour citizens’ entitlement to public information.

(b)                To retain trust in official statistics, the statistical agencies need to decide according to strictly professional considerations including scientific principles and professional ethics on the methods and procedures for the collection, processing, storage and presentation of statistical data.

(c)                To facilitate a correct interpretation of the data, the statistical agencies are to present information according to scientific standards on the sources, methods and procedures of the statistics.

(d)                The statistical agencies are entitled to comment on erroneous interpretation and misuse of statistics.

(e)                Data for statistical purposes may be drawn from all types of sources, be they statistical surveys or administrative records. Statistical agencies are to choose the source with regard to quality, timeliness, costs and the burden on respondents.

(f)                 Individual data collected by statistical agencies for statistical compilation, whether they refer to natural or legal persons, are to be strictly confidential and used exclusively for statistical purposes.

(g)                The laws, regulations and measures under which the statistical systems operate are to be made public.

(h)                Coordination among statistical agencies within countries is essential to achieve consistency and efficiency in the statistical system.

(i)                  The use by statistical agencies in each country of international concepts, classifications and methods promotes the consistency and efficiency of statistical systems at all levels.

(j)                  Bilateral and multilateral cooperation in statistics contributes to the improvement of system of official statistics in all countries.

Experiences of Other Countries

14.4.4                  The Commission examined in detail the present status of the statistical systems of several countries. The statistical system of the United Kingdom has recently been entirely restructured: this is described below. The experience of other countries is given in Annexe 14.6. 

Reforms in the Statistical System in UK

14.4.5                  The following extract from the White Paper: “Open Government” published by Her Majesty’s Government, UK, in July 1993 gives the rationale for reforms in its statistical system:

Official statistics are collected by government to inform debate, decision-making and research both within government and by the wider community. They provide an objective perspective of the changes taking place in national life and allow comparisons between periods of time and geographical areas.            

Vital as this is, open access to official statistics provides the citizen with more than a picture of society. If offers a window on the work and performance of government itself, showing the scale of government activity in every area of public policy and allowing the impact of government policies and actions to be assessed.

Reliable social and economic statistics are fundamental to the Citizen's Charter and to open government.  It is the responsibility of government to provide them and to maintain public confidence in them.

14.4.6                  In April 1996, a new Office for National Statistics (ONS) came into existence seeking to1:

(a)                Establish and gain commitment to Office of National Statistics (ONS) vision and values;

(b)                Link ONS vision to business needs;

(c)                Align individual, managerial and work team behaviours to those required by the vision;

(d)                Identify business needs of the organisation and business units;

(e)                State the process and people skills changes, required to meet those business needs;

(f)                 Promote the development and implementation of action plans to address identified individual and work team skills and competence needs;

(g)                Initiate business and work team based activities to develop and progress business and learning outcomes; and

(h)                Maintain flexibility in the delivery of skills development and learning activities.

14.4.7                  In June 2000, the United Kingdom has appointed a new Statistics Commission2, an independent, non-executive body, which will provide a check on the integrity and independence of National Statistics. It has a Chairman and seven Members, all of whom are part-time and a full-time Chief Executive. This Commission3 is independent of both ministers and the producers of National Statistics. It has its own budget and is able to commission its own activity. The Chairman and Members are appointed by the Chancellor, as Minister for National Statistics, in accordance with the Code of practice published by the Commissioner for Public Appointments. The Commission produces an annual report, which is to be laid before Parliament. It publicly advises ministers on issues of statistical integrity related to National Statistics. The role of the Statistics Commission is fundamental to the new arrangements for Government statistics.  The Commission is required to:

(a)                Consider and comment to Ministers on the programme for National Statistics, drawing on the views of users and suppliers, and will have a role in advising on the scope of National Statistics;

(b)                Comment on the quality assurance processes of National Statistics, as well as being able to carry out spot checks on departmental or other audits of National Statistics and to carry out or commission its own audits in areas of concern;

(c)                Comment on the application of the code of practice for National Statistics and other procedures designed to promote statistical integrity;

(d)                Prepare and lay before Parliament an annual report on National Statistics, incorporating its comments on the report by the National Statistician, and on how the Commission has discharged its functions.

14.5                     Recommendations on Restructuring the Indian Statistical System

14.5.1                  In the light of the experience of other countries, especially the UK to meet the problem of coordination and to ensure public trust in statistics, it is necessary to restructure the Indian Statistical System as per suggestions given in the succeeding paragraphs. The creation of a permanent and statutory National Commission on Statistics (NCS) is envisaged as a pre-requisite for improving the System. This is more or less on the lines of the UK except that the Indian NCS would have more authority in certain respects to cope with the federal structure of the Indian polity.            

National Commission on Statistics 

14.5.2                  As official statistics play a major role in assessing the performance of Government, it is important that such statistics are not only accurate, but are also trusted as such by the layman as well as by its principal users. This is ensured if a high-level policy-making body that has commensurate authority and obligations, oversees the statistical system. This is not a new concept. For instance, in the Nehruvian era, Professor P.C. Mahalanobis, an independent non-official was the Honorary Statistical Adviser to the Cabinet, had this kind of authority, but his authority was entirely personal with no formal institutional arrangement. The Governing Council of the NSSO also had complete autonomy in respect of sample surveys conducted by the NSSO. The NABS was intended to be an institutional arrangement for the statistical system as a whole, but it did not succeed because it lacked legal backing. The Commission recommends creation of a permanent and statutory apex body – the National Commission on Statistics (NCS)  – independent of the Government and responsible to the Parliament in respect of policy-making, coordination and certification of quality of Core Statistics.

14.5.3                  The NCS should be entrusted with functions broadly categorised as follows so as to:

(a)                Serve as a nodal and empowered body for all core statistical activities of the country;

(b)                Evolve, monitor and enforce statistical priorities and standards;

(c)                Ensure strong coordination through a closer linkage between statistical programming and budgeting.

 14.5.4                  The NCS will be constituted through an Act of Parliament. It will determine the areas of official statistics that are to be considered as core or critical to the functioning of the economy and accordingly prioritise the statistical activities of National Statistical System. The statistics defined under critical areas could be called Core Statistics. The Core Statistics will have the following characteristics:

(a)                They should be of national importance;

(b)                It will be mandatory for the Governments at all levels to collect and disseminate them;

(c)                They should conform to prescribed definitions, concepts and standards laid down by the NCS; 

(d)                They should be updated periodically, with suitable periodicity to be determined; and

(e)                They will be available at both aggregate and disaggregate levels, wherever appropriate.

14.5.5                The Core Statistics will be identified and accessed from all critical areas of the economy including agriculture, socio-economic sector, demographic, industrial, labour and employment, finance. The NCS will ensure that the production of statistics and their release are free from Government influence, by designating appropriate statistical institutions or functionaries to be solely and independently responsible for these functions.  In designating the statistical institutions or functionaries for the collection and release of Core Statistics on different subjects, the NCS will adhere to the distribution of subjects in the Union, State, and Concurrent Lists of the Constitution of India. Though the compilation and release of statistics will be the sole responsibility of the agency concerned, the NCS will exercise a statistical audit over the statistical activities to ensure quality and integrity of the statistical products. The Act would empower the NCS to make it binding on all agencies responsible for the Core Statistics to comply with the directives of the NCS. The NCS would be required to submit its Report annually to Parliament regarding its own functioning, and that of other statistical agencies and about the statistical situation in the country.

14.5.6                  The directive principle for the NCS will be that it shall work within the framework of a decentralised National Statistical System, both laterally among Central Ministries and vertically among the State Governments. In this context, it would be pertinent to mention that the National-level Statistics, in most cases, will be merely State -level aggregation of statistics.  Particularly, its orientation shall be that:

(a)                National statistics would mean entire set of statistics collected officially from administrative returns or through sample surveys.

(b)                Statistics at the all-India level are an aggregation of State-level statistics in most cases.

(c)                In advising on the collection of Core Statistics, the NCS would keep in view the optimum use of national resources, in context of essentially decentralised character of statistical system. The national resources mean the resources of both Central and State Governments together.

(d)                Further, the proposed system shall provide a direct approach by the States to the NCS on any statistical issue, and

(e)                The States shall have the opportunity to bring directly to the notice of the NCS, their reservation on any policy decisions taken by the NCS and to request for its consideration.

Mission Statement

14.5.7                The Mission Statement of the Indian Statistical System shall be to provide, within the decentralised structure of the system, reliable, timely and credible social and economic statistics, to assist decision-making within and outside the Government, stimulate research and promote informed debate relating to conditions affecting people's life.

Functions of the NCS

14.5.8                  Within the framework of the decentralised system, the functions of the NCS would be to:

(a)      Evolve and arrange to monitor the nation-wide strategies on: Core Statistics; updating the list of Core Statistics; framing and monitoring the advance release calendar; dissemination of data;

(b)      Evolve and arrange to monitor the nation-wide strategies on: Human Resource Development for Official Statistics; Information Technology and Communication needs of the Statistical System;

(c)      Improve Public Trust in Statistics by: increasing and promoting public awareness of Official Statistics; monitoring and ensuring reduction of respondent burden; arranging interaction with data users;

(d)      Function as Apex Authority on Statistical Coordination: between Central Ministries, departments and other Central agencies; between Central and State Governments;

(e)      Ensure Quality Assurance of Statistical Processes; evolve and enforce appropriate statistical standards; declare quality with statistical releases; audit Statistical Activities; determine modality of the release of data;

(f)       Constitute Technical Committees or Working Groups to assist the NCS in performing various functions;

(g)      Assess legislative requirements periodically.

Constitution of NCS

14.5.9                   The NCS will have a Chairman and four Expert Members. Since the NCS would be primarily a policy-making body working through a number of technical committees and should not be involved in routine administration, given the function of the NCS and the time that the Chairman and members are expected to devote to them, the Chairman and the members should be appointed on a part-time basis. The tenure of both would be 3 years. For administrative purposes, the status of the members should be at least that of Secretary to the Government of India. The Chairman and members will be eminent statisticians or social scientists and represent the following indicative areas of specialisation:

(a)                Agriculture and allied areas including, Meteorology and Environment;

(b)                Industry, Trade, Finance, National Accounts and Infrastructure;

(c)                Population, Health, Education, Level of Living, Labour, Employment and other Socio-economic Sectors;

(d)                Survey Design, Analysis and Statistical Modelling;

(e)                Statistical Information System and Information Technology;

(f)                 State Statistical Systems.

14.5.10               The NCS would be assisted by technical committees in the following illustrative list of subject areas:

(a)                Agricultural Statistics

(b)                Industrial Statistics

(c)                Price Statistics

(d)                Trade statistics

(e)                Social Statistics

(f)                 Infrastructure Statistics

(g)                National Accounts Statistics

(h)                Large-scale Sample Surveys

(i)                  Information Technology.

14.5.11               For budgetary purposes, the NCS would be in the Ministry of Statistics. The Secretary of the NCS would be the head of the National Statistical Organisation (NSO) described below. He will be called the National Statistician and would also be the Secretary to the National Commission on Statistics. He will have the rank of a Secretary to the Government of India. To assist him in the responsibilities, a core secretariat should be established in the NCS. The post of National Statistician will not be reserved for any organised service of the Government of India. Recruitment will be made by open selection from among professional statisticians with long technical and managerial experience in large statistical organisations.

 National Statistical Organisation (NSO)

14.5.12            The NCS will operate through the National Statistical Organisation (NSO), which will be the official agency to implement policy decisions of the NCS. The NSO will function as the single full-fledged Department of the Ministry of Statistics of the Central Government headed by the National Statistician, who would be the Secretary of the Department. Essentially, the NSO would be the restructured form of the present Statistics Wing of the MoS&PI. The National Statistician, will be its technical and administrative Head. The National Statistician, would broadly be responsible for the following activities:

(a)                To provide leadership to statistical activities by promoting coordination with components of the National Statistical System, particularly with the State Directorates of Economics and Statistics;

(b)                Assist in evolving and implementing the National Statistical Strategy;

(c)                Assist the NCS to decide and reallocate statistical priorities;

(d)                Promote reliability and integrity of statistics;

(e)                Formulate and implement plan scheme in statistics (as ex-officio Adviser, Statistics to the Planning Commission).

Functions of the NSO

14.5.13               The envisaged functions of the NSO are enumerated below:

(a)                Implement and maintain statistical standards and coordinate statistical activities of Central and State agencies as laid down by the NCS;

(b)                Compile National Accounts according to the latest international standards at regular periodic intervals;

(c)                Collect or arrange to collect Core Statistics, which have not been collected so far;

(d)                Participate in regional, national, and international statistical forums and meetings;

(e)                Carry out methodological research and studies;

(f)                 Publish Core Statistics at regular intervals together with critical analysis regarding the quality of data and implication of the use of data in policy- making and administration;

(g)                Arrange in-service training course for statistical personnel, in cooperation with universities and research institutes;

(h)                Maintain a “warehouse” for Core Statistics, for dissemination amongst all users in the public and private sectors inside and outside the country and serve as the sole provider of information to foreign Governments, international bodies and United Nations agencies.

Proposed Structure of the NSO

14.5.14               The NSO will comprise the following four offices. The present rank and status of the heads of the first three offices, which now form the Statistics Wing of the MoS&PI, should be maintained. The existing structure of MoS&PI is given at Annexe 14.7.

(a)                Central Statistical Office (CSO), to replace the present Central Statistical Organisation;

(b)                National Sample Survey Office (NSSO), to replace the present National Sample Survey Organisation;

(c)                Data Storage and Dissemination Office (DSDO), to replace the present Computer Centre;

(d)                Consultancy Wing (CW), new.

These will be divided into a suitable number of Divisions.

Central Statistical Office

14.5.15               The Central Statistical Office (CSO) will comprise the following Divisions with functions given below the names:

(a)            Coordination, Standards and Administration Division

(b)            Human Resource Development Division

(c)           National Accounts Division

(d)           Economic Statistics Division

(e)            Social Statistics Division

(f)             Research and Publications Division

National Sample Survey Office

14.5.16               The National Sample Survey Office (NSSO) will comprise the same four Divisions with more or less the same functions as those of present NSSO.

(a)                Survey Design and Research Division (SDRD)

(b)                Field Operations Division (FOD)

(c)                Data Processing Division (DPD)

(d)                Coordination and Publication Division (CPD)

Data Storage and Dissemination Office

14.5.17               The Data Storage and Dissemination Office (DSDO), with no break-up into Divisions, would perform the following functions:

Consultancy Wing 

14.5.18               The Consultancy Wing (CW) would cater to the increasing demand for professional statistical services, especially within the Government. Presently, this demand is being partially catered to by private agencies whose competence could be questionable. It is understood that some Central and State Government departments have been incurring an annual expenditure of the order of a few hundred crores of rupees on account of out-sourced studies, surveys and consultancy work. An indicative list of some such projects assigned by the Government Departments to outside agencies is given in Annexe 14.8.  The Consultancy Wing will aim to cater to the increasing demand for investigations and studies of a statistical nature and provide related professional statistical services that cannot be accommodated in the existing arrangements within the Government. This Wing would essentially function as an autonomous body and aim to be the commercial wing for professional statistical activities. In its nascent stage it will be nurtured by the NSO, before it hives off as an independent corporate entity outside the Government.

14.5.19               To start with, it may undertake projects from Governmental and international agencies regarding data collection, processing, analysis and report generation through sample surveys or other means on topics assigned to it. It could also take-up consultancy services relating to statistical problems including methodological studies and model building. It would carry out detailed analytical reporting, consultancy (both national and international) and compete in the consultancy market on commercial terms. Some of the activities include methodological studies, surveys, macro-econometric modelling and forecasting, and consultancy to the State Governments. It will adhere to the appropriate statistical standards and methodology. The Consultancy Wing would be free to hire skilled personnel for project-specific work including officers of the Indian Statistical Service on deputation.

14.5.20               This being a new activity, the Commission recommends that the organisational structure for it should evolve along with the growth of its activities.

Improvement of lateral coordination at the Centre through Statistical Advisers

14.5.21               At present there is no institutional mechanism through which the MoS&PI can effectively coordinate with different ministries at the Centre in statistical matters. The Commission is of the view that heads of the statistical divisions in the ministries and departments should be responsible for the professional integrity in the statistical activities of their departments and improvement of these activities. In discharging their responsibility, they will work closely with the National Statistician as the head of the national statistical system. They will be responsible to the National Statistician for the professional quality of their work. They will collaborate with the National Statistician in their professional responsibilities while remaining in the administrative organisation of their ministries or departments. The heads of the statistical divisions, to be designated as Statistical Advisers, would thus have dual responsibilities – assisting the concerned ministry in matters of statistics and coordinating with the National Statistician in respect of maintenance of quality standards as laid down by the NCS. Considering the high level of responsibility of their posts, the Commission recommends that the Statistical Advisers in the major ministries or departments, such as Agriculture, Industry, Commerce, Finance, Health, Water Resources, etc. would be of rank one step below the National Statistician, while in other ministries and departments, the Statistical Advisers should be of a sufficiently high rank (see Annexe 14.9). The Statistical Adviser would: 

(a)                Be designated as the ‘Nodal Officer’ with regard to all statistical matters pertaining to the ministry or department;

(b)                Assist the Secretary of the administrative ministry or department in all statistical matters;

(c)                Be associated closely with the National Statistician in implementing the guidelines outlined by the NCS;

(d)                Coordinate flow of information to and from the NSO.

Improvement of coordination with the States by empowering the State Directorate of Economics and Statistics

14.5.22            The close vertical coordination between the Statistical Divisions of the ministries of the Government of India and those of the departments of the State Governments, and between the CSO and the State Directorate of Economics and Statistics (DES), has been the strength of the decentralised Indian Statistical System. The weakness of the lateral coordination so far at the Centre between the CSO and the Government of India ministries had also led to weak lateral coordination in the States (where the statistical system is not fully centralised) between the State’s DES and the statistical units of the departments. The system did not formally provide for a review by the DES of the content and methodology of the statistics collected by other State Government departments.

14.5.23            With the creation of the NCS and establishment of an NSO that will have a wider role than that of the present CSO, the lateral coordination at the Centre between the NSO and Central Ministries will greatly improve with a much more cohesive approach to the statistics of the ministries. The new institutional arrangement will not however loosen the present strong ties of vertical coordination between the CSO and Government of India, ministries on the one hand and the DES and State Government departments on the other. On the contrary, it should lead to a strong lateral coordination at the level of the States.

14.5.24               To ensure that this takes place, the existing coordination of the State DES should be widened to cover technical coordination. The Directorates of Economics and Statistics should be formally entrusted with the responsibility for a periodic review of the content, methodology and output of the statistics of all State departments and to make suggestions for the further improvement of these statistics. The Conference of Central and State Statisticians should be held regularly. A similar forum for a meeting of State Departmental Statisticians should be created by the State Governments to review the performance of the statistical system of each State. The report of the review and the suggestions may be forwarded by the DES to NSO and by the departments to the corresponding ministries, for action at the Centre (see next section).

14.5.25            The enhanced role of the DES and the wider technical discussion of the State’s Statistics will help State Governments take a holistic view of the State’s Statistical system to enhance its utility to the State Governments and indirectly to achieve the same result at the Centre (see next section).

14.5.26            The Commission recommends that a Centrally-sponsored scheme for strengthening the statistical system in the States be drawn up immediately for inclusion in the Tenth Five Year Plan, with the specific objectives of developing a survey and data-processing capability in the States.

14.5.27               The earlier mechanism of setting up working groups in the CSO to formulate the plan schemes in statistics at the all-India and State levels should be revived immediately for the Tenth Five Year Plan.

14.6                     The States’ Statistical Systems

Improving the Administrative Statistical System

14.6.1                  The Administrative Statistical System (AdSS) is in disarray in three sectors: agriculture, industry and labour, of which the first and the third belong to the States’ Statistical Systems (SSSs). In Agricultural Statistics, the system of regular yearly recording of area under crops has almost ceased to operate. Overburdened with all kinds of other work, the village official, the last point of revenue administration, whose basic function is the maintenance of land records, does not find time for this work. For a different reason, the same is true of the system of Labour Statistics. The Office of the Chief Inspector of Factories (CIF), charged with the implementation of the Factories Act (1948), does not even maintain an up-to-date list of factories, and neglects to collect the half-yearly and annual returns, the basic returns to be filed under the Act, from more than a third of the factories. The situation about education statistics is only a little better. The position about the collection of other Administrative Statistics is equally unsatisfactory. It may appear that the failure of the AdSS is restricted only to a few sectors. But its causes are not specific to these sectors and reflect such systemic degeneration as would be common to all. It is only a question of time before the AdSS collapses in other sectors. Therefore, the ills of these sectors have to be urgently remedied.

14.6.2                The causes of the failure are of two different types. The first is the overburdening of the staff with other work to such an extent that they are left with no time for their normal work. The second failure occurs, without these extenuating circumstances, and consists in the neglect of the normal work by Government offices.

14.6.3                The failure of the first type is a process that has started long ago. On one hand, one notes a constant reference being made to burgeoning bureaucracy; on the other, at the farthest end of the Government system, at the lowest level of village, the Government staff, the patwaris, the gramsevaks or primary teachers, is loaded with work as if the staff is a kind of infrastructure. Given our penchant for planning by campaigns, and with little thought paid to the consequences, this staff is burdened continually with ever-new assignments forcing it to neglect normal work. Apparently, in spite of their close bonds with the State Government departments, the ministries of the Government of India have not been able to deal with this problem effectively.

14.6.4                  It is apposite to mention another factor that affects not the AdSS but other statistical projects of the Indian Statistical System such as the Agricultural Census or the Livestock Census. The planners of these projects also adopt the same approach when they assign the project work to the lowest-level staff of patwaris, gramsevaks or primary teachers. Further, since the collection of data by these functionaries, costs almost nothing, reducing considerably the cost of the project, village-level staff become an easy choice as field workers. That also induces the ministries of the Government of India to plan censuses of different types rather freely; i.e. without having to consider the cost of fieldwork, and with freedom to choose the census as the means of collecting data, without having to consider alternative ways of collecting data. The result is that the staff have least respect for statistical work in general. They have also become aware of why they get loaded with such projects and are now hesitant to do such work conveying a message that the days of free data-collection are over.

Ebbing Efficiency of Government Administration

14.6.5                The second cause for the failure of AdSS arises out of the ebbing efficiency and effectiveness of Government administration. As stated before, Administrative Statistics are generated indirectly as a part of Government administration from the quantitative information that departments collect in order to implement the Acts, Rules or Regulations, which also empower and require them to collect information from the relevant units in prescribed forms. Collection of information thus becomes a part of their normal function. The soundness of the AdSS thus depended on the assumption that the department carried out its functions satisfactorily. For example, quantitative and other information is collected from factories because it aids the Office of the Chief Inspector of Factories (CIF) to carry out its functions of inspection of factories to see that they satisfy the provisions under the Factories Act and other related Acts. Factories are required to furnish relevant information to the Office of the CIF in various forms (called factory returns) prescribed under Rules framed under the Acts, and it is the responsibility of the Office of the CIF to ensure that they do so. But when that office carries out its functions without a large proportion of factories having filed their returns, it should be obvious that it is not doing so as satisfactorily as it is expected to do. The growing unsatisfactory condition of factory statistics system is but a reflection of this unsatisfactory working of the Office of the CIF. In general, over the years, the system of Government administration is deteriorating. The failure of the AdSS is but a corollary of this deterioration of the system of Government administration.

14.6.6                Neglect of normal duties by Government functionaries has caused a failure of Government administration in both cases. In the first case, the Government officers or offices responsible for Government statistics are different from those whose officers or staff are responsible for data collection: Agricultural Statistics office and Revenue Department, and State’s statistical office and the Office of the CIF, respectively. Since the problem involves different departments, only the highest level of Government administration, the Chief Secretaries of the States, can resolve it. The State Governments should appreciate that the AdSS is the prime responsibility of the State Governments, that the entire structure of the Indian Statistical System is founded on its basis, that the AdSS is presently in a state of collapse in certain sectors, and that the effects of this collapse will not be restricted to particular States but will affect the completeness and quality of national statistics. They have to take urgent steps to remedy the situation by resolving administrative problems in some cases and toning up the administration in others. The States have played a dominant role in building the National Statistical System known for its soundness all these years. Inaction by States about the AdSS will gravely impair the national system. That will necessarily lead to a creation of alternative systems by the Central Government, which will be centralised, and will produce statistics which the States will have to accept in place of those they have failed to generate from their own systems. The States will thus lose power over the field of statistics, which belongs to the Concurrent List of subjects in the Constitution. The Commission therefore highlights the seriousness of the whole issue, and its implication for not only the States but for National Government as well. Without improvement in Government administration and a resolution of the administrative problem of excess burden of work on the staff, attempts at revamping the statistical system of the whole country in isolation can only be partially successful.

14.6.7                The extent to which this central issue is resolved will set measure for the effectiveness of improvements in interconnected aspects of the States’ Statistical Systems (SSSs) and in turn of the Indian Statistical System. However, independently of the resolution of this central issue, attempts at improvements in other aspects of the statistical system should have their own positive influence of encouraging the State Government authorities to address the central issue of the AdSS.

14.6.8                  Since the subject concerns the States, and further since the situation about the SSSs varies in different States, the Commission is not able to recommend measures for different States. It is also not able to consider the question of resources for implementing its recommendations. In view of this it will restrict itself to suggesting certain main policy guidelines for the improvement of the SSSs. The State Governments may consider setting up commissions or committees to advise them on the manner of implementation of these guidelines and on other issues relating to SSSs. The guidelines that follow have the objective to help State statisticians improve the usefulness of their statistics to the Governments.

Enhancing the Usefulness of the State Statistical System to the State Government

Statistics for Decision-Making

14.6.9                  The most common problem of the States’ statisticians is that their Governments do not provide them with the necessary resources. But its obverse side is that statistical offices will gain importance only by establishing the usefulness of statistics to the State Government. To succeed in raising his status and that of statistics, a State Statistician’s job should, therefore, be that of enhancing both the demand for and the supply of statistics and their analysis. The “mission” of his office should be to improve decision- making of the Government by providing quality statistical service.

14.6.10               Presently, the statistical work programme of the States’ DESs has been more or less standardised in the country. Their publications include statistical abstracts and handbooks for the States, annual economic reviews or surveys, budgets-in-brief, economic-cum-functional classification of the budgets, estimates of State Domestic Product, district statistical abstracts, social and economic indicators of development, municipal year books, periodic releases of price index numbers, and reports based on tabulation and analysis of sample survey data including those based on the survey of the NSS matching sample. These publications do make available to the people and the States’ administrators considerable useful statistical information about the State and its districts. The timely issue of these publications and maintaining a quality of these statistics in all respects will always be the primary function of the DESs. But experience shows that for the DES to be useful to the Government, much more is required to be done.

14.6.11               Broadly there are two kinds of statistics needed by Governments. One, at the macro-level, of national aggregates, such as national income and accounts, poverty, unemployment or such other macro-level economic variables. The other is statistics at the micro-level for operational planning where disaggregated statistics are more relevant and therefore, important. Broadly again, the two types of statistics are identifiable separately with the concerns of and use by the two main levels of Government; the macro-level for use by the Central Government, given its responsibility of economic policy-making, and the micro-level with the State Governments, for operational planning, that is, for the formulation of specific plans and programmes.

14.6.12                A very important characteristic of the statistical requirements for State Government decision-making is that, of the two main types of statistics, the average or the total and the distribution, it is the latter that is more frequently required. This is because a State Government’s decisions about its plans and programmes mostly relate to specific groups of people or units, needing statistical information for such groups defined in a manner relevant to the plans and programmes. Generally, the information is required urgently. The statistician must accept the challenge, for it is only on such occasions that the administrators come in meaningful contact with the statistical system, and these occasions provide an opportunity for establishing the usefulness of the statistical system.

Operational Aspects

14.6.13               In most cases, the State’s Statistical System possesses the data. What is needed is the capability to process this on demand and to synthesise the data from different sources. With computerisation now, both processes can be easily carried out. The first will also require a fundamental change in the orientation of production of results, from a “fixed-product” approach of pre-determined tabulation plans to a “demand-determined-product” of tabulation on demand. The second will require a greater effort at coordination, by including a set of common parameters in all data collection projects, so that data from diverse sources can be linked. For this development the State Statistical Systems should be so developed that they can create the following facilities:

(a)                Unit-level data of one department would become accessible to other departments;

(b)                A central storehouse of unit-level data of all departments should be created in the DES;

(c)                A small set of data elements should be identified (such as permanent village codes) that should be included in all forms of data collection of the AdSS.

Computerisation of Administrative Statistics

14.6.14               The entire system of Administrative Statistics is record-based in the offices of the Government, which are empowered and responsible for the administration of different Acts and Rules of the Government. The PC and information technology revolution have prompted many State Governments to declare their intentions to launch programmes to computerise their administration to achieve the goal of e-governance. In this programme, computerisation of the offices administering the Acts and Rules, (Sales Tax Commissioner, Transport Commissioner, Registrar of Stamps and Duties, Chief Inspector of Factories, and the like), which directly deal with units or people, deserves to be accorded a priority. This will lead to three tangible benefits. First, it will facilitate and systematise the routine, essentially clerical, operation of recording, maintaining, updating and processing of administrative records and data, resulting in an increase in the efficiency of the office, and help in enhancing the quality of information. Second, these offices will be able to speedily process the Administrative Statistics collected by them. Technology is available to link the PCs in their sub-offices to the processing PC in the headquarters office and the Directorate of Economics and Statistics for further speeding up of the process. Third, apart from the improvement of Administrative Statistics, computerisation will also act as a strong catalyst in the improvement of the overall working of these offices to the benefit of the concerned units and people, the recipients of the Government services. The focus of computerisation in the States’ administration should thus be on these offices.

14.6.15               An apparently minor, but in fact vital, point requires to be brought out. Quite often modernisation is interpreted solely in terms of use of sophisticated computer equipment at the top while easily forgetting the fact that the lowest-level functionary, the patwari or gramsevak, is not supplied even with a simple calculator. It is on him that the accuracy of the statistical work basically depends, and the computer becomes useful in processing data fast, only if he supplies accurate data. In the age of computers, and with attempts to inject modern technology, it is a paradox that simple inexpensive “technology” is not introduced where it is most needed, and where it will be most effective. Supplying such simple equipment to these functionaries should be the first priority in the modernisation of the Indian Statistical System.

Potential of the NSS Survey Mechanism and Data

14.6.16               It is indeed a sad state of affairs that although most States carry out the field-work of the matching sample of the NSS, most of them do not tabulate the collected data and publish the NSS findings for the State because of a mismatch of resources for data collection and those that should be devoted to the tabulation and analysis of the same. This is a glaring example of wastage of resources and the States should attempt to immediately correct this imbalance, as the NSS data they possess are a mine of information that would prove extremely useful to the State Governments.

14.6.17               Further, the NSS offers two potentially great advantages to the States. Participation in the NSS provides the States with the best representative sample of their States’ population and a band of well-trained field staff. If a State Government needs any simple type of data, over and above those already included in the NSS schedules of a round, it can easily and quickly collect them by canvassing a simple additional schedule containing them for the same selected sample.

14.6.18               The second is the wealth of quantitative information that the NSS provides. The DES should perceive the immense utility of these data much beyond the production of a set of tables decided upon by the NSSO. In effect, the DES possesses a data-bank containing sets of voluminous data generated by several rounds of the NSS on many social, economic and demographic variables for large representative samples of a State’s households. It is possible to tabulate on demand data on those variables, which are relevant to a particular problem of decision-making, in any manner required. Computerisation will greatly help this process of full exploitation of the potential utility of NSS data.

Strengthening of Sample Survey Capability in DESs

14.6.19               Before their participation on a matching sample basis in the NSS, the DESs of the major States had their own sample survey organisations to conduct surveys of specific interest to them. Subsequently, the States’ own survey organisations languished and could not be developed. Since the NSSs cannot meet all the data needs of the States, complete dependence on the NSS is a handicap from which the States now suffer. It is, therefore, important that the States’ DESs should build their own sample survey capabilities and organisations.

Computerisation

14.6.20               Undoubtedly, the major step that the DESs have to take is the reduction in time lag between collection of data and their tabulation and release. This requires that the time taken for processing the data should be drastically reduced. Voluminous tabulation plans decided upon in advance are one of the reasons for processing delays. With computerisation and proper software design it is possible to easily tabulate data in any manner required on demand at a later date. In the future plans for statistical development, the DESs should give priority to computerisation and creation of necessary in-house software capability.

Development of Analytical Capabilities

14.6.21               To make use of data for the resolution of a particular problem of decision-making will undoubtedly require the development of analytical statistical expertise. The focus will be on generation, from available data, of statistical “information” relevant to a particular problem, and this will require familiarity with and ingenious use of modern statistical methods and operation research techniques. The DES will have to create a small data-analysis unit of “problem solvers”, and man it by one or two statisticians competent in these techniques. In the suggested organisation of the National Statistical Office (NSO), it is proposed that a consultancy wing may be created. The DESs will be able to call upon the statisticians working in this wing, when required. To nurture growth of analytical capabilities for problem solving in all State DESs, it will help if they share their experiences. For this, it would be beneficial if the Conference of Central and State Statisticians devote a technical session for this purpose.

Supportive Institutional Structure

Directorate of Economics and Statistics

14.6.22               Consistent with the expectations that statistics and the director of the DES should be of help to Government in decision-making, certain institutional changes are necessary. The State Government should ensure a working environment in which it is possible to work with professional integrity. The institutional arrangements suggested below derive their rationale from the goal of creating an environment in which a statistical system can provide the best statistical service to the Government and the people.

Independence of DES

14.6.23               A statistical office, as Messrs. Bowley and Robertson averred nearly 70 years ago, should be as nearly independent of department control as administrative requirements permit. If Government statistics are to be seen to be objective and free from the influence of any implementing department, the DES should be independent from the control of any Government department in the matter of its substantive work. For this reason, considering the role that the director of DES is expected to play in the decision-making process of the Government, the Commission is of the view that the DES should be transformed into a separate Department of Statistics with the director of DES as its Secretary to the Government. This will also be consistent with the relative status of the director of DES vis-à-vis the Director General of CSO.

A Professional as Director of DES

14.6.24               For the director of a DES to perform the role expected of him satisfactorily, a professional statistician or a professional economist with considerable experience in the field of empirical analysis, should be appointed, in order to provide the requisite guidance and leadership to the State Statistical System.

Director as Adviser to Government

14.6.25               If the director is to perform his role satisfactorily, the State Government on its part should involve him in the decision-making process. This can be done by making him a member of, or a permanent invitee to, committees and groups, formally or informally formed, dealing with plans, programmes and decisions in substantive fields: agriculture, rural development, irrigation, industry, education and the like. (This should be distinguished from the statistics of these subjects.) State Governments should formally order or establish by convention an arrangement by which the director should be free to participate in such groups and the departments should be free to take the benefit of his help.

Role of DES vis-à-vis Statistical Divisions of Other Departments

14.6.26               To be effective in the larger role envisaged for its director, the functions of the DES must encompass the overseeing of the operations of the entire State’s Statistical System. Presently, its function is that of “coordination” for which, in some States, the DES is identified as a “nodal agency”. Both are rather vague terms and are not of much help in translating them into the operation of the State Statistical System. The role of the DES should be larger and more specific than the two. The DES should be assigned the function of technical coordination for taking a holistic view of the State Statistical System. It should be formally charged with the responsibility of taking an annual technical review of the statistical activities of all Government departments and should submit a report to the Government with its suggestions on the development of statistics in different fields. The DESs should also be asked to make a report to the Government of its comments on and suggestions for these activities. The DESs should also be authorised to convene a biennial conference to take a review of the State Statistical System and its activities. Also, the ministries of the Government of India, which define the statistical work of State Government departments, should give up their compartmentalised approach and recognise that the DES has a leadership role in all fields of statistics at the State level, and should consult the DES in statistical matters relating to those fields. The improvement in lateral coordination at the Centre, implicit in the Commission’s recommendations for the top structure of the Indian Statistical System, will also help in the improvement of lateral coordination at the State and between the Central ministries and the DESs.

Common Statistical Cadre

14.6.27               Since statistics is a discipline common to all fields, there is an advantage in having a common cadre of statisticians to which all statistical posts in the State’s department should belong. There is a further advantage in the common cadre being under the control of the DES, the largest statistical office in the State. On account of this arrangement, a statistician can get the rewarding experience of working in different fields, and that experience is useful to any department where he is posted. Belonging to a common cadre is also reassuring to the statistician. Due to the very nature of his work, a statistician working in a substantive department is exposed to the risk of being influenced or pressurised by the senior officers of the departments where he is posted. This creates a conflict between loyalty to the statistical discipline and that to the department’s interest. As he has a sense of belonging to a larger parent department or service from which he is seconded, he is able to successfully resist such influences of the department that jeopardise his professional integrity. Finally, since the statistical divisions in most departments are too small for career prospects of persons if the departments recruit them independently, posting an officer from a common service or cadre obviates the difficulty of departments in recruiting and retaining good statistical personnel in their statistical divisions.

14.6.28               The next step should be to form a State Statistical Service and statistical posts in all departments, including the DES, should be manned by officers and staff belonging to this service. This will make possible the organisation of a true all-India Indian Statistical Service, so that officers of the States’ Statistical Service can be inducted into it, the officers from this service then can be posted in States, those from the States can be posted in Central organisations, and both Indian Statistical System and State Statistical System will benefit from the exchange of experience that will follow.

Statistical Divisions in Departments

14.6.29               The role of a statistical division in a department should be the same as that of the role of DES in the Government. The statisticians heading the division should be involved in the decision-making process of the department in the same manner as suggested for the DES. For facilitating this, the statistical cells should be headed by at least a Group A officer. More importantly, whatever his status relative to that of the second top-level officers of the department, the head of the cell should work directly under the head of the department.

Block Statistical Organisation

14.6.30            The block statistical organisation (BSO) was created nearly fifty years ago for statistics of the community development programme. It was one of the finest organisational arrangements that established statistical units under the supervision of the DESs on the lowest possible rung of the administrative ladder. Since then, after reorganisation under various forms of democratic decentralisation, the organisation in some cases has been transferred to the “local sector”. The survey taken by the Commission of the State DESs shows the present situation. 

 

Major States*

Other States All
Total number of State DESs reporting 10 11 21
Number of States that      

Had block statistical organisation in the past

9 7 16

Have block statistical organisation now

9 6 15
Have Block Statistical Assistants on DES cadre 5 6 11
Have DES as their controlling authority 3 6 9

*States with a population of one crore or more

14.6.31               Thus, in many major States, the block statistical assistants (BSAs) are on the local cadre, and in most of them, their work programme is not decided by the DESs. Their work has not remained statistical and the purpose of creating that organisation is not always served in all the States. There had been many administrative difficulties in re-establishing the link between the DSOs of the DESs and the BSAs.

14.6.32               Now, with the 73rd and 74th amendment to the Constitution, there is an opportunity to bring the BSO appropriately within the fold of the State Statistical System. The manner in which this can be done might vary from State to State. But the minimum that requires to be done is to make the DSO the technical head (not the administrative head necessarily) of the BSAs in the district. This will ensure a greater degree of their participation in the statistical work. In deciding upon their work, the DESs should see that the BSAs’ work primarily involves local statistics and is relevant for local area planning. The Commission has made recommendations on the development of local areas statistics (see paragraph 9.2.22). The BSO should be made the instruments of this development.

14.6.33               The other manner to bring the BSO in the State Statistical System is to transfer it completely to the State sector. This change will bring life to the languishing BSO and would be generally welcomed by the BSAs whose present career prospects are too discouraging for them to work satisfactorily.

Recommendations

14.6.34               The Commission therefore recommends:

(i)                  The breakdown of the Administrative Statistical System needs the immediate attention of the highest authorities of State Governments. They are urged to take steps to reduce the burden of the additional work given to lowest-level Government functionaries such as patwaris and primary teachers so that they can effectively carry out statistical functions assigned to them.

(ii)                The authorities should also instruct the offices implementing different Acts and Rules to be vigilant that all relevant units file with them regularly the statutory statistical returns required by the Acts and Rules, and take necessary action under the Acts against the defaulting units.

(iii)               The State Directorates of Economics and Statistics (DESs) should develop capabilities to tabulate data on demand and to analyse data from different sources. For this they should organise all the data that the State’s statistical system possesses in an appropriate manner.

(iv)              The State Governments should accord priority to computerisation of administrative offices that generate administrative statistics.

(v)                The DESs should fully exploit the potential of their participation in the National Sample Survey (NSS) programme by using the survey data as a data bank and by utilising the survey mechanism for ad hoc collection of additional simple data required by the Government.

(vi)              The State Governments should support the DESs in the creation of sample survey capabilities by creating sample survey divisions in them.

(vii)             The State Governments should make the necessary resources available to the DESs for computerisation and development of necessary software to make the DESs self sufficient in this respect. This will help them to undertake tabulation of NSS data, which they are collecting in their matching samples.

(viii)           The DESs should develop the necessary analytical capabilities to carry out data-analysis relevant to the problems of decision-making of the Government.

(ix)              For strengthening the effectiveness of the statistical system of the Government, the State Governments should create a separate Department of Statistics by elevating the existing DES to the level of a Department and the Director of the existing DESs to the level of Secretary to the Government. The Department of Statistics should have complete freedom in statistical work. The head of the Department of Statistics should be a professional statistician or a professional economist with experience in large-scale data collection and empirical analysis of data. 

(x)                The State Governments should closely involve the Director of DESs in its decision-making processes by making him a member of or an invitee to committees and groups dealing with plans and programmes in substantive fields.

(xi)              The State Governments should strengthen the role of the DESs as coordinators of their statistical activities by empowering them to take a technical review of the statistical activities of all departments every year.  The DESs should also be asked to make a report to the Government of its comments on and suggestions for these activities. The DESs should also be authorised to convene a biennial conference to review the State Statistical System and its activities.

(xii)             The State Governments should take steps to create a common statistical cadre and State Statistical Service for manning statistical posts in all departments.

(xiii)           The heads of the department of the State Governments should closely involve their departmental statisticians in their decision-making process. To give institutional support to his role, the departmental statisticians should be placed directly under the head of the department.

(xiv)           In view of the renewed importance of the Block Statistical Organisation in the context of local area planning, the State Governments should bring it directly within the fold of the States’ Statistical System by either transferring the organisation to their Directorates of Economics and Statistics, or by making it responsible for its statistical work to the Directorate and bringing it under the Directorate’s technical supervision through the district statistical organisation.

(xv)            The State Governments may consider setting up commissions or committees to advise them on the manner of implementation of these recommendations and on other issues relating to States’ Statistical System.

14.7                     Information Technology in the Indian Statistical System

Historical Background

14.7.1                  Before the introduction of electronic computers in India, Unit Record Machines with 80-column Punched Cards as the medium of data-input were extensively in use during the late forties at the Indian Statistical Institute (ISI) for processing of survey data. The first-ever electronic computer in India - the Hollerith Electronic Computer Model 2M (HEC 2M) from the UK was installed for statistical use at ISI in 1949. Limited input-output capabilities of this and the next computer - the Russian URAL II installed at the ISI made them unsuitable for survey data processing. These were used mainly for complex calculations in other types of statistical applications. Use of computers in the special tabulation work of the NSS began at ISI in 1965 on the more versatile IBM 1401 computer system.

14.7.2                  A Computer Centre was created in the Department of Statistics in 1967. The Computer Centre was initially equipped with a Honeywell-400 computer system. It was used for the processing of survey data and for providing computing support to various ministries and departments of the Government of India. Major jobs carried out by the Computer Centre have already been described.

14.7.3                  Later, with the growth in demands on the use of Information Technology (IT) tools, the Government of India established the Department of Electronics (DoE) and the National Informatics Centre as part of DoE to cater to the needs of different ministries and departments of the Government. The nodal role of the Computer Centre was changed and this was to cater mainly to the need of Department of Statistics. 

14.7.4                  It must be admitted, however, that till the late 1990s, the processing of NSS data was not very successful. There were long delays in completing the tabulation of any round of the survey and the backlog went on accumulating. There are good reasons why it happened. Raw data as obtained by enumerators from the field usually have many defects. Before tabulation can start, these have to be subjected to detailed scrutiny and rectification, a process called "data cleaning". This is a very difficult and time-consuming task, and may take up more than 80 per cent of the total computer time. Deep knowledge of the subject of investigation is essential to make an effective plan for data processing and tabulation. Mere software skills are not enough. This lesson was learnt at great cost, when worried by the delays in completion of tabulation at the Computer Centre, the Governing Council of NSSO entrusted the work of processing of the data of the 35th Round of the NSS to the National Infromatics Centre of the Department of Electronics. Not only was the job completely bungled, the entire volume of data were totally lost in the process. Success came only later when it was realised that survey data processing to be successful would have to be done in-house by the statisticians conducting the survey.

14.7.5                  Earlier, the DPD procured 108 data entry machines of type D20 and three small computers of model UPTRON S-1650 for data cleaning. The DPD was responsible for data transcription and cleaning on the above equipment – later on the newly-acquired Personal Computers (PCs) – and the final tabulation was done on the mainframe H-400 computer at Computer Centre. This was not much of a success mainly because of problems of communication between the Computer Centre and the DPD located at two distant places. This procedure continued up to 50th Round (survey period: July 1993-June 1994).

Current Status

14.7.6                  A new approach was taken for the processing of NSS data from the 51st Round onwards. The entire responsibility was given to the DPD, relieving the Computer Centre of any responsibility in the matter. The Data Preparation Centres of DPD were to transcribe data from schedules in small batches and generate error reports using a set of in-house software developed by the Division. After manual cleaning of the data by the Data Preparation Centres, these were sent to the DPD (Headquarters) located at Kolkata for automated scrutiny and final tabulation. With this new approach of in-house tabulation of data by the DPD, backlog in tabulation and report writing work has now been completely wiped out by the NSSO. 

14.7.7                  Besides routine data processing, computers are used in sample selection and as a desktop printing device in preparing manuals, presentation of tables, etc.

14.7.8                  With the advent of PCs, statistical computation in the Department of Statistics has been largely decentralised. From the reference year 1995-96, summary and detailed tabulation of ASI data are now done at the Industrial Statistics Wing of the CSO at Calcutta. The FOD carries out at Faridabad the processing of agricultural data collected by it under the Timely Reporting Scheme and the Improvement of Crop Statistics Scheme. The National Accounts Division (NAD) has its own system of PCs for compilation of national accounts.

14.7.9                  One important step in the process of computerisation of survey data processing in NSSO is the introduction in 1995-96 of Palmtop computers for the collection of data. As an experimental measure, in the 52nd Round of NSS, socio-economic data were collected in Haryana directly on palmtop computers. The basic idea was to download the data collected on palmtop computers directly on to main computers for processing and thereby avoid the intermediate step of data entry. The palmtop computer had just two lines of display of 16 characters each, and a memory of only 64 Kilo Bytes. The FOD used these gadgets subsequently to collect field data under the scheme of Improvement of Crop Statistics (ICS), Middle Class Price Collection (MCPC), etc. These projects were planned in a hurry. Operational problems under difficult conditions in the field were not examined carefully. Limitations of hardware and software were not taken into account. The project was a total failure.

14.7.10               Wrongly diagnosing the problem as mainly one of limited capacity of the equipment, another experimental project was attempted in the 54th Round of the NSS, this time using more efficient Palmtop computers with larger memory. The data were collected from Orissa and Maharashtra using these newer Palmtop computers. However, with the total failure this time again, the project has apparently been shelved. The 700 or so pieces bought at a cost of more than 1.5 crores of rupees are lying unused and may not be even usable any longer. But, there is now a proposal to undertake a pilot study for using even better and much more expensive modern laptop computers with large memory and hard disk capacity for data collection by the field staff! 

14.7.11               A great step of forward to meet user requirements was the adoption by the Government in 1999 of a National Policy on Data Dissemination. According to this policy, the Government is committed to supply the user, at marginal cost, unit level data, from all surveys after the expiry of three years from the completion of fieldwork or after the reports based on survey data are released, whichever is earlier.  To protect the privacy of information, all identification particulars of the informant would be removed from the data before making these publicly available.

14.7.12               By the above policy, the Computer Centre has been entrusted with the responsibility of creation and maintenance of a National Data Warehouse of Official Statistics. Under this project, the Computer Centre will preserve data generated by various Central and State Government departments and public sector undertakings on electronic media, organise them in the form of databases and provide remote access facilities to end-users through a network.  The Computer Centre has already initiated action for the creation of such a Warehouse.  The Computer Centre has been preserving a large volume of data generated through various socio-economic surveys conducted by NSSO, Follow-up Enterprises Surveys by the MoS&PI, and Annual Surveys of Industries conducted by the CSO. These data are being disseminated regularly to a large number of national and international users on Floppy and Compact Disk (CD).

14.7.13                The Computer Centre has also been given the responsibility of creating and updating the website of the MoS&PI which is hosted by the National Informatics Centre. The site is being regularly updated.

14.7.14               Computers are being used in almost all the Central ministries, departments and organisations in one-way or other. The Directory of Statistics published by the CSO gives information on computerised databases maintained by various organisations at the Centre as well as the State Directorates of Economics and Statistics.

14.7.15               Several attempts were made by the NSSO to use the National Informatics Centre managed communication network (NICNET) for transmission of Core Items of Monthly Progress Report (MPR) and Middle Class Price Collection (MCPC) data. However it was found that in many cases data transmission through NICNET was not at all satisfactory. As a result, the system of sending filled in schedules to the CSO through the traditional postal service was continued and transmission of Core Items of MPRs and MCPC using NICNET was discontinued. Similarly, an attempt by the FOD to transmit ASI summary data through NICNET had also to be given up because of duplication or loss of data in transmission and failure to install revised versions of software in a number of Centres of National Informatics Centre.

14.7.16               During 1998, a decision was taken to install e-mail in all the 172 field offices of FOD, NSSO, through the network of National Informatics Centre. After a period of two years, the connection could be provided only in 116 offices. Even in the offices connected, the transmission of data was not very successful.

General observations on the use of Information Technology in the Statistical   System of India

14.7.17               Statistical data processing involves the following types of work:

(a)                Transcription of data from filled in schedules to a computer readable medium.   This is labour-intensive error prone work;

(b)                Verification of transcription, in which the transcription operation is repeated and the copy is mechanically compared with the original (100 per cent verification is the norm in NSS);

(c)                Computerised check of internal consistency of the transcribed data; preparation of list of errors and their rectification by reference to the original document, or even by revisiting the informant;

(d)                Consolidation of data files and check of completeness of coverage;

(e)                Computerised check for missing or inconsistent data and replacing these by rule-based imputed values;

(f)                 Calculation of weights;

(g)                Preparation and scrutiny of Tables;

(h)                Calculation of standard errors of estimates – optional;

(i)                  Printing the Tables and Survey Report in appropriate lay out.

14.7.18               Currently by and large, in survey data processing, a system of flat files is in use. Imputation of missing or wrong values is done through cold deck methods.

14.7.19                Survey Design involves the following activities:

(a)                Preparation and Maintenance of Sampling Frame of First Stage Units;

(b)                Selection of First Stage Units according to the sampling design;

(c)                Choice of Sampling Design;

(d)                Design of Schedule or Questionnaire.

(e)                Computers are being or can be used in each of these activities.

14.7.20               At present, a standard sampling design is used – two or three-stage stratified sampling using a circular systematic method of selection with probability of inclusion proportional to size. The sample size is determined on the basis of availability of field investigators. There is a great scope for imaginative use of computers in improving the sampling design.

14.7.21               A large number of Statistical Software Packages are now available for sophisticated analytical work. Though some of these are available, official statisticians in India seldom use these. As a matter of fact, routine statistical work involves very little of technical computation.

14.7.22               A sound statistical system should ensure speedy transmission of information at different levels: field to the data preparation centres, data preparation centres to the main data processing centre, main data processing centre to the data warehouse and finally to users. At present only a few of the above offices are electronically connected through a communication network, either though the NICNET or through e-mail provided by various Internet Service Providers. Though networking of all important statistical offices is necessary, it need not be a dedicated one at the present.

14.7.23               A number of standard classifications like the National Industrial Classification (NIC), National Classification of Occupations (NCO), Indian Trade Classification based on Harmonised Commodity Description and Coding System {ITC(HS)}, Standard Classification of Diseases, etc. are presently in use. There is a need to develop a computerised system to facilitate searching the appropriate code of any classificatory variable easily with the help of some key words about it. The system needs to be made available on the Internet through website. Such an arrangement would help the potential users of Classification.

Deficiencies

14.7.24                Though the NSSO has been conducting several rounds of surveys year after year, there has been no specific attempt to build up a specialist group for survey data processing – with deep knowledge not only of computer software and hardware but also of the subject matter of the surveys. On the contrary, the group that had worked hard to acquire the required skill and knowledge and was able to remove the long-standing backlog of unfinished tabulation of NSS data, was unceremoniously split up and transferred to areas where the skills acquired by them may not be of much use to the new organisation.

14.7.25               The system of processing State-level data is very weak. Inadequacy of hardware and software and availability of trained manpower is a problem in many of the States.

14.7.26               Acquiring expensive equipment and embarking upon a large-scale experiment without adequate examination of the pros and cons, as in the case of palmtop computers, is counterproductive and wasteful.

14.7.27               The department has often got to arrange consultation with specialists working in different places. This is done by arranging a meeting at a common place, which costs quite a lot in the form of travel expenses and wasted time.

14.7.28               In the past, the Government constituted ad hoc technical committees to recommend changes in the IT set up of its department. Delays in implementation usually meant that the recommendations were outdated by the time they were implemented.

14.7.29               According to the National Policy on Data Dissemination, the survey results and unit-level data should be made available to data users in India and abroad after the expiry of three years from the completion of the fieldwork or after reports based on survey data are released, whichever is earlier. Naturally, the unit-level data disseminated would be authentic in the sense that if any re-tabulation is done using these data, the new results would agree exactly with the corresponding published table. The Commission carried out an exercise to examine this. Table I of ASI 1995-96 Report was re-tabulated from the publicly available unit-level data. Large unexplainable discrepancies were found between the two. This raises serious question, whether the unit-level data supplied, or, the tables published in the report were wrong. This jeopardises the credibility of the statistical system. It brings out the importance of extreme care in as commonplace a task as preparation of statistical tables.

14.7.30               Understandably, the dissemination policy is likely to put the statistical system under considerable strain but in the larger interest of the user community, the Commission would urge the Government to continue it.

Recommendations

14.7.31               The Commission makes the following recommendations:

(i)                  The Government must develop and nurture expertise and skills in various areas of specialisation - statistical software being one of the most important amongst them. Training and transfer policies must be framed accordingly. Transferring specialist officials to positions, in which their specialised knowledge is of no use, is a waste. A software group consisting of systematically-trained officers in IT tools should be set up in the National Statistical Office, to meet all software requirements of the NSO. When in-house expertise and resources are not available, data processing or software development projects could be given to agencies of proven competence.

(ii)                The area of application of computers should be widened to cover statistical modelling, forecasting, simulation, and other sophisticated “applicable” theoretical methods.

(iii)               It is essential to establish strong communication links between:

(iv)              These offices should be networked through one or more Internet Service Providers, and/or one or more Virtual Private Network. A dedicated computer network is neither necessary nor desirable and would not at all be cost effective.

(v)                Urgent steps must be taken to strengthen computer hardware and software systems in the State DESs.

(vi)              To cut down travel expenses and waste of time, it would be more economic and convenient to go in for video conferencing facilities, which are comparatively inexpensive when held between a pair of participants.

(vii)             Before investing in expensive sophisticated equipment, a feasibility study including cost-benefit analysis must be carried out. When the equipment are to be used by primary workers under field conditions, as in the case of palmtop or laptop computers, practical difficulties of maintenance, repair, local availability of consumables and the procedural problems of handing over expensive Government property to primary workers should be carefully examined.

(viii)           Specifically in respect of palmtop and laptop computers, the Commission is of the view that these are not needed for collection of data in large-scale sample surveys at present except when information content is small. The large number of palmtop computers already purchased, if they are still serviceable, should be used in surveys with small information content - price data collection, for example.

(ix)              However, Methodological studies on Computer Aided Interviews, as a collaborative venture of survey practitioners, software specialists, subject-matter specialists and psychologists is recommended. The first attempt should be to reduce the questionnaire to a reasonable size, which can be honestly answered in less than one hour. The question of development of appropriate software is of second priority. It should be emphasised that software for laptop computers can very well be developed on PCs and no investment on laptop computers would be necessary for this methodological study.

(x)                For mobile applications, a few laptops should be available in each large statistical office.

(xi)              In the ASI or in the envisaged Survey of Non-Manufacturing Industries, attempts should be made to collect information on electronic media from enterprises, which use computers for accounting purposes.

(xii)             The existing practice of publishing survey results in the form of multiple cross-classified tables with, in many cases, a large number of empty cells should be stopped. Only readable reports, with simple tables and their interpretation, should be published. For experts and professionals, the results and unit-level data should be made available in an electronic medium like a Compact or Floppy Disk. Survey results and other important statistical information should be put on the website of the NSO.

(xiii)           There should be regular computer training programmes for statistical personnel at all levels.

(xiv)           The Commission has noted with serious concern that there are occasions when the unit-level data as well as summary tables computed from them, both disseminated under the National Policy on Data Dissemination, do not match. In order to establish its credibility, the Government should investigate the reasons for the discrepancies and assign institutional responsibility for the failure. A case of immediate concern is the data and results of ASI 1995-96.

(xv)            A Standing Technical Committee on IT should be set up in the proposed NCS, to lay down policies and review their implementation.

(xvi)           A website of all classifications, concordance tables along with online database query system should be developed for public use. This system should help the user in identifying a code on the basis of part description or key words.

14.7.32               A recommendation relating to conversion of the Computer Centre as the Data Storage and Dissemination Office is given earlier in paragraph 14.5.17.

14.8                     Legal Provisions for the Statistical System

Introduction

14.8.1                  In any statistical system, data are collected either directly from individuals and institutions or from administrative records or statutory returns. The success of data collection depends to a large extent, on the cooperation of the respondents. Many countries have found it necessary to put the entire data collection within a legal framework, laying down the obligations and rights of the respondents as well as those of the data collectors.

14.8.2                  In our country, there are two such laws: Census Act, 1948 and Collection of Statistics Act, 1953.There is a third set of laws, mainly for administrative purposes, under which statutory returns have to be submitted to specified authorities. These returns are an important source of official statistics. A few of such laws are listed at Annexe 14.10.

14.8.3                  Many of these laws are outdated and may require revision. The Government of India had set up in 1998 a Commission to review such laws, but it felt handicapped by the non-availability of rules, and regulations related to this. Comments from different ministries on some of these laws are given at Annexe 14.11.

Collection of Statistics Act, 1953

14.8.4                  This Act is to facilitate the collection of statistics of a certain kind relating to industries, trade, and commerce. The Act specifies that the Central or a State Government may appoint a Statistics Authority, who in turn may serve on the owner of an industrial or a commercial concern or an individual, a notice requiring him to furnish certain listed information about the concern. It confers the right of access to relevant records or documents and restricts publication of any information disclosing the identity of the concern. It also provides for certain penalties for wilfully refusing to furnish, or, the furnishing of false information. Details of the Act are given in Annexe 14.12.

14.8.5                  However, this law has so far been used for the restricted purpose of conducting an Annual Survey of Industries, to collect information on input, output and employment from a limited segment of the industrial sector. A very large segment of industries remains uncovered and even the penalty clause has not been successful in avoiding non-response. A detailed account of the limitations of this law is given in Annexe 14.13.

Census Act, 1948

14.8.6                  The Population Census, which is a Union subject, is conducted under the Census Act, 1948. The Act empowers the Central Government to take a Population Census of the country after duly notifying its intention to do so. It empowers the Central Government to appoint Census Commissioner and State Governments Census officers to take the census. The Census Act is utilised for fixing primary administrative responsibility, for obtaining necessary funds, for determining the general scope and timing of the Census, for placing a legal obligation upon the people to give truthful answers, and for placing a legal obligation upon the enumerator to record the responses faithfully. While the Act makes it obligatory for the public to answer all questions faithfully, simultaneously, it guarantees confidentiality of information in respect of individuals. It provides penalties for (a) census officers if they fail in their official work, and (b) the respondents if they provide wrong information. The Act has been further amended vide the Census (Amendment) Act 1993   (see Annexe 14.14).

Need for Broader Legislation

14.8.7                  At present, these two Acts cover only limited areas of data collection: (a) basically demographic information through Population Censuses, and (b) industrial information through Annual Survey of Industries. There is no legal basis for other large-scale data collection efforts through censuses or sample surveys. A common feature of these two Acts is to make it obligatory for the respondent to supply correct information and for the data collector to maintain confidentiality of information from a specific individual. There is provision for penalties for failures on either side, but these have seldom been availed of. It is considered necessary that legislation of a general type is required to provide a legal framework for collection of other Core Statistics.

14.8.8                  The Commission has recommended the setting up of a permanent National Commission on Statistics (NCS). Introduction of an entirely new legislation is necessary for creating the proposed Commission, laying down its composition, authority, responsibility and procedure of work.

14.8.9                  In the following paragraphs, legislative requirements for the three sets of Acts are discussed separately. Whether these could be incorporated into a single Act is a matter for legal experts to decide.

 Legislation in respect of proposed National Commission on Statistics

14.8.10               This being a new arrangement, various aspects of the Commission’s functioning, its relationship with different official agencies, other institutions, etc. as also the necessary mechanisms required for its effective functioning, have to be examined in detail and appropriate legislation thereof has to be put in place. However, to ensure that the legislation is actually effective in practice and fulfils its objectives it would be desirable, not to draft it in advance but to let it be evolved by the proposed Commission itself, taking into account the ground realities, the emerging requirements, etc. when it starts to function. It is therefore suggested that the NCS be established as early as possible (within six months) with a modicum of authority, through a Government Order. Thereafter, in consultation with the Law Ministry and other appropriate agencies and in the light of its own operational experience, the NCS could evolve within a short period the appropriate legislation. This is in fact the procedure adopted by the National Statistics Commission of the UK, which came into being on 7th June 2000 by a Government Act. The UK Commission is still working on the legislation to define its status.

14.8.11               The proposed NCS, which is envisaged to be a high-level nodal body accountable to the parliament and having a policy-making, standard-setting and co-ordinating role for the statistical system, has to be empowered for the crucial role of framing legislation.

Legislation for the proposed National Commission on Statistics

14.8.12                Legislation for the proposed National Commission on Statistics (NCS) should inter alia address the following important issues:

(a)                Constitution;

(b)                Status, powers and functions; 

(c)                Terms and conditions of service of Chairman and Members of National Commission on Statistics: 

(d)                Modalities of functioning of National Commission on Statistics:

(e)                Mandate of NCS on Core Statistics:

(f)                 Budget, Accounts and Audit:

Legal Provision for Collection of Statistics

14.8.13               Necessary legal provisions should be made, either by expanding the scope of the present Collection of Statistics Act (1953) or by passing new Act or Acts to:

(a)                Cover any topic under Core Statistics, as defined by the proposed NCS;

(b)                Make it obligatory on the part of individuals, or enterprises, or State and private agencies to provide the information sought for any survey under the aegis of the NCS;

(c)                Provide right of access to records, including the record of Government agencies for statistical purposes;

(d)                Ensure the informant’s right to privacy by making it illegal to publish the identity of the informant, or by requiring him to furnish sensitive information;

(e)                Provide penalties for informants, for their refusal to supply, or for wilfully supplying wrong information;

(f)                 Make it a penal offence for a statistical officer authorised to collect, process, or disseminate information collected from any survey under the Act, to wilfully distort or manipulate the data.

14.8.14               Though clauses 3(a), (b) and (c) of the Collection of Statistics Act (1953) on the one hand gives the State Governments the right to name a Statistical Authority, it seems to be substantially negated by subsequent provisions of the Act under the same clause. As a matter of fact, though there have been occasional requests from other data-collecting agencies to be named as Statistics Authority under the Act, the Government of India had seldom agreed to such a request. The sole Statistics Authority for the Annual Survey of Industries has always been the Head of the Field Operations Division of the NSSO, while the States have also been engaged in collection of ASI data, without any such Statistics Authority. To strengthen the attempts of the States to collect industrial statistics, particularly through the ASI, the Government of India should delegate to them, as was done earlier, the necessary legal authority.

14.8.15               Though “Statistics” is under the Concurrent List of the Constitution and “Surveys” is only under the Union list, it is more important that the collection of statistics on any subject vests in the authority (Central Ministry or State Government Department) that is responsible for that subject according to its status in the Union, State or Concurrent Lists in the Constitution of India. When the NCS determines certain statistics as Core Statistics, in deciding on the agency that should be responsible for their collection, it will have to do so in consistence with the distribution of subjects in the three Lists. Therefore, when proposing legal measures for Core Statistics, the NCS may have to propose different Acts for different subjects according to the List to which the subject belongs. 

Modification of the Census Act

14.8.16               Twice in the past, the Economic Census of India had been carried out as part of the house-listing operation of Population Census. Recently doubts have been raised, whether the Census Act permits this, and consequently the Economic Census has been de-linked from the Population Census. The Commission is of the view that it would be desirable to revert back to the old practice and modify if necessary the Census Act, 1948 and Census (Amendment) Act, 1993 for this purpose.                  

14.9                     Statistical Audit

Introduction

14.9.1                  One of the terms of reference for the Commission is “to examine the need for instituting statistical audit of the range of services provided by the Government and local bodies and make suitable recommendations thereof”. For efficient execution of any such project for providing services to the general public, Statistics are required at every stage – project formulation, execution and evaluation on completion. Apart from the basic information required in setting the targets of the project and assessing resource needs and availability, a benchmark survey is usually carried out to record initial conditions at the start of the project. During the execution of the project, collection of information is also necessary to monitor its progress, while evaluation and audit of the performance conclude the exercise.  Since statistical methods used generally vary from project to project depending on their nature, the Commission is not in a position to make any general recommendation, except that the Government should make use of the statistical expertise of the NSO in planning, monitoring and evaluation of large projects of national importance.

14.9.2                  Instead, the Commission has examined the more pointed issue of control and certification of quality of statistics and the process of their generation.

14.9.3                  Statistical audit is necessary to assure the user not only of the quality of data presented, but also of the soundness of concepts, definitions and of the entire system of collection, processing, summarisation and dissemination of data. Currently, the CSO is responsible for setting up standards in respect of concepts and definitions, but there is no system of certification of either the process of generation, or the generated statistics. It is envisaged that the proposed National Commission on Statistics would be formally responsible for certification of the ‘quality’ of Core Statistics generated by the official statistical system, including the data collected through administrative records, surveys and censuses and would get this accomplished departmentally through the NSO. Certification of the quality of statistics may be extended to private producers of statistics on a voluntary basis.

Experience in Different Countries

14.9.4                  Different countries have different approaches to the assessment of data quality. Since 1988, Statistics Sweden has produced an annual quality report. This report deals with the external factors affecting the quality of the statistical product This provides a basis for the development of the quality in their statistical products. For American Housing Survey, Quality Profiles are produced gathering together into one document, information on all aspects of data quality. The quality profile includes sections on sample design, data collection procedures, non-response error, measurement errors, data processing, weighting, sampling and a comparison of survey data with external data. Canada and Australia have laid down Statistics Quality Guidelines. These guidelines have to be followed before publishing the results. The guidelines require that an assessment of the quality of statistics produced should be published with the results.

Determinants of Quality of Statistics

14.9.5                  The three most important determinants of the quality of statistics are validity, reliability and timeliness. Validity is concerned with the relevance of the collected statistics to the subject of the study. Reliability is assessed in terms of the extent to which the data are free from sampling and non-sampling errors. Statistics are timely if they are available when they are needed.

Internal and External Audit

14.9.6                  Concurrent audit of statistical activities is necessary for early detection of errors and mistakes during the progress of work, and their rectification in time. This is essentially an internal activity of the data-collecting organisation. Assessment and certification of the quality of the end product are done through an audit - by an external authorised auditor - of the final results embodied in a report and other related records and documents. The most important amongst all the items of information that should be made available for such an audit are listed below.

Recommendations

Items for Audit

14.9.7                  A fairly long but not exhaustive list of items, which could be audited is given below:

(a)                Theoretical concepts and their modification into operationally feasible definitions, covering individual respondents, population of such respondents, sampling unit and information unit, frame and its adequacy, information to be collected from the respondents, etc.

(b)                Methods of data collection and handling: Interview, Direct observation or measurement, copied from records, mail enquiry, etc.; Steps to avoid respondents' or interviewers' bias; Treatment of sensitive questions; Deliberate redundancy to check consistency of information; Data transcription and scrutiny; Classification and coding; Choice of reference period and survey period; Design of questionnaire or schedule; Instruction manual.

(c)                In the case of secondary data acquired from administrative records – design of the form for recording and summarising, incomplete coverage and treatment of missing data, definitional consistency, checks on arithmetical errors.

(d)                Sampling design: uni-stage or multi-stage, procedure for stratification and selection at very stage; Formula for estimation of parameters.

(e)                Procedure for control and assessment of sampling and non-sampling errors; Manual and computerised procedures of scrutiny and editing of data; Methods for imputation of missing or rejected observations.

(f)                 Data handling errors - Errors in data capture, editing, coding of open-ended textual responses, data processing, etc.

Obligation of Producers of Core Statistics

14.9.8                  It will be obligatory for every producer of Core Statistics to supply a copy of their report to the NCS for audit. The producer of the report should certify that all standards set by the NCS have been met, or draw attention to deviations, in case there are any. Their reports must contain information on all the above items for audit. In addition, the report should give its own assessment of the magnitudes of sampling and non-sampling errors associated with the more important statistics produced by them.

Audit Procedure

14.9.9                  The NCS may accept the producer’s certificate or carry out an audit of the Report and related documents through the NSO. For the purpose of the audit the NCS may call upon the producer to supply additional records or documents. The result of the audit would be communicated to the producer and would also be included in the annual report of the NCS.

14.10                Human Resource Development

Staffing Pattern at the Centre

14.10.1               At the Centre, the Ministry of Statistics and Programme Implementation (MoS&PI) is the nodal ministry dealing with various aspects of statistics. Most of the ministries and departments have Statistical Units or Divisions manned by staff meant to deal with statistics related to their department. Statistical staff can broadly be classified as technical-cum-supervisory and subordinate. Officers from the Indian Statistical Service (ISS) carry out technical-cum-supervisory work at various levels in the MoS&PI and some other but not all central ministries.

14.10.2               On the other hand, the subordinate staff, engaged in either the primary work of data collection, manual or computerised processing of data, or in lower level supervision of such primary work, do not belong to any organised service. They work in different ministries and departments and usually all promotions are within the respective ministry or department.

Training aspects

Current Activities

14.10.3               As there is a continual change in, and expansion of areas covered by statistics, and constant change and improvement in the methodology of data collection, analysis, interpretation and dissemination, it is essential for the practitioners of statistics to keep abreast of developments in the discipline, through a properly organised training programme. Training is needed at all levels – at senior levels of management, where the main responsibility is to plan, introduce and manage innovations, at the middle level, where the main job is to carry out ongoing jobs efficiently, and at the operational level, where the operational personnel must be trained in the details of the specific job on hand. To meet this need, presently the MoS&PI has the following arrangements:

Training Arrangements of MoS&PI

14.10.4               The MoS&PI has a Training Division in the CSO, which arranges training programmes not only for the officers and staff of the Ministry but also to statistical personnel working in other Government organisations. In NSSO, the FOD has training centres for training its own field personnel mainly in field operations. The other organisations of NSSO like SDRD and DPD lack such training facilities. However they do provide in-house training informally to their officers and staff.  Besides these, various organisations of the ministry, from time to time, organise seminars and talks on important and emerging topics by renowned scholars and experts in the concerned subject fields.

Training Courses organised by the Central Statistical Organisation 

14.10.5               The Central Statistical Organisation in the MoS&PI has a Training Division, which is under the charge of an Additional Director General. A Director along with a small team of officers and supporting staff assists him in his task.

14.10.6               A list of regular training courses run by the Training Division, is given below:

(a)                Two years Probationary training programme for the direct recruits of Indian Statistical Service (ISS) officers;

(b)                Induction Course for the officers promoted to the Indian Statistical Service;

(c)                Junior Certificate Course in Statistics for the statistical staff working in Central and State governments and Public Sector Undertakings;

(d)                Senior Certificate Course in Statistics for the statistical staff and officers working in Central and State governments and Public Sector Undertakings;

(e)                A 4-week Course on “National and International Statistical Systems” for M-Stat students of the Indian Statistical Institute (ISI); and

(f)                 A 6-week Course on “Official Statistics and Related Methodology” for the participants of the International Statistical Education Centre (ISEC), Kolkata.

14.10.7               Apart from the above regular courses, the Training Division organises occasional Refresher Courses for ISS officers for duration of about one to two weeks. A list of such courses organised during the years 1996-97 onwards is given at Annexe 14.15. Subject-wise break-up of the 22 courses so organised, shows that as many as eight of them were on computers, five on sample survey techniques (of which two were on small area techniques), two on management, and the rest on topics of Decentralised Planning, Gender Statistics, Applied Econometrics, Index Numbers, National Accounts, Database for Population Programmes, and Poverty. 

14.10.8               The physical facilities presently available with the Training Division for training purposes are: (a) two class-rooms to accommodate about sixty trainees in all, (b) a few PCs, two LCD projectors, three overhead projectors and an amplifier (sound system), and (c) access to the departmental library. Training arrangements are of three types: One, in the class-rooms of the Training Division, where teaching is done mostly by invited experts; Two, at different Academic, Training and Research Institutes like the Indian Statistical Institute, Indian Agricultural Statistics Research Institute, etc. and at computer organisations like the National Infromatics Centre, Computer Maintenance Corporation, NIIT, etc. and Three, by different departmental statistical officers in their respective offices. The arrangements have a financial component: institutions are paid a course fee, and individuals involved in the training as faculty is financially compensated.

14.10.9               The main responsibility of the Training Division has therefore been of coordination of training activities with the external organisations, statistical offices and individuals. There is no integrated plan of training, no annual calendar, no linkage of training with deployment of personnel. In-service job-linked training is arranged independently by different operating divisions of the National Sample Survey Organisation (NSSO), with which the Training Division is usually not associated.

Training Courses organised by the National Sample Survey Organisation

14.10.10           In-service courses organised by the Field Operations Division (FOD) of the National Sample Survey Organisation (NSSO) at its six Zonal Training Centres are as follows:

(a)                Induction courses for the newly recruited Investigators (14 days);

(b)                Refresher courses for the Investigators (7 days);

(c)                Scheme-Specific courses namely, Socio-Economic Surveys, Industrial Statistics, Agricultural Statistics and Urban Frame Survey for Assistant Superintendents, (7 to 10 days);

(d)                Survey Management Courses for Superintendents (10 days);

(e)                Special Training Programme covering various subjects connected with field work of NSSO Surveys (8 days);

(f)                 Training in Computer Operations (7 to 14 days);

(g)                Field training programme for ISS probationers - orientation at FOD Headquarters in New Delhi, followed by practical training at different zonal centres and regional and sub-regional offices of the FOD (4 to 8 weeks); 

(h)                In addition, FOD officials are sent for computer training at the National Informatics Centre, Government of India.

14.10.11           Besides providing in-service training to the officers and staff, the Survey Design and Research Division (SDRD), NSSO participates in the following training activities:

(a)                Training of field personnel of FOD, NSSO and State Government personnel on concepts, definitions, coverage, sampling design, etc. relevant to the current round of the NSS;

(b)                Annual programme of Training of ISS probationers;

(c)                Special programme for ISS officers on computer software by private agencies;

(d)                Organising, from time to time, seminar or talks on important, relevant, current and emerging topics by renowned scholars and experts in the concerned subject fields;

(e)                Organising the training programmes on “sample surveys and related topics” for students of JCCS and ISEC.   

14.10.12           The Data Processing Division (DPD), NSSO organises Workshops on Data Processing at various Data Processing Centres in each NSS Round to train its officials on scrutiny of schedules, data entry, data validation, preparation of ‘directory’ and ‘multiplier’ files, etc. Further, the Division imparts in-house training on different relevant topics such as administration, statistics, computers and allied subjects to its officers and staff, besides imparting training on “data processing” to officials of State Governments. 

14.10.13           The Computer Centre has been conducting Electronic Data Processing (EDP) courses for various States and Central Government Departments and International agencies. Over a period of time, the Centre has trained a large number of officers in Electronic Data Processing.  Besides, it has conducted six "Programmer" level courses and two "Training of Trainers" level courses in EDP for UN-sponsored candidates from the ESCAP region countries under the United Nations Household Survey Capability Programme between 1983 and 1991. Currently, the Centre is conducting courses on Information Technology (IT) for officers under Junior Certificate Courses in Statistics (JCCS), software packages for middle level ISS officers in addition to in-house training courses on new topics of IT for its own officers and staff.

Training arrangements at State Statistical Organisations

14.10.14           State Statistical Organisations have, by and large, very limited and inadequate facilities for organising training programme for their statistical officers and staff. They send their officials usually to take part in Junior and Senior Certificate Courses in Statistics arranged by the CSO.

Deficiencies

14.10.15        The main deficiencies in the present training programmes of MoS&PI are as follows:      

(a)                There is a marked lack of infra-structural facilities – class rooms, computers, hostel for trainees, and a regular, committed, well equipped and full time faculty and team of trainers. 

(b)                The main task of the Training Division up till now has been one of coordination of training, rather than imparting actual training. Consequently, there is a scarcity of expertise in teaching hard technical courses.

(c)                There is no systematic arrangement for induction training on transfer to a new position that requires new knowledge.

(d)                There is no long-term plan of development of the statistical system and consequently no long-term assessment of training needs, particularly in the highly applicable technical areas. Apart from the regular courses run by the Training Division and the in-service courses run by Field Operations Division of NSSO, which have been more or less standardised, there is no organised plan for Refresher Courses for ISS officers.

(e)                There is no Advance Training Calendar to help the trainees choose a training programme of their interest. 

(f)                 At present, there is often no linkage between the kind of training received by an officer and his or her assignment. The fact that Statistics, like Medicine, is a vast area where specialisation is absolutely necessary does not seem to be appreciated in the statistics officialdom. That it is a great wastage of manpower resources, if for example, an officer who has long experience of survey data processing is sent for training on time-series analysis and then transferred, on successful completion of training, to work on compilation of national accounts, is not realised.

(g)                Methods used in official statistics are of two different kinds: (1) Methods that are subject matter specific, and (2) General analytic methods not restricted to any specific subject matter. Concepts, definitions, summary measures, tabulation plans, etc. are very much subject matter specific, whereas statistical methods used in sampling design, estimation of survey parameters, imputation of missing values, use of auxiliary information, analysis of time series and forecasting, classificatory techniques, statistical modelling, etc. are, by and large, not so. Training on advances in general analytical methods of statistics seems to have received less attention.

(h)                Training in basic principles of economics as well as in the development of the skill in communications is a must for every statistical officer. But there is no training in these areas.

Recommendations

14.10.16           The Commission makes the following recommendations for immediate implementation:

(i)                  A suitable Central Training Facility for Indian Statistical Service officers and senior statistical officers of State Governments, equipped with accommodation, arrangement for food, lecture rooms, computer laboratory, and library should be constructed for trainees and visiting teachers.

(ii)                There is an immediate need for extensive arrangements for training of trainers so that when availability of qualified trainers is assured, eventually the Central Training Facility could be transformed into a Training Academy.

(iii)               Arrangements for training of subordinate operational staff should be decentralised and separate. The existing in-house training facilities of NSSO should be further strengthened. 

(iv)              It should be made mandatory for each ISS officer to undergo Refresher Training for a period of at least four weeks every two years. The training could be either in-house at the MoS&PI, or at any other Institute in India or abroad. Training in respect of new practices in official statistics could be arranged in-house or through participation in training programmes arranged by professional agencies like the International Association of Survey Statisticians, the U.S. Bureau of Census, Statistics Canada, etc. For broadening knowledge of ‘applicable’ statistical theory, training arrangements could be made with universities in India or abroad or with reputed research and training organisations like the Indian Statistical Institute, Indian Agricultural Statistics Research Institute, etc. Study leave with financial support should be provided to promising ISS officers working for doctorate degrees in relevant subjects.

(v)                Refresher Training Courses should be arranged in the following illustrative list of areas:

(a)          Principles of Economics

(b)          Communication skills  

(c)          System of National Accounts

(d)          Time Series Analysis, Forecasting and Modelling

(e)          Small Area Estimation

(f)           Geographic Information System

(g)          Management

(h)          Information Technology

(i)            Classificatory Analysis

(j)            Market Research.

(vi)              ISS officers should be eligible for Sabbatical leave for pursuing advanced studies related to their area of specialisation.

(vii)             Training and deployment should be linked. The Cadre Management system should be suitably streamlined for this purpose.

(viii)           The Annual Training Calendar should be announced in advance.

(ix)              The content of the Junior and Senior Certificate Courses in Statistics should be reorganised into smaller modules and offered on a large scale to the supporting statistical personnel (both from the Central and State Governments) who need training at this level. The training also should be decentralised and organised by State Directorates of Economic and Statistics. The MoS&PI should organise Training of Trainers Sessions for this program.

(x)                In order to achieve closer collaboration between academicians and professionals, a suitable system should be developed to enable teachers and researchers from academic institutions to work in the Ministry of Statistics and Programme Implementation, Government of India and vice-versa.

14.10.17           The Commission further recommends that a high-level committee should be set up by the Government of India to evolve a long-term plan for assessing and effectively meeting the training needs for the Central and State Statistical Systems, consistent with what would be expected from the system. The said committee should also be required to examine in this connection whether a Staff Training Institute is necessary and feasible, or, whether the need could be met through cooperation with existing organisations. Such a Committee could comprise as its members, amongst others, the Director of the Indian Statistical Institute as also the envisaged National Statistician besides eminent statisticians with proven academic and professional credentials.

Career Management

14.10.18           The effectiveness of career planning in an organisation or in a system largely depends upon the extent to which training and development opportunities are made available to the employees to enable them to realise their growth potential and to contribute towards the achievement of the organisational goals and objectives.

14.10.19           The basic information needed for cadre management relates to: (a) goals of the organisation and job requirement to meet the goals, on the one hand, and (b) personnel available and their capacity to carry out the jobs, on the other.

14.10.20           Like medicine, the discipline of statistics is an agglomeration of various areas of specialisation, though there is also need for the general practitioners. The Commission recognises the following broad areas of specialisation in the statistical system:

(a)                Applied Statistics – sampling design, time–series analysis and forecasting, statistical modelling and inference, classificatory techniques, etc.;

(b)                Computer linked areas – Systems analysis and software development, survey data processing, data bank and data warehouse management, desk top publishing, computer management, etc.;

(c)                National Accounts;

(d)                Agricultural Statistics;

(e)                Industrial and Commercial Statistics; 

(f)                 Population and Socio-economic Statistics;

(g)                Field Operations.

14.10.21           The career of each ISS officer should be planned so that he or she can specialise in a number of the above areas, gradually narrowing down the areas of specialization over the years. Concerted efforts of the ISS Cadre Controlling Authority and the Training Division of CSO would be necessary to achieve this. 

Recommendations

14.10.22        The Commission makes the following recommendations:

(i)                  During the first 15 years or so of the career, every ISS officer should work in about four of the following areas: Applied Statistics (sampling design, time–series analysis and forecasting, statistical modelling and inference, classificatory techniques, etc.), Computer-linked areas (Systems analysis and software development, survey data processing, data bank and data warehouse management, desk top publication, computer management etc.), National Accounts, Agricultural Statistics, Industrial and Commercial Statistics, Population and Socio-economic Statistics, and Field Operations.

(ii)                Gradually during the next 5 years or so, he or she should be required to narrow down the area of specialisation to only one or two of these areas. The role of the cadre management should be to help harmonise the choice of specialisation by the officer with the goals of the organisation. All transfers and training of an officer should be consistent with the goals of specialisation.

(iii)               A computerised database of details of qualifications, job experience and training undergone by every officer must be maintained up-to-date.

Improving Morale of Staff and Officers

14.10.23           The Commission took cognisance of the representations from the ISS officers’ Association and a number of subordinate staff associations regarding severe lack of prospects of career advancement in their respective categories. The Commission is convinced that their grievances are genuine and urges the Government to take necessary steps to redress their grievances.

Subordinate Staff

14.10.24           For subordinate staff, the Commission was informed by their representatives that the percentage of officials working at the same post or scale for 13 years or more is about 39 per cent in case of Senior Investigators of the Central Statistical Organisation, 80 per cent for Data Processing Assistants of the Computer Centre, 46 per cent for Data Processing Assistants of the Data Processing Division of the National Sample Survey Organisation, 34 per cent for Investigators and more than 60 per cent for Assistant Superintendents of the of the Field  Operations Division of the same organisation. To improve the morale of the staff, the Fifth Central Pay Commission recommended upgradation of certain posts and the constitution of a Subordinate Statistical Service, bringing under one umbrella two separate categories of staff: field workers and statistical investigators. There was no suggestion to bring the third category namely, data processing staff under the same umbrella. The subordinate service will have two levels of workers to be called Statistical Investigator Grade I and Grade II, respectively. It will offer the subordinate cadre an entry path through promotion to the ISS in the ratio of 40:60 for promotions against direct recruitment. A common seniority list will be maintained for this purpose. However, no such entry path to the ISS is provided for the data processing staff.

14.10.25           The MoS&PI informed the Commission about the progress in the implementation of the Subordinate Statistical Service. Draft recruitment rules governing the Subordinate Statistical Service have been prepared and circulated amongst concerned Ministries and Departments for comments. Information about posts, pay scales, number of incumbents and recruitment procedures in participating Ministries has been collected. Pending constitution of the Subordinate Statistical Service, upgraded replacement scales of pay have been granted with effect from 1 January 1996 to Group B and C statistical functionaries as recommended by the Fifth Central Pay Commission. Abolition of Assistant Superintendent post in the FOD (NSSO) is also under consideration of the Government so that an overall two-tier structure can be adopted in the Service.   

14.10.26           The Commission is convinced that there is no need to maintain the present distinctions amongst the subordinate staff: field workers, statistical investigators and data processing staff. Computers are user-friendly now and do not require much of specialisation at lower levels of use. A whole life spent in data collection from the field is monotonous and breeds too much familiarity and a know-all attitude. However, the main purported reason for creating the subordinate service is not unification of functions, but to provide better opportunity of career advancement for lower level statistical personnel working in small or outlying offices.  To what extent this can be achieved should be examined.

14.10.27           It appears that to make the Subordinate Statistical Service viable, quite a few ex-cadre positions would have to be created. The Commission would like to point out that the main difficulty in ISS cadre management had arisen out of recruitment or placement of a large number of persons in an ad hoc manner in the ISS and subsequent failure to regularise the placements. The affected persons went to the Supreme Court for redressal. According to the dispensation by the court, they had to be absorbed in the cadre with retrospective effect. The base of the service was thereby extensively widened. The number of positions at higher levels has however remained the same reducing the prospects of promotion in the ISS. It does not appear to be a logical proposition to create a number of posts outside a proposed new cadre to make the new cadre viable. Also, since it appears that for the relatively small number of positions in the ISS likely to be available annually, there would be a very large number of aspirants in the subordinate service, the extent of improvements of promotion may have to be examined carefully.

Recommendation

14.10.28           The Commission recommends that:

(i)                  As the constitution of the Subordinate Statistical Service has been recommended by the Fifth Central Pay Commission and the related issues are under active consideration of the Government, all relevant aspects as mentioned in the body of the report should be considered by the Government while constituting the service.

Indian Statistical Service

14.10.29           The Indian Statistical Service (ISS) was constituted as a Group A Central Service on 1 November 1961 by pooling together statistical posts offered voluntarily by different ministries and departments of Government of India and en-cadering them. The service was created with the laudable objective of unification of all statistical work, and providing its officers good opportunities of career advancement. But it was born with a congenital defect – a bottom-heavy structure, with very few positions at the top to match career aspirations – as only low-level positions were initially offered for en-cadrement. Recruitment to the ISS and its sister Indian Economic Service (IES) is made on all-India basis through the ISS/IES Examination conducted by the Union Public Service Commission every year.

14.10.30           For ISS officers, the Commission noted with concern that management of the ISS cadre by the Cadre Controlling Authority – MoS&PI – has not been satisfactory. There has been only a solitary cadre review since its creation. Meetings of Departmental Promotion Committees (DPC) have not been held in time to fill up existing vacancies.

14.10.31         Due to the absence of cadre reviews, many important statistical positions, which should normally come under the ISS, have not been brought into the service. Therefore, it has not been possible to change the original bottom-heavy structure of the service. As a result, deserving ISS officers have been denied opportunity of promotions for long periods so much so that some ISS officers have fallen behind their compatriots from sister services by as many as two promotions.

14.10.32           Because meetings of DPC have not been held in time, for the purpose of promotions, annual confidential reports of several batches have been considered together at one time resulting in supercession of officers of a senior batch by their juniors. Had the meetings of the DPC been held regularly, or promotions for different batches been considered separately, there would not have been so many supercessions as they exist in the ISS today. Supercessions have badly affected the morale of the ISS officers. 

14.10.33           The Fifth Central Pay Commission (CPC) had recommended regular cadre reviews for all Group A central services to identify and upgrade positions so that ultimately the following model composition of the Service is attained. The following table shows the model composition vis-à-vis the actual condition of the ISS. The preponderance of positions at the lower levels is starkly evident from the table.

ISS Cadre Structure vis-à-vis the Model Structure Recommended by 5th CPC

Scale or Grade As recommended by Fifth Central Pay Commission Percentage of ISS posts in various grades
Mandatory eligibility for grant of the grade Percentage of officers

Senior Time Scale (STS)

5th year 30 57.1
Junior Administrative Grade (JAG) 9th year 30 6.6
Non-Functional Selection Grade (NFSG) 14th year 20 30.8
Senior Administrative Grade (SAG) 17th year 17 4.7
Higher Administrative Grade (HAG) 25th year 3 0.8

14.10.34           In the opinion of the Fifth Central Pay Commission, ideally all Group A officers should reach Senior Time Scale (STS) in the fifth year, Junior Administrative Grade (JAG) in the ninth year and Non Functional Selection Grade (NFSG) in the fourteen year of their service.  However, a study conducted by ISS Association, Kolkata, gives the following gloomy picture of promotion prospects for ISS officers:

Years of Service for Different Promotion in the Present Scenario

Promotion to level Year of Joining
77 79 80 81 82 83 84 85 86 87 90 92 93
STS 5 13 12 12 11 13 12 11 12 11 8 8 8
JAG 16 16 16 17 16 24 28 29 30 30 - 25 25
NFSG 21 23 27 32 33 33 33 34 33 33 - 29 29

14.10.35           Much of the problems could have been solved, had appropriate steps been taken in the past through periodic cadre reviews as per the guidelines of the Department of Personnel and Training (DoPT). 

14.10.36           The ISS has suffered a lot because of poor cadre management and is in a pitiable condition. The Commission has learnt that the Fifth Central Pay Commission had recommended a scheme for Assured Career Progression for Government employees in order to mitigate the effects of stagnation. The Government had not accepted the recommendation for the Group A officers in the hope that regular cadre reviews would take care of the problem. However, in the case of the ISS this has not been done.

Recommendations

14.10.37           The Commission therefore recommends that:

(i)                  As a one-time ameliorative measure, ISS officers should be given the benefit by awarding them the Senior Time Scale, Junior Administrative Grade and Non Functional Selection Grade in the 5th, 9th and 14th year of their service. The Commission is aware that quite a few officers would miss this benefit because the length of their service is only marginally shorter than the specified limits set above. The Commission hopes that the next two recommendations, if implemented promptly and properly, would resolve the problem. 

(ii)                MoS&PI should immediately carry out a Cadre Review of the ISS and thereafter every five years to assess the statistical needs of different ministries and departments as per the guidelines of the DoPT.

(iii)               In addition, all ad hoc promotions should be regularised, and vacancies should be filled up immediately. A panel for promotions to fill up vacancies during the following year should be drawn up in advance in the current year itself.

(iv)              The ISS cadre should be restructured to narrow down the base so as to achieve the model cadre structure as recommended by the Fifth Central Pay Commission.

(v)                The stature of the officer responsible for management of the Cadres of Indian Statistical Service as well as Subordinate Statistical Service should be sufficiently high for better management of the cadres.



1 Source: The creation of the Office for National Statistics, 1997 by Her Majesty’s Government UK.

2               HM Treasary News Release, June, 2000 www.statistics.gov.uk

3               Framework for National Statistics, June 2000 by Her Majesty’s Government UK