This part revisits the various components of an information system and outlines minimum requirements to operationalize an information system. The responsibilities of each stakeholder are set out as well as guidelines for each of the three components based on the identified four dimensions.
The objective of the sub-system is to systematically produce and manage indicators on the living conditions of households and/or basic information for each of the priority sectors identified in PRSPs and MDGs.
Indicators
A set of minimum performance and impact indicators is proposed in annex 1. It was based on the PRSP list of indicators of eighteen countries and from the list of 48 MDG indicators on account of their relevance and statistical operational capacity. It covers the twelve areas identified in the PRSPs: macroeconomics, poverty and social inequalities, health and nutrition, education, employment, living conditions, communication and information, agriculture, infrastructure, private sector, environment, governance and participation.
The list is restrictive in the sense that it does not take into consideration certain national specificities. Each indicator indicates the data source, expected timelines, implementing body and disaggregation level.
The data used to calculate the indicators are drawn essentially from three sources:
Statistical data sources
National accounts
National accounts outline the vast majority of macroeconomic indicators. Macroeconomic indicators should be produced in a timely manner as applicable to all other indicators in keeping with international production (1993 System of National Accounting or SNA93) and dissemination (General Data Dissemination System or GDDS) standards.
Administrative records
Few of the proposed indicators are extracted from administrative records. The line government services are typically health, education, infrastructure and finance agencies.
In the short term, the statistical services of the relevant sector-specific ministries require support, particularly from NIS, to improve their statistical production which is vital in annual indicator tracking. In the medium term, such sources will become popular with improved reliability and timely dissemination of data.
Statistical surveys
The majority of indicators are extracted from the following statistical operations:
It is worth noting that such statistical surveys are based on worldwide established and agreed methods. They are conducted in most African countries. They are recommended especially for tracking MDGs though their timeliness is dependent on funding which generally comes from foreign partners, and on the capacities of States to implement them on a regular basis. Appendix 4 sets out, for each of the statistical operations, the objectives and scope.
The table below shows a tentative five- year calendar for statistical operations. It is highly dependent on available funding which is difficult to programme as evidenced by previous experience.
Year n |
Year n+1 |
Year n+2 |
Year n+3 |
Year n+4 |
|
General population and household survey (GPHS) | Every ten years | ||||
Survey on household living conditions (SHLC) | |||||
Population and health survey (PHS) | |||||
Multiple indicator cluster survey (MICS) | |||||
1-2-3 Surveys (Phases 1 and 2: employment, informal sector, education, subjective poverty, governance, democracy) | |||||
CWIQ Survey | |||||
Annual agricultural survey | |||||
Annual business survey |
Although the timeline of major statistical operations (MICS, PHS, SHLC) is often five years, it should however be pointed out that an indicator can be produced within a shorter timeline where its basic data may be derived from several statistical operations (e.g. net enrolment rate provided by GPHS, SHLC, PHS, 1-2-3 and CWIQ Surveys; infant mortality drawn from GPHS, PHS and MICS).
In a given year, the survey on household living conditions may be a combination of a CWIQ module on household living conditions and a module on expenditure on consumer goods akin to phase 3 of the 1-2-3 survey.
For real comparability between the values of an indicator calculated from different statistical operations, it is fundamental that the statistical variables and data collection methods, details and question formulation be standardized. In this connection, it is recommended that within each country, NIS should design a guide on concepts and nomenclatures used in the national statistical system.
One-off participatory surveys gather qualitative data that enhance knowledge on poverty thus enabling the adjustment of poverty reduction policies.
Products of the sub-system include:
The above products should be fully accessible to all.
The indicators and their metadata may be stored in any of the databases (such as DevInfo or 2gLDB) chosen by the country for establishment in the NIS.
Such information, including the above products, should be widely disseminated, specifically through websites, including NIS and PRSP Monitoring Unit website.
Producers of statistics
The compilation of statistics is overseen by many bodies belonging to different ministries. NIS is the central structure of the system.
Generally, NIS is responsible for compiling national accounts, business statistics, other economic statistics (pricing, foreign trade), population and social statistics.
The other major producers of statistics in the area of poverty and household living conditions monitoring are essentially services in charge of statistics in line ministries (Health, Education, Employment, Agriculture, etc.). The Central Bank (monetary statistics), the Department of Forecast (public finance) and other Departments of the Ministry in charge of finance produce statistics used mostly for calculating macroeconomic indicators.
The key line ministries classified under the subsystem on "tracking poverty and
household living conditions" are respectively responsible for the following areas:
These structures produce administrative statistics. These statistics, apart from those on agriculture and employment, are specific because they are compiled after exhaustive data collection, that is, from all the statistical units under their control (schools and universities, health facilities, courts and prisons, telecommunication companies, local governments, etc.). Data collected from statistical units must be drawn from reliable and regularly updated databases.
Stakeholder coordination
Coordination of all statistical stakeholders generally rests with a body, namely the "National Statistics Council" (NSC). Such coordination is unfortunately lacking or moribund in most States. This explains the difficulty in using harmonized standards, methods and nomenclatures, which does not facilitate data comparability but causes duplication.
Coordination of the public statistical system requires a more active NSC, running several thematic groups and a full-time secretariat. It also entails regular interaction with statistical services of ministries to track their activities and manage officers placed on secondment by NIS.
Centralizing, storing, analyzing and disseminating information
Poverty monitoring information, generally produced by diverse administrative structures, should imperatively be centralized, validated and stored by a single structure.
The administrative structure responsible for such activities may be a unit within NIS or a poverty observatory. Where the observatory is not lodged within NIS, specific relations should be established to define roles and thus ensure sound monitoring.
It is desirable for the structure to be sustainable and have permanent staff. It may however receive special support at the beginning of activities. Such structure will be responsible for collecting data from producer services for the purpose of storage, analysis and user dissemination.
The unit will disseminate collected information primarily through its website or the NIS website and in small periodical publications of 4 to 8 pages. The unit will also play the key role of circulating information to users through an appropriate medium, primarily to PRSP and MDG monitoring units, but also, after the validation of such information, to all others. Moreover, the implementation of concrete activities, particularly data collection, will lead to improved coordination of all stakeholders of the information tracking system on poverty and household living conditions.
Capacity building needs
Capacity building needs are numerous and span many areas of the "Poverty and
household living conditions tracking" subsystem, namely:
The objective of this sub-system is to monitor the implementation of poverty reduction programmes and projects.
Such a tracking system which differs from the traditional statistical system seeks to appraise the outcome of implemented strategies based on funded actions.
According to PRSP and MDG monitoring requirements, each strategic component should indicate programmes, projects and planned actions. For these actions, it is required to set specifically measurable objectives (SMO and, to programme them using timelines and budget resources.
SMO is a key variable for improving the welfare of the population; indicators
are used to measure quantities while specific values set the timelines. SMOs can
be determined through strategic areas and sub-areas of the PRSP. A general PRSP
bridge table highlighting such an allocation must be validated by the various
PRSP monitoring stakeholders. Thereafter, there is need to estimate
implementation costs and ensure that they match financial resources. This is a
long and complex process that seeks to:
Unlike poverty and household living conditions monitoring, monitoring the implementation of policies, programmes and projects is conducted initially at sector level. However, some indicators used in monitoring poverty and household living conditions (which are generally performance indicators) will appear again at the end of this monitoring subsystem.
In this sub-system, monitoring will involve calculating input and output indicators, bearing in mind that performance indicators are already dealt with in the "Poverty and household living conditions monitoring" subsystem.
Such indicators are financial and practical achievement indicators relating to the various strategic aspects of the PRSP.
Financial achievement indicators can also be calculated in relation to the action of a ministry (share of national education budget in the national budget, share of primary education budget in the national education budget, etc.).
A list of indicators of sub-system 2 is not proposed unlike sub-system 1, since the input and output indicators are directly linked to the programmes and projects implemented by each country.
To attain the poverty reduction objectives, it is necessary to align national budget preparation and management systems on PRSP needs.
The collection of reliable budget data, the codification according to targets and their disaggregation per scale contributes to feeding the poverty and social indicator database. This is essential in building national capacities to ensure timely tracking of poverty reduction policies.
This seeks to ensure that policy objectives and priorities stated in the programmes formulated by the government are well reflected in annual budgets and that the budgeting method explicitly matches targets of ministries with available resources.
In this respect, there is a need to make various expenditure classifications: Item Classification, Administrative Classification, Functional Classification, Economic Classification, Programme Classification, Geographical Classification (see annex 7).
This can be achieved either through the Medium-Term Expenditure Framework (MTEF) [2] established in ministries or through an objective-based budget expenditure tracking system using financial and non-financial indicators to be specified. This presupposes systematic cost assessment. The following four approaches can improve such cost assessment: full cost accounting, expenditure analysis per institutional structure, unit cost analysis, cost analysis by activity (see annex 7).
This sub-system has several products:
Directing the budgetary system towards PRSP activities in a participatory and ownership process requires the commitment and determination of priority ministries of the PRSP at all levels of the system (national, provincial, etc.). The active support of the ministry in charge of finance is instrumental throughout the process, as it determines the model fiscal incentive system to be used by other bodies in preparing their budgets.
The process operates in two ways:
There are many stakeholders in this sub-system:
Stakeholders coordination should be performed by the appropriate structures of the ministries in charge of finance and planning in conjunction with the PRSP unit.
Capacity building should take several forms:
Tracking poverty and household living conditions provides information for an overall assessment of progress in achieving poverty reduction objectives and understanding various changes made so far. It is therefore necessary to have more targeted tools, such as impact assessments for the sake of reporting to decision-makers and the population, successful or unsuccessful government poverty reduction actions.
Like monitoring, evaluation is a tool that helps in identifying and measuring PPPs. While monitoring is a yardstick for continuously tracking output, assessment is periodic and seeks to measure the impact and efforts of PPPs. As such, monitoring is a set of performance indicators which provides regular information, while assessment is conducted over a period of time and requires more targeted and in-depth investigations.
Impact assessment seeks to identify changes in the well-being of individuals belonging to a specific group induced by a given programme or policy. The information obtained from impact assessment can influence decisions on the extension, adjustment or scrapping of a given policy or programme and serve in rating public policies by order of priority. It is a tool that guides decision-making which furthers programme visibility to the population.
The first two sub-systems are mechanisms for continued PPP output tracking, while the "impact assessment" sub-system measures PPP efforts.
Within the context of the implementation of the PRS, the major questions raised
by impact assessment include:
To sum up, impact assessment seeks to check the relevance, implementation and performance of programmes and policies by way of comparison with a set of explicit or implicit standards for the purpose of improving public policies.
There are four major assessment methods:
Assessment by comparison with a corresponding control group is genuinely experimental and is the ideal method.
This method is ideal albeit quite cumbersome and raises ethical problems. It is described in annex 8 alongside the other three most common methods.
Other types of assessments (process and theory-based assessments for instance) also help to improve management capacities. Such methods would be used depending on the type of assessment. Meanwhile, it is important to point out that they neither measure the scale of the impact nor establish a causal relation. Cause analysis is essential in understanding the efficiency of other programmed poverty reduction interventions and hence tailoring poverty reduction strategies.
Selecting communities and/or specific programmes for impact assessment is based on a specific number of impact indicators contained in sub-system 1.
Impact assessments are not conducted systematically. There is need to select policies and interventions for impact assessment. Such selection is based on the importance of the specific policies in terms of lessons to be drawn for poverty reduction. Generally, impact assessment aims to ascertain the relevance of implemented policies and guide future poverty reduction choices.
Policies or programmes selected for impact assessment should provide a positive answer to one of the following three questions:
Policy or programme impact assessment refers to specific operations and it is unlikely to assess all policies and programmes. They require a great deal of statistical data and technical capacities, as well as lengthy timelines and attendant funding.
Impact assessment, notably performance assessment, compares the welfare variables and indicators of communities affected by the programmes and those who are not. The numerous methods used engender manifold information needs.
Strictly speaking, the data needs and corresponding collection systems of potentially selected programmes should be defined before the implementation of the programmes for the purpose of adopting the collection tools in due time.
Small coverage programmes and projects which introduce monitoring and evaluation at beginning of activities will have little trouble in compiling appropriate information.
For nationwide programmes and policies, the core issues relate to:
Considering the multiplicity of poverty reduction policies and programmes, designing such surveys are quite intricate. It is difficult to design a household survey that covers all cross sections of the target communities of such policies and programmes.
Household surveys conducted for the sake of poverty reduction policy assessment
should therefore be designed according to the following criteria:
The ensuing products are PPP impact assessment reports. Such evaluation reports must be stored and widely disseminated, especially through the NIS web site.
In consideration of the multitude of poverty reduction policies and programmes covering all the sectors of economic and social life, the procedure to choose the policies or programmes to be assessed is absolutely important.
Since the role of impact assessments is to establish the relevance of poverty
reduction policies and justify specific policy choices made by decision-makers
for the sake of accountability to all partners, the selection of policies to be
assessed must follow a clear process. In particular, the following institutional
questions must be answered:
The policies or programmes to be assessed must be chosen by sector-specific ministries under the coordination of the PRSP unit.
Impact assessment requires vasttechnical capacities; hence, they can be conducted only by institutions with such capacities. Such bodies may hail from the public sector (NIS, line statistical services, university, research institutes, etc.) or civil society (non-governmental organizations, consultancies, etc.).
Monitoring, centralizing and disseminating the gamut of evaluations should be under the responsibility of the PRSP unit.
Capacity building needs are critical in introducing these rather innovative techniques in Africa.
The link between the three sub-systems is found in the logical performance matrix of a programme or project. In fact, there is a need to define a reference situation (sub-system 1) before setting targets. Thereafter, strategies are designed to meet the set targets. Such strategies are implemented through programmes and projects that require means (human, material, financial …) to run the activities (constructing, training, organizing, treating …) [sub-system 2] whose direct results (buildings constructed, persons trained, laid down procedures, enacted legal instruments…) must produce effects (more children at school, more efficient public services, more efficient judicial system …) [sub-system 1] and an impact on development (increased literacy rate, longer life expectancy, improved rule of law …) [sub-systems 1 and 3]. Such sequencing is based on a series of logical relations (if … then) known as "cause/effect." The effects represent the medium-term results achieved through a combination of direct results while direct results are the immediate and tangible products of PPP activities.
The impact assessment sub-system seeks to analyze consistency between the products of the "Projects and programmes implementation monitoring" and "Poverty and household living conditions tracking" sub-systems, particularly the links between input and output indicators on the one hand, and results and impact indicators on the other hand. Within the context of scarce resources, the relationship between the three information sub-systems falls in line with the PRSP review that aims to better target future policies, programmes and projects so as to achieve performance-based management.
The diagram above shows the role of NIS (in green) in the iterative poverty monitoring process, as well as in designing and setting the macroeconomic benchmarks of the MTEF, in conjunction with the ministries in charge of the Plan, the Economy and Finance. The PRSP unit is responsible for initiating specific studies that will be used assessing the impact of poverty reduction policies and projects: the diagram highlights the need to include these three components in a single information system where each key stakeholder can collect data appropriately.
Each of the three sub-systems of the PRSP and MDG monitoring systems must be coordinated, as mentioned earlier:
The activities of the entire PRSP and MDG monitoring system should be obviously coordinated by a permanent structure.
The most appropriate coordination body is the National Statistics Board which is responsible for coordinating all statistical activities in a country.