III. Content of a PRSP and MDG information tracking system

This part revisits the various components of an information system and outlines minimum requirements to operationalize an information system. The responsibilities of each stakeholder are set out as well as guidelines for each of the three components based on the identified four dimensions.

III.1. Sub-system on "tracking poverty and household living conditions"

III.1.1.Objective

The objective of the sub-system is to systematically produce and manage indicators on the living conditions of households and/or basic information for each of the priority sectors identified in PRSPs and MDGs.

III.1.2. Content

III.1.2.1. Methodology

Indicators

A set of minimum performance and impact indicators is proposed in annex 1. It was based on the PRSP list of indicators of eighteen countries and from the list of 48 MDG indicators on account of their relevance and statistical operational capacity. It covers the twelve areas identified in the PRSPs: macroeconomics, poverty and social inequalities, health and nutrition, education, employment, living conditions, communication and information, agriculture, infrastructure, private sector, environment, governance and participation.

The list is restrictive in the sense that it does not take into consideration certain national specificities. Each indicator indicates the data source, expected timelines, implementing body and disaggregation level.

The data used to calculate the indicators are drawn essentially from three sources:

Statistical data sources

National accounts

National accounts outline the vast majority of macroeconomic indicators. Macroeconomic indicators should be produced in a timely manner as applicable to all other indicators in keeping with international production (1993 System of National Accounting or SNA93) and dissemination (General Data Dissemination System or GDDS) standards.

Administrative records

Few of the proposed indicators are extracted from administrative records. The line government services are typically health, education, infrastructure and finance agencies.

In the short term, the statistical services of the relevant sector-specific ministries require support, particularly from NIS, to improve their statistical production which is vital in annual indicator tracking. In the medium term, such sources will become popular with improved reliability and timely dissemination of data.

Statistical surveys

The majority of indicators are extracted from the following statistical operations:

It is worth noting that such statistical surveys are based on worldwide established and agreed methods. They are conducted in most African countries. They are recommended especially for tracking MDGs though their timeliness is dependent on funding which generally comes from foreign partners, and on the capacities of States to implement them on a regular basis. Appendix 4 sets out, for each of the statistical operations, the objectives and scope.

The table below shows a tentative five- year calendar for statistical operations. It is highly dependent on available funding which is difficult to programme as evidenced by previous experience.

Year
n
Year
n+1
Year
n+2
Year
n+3
Year
n+4
General population and household survey (GPHS) Every ten years
Survey on household living conditions (SHLC)          
Population and health survey (PHS)          
Multiple indicator cluster survey (MICS)          
1-2-3 Surveys (Phases 1 and 2: employment, informal sector, education, subjective poverty, governance, democracy)          
CWIQ Survey          
Annual agricultural survey          
Annual business survey          

Although the timeline of major statistical operations (MICS, PHS, SHLC) is often five years, it should however be pointed out that an indicator can be produced within a shorter timeline where its basic data may be derived from several statistical operations (e.g. net enrolment rate provided by GPHS, SHLC, PHS, 1-2-3 and CWIQ Surveys; infant mortality drawn from GPHS, PHS and MICS).

In a given year, the survey on household living conditions may be a combination of a CWIQ module on household living conditions and a module on expenditure on consumer goods akin to phase 3 of the 1-2-3 survey.

For real comparability between the values of an indicator calculated from different statistical operations, it is fundamental that the statistical variables and data collection methods, details and question formulation be standardized. In this connection, it is recommended that within each country, NIS should design a guide on concepts and nomenclatures used in the national statistical system.

One-off participatory surveys gather qualitative data that enhance knowledge on poverty thus enabling the adjustment of poverty reduction policies.

III.1.2.2. Products

Products of the sub-system include:

The above products should be fully accessible to all.

The indicators and their metadata may be stored in any of the databases (such as DevInfo or 2gLDB) chosen by the country for establishment in the NIS.

Such information, including the above products, should be widely disseminated, specifically through websites, including NIS and PRSP Monitoring Unit website.

III.1.3. Institutional aspects

Producers of statistics

The compilation of statistics is overseen by many bodies belonging to different ministries. NIS is the central structure of the system.

Generally, NIS is responsible for compiling national accounts, business statistics, other economic statistics (pricing, foreign trade), population and social statistics.

The other major producers of statistics in the area of poverty and household living conditions monitoring are essentially services in charge of statistics in line ministries (Health, Education, Employment, Agriculture, etc.). The Central Bank (monetary statistics), the Department of Forecast (public finance) and other Departments of the Ministry in charge of finance produce statistics used mostly for calculating macroeconomic indicators.

The key line ministries classified under the subsystem on "tracking poverty and household living conditions" are respectively responsible for the following areas:

These structures produce administrative statistics. These statistics, apart from those on agriculture and employment, are specific because they are compiled after exhaustive data collection, that is, from all the statistical units under their control (schools and universities, health facilities, courts and prisons, telecommunication companies, local governments, etc.). Data collected from statistical units must be drawn from reliable and regularly updated databases.

Stakeholder coordination

Coordination of all statistical stakeholders generally rests with a body, namely the "National Statistics Council" (NSC). Such coordination is unfortunately lacking or moribund in most States. This explains the difficulty in using harmonized standards, methods and nomenclatures, which does not facilitate data comparability but causes duplication.

Coordination of the public statistical system requires a more active NSC, running several thematic groups and a full-time secretariat. It also entails regular interaction with statistical services of ministries to track their activities and manage officers placed on secondment by NIS.

Centralizing, storing, analyzing and disseminating information

Poverty monitoring information, generally produced by diverse administrative structures, should imperatively be centralized, validated and stored by a single structure.

The administrative structure responsible for such activities may be a unit within NIS or a poverty observatory. Where the observatory is not lodged within NIS, specific relations should be established to define roles and thus ensure sound monitoring.

It is desirable for the structure to be sustainable and have permanent staff. It may however receive special support at the beginning of activities. Such structure will be responsible for collecting data from producer services for the purpose of storage, analysis and user dissemination.

The unit will disseminate collected information primarily through its website or the NIS website and in small periodical publications of 4 to 8 pages. The unit will also play the key role of circulating information to users through an appropriate medium, primarily to PRSP and MDG monitoring units, but also, after the validation of such information, to all others. Moreover, the implementation of concrete activities, particularly data collection, will lead to improved coordination of all stakeholders of the information tracking system on poverty and household living conditions.

Capacity building needs

Capacity building needs are numerous and span many areas of the "Poverty and household living conditions tracking" subsystem, namely:

III.2. "Programme and project implementation tracking" sub-system

III.2.1. Objective

The objective of this sub-system is to monitor the implementation of poverty reduction programmes and projects.

Such a tracking system which differs from the traditional statistical system seeks to appraise the outcome of implemented strategies based on funded actions.

According to PRSP and MDG monitoring requirements, each strategic component should indicate programmes, projects and planned actions. For these actions, it is required to set specifically measurable objectives (SMO and, to programme them using timelines and budget resources.

SMO is a key variable for improving the welfare of the population; indicators are used to measure quantities while specific values set the timelines. SMOs can be determined through strategic areas and sub-areas of the PRSP. A general PRSP bridge table highlighting such an allocation must be validated by the various PRSP monitoring stakeholders. Thereafter, there is need to estimate implementation costs and ensure that they match financial resources. This is a long and complex process that seeks to:

Unlike poverty and household living conditions monitoring, monitoring the implementation of policies, programmes and projects is conducted initially at sector level. However, some indicators used in monitoring poverty and household living conditions (which are generally performance indicators) will appear again at the end of this monitoring subsystem.

III.2.2. Content

III.2.2.1. Methodology

In this sub-system, monitoring will involve calculating input and output indicators, bearing in mind that performance indicators are already dealt with in the "Poverty and household living conditions monitoring" subsystem.

Such indicators are financial and practical achievement indicators relating to the various strategic aspects of the PRSP.

Financial achievement indicators can also be calculated in relation to the action of a ministry (share of national education budget in the national budget, share of primary education budget in the national education budget, etc.).

A list of indicators of sub-system 2 is not proposed unlike sub-system 1, since the input and output indicators are directly linked to the programmes and projects implemented by each country.

To attain the poverty reduction objectives, it is necessary to align national budget preparation and management systems on PRSP needs.

The collection of reliable budget data, the codification according to targets and their disaggregation per scale contributes to feeding the poverty and social indicator database. This is essential in building national capacities to ensure timely tracking of poverty reduction policies.

This seeks to ensure that policy objectives and priorities stated in the programmes formulated by the government are well reflected in annual budgets and that the budgeting method explicitly matches targets of ministries with available resources.

In this respect, there is a need to make various expenditure classifications: Item Classification, Administrative Classification, Functional Classification, Economic Classification, Programme Classification, Geographical Classification (see annex 7).

This can be achieved either through the Medium-Term Expenditure Framework (MTEF) [2] established in ministries or through an objective-based budget expenditure tracking system using financial and non-financial indicators to be specified. This presupposes systematic cost assessment. The following four approaches can improve such cost assessment: full cost accounting, expenditure analysis per institutional structure, unit cost analysis, cost analysis by activity (see annex 7).

III.2.2.2. Products

This sub-system has several products:

III.2.3. Institutional aspects

Directing the budgetary system towards PRSP activities in a participatory and ownership process requires the commitment and determination of priority ministries of the PRSP at all levels of the system (national, provincial, etc.). The active support of the ministry in charge of finance is instrumental throughout the process, as it determines the model fiscal incentive system to be used by other bodies in preparing their budgets.

The process operates in two ways:

There are many stakeholders in this sub-system:

Stakeholders coordination should be performed by the appropriate structures of the ministries in charge of finance and planning in conjunction with the PRSP unit.

Capacity building should take several forms:

III.3. "Impact assessment" sub-system

III.3.1. Objective

Tracking poverty and household living conditions provides information for an overall assessment of progress in achieving poverty reduction objectives and understanding various changes made so far. It is therefore necessary to have more targeted tools, such as impact assessments for the sake of reporting to decision-makers and the population, successful or unsuccessful government poverty reduction actions.

Like monitoring, evaluation is a tool that helps in identifying and measuring PPPs. While monitoring is a yardstick for continuously tracking output, assessment is periodic and seeks to measure the impact and efforts of PPPs. As such, monitoring is a set of performance indicators which provides regular information, while assessment is conducted over a period of time and requires more targeted and in-depth investigations.

Impact assessment seeks to identify changes in the well-being of individuals belonging to a specific group induced by a given programme or policy. The information obtained from impact assessment can influence decisions on the extension, adjustment or scrapping of a given policy or programme and serve in rating public policies by order of priority. It is a tool that guides decision-making which furthers programme visibility to the population.

The first two sub-systems are mechanisms for continued PPP output tracking, while the "impact assessment" sub-system measures PPP efforts.

Within the context of the implementation of the PRS, the major questions raised by impact assessment include:

To sum up, impact assessment seeks to check the relevance, implementation and performance of programmes and policies by way of comparison with a set of explicit or implicit standards for the purpose of improving public policies.

III.3.2. Content

III.3.2.1. Methodology

There are four major assessment methods:

Assessment by comparison with a corresponding control group is genuinely experimental and is the ideal method.

This method is ideal albeit quite cumbersome and raises ethical problems. It is described in annex 8 alongside the other three most common methods.

Other types of assessments (process and theory-based assessments for instance) also help to improve management capacities. Such methods would be used depending on the type of assessment. Meanwhile, it is important to point out that they neither measure the scale of the impact nor establish a causal relation. Cause analysis is essential in understanding the efficiency of other programmed poverty reduction interventions and hence tailoring poverty reduction strategies.

Selecting communities and/or specific programmes for impact assessment is based on a specific number of impact indicators contained in sub-system 1.

III.3.2.2. Products

Impact assessments are not conducted systematically. There is need to select policies and interventions for impact assessment. Such selection is based on the importance of the specific policies in terms of lessons to be drawn for poverty reduction. Generally, impact assessment aims to ascertain the relevance of implemented policies and guide future poverty reduction choices.

Policies or programmes selected for impact assessment should provide a positive answer to one of the following three questions:

Policy or programme impact assessment refers to specific operations and it is unlikely to assess all policies and programmes. They require a great deal of statistical data and technical capacities, as well as lengthy timelines and attendant funding.

Impact assessment, notably performance assessment, compares the welfare variables and indicators of communities affected by the programmes and those who are not. The numerous methods used engender manifold information needs.

Strictly speaking, the data needs and corresponding collection systems of potentially selected programmes should be defined before the implementation of the programmes for the purpose of adopting the collection tools in due time.

Small coverage programmes and projects which introduce monitoring and evaluation at beginning of activities will have little trouble in compiling appropriate information.

For nationwide programmes and policies, the core issues relate to:

Considering the multiplicity of poverty reduction policies and programmes, designing such surveys are quite intricate. It is difficult to design a household survey that covers all cross sections of the target communities of such policies and programmes.

Household surveys conducted for the sake of poverty reduction policy assessment should therefore be designed according to the following criteria:

The ensuing products are PPP impact assessment reports. Such evaluation reports must be stored and widely disseminated, especially through the NIS web site.

III.3.3. Institutional aspects

In consideration of the multitude of poverty reduction policies and programmes covering all the sectors of economic and social life, the procedure to choose the policies or programmes to be assessed is absolutely important.

Since the role of impact assessments is to establish the relevance of poverty reduction policies and justify specific policy choices made by decision-makers for the sake of accountability to all partners, the selection of policies to be assessed must follow a clear process. In particular, the following institutional questions must be answered:

The policies or programmes to be assessed must be chosen by sector-specific ministries under the coordination of the PRSP unit.

Impact assessment requires vasttechnical capacities; hence, they can be conducted only by institutions with such capacities. Such bodies may hail from the public sector (NIS, line statistical services, university, research institutes, etc.) or civil society (non-governmental organizations, consultancies, etc.).

Monitoring, centralizing and disseminating the gamut of evaluations should be under the responsibility of the PRSP unit.

Capacity building needs are critical in introducing these rather innovative techniques in Africa.

III.4. Relationship between the three sub-systems

The link between the three sub-systems is found in the logical performance matrix of a programme or project. In fact, there is a need to define a reference situation (sub-system 1) before setting targets. Thereafter, strategies are designed to meet the set targets. Such strategies are implemented through programmes and projects that require means (human, material, financial …) to run the activities (constructing, training, organizing, treating …) [sub-system 2] whose direct results (buildings constructed, persons trained, laid down procedures, enacted legal instruments…) must produce effects (more children at school, more efficient public services, more efficient judicial system …) [sub-system 1] and an impact on development (increased literacy rate, longer life expectancy, improved rule of law …) [sub-systems 1 and 3]. Such sequencing is based on a series of logical relations (if … then) known as "cause/effect." The effects represent the medium-term results achieved through a combination of direct results while direct results are the immediate and tangible products of PPP activities.

The impact assessment sub-system seeks to analyze consistency between the products of the "Projects and programmes implementation monitoring" and "Poverty and household living conditions tracking" sub-systems, particularly the links between input and output indicators on the one hand, and results and impact indicators on the other hand. Within the context of scarce resources, the relationship between the three information sub-systems falls in line with the PRSP review that aims to better target future policies, programmes and projects so as to achieve performance-based management.

III.5 Sub-system coordination

The diagram above shows the role of NIS (in green) in the iterative poverty monitoring process, as well as in designing and setting the macroeconomic benchmarks of the MTEF, in conjunction with the ministries in charge of the Plan, the Economy and Finance. The PRSP unit is responsible for initiating specific studies that will be used assessing the impact of poverty reduction policies and projects: the diagram highlights the need to include these three components in a single information system where each key stakeholder can collect data appropriately.

Each of the three sub-systems of the PRSP and MDG monitoring systems must be coordinated, as mentioned earlier:

The activities of the entire PRSP and MDG monitoring system should be obviously coordinated by a permanent structure.

The most appropriate coordination body is the National Statistics Board which is responsible for coordinating all statistical activities in a country.


[1] Classification of Functions of Government
[2] A Medium-Term Expenditure Framework (MTEF) is a coherent framework for the strategic planning of objectives aimed at a improving the allocation of financial resources to the various sectors (see appendix 6).