: / Analytics / Local Governance Assessment in Kyrgyzstan (2005)

Kyrgyzstan Review, 10 years ago

2.4 Report Cards

General introduction to the system
The Report Card system is an assessment of AO public services from the perspective of its staff and citizens that live in the community. The latter are the users of these services and can provide authentic feedback on the quality, efficiency and adequacy of the services and the problems they face in their interactions with local government representatives. They may not be able to comment on the technical features and standards of the services or evaluate the overall performance of a provider, but they are qualified to say whether the services satisfy their needs and whether the agency is responsive, corrupt, reliable, etc.
The report cards are not simply an opinion poll. They reflect the actual experience of people with a wide range of public services. The survey on which a report card is based covers not only those who have had experiences in the use of specific services and interactions with the relevant public agencies, but also those who provide these services. The users possess fairly accurate information, for example, on whether local governments actually solve their problems or whether they had to pay bribes for services. The service providers indicated their priorities and barriers that do not allow for better service delivery. The report cards solicit user perceptions of the quality, efficiency and adequacy of public services and provide collective feedback on user satisfaction and ways to improve services intended to benefit them.
Stratified random sample surveys using well-structured questionnaires are the basis on which report cards should be prepared. It is generally assumed that people from similar backgrounds, in terms of education, culture, and so forth, are likely to use comparable standards in their assessments. Errors of recall cannot be ruled out, but a significant number of responses should be collected to generate and lend credibility to the findings.
Previously, the report card system was used in Bangladesh, where it was called a Citizens Report Card. The concept of the report card and client surveys are new to most local governments and their leaders, but private firms operating in a competitive environment make use of this approach in many countries.
Experience using a report card system to assess local governance
For the first time the report card system was introduced for assessment of local governance in Kyrgyzstan. This tool, which acts as a civic engagement tool in the process of managing and monitoring local expenditures and services, was used in the assessment of a local governments capacity to do effective planning, implementation and management of their budgets.
Initially, the report cards had undergone pre-tests in pilot and non-pilot AOs and were then refined by the consultant. During the actual assessment stage, the AO staff and AK deputies were asked to provide their scores of basic budgeting processes. Another point to score was the organization of the budgeting process in the AO. The citizens of the same community were also asked to make judgments about the same assessment categories by rating them, based on their personal experience and perception. The report card was developed from the list of the different assessment categories, where each corresponds to the performance indicator. The respondent provides a score to the each of the assessment criteria listed in a group that corresponds to a key performance indicator.
Additionally, the facilitators, whose responsibility was to organize the scoring process, had completed report cards as well. This approach helped to identify the different views and perceptions of the key stakeholders. This was particularly helpful when citizens from non-pilots had put high scores to the budget capacities of the AOs, as they were probably afraid of the bad consequences from the AO leadership for low scores. The scores of the facilitators who had noticed this were therefore relatively low.
An important part of the system is that it gives an opportunity to evaluate changes in the citizens and local governments views on budget efficiency within the selected time period. Following the analysis of the changes in their views, it is possible to make judgments on te efficiency of external interventions or internal initiatives that might appear during the period in between two assessments.
The present report card system proves to be an effective accountability tool in the local governance assessment when it is used by a combination the focus group discussions.
The current system was developed using a 0 to 4 scale, which gives the respondent a chance to omit an assessment category if he/she having difficulty with scoring it or has little information. Please refer to the scale below for reading the results of the data analyses using the report card system.

Assessment Scale


0 Information is not sufficient for assessment of the criteria


1 The criteria is expressed week, radical improvement is needed


2 Criterion is expressed weak, strong improvement is needed


3 Criterion is expressed adequately, slight improvement is needed


4 Criterion is expressed in great extent

The findings of the use a report card in Kyrgyzstan are most striking. A majority of the AOs where the pilot project had not worked before received low ratings from ordinary citizens. It is interesting to mention that AO staff and AK members seemed very critical of these estimations. This can clearly be seen if the scores of the South and North regions are compared. The detailed analyses of the assessment of local governance by use of report card are described in the next chapter.