Gains and losses of structured information collection in the evaluation of 'rehabilitation in the community' programmes: ten lessons learnt during actual evaluations.

PURPOSE: Previously we have explored definitions of community-based rehabilitation (CBR) and proposed a way of classifying rehabilitation programmes by describing their essential characteristics. As the next step, we proposed two flow charts that guide the evaluator through a characterization of the programme and then indicate the information that should be collected. The present paper describes the application of this methodology in five actual evaluations of programmes aiming at socio-economic rehabilitation of persons affected by leprosy.

METHOD: We compared the information as required by the flow charts to the information presented in the evaluation reports and asked: "Does this methodology adequately describe and reveal all relevant aspects of the rehabilitation programme?"

RESULTS: Use of the flow charts led to discussion between evaluators and programme staff about how each would characterize the programme; this was a valuable step in the evaluation process and provided insight to the staff into their current practices and aspirations. The rehabilitation services as such were always well-described in the evaluation reports. More attention could have been given to the programme environment and provider-client relationships. More or more explicit attention than required by the flow charts could be given to linkages with other rehabilitation programmes and community organizations; questions of organizational capacity; systems to maintain and increase the quality of services; and conditions and constraints imposed by donor organizations. In order to show their effectiveness, rehabilitation programmes need to develop simple information systems which show progress of clients towards the rehabilitation goals defined for them, with them or by them. Impact can be demonstrated by an assessment one year after ending the intervention. This should include assessments of clients' psychological and social status.

CONCLUSIONS: The original theoretical framework has proven its value in evaluation practice. The flow charts accommodate a variety of programmes and address the specific aims, contexts and developmental stages of the programmes evaluated. Taking the lessons learnt here into account will further improve the usefulness and practical relevance of the methodology we proposed.