In monitoring and evaluation (M&E), the OECD Development Assistance Committee (DAC) criteria provide a robust foundation for assessing development interventions. These six principles—relevance, coherence, effectiveness, efficiency, impact and sustainability—are widely recognized for their ability to guide comprehensive evaluations. However, integrating these criteria with other evaluation frameworks can enhance the depth and breadth of analysis, ensuring a more holistic understanding of program outcomes. This blog post explores the concept of integrating DAC criteria with other evaluation frameworks, offering practical examples and insights into its application.

DAC Criteria Definition

The DAC criteria are a set of six key principles that guide the evaluation of development interventions:

• Relevance: The extent to which the intervention addresses the specific needs and context of the target population.
• Coherence: The compatibility of the intervention with other interventions in the same context, ensuring that they complement and reinforce each other.
• Effectiveness: The degree to which the intervention achieves its stated objectives.
• Efficiency: The optimal use of resources to achieve desired outcomes.
• Impact: The broader changes attributable to the intervention.
• Sustainability: The long-term viability of the intervention and its outcomes without ongoing external support.

Integrating these criteria with other evaluation frameworks involves harmonizing these principles with additional or complementary evaluation criteria commonly used in various sectors and contexts.

Practical Example: Evaluating a Public Health Program

Context

A government health department implements a comprehensive public health intervention aimed at reducing the incidence of diabetes through community education, improved access to medical care, and lifestyle modification programs.

Application in M&E

1. Conceptualization and Planning

Alignment of Objectives: The health department aligns the objectives of their public health intervention with the DAC criteria and sector-specific criteria such as equity and inclusiveness.

2. Evaluation Design

Mixed-Methods Approach: The evaluation employs a mixed-methods approach, combining quantitative measures with qualitative assessments to provide a comprehensive view of the program’s effectiveness.

• Quantitative Measures: Data on changes in diabetes incidence, blood sugar levels, and other health indicators are collected to measure the intervention’s effectiveness.
• Qualitative Assessments: Interviews, focus groups, and surveys are conducted to gather insights on community perceptions, experiences, and satisfaction with the program.

3. Baseline Data Collection

Comprehensive Data Gathering: Baseline data on diabetes prevalence, community health behaviors, and access to healthcare services are collected before the intervention starts.

• Data Sources: Health surveys, medical records, and community assessments are used to establish a baseline for comparison.
• Analysis: The baseline data provide a point of reference to measure the intervention’s impact over time.

4. Implementation and Monitoring

Routine Monitoring: Ongoing monitoring ensures the intervention is implemented as planned and provides continuous data collection on key indicators.

• Immediate Indicators: Participation rates, community engagement levels, and utilization of medical services are tracked to ensure the program reaches its intended audience.
• Monitoring Tools: Digital health records, mobile health applications, and regular field visits are used to collect and analyze data in real-time.

5. Integrating DAC Criteria 

Relevance

Linkage: Ensures that the intervention addresses the specific health needs and priorities of the target population. 

Example: The program targets regions with the highest diabetes prevalence. Community education sessions are designed to address local dietary habits, cultural practices, and specific risk factors prevalent in the area.

Coherence

Linkage: Ensures the intervention is aligned and compatible with other ongoing health initiatives and policies. 

Example: The diabetes intervention is coordinated with existing public health initiatives, such as campaigns for healthier eating and increased physical activity. This alignment ensures that all efforts are complementary and reinforce each other, creating a more effective overall health strategy.

Effectiveness

Linkage: Measures the degree to which the intervention achieves its stated objectives. 

Example: The intervention sets clear objectives, such as reducing the incidence of diabetes by 20% over five years. Regular monitoring and evaluation indicate significant reductions in new diabetes cases, improved patient health outcomes, and high levels of participant satisfaction, demonstrating the intervention’s effectiveness.

Efficiency

Linkage: Assesses the optimal use of resources to achieve desired outcomes. 

Example: The program implements a mix of cost-effective strategies, such as using community health workers for outreach and education, leveraging existing healthcare facilities for screenings and check-ups, and utilizing mobile clinics to reach remote areas. These strategies ensure resources are used efficiently while maximizing impact.

Impact

Linkage: Evaluates the broader changes attributable to the intervention. 

Example: Beyond the primary goal of reducing diabetes incidence, the program’s broader impacts include increased health awareness, reduced healthcare costs due to fewer complications, improved overall community health, and enhanced economic productivity as fewer people suffer from diabetes-related health issues.

Sustainability

Linkage: Considers the long-term viability of the intervention and its outcomes without ongoing external support. 

Example: The intervention includes training local healthcare providers and establishing community health committees to continue diabetes education and support activities. Additionally, partnerships with local organizations and government agencies help secure ongoing funding and resources, ensuring the program’s sustainability.

By linking the DAC criteria principles with practical examples in the context of a public health intervention, this approach demonstrates how each principle contributes to a comprehensive and effective evaluation. Integrating these principles with other evaluation frameworks ensures a holistic assessment that can guide future policy-making and program design for sustained impact and improvement.

6. Data Analysis

Comparative Analysis: A combination of statistical techniques and thematic analysis is used to compare pre- and post-intervention data, providing comprehensive insights into the program’s effectiveness.

• Statistical Techniques: Regression analysis, difference-in-differences, and propensity score matching are used to establish causality and measure changes over time.
• Thematic Analysis: Qualitative data from interviews and focus groups are analyzed to identify recurring themes and insights into community experiences and satisfaction.

7. Reporting and Learning

Detailed Reporting: The findings are documented in a comprehensive report that includes assessments based on DAC criteria and additional framework-based evaluations, offering a holistic view of the program’s performance.

• Content: The report includes quantitative data, qualitative insights, case studies, and recommendations for future interventions.
• Distribution: The report is shared with the health department, community leaders, funding agencies, and other stakeholders to inform policy-making and program design.

Stakeholder Engagement: Results are communicated to all relevant stakeholders through presentations, workshops, and community meetings to ensure transparency and foster collaboration.

8. Decision Making

Evidence-Based Adjustments: The health department uses the evaluation findings to make informed decisions about scaling up, modifying, or discontinuing the intervention.

• Adjustments: Based on the findings, the program might be expanded to new areas, modified to address identified gaps, or integrated with other successful initiatives.
• Policy Development: The insights gained from the evaluation are used to develop policies and strategies that enhance the overall effectiveness of public health interventions.

Key Points in Application

• Holistic Evaluation: Integrating DAC criteria with other evaluation frameworks provides a more comprehensive assessment of the intervention.
• Stakeholder Alignment: Involving diverse stakeholders ensures that the evaluation criteria are relevant and applicable to the specific context.
• Enhanced Learning: Combining multiple frameworks facilitates greater learning and improvement by providing diverse perspectives on program performance.
• Actionable Insights: The integrated approach yields actionable insights that can guide program modifications, optimization, and policy development.

Additional Elements and Resources

• Theory of Change: Ensures clarity in how different components of the intervention influence outcomes, guiding the evaluation design.
• Logic Models: Visual tools to map out the expected sequence of events, enhancing stakeholder understanding.
• Qualitative and Quantitative Methods: A balanced approach to measure and understand outcomes.
• Counterfactual Analysis: Helps establish causality and understand what would have happened in the absence of the intervention.
• Data Collection and Management: Ensures reliable and consistent data for accurate evaluation.
• Ethical Considerations: Adhering to ethical standards protects participant rights and enhances credibility.
• Capacity Building: Empowers local teams to sustain evaluation practices for long-term benefits.
• Utilization-Focused Evaluation: Ensures the evaluation design is aligned with the practical needs of stakeholders.

This practical example illustrates how DAC criteria can be effectively integrated with other evaluation frameworks to provide a comprehensive and actionable understanding of a public health intervention. By leveraging diverse evaluation frameworks, organizations can enhance their ability to measure, understand, and improve their interventions, ultimately leading to better outcomes and more sustainable impacts.

Resources and Further Reading

1. Book:
   • Utilization-Focused Evaluation by Michael Quinn Patton
2. Online Course:
   • Evaluate health program effectiveness and impact. Master the tools and skills for effective health program evaluation.
3. Academic Journal:
   • American Journal of Evaluation
4. Guidelines and Frameworks:
   • OECD DAC Evaluation Criteria
   • World Health Organization: Evaluation Practice Handbook
5. Website:
   • BetterEvaluation