Developing knowledge on monitoring and evaluation of progress through analogues in the health and hydroelectricity sectors

Knowing the effectiveness of specific climate change adaptation (CCA) measures is a critical part of expanding knowledge about how to adapt to the impacts of climate change. Tools to monitor and measure the effects of adaptation measures are needed to develop this knowledge, but are challenging to develop. In Canada, there is limited experience in implementing concrete and specific adaptation strategies, and therefore also in measuring progress in this area. At a National Workshop organized by Ouranos in March 2012 to further discussions across Canadian institutions in this area, one of the key recommendations was for the adaptation community to learn from other areas with extended experience in monitoring and evaluation of programs, a common tool for measuring progress. This project builds upon this recommendation and describes two monitoring and evaluation (M&E) activities, one in the health sector and one in the hydropower sector, and draws lessons from these which can be applied to adaptation to climate change.

• Describe and learn from two monitoring and evaluation (M&E) activities in two analogues studies, one in health sector and one in the hydropower sector; and to identify generalizable good practices for monitoring and evaluation.

• Conduct a review of the literature on monitoring and evaluation of adaptation to climate change and identifying key challenges.

• Collate supplemental background from the evaluation literature, in particular with respect to theories of change;

• Collect data on the two analogue cases through interviews and document from institutes responsible for the analogues in the hydropower and health sectors;

• Document the monitoring and evaluation of the objectives, procedures and implementation arrangements.

• Assess these analogues in the context of pre-identified requirements and challenges in adaptation measurement, drawing observations that may applied to adaptation to climate change

Two important concepts applicable to adaptation implementation emerge from the two analogue studies: firstly, causal logic models and theory of change and, secondly, the role of social participation in adaptive management approaches. These are discussed in turn below.

Causal logic models and theory of change

Applying the causal logic model, illustrated in the figure below, can help in both designing an adaptation measure and measuring its progress. It involves identifying the desired Impacts and Outcomes and then working backwards to identify the Outputs, Activities and Inputs required in order to result in the desired change, also known as the theory of change, or the “progress” being measured. The cause and effect relationships can be broadly identified during project design, and those that will be addressed within the scope of a given project isolated. Using this system to learn and adjust efforts towards expressed and desired impact was achieved by the analogues by:

• Identifying High level desired Impacts and Outcomes to guide the identification of inputs, activities and outputs required to achieve those;

• Establishing observable baseline conditions against which to compare changes over time. The baseline conditions in each analogue were established before the start of the projects, up to two years before, and served not only as a benchmark, but as valuable input for isolating where inputs were required by the project; 

• Measuring confounding variables in order to confirm that results achieved were in fact related to the project activities. For example, in the case of the health sector analogue, other sources of allergens than ragweed, such as the pets at home, were also monitored to confirm that progress was attributable to ragweed control measures put in place.

Social participation and adaptive management

M&E systems are tools for supporting adaptive management, that is, flexible decision making so that activities can be adjusted over the course of a project based on regular self-monitoring of selected indicators, and against a baseline established before the outset of a project. These tools provide an evidence-based method for applying adaptive management, especially important in cases where uncertainties exist, as is the case with CCA. The analogues emphasized consulting with and incorporating measurements of social impacts and traditional knowledge as key to determining the success of a project, and indeed, to confirm the appropriateness of the planned impacts at the outset.

Consultation and involvement by affected communities were essential components to both analogues. Biophysical indicators were joined by social indicators, as well as validated through community consultation. For example, in the case of the hydro sector project, the monitoring and evaluation program integrates traditional knowledge in order to improve and adjust the management of environmental risk mitigation measures. This has helped adjust environmental management measures to improve their effectiveness from an environmental and social perspective and took place at each stage of the project.

In this way, community participation was essential to defining, refining and validating the targets and the data collected during the course of the projects. Overall, flexibility that results from adaptive management can become a management trait that becomes more valued given the unknowns related to the impacts of climate change and the effectiveness of adaptation.

• Agence de la Santé et des Services sociaux de la Montérégie

• Direction de la santé publique

• Hydro-Québec

560005