Alternate Version: PDF version (4 pages, 853 KB) 2016-0275-tip_sheet_evaluate-eng.pdf
If the group identifies more change drivers1 than can feasibly be used in modules 5 and 6 activities, it may be necessary to cull some to obtain a more manageable set of the best change drivers. In module 5, change drivers are explored through cascade diagrams and cross-impact analysis. In module 6, change drivers will serve as the scaffold for the scenario exercise. The choice of how many change drivers to include will depend on the time allowance. As an indication, by the time Horizons reaches the scenarios phase, the scenario scaffold would typically be limited to 5–10 change drivers.
The best change drivers are those understood to be potentially disruptive or transformative for the system, on their own and through their interaction with other change drivers as well as under various scenarios.
Assessing change drivers across system elements
The first and most important test of a change driver’s value is to estimate the extent and significance of the impact it could present to the system under study.
As an example, the image below shows the final domain map (14 elements in boxes) selected for Horizons’ Future of Asia study. Recorded over the map are notes on how participants expected the change driver “Job-killing Automation” to affect each element, as well as a rating of impact.
Summary: Image description
This image is an example of a roughly drawn domain map. The image below shows the final domain map (14 elements in boxes) selected for Horizons’ Future of Asia study. Recorded over the map are notes on how participants expected the change driver “Job-killing Automation” to affect each element, as well as a rating of impact.
To do this exercise, the facilitator needs a final map of domain elements. For each change driver, print a copy of the domain map on large paper to use as a worksheet. With the group, spend time on one change driver worksheet at a time, evaluating the driver’s impacts for the system in 10-15 years.
Discussion with the group:
- How is the change driver anticipated to affect each of the elements (if it does)? Move quickly across the map for this exercise (a couple of minutes for each element); there will be other opportunities to discuss the impacts in greater depth. Record in two or three words the biggest disruptions suggested for each element.
- For each element, note also whether the impact is significantly disruptive. In the example above, the key question was: “Would you tell a senior decision maker about this change?” Those impacts considered sufficiently disruptive were then marked “D.” The change driver “Job-Killing Automation” was considered highly disruptive, with many Ds across the system. That worksheet took about 40 minutes to complete.
Since the system elements are also part of the scenario scaffold, this exercise gives an early sense of which change drivers are likely to create interesting results (independently) in the scenarios. Repeat the exercise for each of the change drivers.
When all worksheets are completed, combine them, and use the results to rank which drivers appear to cause the most interesting, strategic and disruptive changes. If two change drivers are related or present relatively similar impacts, consider whether one can be dropped.
Further reducing the number of change drivers
The change driver exercises in module 5 will both develop the group’s understanding of the drivers (in terms of plausible future implications), and provide information that may help them discard some change drivers along the way, if needed. Each of the activities below contributes to this process in different ways.
- Developing a cascade diagram. This exercise explores a change driver’s plausible impact in more depth by considering cascading consequences, beyond what is immediately apparent. If participants have difficulty arriving at new, surprising consequences2, the facilitation team should ask themselves whether the change driver is a good candidate for the study. After the exercise, the group can reflect on the extent of the impact discussed. Are disruptive changes anticipated in only a few limited areas? If so, are those few consequences significant enough for the change driver to merit inclusion in the study?
- Cross-impacting change drivers. This activity (Cross-Impact Facilitator Guide) identifies which change drivers might create interesting results if they happen simultaneously with other change drivers. (See slide 14 of the Change Drivers presentation (module 5) for an example.)
- Determine the number of cross-impact discussions: It may be necessary to first decide how many cross-impact discussions are needed and can be held. The number of possible cross-impact discussions grows exponentially with the number of change drivers, which can make it impractical to spend time on all combinations if the number of drivers is high (e.g. 10 drivers means up to 45 cross-impacts). In our experience, a good cross-impact discussion can take 15 minutes per pair of drivers. To save time, a filtering exercise beforehand can take only 2–3 minutes per pair and be done by the core facilitation team on behalf of participants.
- Create a cross-impact spreadsheet: Using a spreadsheet of all possible change driver interactions, the core team can simply ask: which pairs of change drivers have the potential to produce a highly disruptive impact because of the way they might interact with each other? Mark each cross-impact cell as L(ow), M(edium) or H(igh) impact. You may also want to record some notes, but the aim is to move through the pairs quickly to complete the spreadsheet. Looking across each change driver row, sum up the number of lows, mediums and highs. Is there a change driver that is not expected to produce a high-impact change with almost any other change driver? If so, this change driver may not be worth exploring further. After ranking all change drivers according to the number of Ls, Ms, and Hs, the core facilitation team can decide how many cross-impacts to prioritize for a longer discussion.
- Coordinate the cross-impact discussions: Having participants discuss the highest-impact change driver pairings is a helpful intermediate step to exploring the full complexity of a scenario. Interacting two drivers at a time may help participants identify some points more easily than when they are overwhelmed by many variables coming together all at once. If some of these cross-impact conversations suggest the interactions between certain change drivers are weak, this is also an opportunity to drop a change driver or two.
- Building the scenario scaffold: The final test of a change driver’s contribution to the study is in the creation of scenario scaffolding. After the basic scenario logic has been developed, the core facilitation team or participants will fill in a sentence or two describing the character of a change driver under each of the four scenario conditions. If the change driver does not seem at all modified (or little) under the influence of each of the four scenarios, the change driver is not offering much added value to the scenario exercise. It may not be worth discussing further.
- See Tip Sheet on How to Identify Change Drivers.
- For example, “population aging” is a change driver that can lead to dull conversations, as many in the policy community have heard of and reflected extensively on this subject (at least as it relates to the labour force, pensions, and tax base). Note that a different problem arises when the group is simply unfamiliar with a change driver, such as a new technology. It could be an excellent change driver, but participants may need additional preparation to discuss it.