On Theories of Change
Recently, a friend asked me if I would take a look at a draft of the mid-term review being undertaken for the programme he manages. It's a large programme, some USD 35 million covering a number of countries. MTRs being, in theory, pretty straightforward exercises, I was knee to know why he needed my opinion. 'It's the theory of change. The consultant has created a new one and I'm at a loss as to why. We have one - it's in the project design document.' Indeed. Weird. Consultant myself, I sometimes have to wonder why consultants feel the need to prove that they have better ideas than literally everyone else, and also why they feel that it's somehow in the scope of their job description to literally try to change a project (unless, of course, that actually is the job description).
The draft report which I admittedly skimmed (it was 70+ pages, also a big no-no in my book) spent a lot of time discussing the programme's theory of change (ToC). Not the one that was detailed in the project design, but one the consultant prepared. A good one-third of the report was dedicated to why the ToC was important. And in the end the 'new' ToC proved to be an overly-complex flow chart which was simply the log frame in new clothes. There was no added value. The result chain of the project hadn't changed (*sigh of relief*). And there was no narrative to support this flow chart. No reference to necessary preconditions for changes, etc. I advised my friend not to worry about it (getting the consultant to rewrite the draft would be epically time consuming). But this incident does raise a couple of points.
First, there does not seem to be an overarching consensus on what ToC should constitute. I have seen no less than five different 'types' or 'layouts' of ToCs, and never have I heard (or read) it explained to me the same way twice. This poses big questions about the rationale and added value. Personally, I can see an added value - in some cases, ToC can fill in the blanks about what needs to be done and why, those 'necessary preconditions for change' I mentioned above, that a log frame simply cannot accommodate and are likely to be missed in the narrative part of a project proposal simply because we often focus on why we are best placed to implement a project more than on what needs to be achieved and why.
Second, there is the issue of new trends. ToCs are really all the crack these days. So we spend a lot of time focusing on them when it comes to activities like MTRs, and thus spend less time on really evaluating change, and what that change means for the future of a programme. I had a long discussion with my friend about the need for the MTR to be forward looking rather than entirely reflective. We can't change what's happened in the past in terms of project implementation, but we can understand what changes we need to make in the future and, more precisely, where we want to go in the future based on any changes already visible due to regular monitoring and reporting. So review of a ToC should be less about 'did it make sense at the time?' and more about 'does it still make sense now?'
I'll be honest, as much as I like to see innovation and as much as log frames are painful and pretty boring tools, I can't see them going anywhere anytime soon. There are new tools, like ToCs and outcome mapping which supplement log frames, but they need to have added value rather than just replicate the content of a log frame in a new format. At the end of the day, it's about the intended results of a project and the best way to both monitor and evaluate those results. Project monitoring can track the results, and MTRs can determine if they are still relevant for the future (as opposed to two years previous), and what, if anything, needs to change about the project.
It is important that we understand that one tool does not supplant another. And we should no get caught up in evaluating the tool, as we did with log frames and now do with ToCs. It's less about the tool and more about the process.
The draft report which I admittedly skimmed (it was 70+ pages, also a big no-no in my book) spent a lot of time discussing the programme's theory of change (ToC). Not the one that was detailed in the project design, but one the consultant prepared. A good one-third of the report was dedicated to why the ToC was important. And in the end the 'new' ToC proved to be an overly-complex flow chart which was simply the log frame in new clothes. There was no added value. The result chain of the project hadn't changed (*sigh of relief*). And there was no narrative to support this flow chart. No reference to necessary preconditions for changes, etc. I advised my friend not to worry about it (getting the consultant to rewrite the draft would be epically time consuming). But this incident does raise a couple of points.
First, there does not seem to be an overarching consensus on what ToC should constitute. I have seen no less than five different 'types' or 'layouts' of ToCs, and never have I heard (or read) it explained to me the same way twice. This poses big questions about the rationale and added value. Personally, I can see an added value - in some cases, ToC can fill in the blanks about what needs to be done and why, those 'necessary preconditions for change' I mentioned above, that a log frame simply cannot accommodate and are likely to be missed in the narrative part of a project proposal simply because we often focus on why we are best placed to implement a project more than on what needs to be achieved and why.
Second, there is the issue of new trends. ToCs are really all the crack these days. So we spend a lot of time focusing on them when it comes to activities like MTRs, and thus spend less time on really evaluating change, and what that change means for the future of a programme. I had a long discussion with my friend about the need for the MTR to be forward looking rather than entirely reflective. We can't change what's happened in the past in terms of project implementation, but we can understand what changes we need to make in the future and, more precisely, where we want to go in the future based on any changes already visible due to regular monitoring and reporting. So review of a ToC should be less about 'did it make sense at the time?' and more about 'does it still make sense now?'
I'll be honest, as much as I like to see innovation and as much as log frames are painful and pretty boring tools, I can't see them going anywhere anytime soon. There are new tools, like ToCs and outcome mapping which supplement log frames, but they need to have added value rather than just replicate the content of a log frame in a new format. At the end of the day, it's about the intended results of a project and the best way to both monitor and evaluate those results. Project monitoring can track the results, and MTRs can determine if they are still relevant for the future (as opposed to two years previous), and what, if anything, needs to change about the project.
It is important that we understand that one tool does not supplant another. And we should no get caught up in evaluating the tool, as we did with log frames and now do with ToCs. It's less about the tool and more about the process.
Comments
Post a Comment