Theory in Practice

In my previous post on designing and implementing adaptive MEL frameworks , I discussed the importance of a program’s Theory of Change (TOC) in facilitating understanding among your team and your stakeholders on what change looks like. The biggest challenge facing most programs, however, is that TOC is often mistaken for another form or version of a results chain/log frame/results framework. While they are not mutually exclusive, they are not the same. The important distinction is that the TOC needs to iterate the theory behind the change you are seeking to enable, not just the process. The process is iterated in a results chain that specifies the tangible results you are pursuing driven by the theory. In an adaptive program, your results chain (i.e.: activities and outputs) can change depending on the initial results you are observing and the reflections that are being undertaken about what should happen next. The ‘what comes next’ reflections need to be based on something, and...

*This is an adapted version of an article I drafted for the SIAP SIAGA program, which can be found here .  In 2019, I began working on a program that set itself the challenge of implementing a systems change, adaptive approach in order to leverage the results of previous programs in that sector, as well as to take into account the increasing complexity of development programming in a middle-income country. This type of program required a different type of monitoring framework – one that could capture changes in the system based on the results of the program, as well as be adaptive to the adjustments in program implementation over time. With adaptive monitoring still in its infancy (in practice, despite the plethora of theoretical information available on line), there were few practical experiences and good practices to draw on to facilitate the design of the monitoring, evaluation and learning (MEL) framework. There were more resources on monitoring systems change, but as with an...

Back in November 2022, I attended the Australasian Aid Conference (AAC2022) where my colleague, Ancilla Bere, and I presented the work of a program I am working on related to the learning and good practice we have accumulated on systems change in disaster management in Indonesia. The presentation assessed some of the methodologies and approaches that we have found to be successful for shifting mindsets around systems thinking, as well as some of the higher profile changes that we have begun to witness.  You can read the full article which was prepared following the conference on DevPolicy here . Can a systems change approach improve disaster management?   By Denika Blacklock and Ancilla Bere on Feb 17, 2023 06:00 am The 2004 Indian Ocean tsunami triggered a shift in thinking in disaster management in Indonesia, from reactive response and recovery towards preparedness, mitigation and risk reduction. It was a major turning point for the Government of Indonesia in addressin...

 

As the COVID-19 economic recovery progresses, and the world works towards resuming some sort of normalcy in our daily lives (and I get back to writing on this blog!), more emphasis needs to be placed on reflecting on what happened during COVID-19. Not just the stress of online schooling (oh boy), the working from home (video off as I was usually still in my pjs), and the loss of time with friends and family (I haven’t seen my family in 2.5 years… soon to be rectified!). No, what we need to focus on is what we have learned about our work – no matter what sector or what type of work you undertake, we all learned that business as usual (BAU) practices simply could not cope with the shock to the economic and social systems we live in. In my line of work, and in my perspective, one of the biggest questions that has emerged is whether the development community is capable of truly improving the resilience of communities – whether to climate, disaster, economic or health shocks. I recently p...

One of the issues that I’m struggling to wrap my head around is meeting donor requirements while trying to implement an adaptive project. One the one hand, everyone from the donor to the government is on board with the approach, although the government is still a bit wary about the amount of risk they will need to take on in order to see this approach through to the end. On the other hand, donor templates and guidance are still very prescriptive, and examples are entirely linear in their approach. Moreover, examples used in guidance are very concrete (vaccination interventions seem to be a favourite), unlike the less-defined, system-level intervention my colleagues and I are working on.

With so much theory around adaptive management available, it can be daunting to try to put it into practice in a holistic way. Much of what I know about adaptive management relates to the actual implementation of a program or project itself. Who knew that the planning process would end up being complex and adaptive in its own right? 

I recall the first time I had to develop a logframe. I had no idea what I was supposed to do. I couldn’t see how the boxes all linked together, I felt that the information it contained was repetitive, I had no idea what my colleagues were talking about when they carried on about ‘assumptions.’ I wanted to cry. I wanted to quit. I was pretty sure this was something I never wanted to have to work on again. 15 years later, I have a far different view of logframes. Once I had understood them – once someone had decoded them for me – I understood their utility. They are a project organizational tool; they are an accountability tool; and they are basically the (over)simplification of projects aiming to address complex issues. However, while I appreciate their utility in some respects, I am also aware of just where they fall short.

A few months ago, my colleagues and I invested significant amount of time to design a project that championed local ownership, accountability to beneficiaries and adaptive management. We were explicit about our methodologies and about how the results would be defined by the beneficiaries during the inception phase of the project: our philosophy was that success would be more likely to be achieved if the beneficiaries could decide what success looked like – for them. Beyond the jargon, the proposal was good. Good enough to be approved by the donor and for us to get excited about a project with a truly adaptive management approach. But when it came time to sign the contract, the dreaded ‘payment by results’ phrase magically appeared. Out of nowhere, we were told we needed to have ‘pre-defined results’ before the project could move ahead.

It’s amazing how I feel completely inundated with blog posts and academic papers and policy briefs on moving M&E in a direction that accommodates complexity and systems thinking; embracing adaptive management and political economy analysis (PEA). Only a year ago, what felt like a handful of knowledgeable people testing the waters with some (much needed) discussion on how M&E in development programming needs to change now feels like a bit of a tidal wave. Or maybe it’s only me, because I’m actively looking for information and have gone down a bit of a rabbit hole on adaptive management in development.

Back with some exciting news!  'Making M&E Work in Development Programme: A Practitioner's Handbook' is a product of more than 15 years in development, and 10 years in M&E. It's purpose is many: as an introduction to M&E in development programming to people new to the field; as a training handbook; and as go-to guide for established development practitioners that understand M&E but struggle with operationalizing the M&E framework for their project or programme.  The book is available through Amazon and other online websites. You can purchase it here: https://www.amazon.com/Making-Work-Development-Programmes-Practitioners-ebook/dp/B07PK5RBKS/ref=sr_1_1?keywords=denika+blacklock&qid=1556249243&s=gateway&sr=8-1-spell

You can now purchase my new book on Amazon Kindle edition here Paperback edition here Happy reading!

Originally posted on the Australian Evaluation Society Blog ( AES Blog )   November 2018 By Denika Blacklock I have been working in development for 15 years and have specialised in M&E for the past 10 years. In all that time, I have never been asked to design an M&E framework for or undertake an evaluation of a project which did not focus entirely on a logframe. Understandably, it is a practical tool for measuring results – particularly quantitative results – in development projects. However, as the drive for increased development effectiveness and, thankfully, more accountability to stakeholders has progressed, simply measuring what we have successfully done (versus what we have successfully changed or improved) requires more than just numbers. More concerning is the fact that logframes measure linear progression toward preset targets. Any development practitioner worth their degree can tell you that development – and development projects – is never linear, and a...

I recently had the unique experience of spending time in DPRK (North Korea). I was evaluating two separate projects – standard stuff related to OECD DAC, results, gender and inclusion, lessons learned etc. After my first visit to the communities where activities were being implemented, I was asked what I thought. ‘Same thing, different country,’ was my response. To be fair, once you get past the hoopla about DPRK, the development needs at the community level are pretty much the same as any other least developed country or fragile state. Geography and culture influence what you do and how, but the issues that need addressing are similar.

I ’ve recently completed a number of missions for M&E purposes across Asia and the Pacific, and although there is always a sense of satisfaction of ticking the box on all of your deliverables, of knowing your client is pleased with the results, there is perhaps a greater satisfaction in what you can take away from each experience to build your own knowledge and try to use in your next assignment, to make development work just a bit more accountable and a bit… more. Measuring Innovation in Laos Nearly 10 years after drafting guidelines for the country office of an international organization on how to implement results-based monitoring and evaluation, I’ve discovered that a) not only have those guidelines been disseminated and used regionally, but b) they haven’t been updated (10 years!!??) and c) we’ve changed what we want to measure but not the tools we are allowed to measure the changes we effect with. Sigh.

Yo u have to admit, there is a lot of good writing out there. Such is the wonder of the internet and the ability of people to give their ideas and opinions a voice. I’m a proponent of this, particularly of those writers (professional and ad hoc) who have something truly meaningful to say. 2017 was, to my mind, a seminal year for some very excellent writing, largely to the shifting world around us. Below is the list of the top five articles which both inspired and challenged me last year, despite the fact that we are well into 2018. Coming in a number five, ‘ Why Expertise Matters .’ What with ‘fake news’ and, sadly, the terrible opinions based on conjecture and absolutely no fact, people with actual expertise have to compete not just with other people with expertise, but more often than not, people who know nothing about anything but like to feed their egos. Differentiating fact from fiction used to be a simile, but now? It is a full-time job and rarely does that mean that fact (and...

Originally Published in Al Jazeera http://www.aljazeera.com/indepth/opinion/arms-embargo-myanmar-save-rohingya-170924081030160.html The past couple of weeks have seen a flurry of statements and op-eds demanding that the international community do something to stop the ethnic cleansing of the  Rohingya  in Myanmar. Among the demands made of the international community was the implementation of an arms embargo by the UN Security Council. In a report published on 17 September,  Human Rights Watch  requested that "The Security Council should urgently place a travel ban and asset freeze on those responsible for grave abuses and impose a comprehensive arms embargo against Burma, including prohibiting military cooperation and financial transactions with key military-owned enterprises."

I try to keep up to date on developments and trends on a variety of international development topics - my ties to the Pacific mean that I spend a significant amount of time exploring ideas and opportunities around climate resilience. So there’s a backlog of articles on the topic bookmarked on my computer. This morning I sat down to go through a few of them and was less than impressed. So much of what is discussed in terms of climate resilience focuses on how advanced government plans and programmes are. In truth, government plans, programmes and financing make a huge difference in the lives of the most vulnerable and in the most climate-vulnerable countries like the Pacific and other LDCs. Good points were made in the article I read this morning, breaking down the essentials, keeping it simple. The problem, for me, was that it didn’t touch on the human element. Those of us working in international development sometimes forget that - caught up as we are in our own plans and budgets ...

I read recently that we see diminishing creativity in society because we are too busy. We never take time to let our minds empty to allow those ‘ah ha!’ moments in. This is probably true. But I had a ‘seriously, it took me that long to realize that?’ moment this morning. Like, full on, I’d-like-to-go-back-and-rewrite-an-entire-evaluation kind of moment. And it all hinged on the confluence of events. The first took place over my morning coffee, reading a recent Guardian Development Pros article ‘ Development is Not a Science and Cannot be Measured .’ Indeed, it is not (as I’ve argued here before). Despite all the tools and academic and policy maker claims to the contrary, there are too many variables in development for any of us to explicitly state ‘A therefore B’. And we know this (those of us who do the work day in and day out and try to meld development policy with practicality and bare bones reality). The second took place at the gym a short while later as I was reading ‘Aid on the ...