Evaluation The Forbidden Truth

By Backson Sibanda

This book is a journey through Dr Backson Sibanda’s 30 years’ experience in the frequently politicised role of evaluation where principals expect evaluations to not recognise their shortcomings but praise them even for poor performance. Dr Sibanda outlines the challenges facing evaluation professionals who often possess the technical capability but not the political savvy and know-how to deliver evaluation results to a self-interested stakeholder grouping. It is a challenge and resource to executives of international organisations, NGOs, Governments, evaluators, academics, students, researchers and aid organisations. It is not a criticism of executives of international organisations but rather an assessment of the value of evaluation, as well as to evaluate the evaluators.

Through Dr Sibanda’s first-hand experience at the helm of evaluations, 23 of which were in four different United Nations agencies, this book demonstrates that evaluation drives accountability not only for the myriad of resources availed to ensure that development takes place, but also for the results or outputs, outcomes, impacts and changes in people’s lives as a result of development efforts. Dr. Sibanda raises the futility of undertaking “ritualistic or tick box” evaluations which do not result in learnings, whether positive or negative, being adopted and implemented to “avoid re-inventing failure”.

The book notes that notwithstanding that acceptance and willingness to learn results in a win-win situation, most executives of international development agencies, NGOs and even Governments suffer from the King Lion Syndrome - denial that the King’s den does not smell like roses leads to the lion devouring the truth telling dog – which results in Evaluation as the Forbidden Truth as executives live in their denial of the truth.

• Language: English
• Publisher: Backson Sibanda
• Release date: Jul 1, 2019
• ISBN: 9780463787700

Book preview

EVALUATION

The Forbidden Truth

Evaluations demonstrate and conclude that where there is willingness to learn, there are no losers

Dr Backson Sibanda

Copyright © 2019 Backson Sibanda

First edition 2019

Published by Backson Sibanda Publishing at Smashwords

All rights reserved. No part of this book may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording or any information storage or retrieval system without permission from the copyright holder.

The Author has made every effort to trace and acknowledge sources/resources/individuals. In the event that any images/information have been incorrectly attributed or credited, the Author will be pleased to rectify these omissions at the earliest opportunity.

Published by Backson Sibanda,

using Reach Publishers’ services,

Edited by Gerard Peter for Reach Publishers

Cover designed by Reach Publishers

P O Box 1384, Wandsbeck, South Africa, 3631

Website: www.reachpublishers.co.za

E-mail: [e-mail]

Dedication

To my wife Shirley Thabani Sibanda and to our two sons Bhekuzulu and Mzwandile and our daughter-in-law Iris Pilane Sibanda for all their support and the sacrifices they made to enable me to write this book.

Contents

Dedication

Foreword

Acknowledgements

Prologue

Preface

1. Background

2. Evidence-Based Decision Making and Policy Development – Do Managers Utilise Evaluation Results?

3. Evidence and Substantive Support

4. Drug Control – The Nightmare of Evaluating Programmes and Initiatives

5. Change and Power Relations

6. What is the Way Forward?

7. Inclusive Way Forward – The Future

8. Conclusions

Epilogue

References

Foreword

This book is fundamentally about the ‘use or usefulness of evaluation’. Dr. Backson Sibanda has succeeded in capturing what many evaluators are aware of but have not been able to express in such a clear manner.

The overall theme of the book revolves around the following: the weaknesses associated with ‘learning’ from evaluation; the winners and losers from the findings of evaluation; the lost opportunities for the delivering of results, in cases where ‘good’ recommendations from evaluations are not accepted and / or implemented; the difficulties associated with some projects / programmes, in that by their very nature, the knowledge base on which to determine success or failure has not yet been established; and that the education and training of evaluators covers tools and techniques extensively, but is very weak on issues such as organisational politics, power dynamics, and effective communication.

In addressing the above, the author has brought some thirty years of experience as an evaluator to bear on the analysis and interpretation of the data and information presented. A central aspect of the book emphasises that various stakeholders are primarily interested in their own ‘winning’, and not necessarily in the success of their organisations. Hence, evaluation findings that are deemed to be ‘positive’ are accepted, while those that are deemed to be ‘negative’ are contested. He writes: Most evaluators have very good if not excellent technical evaluation skills and knowledge …The majority do a great technical job and produce professional, credible, and evidence-based high-quality evaluations. In general, disputes and non-acceptance of evaluation results is not because of their quality but one of contesting what is viewed as negative and fault-finding due to non-delivery of some planned results. He goes on to highlight that ‘disputes’ over some evaluation findings consume efforts that could otherwise have been used for improving the delivery / achievement of results in given organisations. Several of the cases presented show this quite clearly.

This is a ‘bold’ piece of work! And, I congratulate the author. The book touches on important issues about evaluation. It proposes some solutions which will be quite beneficial to all involved: from governing boards of international organisations to programme / project managers; and from governments and their officials right through to civil society organisations.

E. Alaphia Wright, PhD.

Retired UNESCO Director and Representative

Acknowledgements

My sincere thanks first and foremost go to Prof E. Alaphia Wright and Mr Roger Miranda for reviewing the manuscript and providing me with the most helpful comments and input and encouraging me to remain focussed and bold on a very sensitive subject. Further I would like to thank the many evaluation professionals especially those that I worked with in the United Nations for 22 years in four different agencies who inspired me and shared with me their ideas and support during some of the most difficult times when evaluation was still a very new science generally viewed as police work. Also, my sincere thanks go to many professionals who participated in many of the training workshops that I facilitated for government and civil society employees across the world from whom I learned a lot from their work and exposure. Thanks also go to thousands of programme managers and implementers and executives who I worked with in the UN and outside the UN and who through our various interactions shared with me many joyful and sometimes difficult experiences as we all laboured to contribute to a better world.

Many thanks go to Mrs Iris Pilane Sibanda for providing the initial editing and suggesting ways of how to present these very challenging ideas in a clear and understandable way even to those who are not evaluators. Her encouragement enabled me to remain focussed and reflect on what and how I communicated the messages.

Finally, my sincere thanks go to my wife Shirley Thabani for her love and support over many years and for enabling me to pursue my passions and sacrificing a lot of time that should have been devoted to her and the family.

Prologue

Your evaluations are destroying this organisation and I will not let this happen even if it means that you and your team must leave the organisation, said one Under-Secretary-General (USG) of the UN to a mere P5 Chief of Evaluation. He continued, I will not allow you to continue to destroy the organisation. I will get rid of you and your team and bring on board those who will be sensitive to the organisation’s needs, he closed the discussion with finality.

I was left numb remembering five years earlier when the same Under-Secretary-General interviewed me after the technical committee had recommended that I be appointed Chief of Evaluation. At that point he said, I need a man of your vision and conviction with the knowledge and expertise to introduce a new culture of accountability to the organisation. That closed the deal. The Under-Secretary-General waited for me for four months while I finished my assignment with another UN agency. After this meeting I woke up from a five-year slumber and realised that things had changed. When he interviewed me five years earlier, he was very new at the UN and had an open mind. Now, the organisational cultural fever had finally caught up with him. It also became very clear to me that the excellent professional relationship I had enjoyed with the boss was gone and a new phase and new reality had just begun. What needs to be learned upfront from this example is that sometimes when changes come some people in the organisation may not be ready for it, hence feel threatened especially when they feel that those changes threaten either their positions or influence or both. As the story unfolds, these issues will become clearer. If this example of my experience was an isolated case it really would not matter much; the world could go on without raising an eyebrow. Unfortunately, this is not an isolated case and with hindsight 20 years later, it is now clear to me that most evaluators have not been trained to deal with issues of how to manage change that comes because of evaluations. They are not trained to deal with the King Lion syndrome.

Three years before this particular USG had interviewed me, another UN agency had been so excited to hire me to develop a six-year Medium Term Strategy. That agency had seen the Medium-Term Strategy which I had developed for another UN agency which had led to it being funded for six years (Medium Term Strategy) rather than the normal two years (bi-annual budget). Within 90 days of joining that particular UN Agency, I presented the eagerly awaited Medium Term Strategy and an accompanying Results Framework as well as a Monitoring, Reporting and Evaluation Framework to the Governing Body. This was met with a standing ovation. As we left the General Assembly the Assistant Secretary-General followed me to my office, closed the door behind him and said, Young man, since you came here, you have made me sleepless. I have been here for 40 years and you are disturbing my sleep just before I retire. Even the Director-General has never been given that honour by the GA. If you want to stay here and get promoted just do as you are told. He said that with finality and yet it was he who had asked me to prepare the Strategy and I thought I had done as I had been told. Further, at this point I did not even understand the culture of the organisation and so could not really comprehend what the issues were. I tried to remain professional but after 12 months it was clear I could not stay. I agreed that I would not renew my contract when it expired in nine months’ time and I was also forced to forego my promotion to a higher level as per probation provisions. I was given a full package and allowed to leave the organisation, so I could