Planning Monitoring and Evaluation for Public and Private Sectors

By Dr Christopher Phiri

How often have we ignored monitoring and evaluation – as if appreciating our little efforts, is of no significance? This book is a timely contribution to an ongoing debate about the need for accountability and transparency in whatever we do – whether we are development practitioners, policy makers, researchers and all those working in the public, private and third sector organisations. A truly stimulating piece of work which students of Development Studies will find it equally informative. The Covid-19 Outbreaks have demonstrated that we are not yet there where systems and processes are in place to plan, monitor and evaluate. If empowerment programmes are managed, monitored and evaluated in ways that are top-down, technocratic and controlling, governments’, private sectors’ and donors’ actions can have unintended disempowering effects.

Planning Monitoring and Evaluation continue to be a fundamental 2063 Africa agenda promoting accountability, effectiveness and efficiency in Local government and all spheres of government. Skilling, up skilling and reskilling in Planning M and E should become business as usual in any organisation.

• Language: English
• Publisher: Dr Christopher Phiri
• Release date: Feb 17, 2022
• ISBN: 9781005751722

Book preview

Planning Monitoring and Evaluation for

Public and Private Sectors

Planning Monitoring and Evaluation for

Public and Private Sectors

Up-Skilling and Re-skilling of 21st Century Government, Academic, NGOs and Private practitioners

Christopher Phiri PhD

Charles Chimombo MSc Strategic Management

Copyright © 2021 Christopher Phiri PhD and Charles Chimombo

First edition 2021

Published by Christopher Phiri PhD and Charles Chimombo Publishing at Smashwords

All rights reserved. No part of this book may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording or any information storage or retrieval system without permission from the copyright holder.

The Author has made every effort to trace and acknowledge sources/resources/individuals. In the event that any images/information have been incorrectly attributed or credited, the Author will be pleased to rectify these omissions at the earliest opportunity.

Editors

Professor Learn more Kambizi Cape Peninsula University of Technology

Professor Serestina Viriri University of KwaZulu Natal

Ms. Desiree Johnson National Department of Social Development

Professor Jonathan Makuwira Deputy Vice Chancellor Malawi University of Science and Technology

Authors

Dr Christopher Phiri is a seasoned Author, a Strategic Planning, Design, Research, Monitoring and Evaluation practitioner in the Public sector, Academic and NGO space. He has extensive experience in strategy planning, M and E for socio-economic development. An Anthropologist, Philanthropist and A Policy Analyst who has trained, mentored, and coached professionals, communities and Public and private sector on M and E and its application. He has provided strategic support to Public and Private sector in South Africa and is a seasoned Academic in Monitoring and Evaluation.

Charles Chimombo is a humanitarian by heart and actions, an international monitoring and evaluation maestro as well as an expert in community development with more than ten (10) years of work experience in international nongovernmental organizations. He has left indelible footprints in Public Health and Nutrition, Disaster Risk Management including Natural Resource Management, Livelihoods, Agriculture and Food Security, Education and Literacy, Gender and advocacy, Water, Sanitation and Hygiene (WASH), Child protection and HIV/AIDS. The Director of Operations for World Vision Malawi and holds Bachelors degree in Human and Social Studies, Masters of Science degree in Strategic Management and currently enrolled for a Masters in Public Health.

TABLE OF CONTENTS

Acknowledgement

Declaration

Foreword

Acronmys

1. Introduction to Monitoring and Evaluation

1.1 Introduction

1.2 The Changing Context of M&E

1.3 Common Tools and Approaches

1.4 Conclusion

2. Monitoring and Evaluation in Government and Development Sectors

2.1 Monitoring and Evaluation in development

2.1.1 The Process of developing the M and E book

2.1.2 Institutional M&E System Assessment

2.1.3 The design of the M and E Book

2.1.4 By Who and by when should this M&E book be used?

2.2 Conclusion

3. Theory of Change and Its Applicability

3.1 Introduction

3.1.1 The Politics in Monitoring and Evaluation

3.1.2 Importance of Assumptions in developing TOC

3.2 Defining Theory of change from our perspective

3.2.1 Benefits of Theory of Change

3.2.2 Key principles for developing a theory of change

3.2.3 Key steps for developing a theory of change

3.2.4 Key steps for developing a theory of change that we have used over time

3.2.5 Step 1: Revise Negative Statements:

3.2.6 Step 2: Rewrite into possible solutions:

3.2.7 Step 3: Identify positive solutions:

3.3 A Theory of Change

3.4 This can be Descirbed as

3.4.2 Illustration of Theory Of Change

3.5 Theory of Change statement.

3.6 Development of Indicators in the Planning Process

3.6 Strategic Plans

3.7 Conclusion

4. Agenda 2030 National Development Plan

4.1 Importance of Planning, Training, Monitoring and Evaluation

4.2 Improved leadership, governance and accountability

4.3 Functional, efficient and integrated government

4.4 Professional, meritocratic and ethical public administration

4.5 Social compact and engagement with key stakeholders

4.6 Mainstreaming of gender, empowerment of youth and people with disabilities

4.7 The role of M and E in catalysing Performance Management and service delivery

4.8 Conclusion

5. Project Management Monitoring And Evaluation

5.1 Introduction

5.2 The Changing Context of M&E

5.3 What is Project Monitoring?

5.4 What is Project Evaluation?

5.5 What is the difference between Monitoring and Evaluation?

5.6 Why do we need to Monitor and Evaluate Projects?

5.7 What will contribute to successful Project Monitoring and Evaluation?

5.8 Conclusion

6. Project Planning

6.1 Introduction

6.1.1 The benefits of planning

6.1.2 Frustrations of Middle Management and Junior officials

6.2 What is Project Planning?

2.3 What are the Steps that are followed in Project Planning?

6.4 What are the key considerations during Project Planning?

6.5 Conclusion

7. Project Implementation And Monitoring

7.1 Introduction

7.1.1 What is Project Implementation?

7.1.2 Project Monitoring

7.1.3 The importance of Project objectives

7.1.4 What indicators will measure project success?

7.1.5 What is the process of developing indicators?

7.1.6 What methods and tools will be used to gather project data or information?

7.1.7 Steps in collecting Monitoring data and Evaluation data or information

7.2 Conclusion

8. Project Evaluations

8.1 Introduction

8.2 Steps involved in the Planning and Implementation of a Baseline or Evaluation Survey

8.2.2 Step 1: Planning for Baseline and Evaluation Surveys (Ante evaluation, Summative, Formative and impact evaluation)

8.2.3 Step 2: Training of Baseline/Evaluation Survey Team

8.2.4 Step 3: Pre-testing and sampling approach

8.2.5 Step 4: Collect the data

8.2.6 Step 5: Data coding, Entry and Cleaning

8.2.7. Step 6: Analyse the Data

8.2.8 Step 7: Compile the Key findings and Draft Report

8.2.9 Step 8: Share the results

8.2.10 Step 9: Prepare Final Baseline or Evaluation Report

8.2.11 Step 10: Revise Program Planning/Design

8.3 Types of evaluation and evaluation processes

8.3.2 Impact evaluation

8.3.3 Ante ex evaluation

8.3.4 Operational evaluation

8.3.5 Process evaluation

8.3.6 Evaluation Processes

8.3.7 Outcome evaluation

8.4 Conclusion

9. Developmental Evaluation

9.1 Introduction

9.2 Clarifications on Developmental Evaluation Myths

9.3 Collecting data through Developmental Evaluation

9.4 Sustainability through building evaluation capacity

9.5 Conclusion

10. Managing Issues During Monitoring And Evaluation

10.1 Introduction

10.2 Use of the Monitoring and Evaluation Information

10.3 What is a project Logframe?

10.4 Why do you need a project Logframe?

10.4.1 10 Steps you need to follow in the development of a Logframe

10.5 What is a Performance Monitoring or M & E Plan?

10.6 Steps you need to consider in the development of a PMP

10.7 Steps you need to consider in the development of an IPTT

11. Results Based Management for Public and Private Sectors

11.1 Introduction

11.2 Some Background

11.3 Over The 13 Years I Have Learnt That Too Much Complexity In The Data Collection System And Performance Measurement Is The Major Obstacle To The Efficient Implementation Of Rbm.

11.4 Result Based Management approach

11.5 Tips for priority setting

11.6 Crafting results statements

11.7 Importance of Indicators

11.8 Risk Management

11.9 Conclusion

12. Capacity Building for Monitoring and Evalaution

12.1 Introduction

12.2 Integrated, Multi-level Strategies for Capacity Building

12.3 Conclusion

13. Participatory Monitoring And Evaluation

13.1 Introduction

13.2 Government or Donor pressure

13.3 Participatory M and E model

13.3.1 Approaches to PM&E

13.4 Designing PME Systems

13.5 Conditions Necessary for PME

13.6 Checklist for Designing a PME System

13.7 Conclusion

14. Monitoring the Implementation of Collective Agreements in the Public Service

14.1 Introduction

14.1.1 The Labour Relations Act

14.1.2 The role of Trade Unions in the Public service

14.1.3 The role of the SA government as employer in Labour Relations

14.1.4 The establishment and role of PSCBC

14.1.5 The Collective Agreements

14.2 Discussions

14.2.1 The future of Labour Relations

14.3 Conclusion

15. The Role of Private and Public Sectors in Advancing M and E

15.1 Introduction

15.2 Difficulties with donor procedures

15.2.1 Government and Donors should be more transparent

15.2.2 Guiding principles on providing coordinated aid

15.2.3 Multi-year funding commitments

15.2.4 An open process for managing any concerns

15.2.5 Consult with partner governments:

15.3 Conclusion

16. Challenges in Implementing Monitoring and Evaluation ProcesseS

16.1 Introduction

16.2 M and E challenges faced by the government

16.2.1 Reflective Practice Competence challenges

16.2.2 Lack of adequate and skilled personnel in M&E

16.2.3 Absence of a learning culture

16.2.4 Monitoring and evaluation is ‘donor driven’

16.2.5 Inadequate Attention To M And E During Program/Project Design

16.2.6 Lack of participation of stakeholders in the development of M&E systems

16.2.6 Lack of contextualizing M and E systems

16.3 Managing practice competence challenges

16.4 Conclusion

17. Government Wide Monitoring and Evaluation

17.1 Introduction

17.2 System users and their needs

17.3 System aims, objectives and intended results

17.4 Performance indicators and sources of information

17.5 System Reports and their utilisation

17.6 Roles and responsibilities

17.7 Critique of GWME as applied in South Africa

17.8 Conclusion

18. Change Management ‘MYTH or REALITY’

18.1 Introduction

18.2 Critical questions to ponder:

18.3 The Prosci 3-Phase Process

18.3.1 Phase 1 – Prepare Approach

18.3.2 Phase 2 – Manage Change

18.3.3 Phase 3 – Sustain Outcomes

18.3.4 Get the Buy-in You Need

18.3.5 Track the Change

18.3.6 Support Open Communication

18.3.7 Monitoring the implementation of change

18.3.8 Reflection point

18.4 Eight ways to manage change in the workplace effectively

18.5 Six Steps to Effective Organizational Change Management

18.6 Conclusion

19. COVID 19 Pandemic, Disaster Management and Monitoring

19.1 Introduction

19.2 Strengths and weaknesses

19.3 Overview on M and E during disaster

19.4 Country Case Studies

19.4.1 South Africa During the Covid-19 pandemic

19.4.2 Lessons Learnt from the Ebola Experience

19.4.3 The politics of Covid 19 across the globe

19.4.4 Reflections

19.4.5 Monitoring Process from an NGO sector (adopted from World Vision International 2020)

19.4.6 Conclusion

20. Conclusion

20.1 External context

20.2 Beyond policy formulation

20.3 Conclusive comments on M and E Training

Bibliography

List of Tables

Table 1: Guideline of Theory of Change Testing Guidelines

Table 2: Stakeholder matrix

Table 3: Stakeholder Analysis Matrix

Table 4: The importance of different stakeholders to the potential Food security project

Table 5: Example of a Project Planning Matrix Model for WALA Project

Table 6: characteristics of good indicators

Table 7: Developmental evaluation application guide

Table 8: Types of Indicators

Table 9: Conventional and Participatory Evaluations

Table 10: Monitoring Process from an NGO sector

List of Figures

Figure 1: Theory of Change template

Figure 2: Source: World Vision Change management 2012

Figure 3: The Seven Steps of Project Planning Process

Figure 4: Problem tree Source: World Vision 2020 Child protection model

Figure 5: Objective tree

Figure 6: Example of a logical framework

Figure 7: Target Framework

Figure 8: Social Return of Investment illustration (source Miller and Hall, 2013)

Figure 9: Detailed Implementation plan sample Source: World Vision International

Figure 10: Logical Framework

Figure 11: The performance Monitoring Plan (PMP)

Figure 12: Example of IPTT for a MCHN Project

Figure 13: Source: (Almazan, 2012; Kusek and Rist, 2004; Meier, 2003; Spreckley and hunt, 2009).

Figure 14: Source: (Gamble, 2008).

Figure 15: Appreciative inquiry

Figure 16: Source: PEPFAR Capacity building and strengthening framework, 2012

Figure 17: Strategic planning cycle

Figure 18: Source: (PEPFAR Capacity building and strengthening framework, 2012).

Figure 19: Participatory Monitoring and Evaluation

Figure 20: Analysis of Public Service Collective Agreements Implemented

Figure 21: New Partnership model

Figure 22: Source: (OECD, 2019).

Figure 23: Change Process

Figure 24: Change Management Triangle

Figure 25: Change Management Processes

Figure 26: Prosci 3 Phase Process

Figure 27: Change Management Flow chart

Acknowledgement

To God be the glory for the things He has done and for guiding us to complete the book. This book would not have been possible to publish without the necessary moral support from family members of the authors (Dr Phiri’s wife Maria, Mr Chimombo’s wife Evelyn and their children. Thanks to the book editing team for the splendid work and to Public Service Coordinating Bargaining Council for the financial sponsorship that made it possible to publish the book. To all who contributed directly and indirectly to internal review, information-sharing, thank you.

Declaration

This book is a collection of our lived experiences in the Planning Monitoring and Evaluation space for the past 13 years and existing literature on M and E broadly. The book refers in every strategic process of the project management cycle our lived experiences as practitioners in the Planning Monitoring and Evaluation where we provided leadership and were hands-on. Where information was collected and used from different sources, as well as other authors, these sources, have been acknowledged through citation and referencing. Sources include academic literature, government strategic planning, monitoring and evaluation reports and documents from different institutions

Foreword

In the 21st century any project including Human life needs thorough planning, monitoring of the project and decision gates along the way. Outbreaks such as Covid 19 pandemic have demonstrated that we are not yet there where systems and processes are in place to plan, monitor and evaluate. This is often blinded by wanting quick wins at the expense of due diligence. Understanding the project management life cycle often helps the project owners to plan and implement their project (s) or programmes well. More often projects fail because stakeholders have no clear picture of the vision, mission, and goal of the pathway of change the project set to achieve. This has made scholars, academics, government, Philanthropy and development practitioners to embrace the discourse of results based management framework within the project life cycle which include at all stages the Monitoring and Evaluation processes. Many books have been written about Monitoring and Evaluation since its introduction by donor agencies to today. However, it is important to note that project management has been revolving and new dynamics are emerging which need to be identified and debated upon in government as well as other public and private platforms.

While the process of promoting Monitoring and Evaluation involves identifying respective tools, guidelines as well as empowerment of staff and communities, the whole process is not homogeneous and is not one size fits all. As institutions plan to be effective and efficient thus ‘doing things right’ and doing right things’ M and E is no longer an area of technical teams alone but for all involved in any programme or project, from senior leadership to middle management. This therefore need continuous up skilling and reskilling of staff as an empowerment process to ensure the organisation achieves its set objectives. However, as the book outlines in some of the chapters, it is not an easy road due to organisational politics and leadership style.

Understanding the ‘whole’ is so critical before any plans to empower the team thus from identifying the root cause of the existence of the organisation, its vision, mission, its strategy, the theory of change, business plan, operational plan and the monitoring and accountability systems as well as the evaluation process. Half-baked management systems and approaches of ‘them and us’ do not apply in the 21st century development agenda in both public and private sectors. The process of planning from design phase, development of theory of change, to strategy development, as well as business plans, medium term strategic frameworks as well as operational plans has been externally controlled and managed inside as some institutions prefer to hire consultants to do their Monitoring and Evaluation (M and E) work to fast track the process. Such management has dire consequences to the organisation. This has been evident in government departments, corporate sector and NGOs’ environment. This requires strong internal process to empower management from higher to lower level to embrace the M and E culture.

Holland and Ruedin (2012) has argued that empowerment is a process that cannot be externally controlled and managed. If empowerment programmes are managed, monitored and evaluated in ways that are top-down, technocratic and controlling, governments’, private sectors’ and donors’ actions can have unintended disempowering effects. While monitoring and evaluation tools are useful for learning and improving effectiveness, in the context of empowerment work they must be designed and used with care, so that the monitoring and evaluation process itself can be supportive of empowerment outcomes

If government, corporate, NGOs and donors use methods that are technocratic and controlling, they can have unintended disempowering effects. Monitoring and Evaluation (M&E) must be used with care in the context of empowerment. Government, corporate, and donors have to strike a balance between the flexibility required in supporting empowerment and social change and their need to demonstrate results and impact. Monitoring and assessing empowerment can encourage governments, private sectors and donors to recognise that they themselves exercise power and reflect on the impact this has on the empowerment process. Government, corporate, NGOs and other donors should also recognise the limits to their power within larger political structure. Participatory methods for monitoring and evaluation can be efficient, producing data for analysis and action to support empowerment if adopted and utilised fully.

There is tension for governments, private sector and donors between adopting a flexible and enabling role in supporting an empowerment process of social change and the obligation to demonstrate results and fulfil internal rules and regulations. This has been demonstrated in different countries during the Covid-19 pandemic and how the disaster was monitored and managed. Countries have been caught unaware by the pandemic and have been struggling to empower citizens, change behaviours, promote better health systems and reduce the numbers of ordinary citizens affected and combat Covid-19. Some countries have shown strong monitoring systems than others.

Understanding ‘what works, what does not and why’, is central to this evolving and deepening self-awareness of an organisation and to the process of changing governments or donor/partner relationships. Monitoring and assessing empowerment can encourage government or donors to recognise that they themselves exercise power in their relationship with local municipality, district municipality, provincial and national and to reflect on the impact that they have on empowerment processes of staff at different levels. It is important to outline that the book is based on our experiences in the strategic planning, Monitoring and Evaluation space working with government at National and local level as well as with NGO sector for more than 15 years. So the debates we put together and the guide giving to practitioners is strongly linked to the experiences we went through being hands on providing strategic leadership and training of staff and communities.

This book attempts to assist both private and public sectors to seriously reconsider putting enough resources towards the empowerment process on M and E as demonstrated by the Covid-19 pandemic that strong M and E systems are needed at every stage of a programme, project or a pandemic. Planning Monitoring and Evaluation continue to be a fundamental 2063 Africa agenda promoting accountability, effectiveness and efficiency in this uncertain future of development. Skilling, up skilling and reskilling in M and E should become business as usual in any organisation.

Professor Jonathan Makuwira Malawi University of Science and Technology

Acronmys

COGTA Corporate Governance Traditional Affairs

DAC Development Assistance Committee

DE Developmental Evaluation

DPLG Department of Provincial Local Government

DPME Department of Planning Monitoring and Evaluation

DPSA Department of Public Service and Administration

GIS Geographic Information system

GWM&ES Government-wide Monitoring and Evaluation System

IPTT Indicator Performance Tracking Table

IT Information Technology

LFA Logical Frame Work Analysis,

M and E Monitoring and Evaluation

MIS Management Information System

MTEF Medium Term Expenditure Framework

MOV Means of Verification

NDP National Development Plan

NGO Non-Governmental Organisation

NSG National School of government

OECD Organisation for Economic Co-operation and Development 

OVI Objectively Verifiable Indicators

PCM Project Cycle Management

PLA Participatory Learning and Action

PME Participatory Monitoring and Evaluation

PMP Performance Monitoring Plan

PPEs Personal Protective Equipment

PPM Project Planning Matrix

PRA Participatory Rural Appraisal

RBM Results Based Management

RCCE Risk communication and community engagement

SAMEA South African Monitoring and Evaluation Association

SDGs Sustainable Development Goals

SROI Social Return on Investment

TLC Total Land Care

TOC Theory of change

WASH Water, Sanitation and Hygiene

WHO World Health Organisation

CHAPTER 1

Introduction to Monitoring and Evaluation

"Whose fault was it that the targets were not met and no impact at all after 3 years of hard work? Was this the problem of senior management, technical team or implementers or the stakeholders?"

1.1 Introduction

Whose fault was it that the targets were not met and no impact at all after 3 years of hard work? Was this the problem of senior management, technical team or implementers or the stakeholders? These are critical questions that we have encountered in the Planning, Monitoring and Evaluation space as practitioners which we need to answer authenticated by our experiences. The answer we found it in the organisation, how the organisation or department is positioned in terms of its Result Based Management.

The chapter outlines debates around the rise and importance of Monitoring and Evaluation (M&E) in development, corporate and government institutions. The debates encompasses both academic as well as our own first-hand experiences in the field leading international and national programmes. The process of promoting Monitoring and Evaluation involves identifying respective tools, and guidelines as well as empowerment of staff and communities. However, the whole process is not homogeneous and is not one-size fits all.

Staff empowerment in M and E as well as the M and E Culture are some parts that are often compromised and left to the technical team yet have a lot of effect on the failure or success of an organisation in achieving the set goals and objectives, long term and short term. The process of empowerment cannot be externally controlled and managed. If donors use methods that are technocratic and controlling they can have unintended disempowering effects. Monitoring and Evaluation (M&E) must be used with care in the context of empowerment as it is no longer an option but priority. Governments, private sector and donors have to strike a balance between the flexibility required in supporting empowerment and social change as well as the need to demonstrate results. Monitoring and assessing empowerment can encourage governments, donors to recognise that they themselves exercise power and reflect on the impact this have on the empowerment process. What we experienced is that, governments, private sector and donors should also recognise the limits to their power within the larger political structure. Participatory methods for monitoring and evaluation can be efficient, producing data for analysis and action to support empowerment.

Holland and Ruedin (2012) argued that empowerment is a process that cannot be externally controlled and managed. If empowerment programmes are managed, monitored and evaluated in ways that are top-down, technocratic and controlling, governments’, private sectors’ and donors’ actions can have unintended disempowering effects. While Monitoring and Evaluation tools are useful for learning and improving effectiveness, in the context of empowerment work they must be designed and used with care, so that the monitoring and evaluation process itself can be supportive of empowerment outcomes. There is tension for governments, private sector and donors to adopt flexible and enabling role in supporting an empowerment process of social change and the obligation to demonstrate results and fulfil internal rules and regulations.

What we learnt in the governance space was that doing things right and doing right things remain a challenge. Understanding what works, what does not work and why is central to this evolving and deepening self-awareness and to the process of changing donor/partner relationships. Monitoring and assessing empowerment can encourage governments, private sector and donors to recognise that they themselves exercise power in their relationship with partners at different levels and to reflect on the impact that they have had on empowerment processes.

Local government, Provincial and National governments, private sector and donors should recognise the limits to their power within bigger political structures and processes. Participatory methods can be quick and efficient, producing data in a timely fashion for evidence-based analysis and action in ways that support empowerment. Many government and Public sector staff lack M and E skills from senior leadership to middle management and junior level.

Governments, private sector, donors and local stakeholders increasingly recognise the importance of monitoring and evaluating development policies and programmes. Monitoring and Evaluation has become so important in any development space contributing to insights about what does and does not work and why. Furthermore, it enables programme changes that will make donors and partners more effective at