The JLN Monitoring & Evaluation (M&E) approach systematically documents the JLN’s role in changes at the individual, interpersonal, and team levels and contributions to organizational and health system-level shifts. The M&E system is a living, adaptable system that builds on the historical work done by the network while also pushing the network towards more rigorous and credible linkages to longer term outcomes.
What We Know
The JLN is a country-driven network of practitioners and policymakers from around the globe who come together to problem solve, co-develop global knowledge products, and implement solutions that help bridge the gap between theory and practice.
Core to the JLN is the cultivation of a “safe space” for practitioners and policymakers to discuss successes and failures in an honest and productive way to learn, adapt and enact changes that will address the root causes of a problem. In the spirit of joint learning, this page illustrates evidence to support the JLN’s approach and areas where there is room to further strengthen the evidence-base.
- We know that: practitioners and policymakers that participate in the JLN’s technical teams report changes in personal knowledge, attitude, and confidence
- We know that: practitioners and policymakers strengthen relationships and build professional networks that oftentimes extend beyond formal engagements with the network. Anecdotally, we hear that long term engagement in the network helps practitioners around the global advance their careers, bringing in perspectives beyond country borders.
- We know that: Many JLN teams produce knowledge products that are global public goods available on our website and there are 60+ examples of use of knowledge products in countries around the world. More recently, teams of shorter duration have developed 15+ sharable outputs.
- We know that: JLN knowledge products are often used by JLN countries and can result in country change (at the national and sub-national level)– though timing and political will are integral to how and where knowledge products are applied.
- We know that: We know that JLN knowledge products are often utilized but sometimes are not enough to result in change. In those cases, countries have successfully implemented when additional resources including peer-learning style facilitation or financial resources are available.
- We know that: Virtual engagement can work for the network and most participants are interested in a hybrid model in the future.
- We know that: More recently, the network has pivoted to real-time implementation as part of a technical team. This approach has resulted in country level health system changes. This approach is useful not just for the implementing country team, but also for those learning from the experience in real time. These non-implementing countries participating virtually report learning practical lessons that can likely be applied in their country-contexts in the future.
- We know that: Often JLN countries engage over a long period of time to advance a single UCH priority in their country. They join many different JLN technical teams on the journey to enacting this priority.
Where We're Going
The JLN recognizes that there are different types of evidence needed to answer different types of programmatic questions. Evidence exists along a continuum from anecdotes (far left), to more rigorous designs, including case-control or quasi-experimental designs. Some questions are best answered using information from testimonials or anecdotes, whereas other questions require different levels of evaluative rigor to be thoroughly answered.
In this grant cycle, the JLN Network Manager has started to push the JLN rightward along the path to answering more complex questions on behalf of the network. Activities embedded in the technical initiative M&E function have focused on generating comparative data over time to document participant knowledge, attitude, and practice shifts (see Comparative data in the graphic above). Understanding self-reported changes at the technical team level will clarify the link between participation in JLN learning activities and in-country advocacy and change.
Additionally, while retrospective documentation of knowledge product use (the country case study section below) has some methodological flaws (recall bias, etc.), building in intentional validation processes– like triangulation of data across sources and internal agency reviews– helps us answer more complex questions. More recent work has included documentation of real-time facilitated implementation.
The Network Manager team is constantly working to improve and articulate the network’s impact. Questions like how we think JLN’s implementation efforts contribute to system changes at the organizational, institutional, or system level, help to articulate the network’s impact and improve the JLN model. Continuing to collect data at the right stage of the continuum is an ongoing priority to answer the most critical questions for the network. In combination with pushing to further expand the evidence base of the JLN’s approach, the Network Manager team continues to strive towards an evidence-based, ever improving learning approaches.
Like many networks, the JLN still has gaps in its evidence base. In the spirit of learning, the network is actively working to prioritize new methods–like contribution tracing and more longitudinal data collection– and push boundaries to answer important questions for the network.
CONCEPTUAL FRAMEWORK
KNOWLEDGE PRODUCT UTILIZATION
INNOVATIVE JOINT LEARNING
REAL-TIME FACILITATED IMPLEMENTATION
TECHNICAL TEAM DASHBOARD
CONCEPTUAL FRAMEWORK
The JLN is a country-driven network of practitioners and policymakers from around the globe who come together to problem solve, co-develop global knowledge products, and implement solutions that help bridge the gap between theory and practice.
Practitioners and policymakers work together with technical facilitators to learn together (“joint learning”), to analyze challenges, share experiences, and co-create the development of how-to’s to address challenges based on global evidence and best practices. Technical facilitators play a background role, bringing in organizational capacity and analytic rigor to help countries frame issues and articulate their insights in a structured manner.
The JLN encourages flexible thinking, enabling practitioners to synthesize new knowledge into knowledge products – including tools, assessments, policy analysis frameworks, decision-making tools, implementation guidance, and case studies– that both serve the needs of the country participants who co-created them, and become global public goods for the global health community.
The JLN Theory of Change predicts that JLN outputs will lead to knowledge dissemination, increased advocacy, and knowledge application at the country level. Directly engaging in problem-solving and creating new knowledge with peers in other countries will build the capacity and motivation of local leaders who are well-positioned to implement reforms, leading to changes in policy and practice in their countries.
KNOWLEDGE PRODUCT UTILIZATION
The JLN approach has historically included the co-development of knowledge products as part of the JLN’s longer technical teams. These documents are written collectively by the JLN country members and shepherded along by the technical facilitation team. The JLN has developed over 30 knowledge products in its history and has many examples of application of these tools in JLN countries. The JLN has two terms for knowledge product utilization– adaptation and implementation. What do we mean by these terms?
Some of these examples have been documented in case studies by the network highlighting how JLN knowledge products have resulted in changes in various countries. Countries highlighted in orange have participated in JLN Adaptation & Implementation activities, countries with blue dots have documented case studies. Click on each of the countries to read more about each knowledge product utilization.
Medical Audits in India
India drew on its participation in the JLN’s Medical Audits Collaborative and adaptation of the Medical Audit Toolkit to implement a medical audit system that could be decentralized and scaled by India’s states to review claims, identify fraud, and ensure the quality of health services being provided to patients.
Using Strategic Communications in Nigeria
Nigeria’s implementation of the Strategic Communications Practical Guide and Strategic Communications Planning Tool to address underlying facilitators and barriers to the BHCPF led to the first appropriation for the fund and an increase in interest in BHCPF services. The communication plan is thought to have improved the targeting and perception of the rollout communication processes.
Medical Audits in Malaysia
Malaysia’s use of the Toolkit to Develop and Strengthen Medical Audit Systems shaped the operations for a new Medical Audit Section responsible for auditing PeKa B40 claims. The SOPs for the unit support identifying and deterring fraud and review of provider perfomance resulting in improved quality of health care screenings.
Costing of Health Services for Provider Payment in Kenya
Kenya’s use of the Costing of Health Services for Provider Payment toolkit produced findings that were used to change county-level reimbursement models to lower costs, increase efficiency and improve financial sustainabilty. This case study profiles the use of Costing of Health Services for Provider Payment in Kenya. In Makueni County, the government has the mandate […]
UHC PHC Self-Assessment in Malaysia
Malaysia’s use of the JLN UHC PHC Self-Assessment produced findings that were used to improve public private partnerships to increase PHC coverage. This case study profiles the use of the Universal Health Coverage Primary Health Care Self-Assessment Tool in Malaysia. Malaysia piloted this self-assessment tool as a cross-sectional study collecting both qualitative and quantitative data […]
INNOVATIVE JOINT LEARNING
The JLN is continuously improving and iterating on its approach to joint learning. In the most recent grant cycle, there have been two innovative approaches the network has focused on. Both are smaller group modalities with an intensive focus on one country’s challenges, with other countries serving as resource countries, often alongside a technical facilitation team. These newer models are important to countries progress on key UHC objectives by end users and therefore appear to be worthwhile investments for the network.
Other promising new joint learning approaches that have been tried by the network but need to be evaluated for efficacy and results in the future including:
- Facilitated Thematic Dialogues: A facilitated thematic dialogue is a structured knowledge exchange between technical experts from different countries with the purpose of elucidating answers to a specific technical issue. Most of the technical facilitation support is provided before the exchange itself, including selection of participants and production of background materials. The modality prioritizes safe space for inter-country discussions and learning- and does not use any video recording, relying instead on notes from the facilitation team for broader lessons and record. The objective is to provide a responsive format that supports immediate exchange on pressing topic at country level, as requested by a country or a group of countries. These are structured on a demand basis, and could involve asks to pair with particular countries, or having facilitators seek a country pairing depending on the nature of the request. The point is to keep the majority of the session focused on a back-and-forth exchange and nurturing dialogue between the countries involved (country A and country B). Questions will be used to prompt and prepare a back and forth at the outset of the discussion, with the majority of time for open exchange between participating countries. Those joining will be in the form of an adapted fishbowl- the discussion will be closed at the outset with participants in listening mode for the majority of the discussion, and able to ask questions at the end of the dialogue.
- Country Facilitator Model: Strong facilitation is a cornerstone of the JLN. This approach was intended to test sustainable models of facilitation of high-impact, country led learning platforms and to explore alternative modalities for collaborative learning by selecting technical facilitators from JLN member countries. This complements external facilitation conducted by global partners, while transitioning technical and facilitation capacities to JLN members. The approach sustains joint learning activities at both the global and country levels as well as nurtures a cadre of regional experts to support capacity building in-country. Under this new approach, country-based facilitators take on a dual role in the collaborative, engaging with members both as a country participant and as a technical expert. In this way, country-based facilitators served as an important resource, leveraging their deep technical knowledge to identify opportunities for shared learning among collaborative members and bringing insights back to the core facilitation team on how these opportunities could be leveraged to generate shared learnings and practical knowledge products.
REAL-TIME FACILITATED IMPLEMENTATION
In the past few years, the JLN has intentionally shifted from a heavy focus on co-created products into an era that also emphasizes the importance of real-time facilitated implementation. This section details real examples of this approach. The countries below have participated in real-time facilitated implementation cases that are currently ongoing (orange only) or have concluded to date (blue dots). Click on each blue dot to read more: