Beeline Reader
Executive Summary
Project Host:
Project Host: Teach the world Foundation
Fellows:
Christopher Geary, Team Lead, Social Entrepreneur Fellow
Catherine Lebel, Research Fellow
Francis Bizoza Bigirimana, Social Entrepreneur Fellow
Lea Moersdorf, Research Fellow
Introduction
Teach the World Foundation (TTWF) is a non-profit headquartered in Pakistan, that designs, develops and operates MicroSchool, In-School and SmartPhone programmes to enable young people to learn basic and functional literacies. Aside from broader corollary societal benefits of basic literacies, the programme enables students in Pakistan to gain access to the formal education system and consequently embark on their journey of lifelong learning and all the accompanying benefits that affords. TTWF has established evidence, through existing studies, that students achieve intended learning gains within the TTWF programmes and through social proof that students are able to transfer their basic literacies in their communities to functional literacies.
The MicroSchool programme is a ground-breaking initiative that enables the affordable setup and implementation of a school for 100 students led by a facilitator in a community within a week. The programme occupies existing available buildings. Learning by students is self directed and delivered through educational apps on tablets that are able to function in online and offline environments. Data about learning is available through an evolving set of app dashboards and is also collected through periodic EGRA, EGMA and EGRA-Urdu testing.
TTWF are now embarking on a first step in scaling their models through an initial partnership with the Sindh Government to establish 100 MicroSchools and 25 InSchool programmes against a schedule of operational and educational KPIs.
As TTWF progresses into the execution of the Sindh project, with an eye on the future, it now seeks to establish longer term KPIs that can demonstrate meaningful and robustly evidenced outcomes for learners and communities as well as meaningful, confidently defined and compelling operational and educational outcomes for future partners and stakeholders that will be key to the long term growth and impact of its initiatives.
TTWF has been paired with a team of four LEAP Fellows with experience in social entrepreneurship (2 Fellows) and research (2 Fellows) and whose experience in data analysis, scaling educational programmes and data collection will support the aims of TTWF in this project.
Organization’s role & strength
With an existing base of evidence and research, TTWF came into the 2023 LEAP programme with an eye on scaling the breadth of impact that can be measured in their programmes. Organisationally TTWF is well organised with dedicated and experienced team members focused upon the evaluative, educational, technological, operational, business development and strategic aspects of programme implementation, measurement and growth. There is an existing culture of seeking to understand the efficacy and evidence of programmes that is pervasive across the organisation and drives decision making. TTWF has begun to consider the operational realities as well as the practicalities of a broader spectrum of impact measurement and is seeking to implement an enhanced technology architecture to support this.
Need Summary
The first four weeks of the project with TTWF began with Fellows examining existing data collection practices, available data and its corresponding analysis from across prior programmes and assessment exercises that had been previously conducted. Fellows met with TTWF team members from strategy, education, evaluation, technology, operations and business development teams. This gave a clearer understanding of the objectives and vision behind an expanded impact measurement, considering the goal through the lens of different groups within TTWF who are tasked with meeting different immediate and longer term programme requirements (e.g. operational considerations for communities accepting a Microschool implementation), as well as existing stakeholder KPIs (e.g. Sindh Government KPIs for operational outcomes of the Microschool and In-School interventions).
It became apparent through this discovery period that there was a need to establish clear, consistent and robust benchmarks for the analysis of existing learning progress data within the programmes. TTWF uses EGRA, EGRA-Urdu and EGMA assessments, which whilst being well established across literacy development, do not come with a consistently comparable group of international benchmarks from peer initiatives, or that apply to the learning of Urdu. Therefore, it would be important to understand how to establish these during the LEAP project and to set scalable practices and benchmarks for the collection and analysis of future learning progress assessment going forward.
Many of the broader spectrum of impacts that are often cited from educational programmes quickly convert into economic measurements. However, as we developed an understanding of the specificities of the stakeholders that are and will be instrumental in the sustainable scaling of the TTWF programme in the long term, it became apparent that the societal impact must relate to expressed needs and therefore those perceived as compelling outcomes by learners, communities, governments, donors and future non-profit partners. This also serves the opportunity to focus the scope of outcomes to be considered, their potential for measurement and required interdependencies thereof.
Solution summary & next steps
At the conclusion of the discovery phase of the project the deliverables were concluded as follows:
How can TTWF scale its research roadmap to capture both specific and broad impacts in line with identified needs of current and future stakeholders, including learner competencies, community impact, and overall societal benefit? How and when can these be measured in a way that is scalable and alongside product development? How can other comparable programmes be used to better understand potential benefits of TTWF?
In line with #1, how can existing data on learning outcomes be used to make a compelling case for learning gains associated with TTWF programmes? How can other existing data (not learning outcomes) be used to help persuade communities and governments of the value of TTWF programmes?
What further data collection possibilities (concerning the learner and beyond) will be required to help respond and be compelling to the needs of the different stakeholders?
The outputs of these deliverables will seek to place TTWF in a position for its teams to proceed in existing programmes with a consistent and robust means of assessing and analysing learner outcomes as well as establishing a data collection and evidentiary framework for sustainably measuring and demonstrating the broader impacts of the TTWF programmes on society at large.
The benchmarking of learner outcome assessments, particularly within the context of an organisation assessing progress through EGRA and EGMA, will serve as a valuable reference for other programmes seeking to establish similar consistency and robustness in evaluating learning progress.
The broader societal impact framework begins from a consideration of societal impacts that can have relevance to a cross section of educational initiatives. The process through which its measurements can be focused to be implementable and relevant to an individual organisation’s programmes and stakeholder considerations is something that other organisations can implement as a transferable approach to demonstrating a broader spectrum of societal impact themselves.