Learner//Meets//Future: AI-Enabled Assessments Challenge
Chat2Learn AI Suite
Provide a one-line summary of your solution.
Chat2Learn AI Suite is an AI-enhanced conversation guide for teachers and parents that promotes and assesses early learners’ curiosity.
What type of organization is your solution team?
Nonprofit (may include universities)
What is the name of the organization that is affiliated with your solution?
University of Chicago
Film your elevator pitch.
What is your solution?
The Behavioral Insights and Parenting Lab at the University of Chicago (BIP Lab) developed Chat2Learn for parents in 2020 to engage parents and children in skill-building conversations with a messaging-based platform. We are now developing the Chat2Learn AI Suite, which includes Chat2Learn (for parents) and Chat2Learn for Classrooms (for teachers). Our new multi-modal program is AI integrated and is offered in English and Spanish. Parents and teachers can use the program separately, but the full benefits are realized when they are linked.
As with the current version for parents, Chat2Learn for Classrooms is an interactive program for teachers. It provides conversation prompts for engaging Pre-K and Kindergarten students in developmentally appropriate, open-ended inquiry and conversation. The program includes behaviorally informed features to help teachers build positive habits of conversation with young children that build learning and curiosity. We know that teachers are busy; research shows programs for teachers suffer from problems of fidelity and persistence. Thus Chat2Learn promotes the most natural and frictionless form of learning interaction.
Chat2Learn includes four interactive modalities: SMS, mobile app, WhatsApp, and a web-based platform. The user chooses their preferred modality and sets personalized goals and preferences. Consequently, Chat2Learn for Classrooms delivers conversation prompts on demand to teachers based on teacher input on topics of relevance to current classroom discussions e.g. animals, nature, adventure, community, etc. Prompts are open-ended, imaginative, and accompanied by storybook-like illustrations that aid conversation and inquiry. Teachers can use the prompts to lead group discussions with the whole classroom or individual discussions with a student. Teachers submit student responses to prompts in the program and the AI integration allows the program to respond with new probing, open-ended questions based on that input. Should the teacher wish to steer the conversation in new directions, the AI integration allows them to direct the program to generate a new prompt about a specific topic of interest. With the full Chat2Learn AI Suite, teachers and parents can share their conversational interactions with students with one other. However, Chat2Learn (for parents) and Chat2Learn for Classrooms (for teachers) can both be used as stand-alone programs to aid the adults in building a habit of having creative, curiosity-promoting conversations with children.
A key feature of Chat2Learn is its embedded assessment of children’s curiosity. This assessment is easy to use and can be administered to a child by a parent or a teacher. Parents and teachers can share student curiosity scores with one other and these synergies can amplify children’s learning. The BIP Lab has field tested and validated this measure of early learners’ curiosity in a national randomized control trial with over 700 low-income children on a web-based platform. This measure is integrated into the ChatLearn AI Suite.
How will your solution impact the lives of priority Pre-K-8 learners and their educators?
The Chat2Learn AI Suite is designed to narrow advantage-based skill gaps by improving children’s interactions with both their teachers and parents. This is important because early childhood classrooms that serve underprivileged students are likely to be more teacher-directed and less inquiry- and play-based than classrooms serving more privileged students. Underprivileged students also have less language- and conversation-rich home environments.
With funding from J-PAL at MIT, we evaluated Chat2Learn and tested our curiosity measure in a randomized control trial of over 700 pre-school age children from low-income families. With an eye toward scale, we designed and tested a web-based assessment that measures novelty preference and the urge to learn, two core dimensions of curiosity. We also surveyed parents on their beliefs about their children’s curiosity and found that parents vastly overestimate it. This represents an information friction that could be solved by implementing a curiosity assessment in schools. Feedback on surveys showed that parents and children alike enjoyed Chat2Learn and used it regularly. Virtually none of the 700 families in our study dropped out of the 6-month long program, a key validation of its acceptability among the families for whom it was designed.
The assessment of curiosity is practical and adaptable for integration into our AI-enhanced Chat2Learn program. The adult administrator needs only to follow a simple script; no formal training is required. The assessment presents children with ten trials using visual prompts only (thus mitigating any literacy challenges). Each trial contains two parts. First, the child is asked to choose to reveal either a novel or familiar picture – this is the novelty preference construct. Separately, the child is asked if they would like to learn a fact about the pictured subject matter that is unknown to both children and adults – this is the “urge to learn” construct.
In the randomized control trial, we measured children’s vocabulary skill with the Peabody Picture Vocabulary Test (PPVT) and we measured children’s curiosity with our web-based assessment. Over a 6-month period, we find a high degree of stability in children’s novelty preference and urge to learn (p<.01). Children’s PPVT scores are also significantly correlated with these two curiosity constructs (p<.01). We also find that children from more economically advantaged households score significantly higher on both constructs. This suggests that young children’s curiosity behaves similarly to other measures of skill for which gaps by family background appear early in life (Moullin et al., 2018).
In designing the AI-enhanced version of our curiosity measure, we will further test and validate the personalization of AI-generated assessment trials on a variety of themes of interest to children. Because these generated prompts will adapt to children’s interests and prior responses, we expect the assessment will be more engaging and inclusive of children’s varied baseline characteristics. Through an existing partnership with the University of Chicago Booth School of Business Mindworks Museum of Behavioral Science we will first field pilot experiments to inform the final design of our assessment before moving to integrate it with the Chat2Learn platform.
How are you and your team (if you have one) well-positioned to deliver this solution?
The BIP Lab has long-standing experience in community-informed program design and implementation. We have worked with over 500 community partners serving low-income families with young children in Chicago and across the US. Chat2Learn was informed by the systematic data collection and analysis we have conducted in partnership with these institutions and the 15,000+ families they serve who participated in our research programs over 10 years. These partner institutions and their families are involved in the co-creation and co-visioning of all programs we design. This is accomplished by the in-depth field work, interviews, and focus groups we routinely conduct before we embark on any new project. We also work closely with our funders to identify neighborhoods that are most in need of the programs we offer.
Currently, we are working closely with the McCormick Foundation to embed Chat2Learn in two of the most impoverished neighborhoods in Chicago – Little Village and Englewood. Within 4 months of beginning program enrollment, we enrolled 10% of eligible families in these two communities and expect to enroll many more. Many parents have shared that they love getting inspiring conversation starters from the program, are eager to see how their child responds to prompts, enjoy learning things they never knew about their child, and say the program feels joyful rather than like another thing they have to do for their child.
We have completed the design phase of Chat2Learn and as we move into the build and pilot testing phase, we will continue to seek the advice of our partner schools on the feasibility of program and assessment implementation in Pre-K-K classrooms. We are currently conducting a nation-wide survey of over 1500 public school educators to inform the adaptability of Chat2Learn for Classrooms in public schools. We are also collecting engagement data from current Chat2Learn parent users.
We understand that wide adoption of a program and/or alternative assessment may face challenges including resistance to change by educators and families. We have anticipated this potential roadblock and will synthesize the qualitative and quantitative data we are collecting to offer users practical recommendations for seamless adoption of Chat2Learn in Pre-K-K classrooms. We will also create a guide for teachers and parents on how to interpret and use the curiosity assessment results in tandem with other measures of children’s academic progress.
At the University of Chicago we partner on this project with the Polsky Center for Entrepreneurship and Innovation, the Rustandy Center for Social Sector Innovation, and the Center for Decision Research at the University of Chicago Booth School of Business.
The team behind MJ Ventures led by CEO Marley Rosario and Managing Partner of Design Sam Aguilar are our technical partners. MJ Ventures is a Chicago-based product development agency committed to building AI-based products for positive social impact. Marley and Sam and their team have built over 25 different award-winning products, workflows, and agents using OpenAI, Google or Anthropic based technologies for organizations.
Which dimension(s) of the challenge does your solution most closely address?
Which types of learners (and their educatiors) is your solution targeted to address?
What is your solution’s stage of development?
PrototypeIn what city, town, or region is your solution team headquartered?
Chicago, IL, USAIs your solution currently active (i.e. being piloted, reaching learners or educators) within the US?
Yes
In which US states do you currently operate?
Illinois
Who is the Team Lead for your solution?
Michelle Park Michelini
What makes your solution innovative?
Research shows that curiosity is a key driver of long-run academic achievement, attainment, and mobility. Our team is the first to develop a validated measure of curiosity suitable for early learners in disadvantaged circumstances. We are the only ones who have designed an easy-to-used measurement tool that leverages cutting-edge technology. Our solution will make a substantial impact on teachers’ and parents’ ability to track and support this key dimension of child skill development. This game-changing approach is well-positioned for adoption by schools and families across the country. We are also currently working with partner stakeholders in Australia and in Peru (with our Spanish-language version of the program) for global adoption of Chat2Learn.
The proposed project will make significant improvements on our current product. We will refine and improve the existing, validated curiosity measure, extend the assessment use case to Pre-K-K students in classrooms, and integrate the assessment with the Chat2Learn AI Suite, which will help us track children’s skill and curiosity over time, thus offering a unique developmental perspective on our tool. As the solution scales in schools, we will establish construct validity by assessing associations between the curiosity measure and indicators from school administrative data on academic achievement (i.e., math, literacy) and student behavior (i.e, student absences). We will also extend our understanding of parent and teacher beliefs about children’s curiosity and collect meaningful information from educators about adoption of the measure at scale. This project will establish a model, use case, and recommendations for successfully scaling a valid and robust measure of young children’s curiosity in classrooms across the country and around the world.
The potential impacts of this project are vast. The measure can be adopted (in tandem with or apart from the Chat2Learn AI Suite) as a key metric of children’s school readiness. It can be linked to administrative data that can track children’s achievement and behavior over the long term. It can catalyze educators and researchers alike to re-think and prioritize the systematic measurement of important but little-measured skills including curiosity. All told this can have major impacts on child development. We will also catalyze new interventions that boost children’s curiosity and enthusiasm for learning. We will collaborate with technology and design partners over time to integrate a successful measure of curiosity into other platforms to scale its use. Eventually we will adapt our measure for older children and adults. This frontier achievement will be the first of its kind to track curiosity across the life span.
Describe the core AI and other technology that powers your solution.
The Chat2Learn AI Suite employs a series of AI processing subdomains including Natural Language Processing (NLP), Cognitive Computing, and Speech Recognition. The backend architecture is multi-modal and accommodates communication in four different platforms with some user experience modifications: mobile app, SMS, WhatsApp, and a web-based experience.
Central to our technology is the use of Natural Language Processing (NLP) which refines and personalizes conversation prompts based on validated research around prompt structuring. Complementing the prompt and follow-up response to the prompt, our system employs generative AI for images, producing thematic visuals to engage children. GPT-4 (and Dall-E for image generation) will power our tool and each LLM layer in our mixture of experts structure will be fine-tuned with content our research team has created and data that we have captured as a part of our experiments and from current and past users of Chat2Learn.
We will integrate speech recognition technology to enable a "hands-free" mode, allowing users to interact verbally with the program. This feature will enhance accessibility and ease of use by allowing users to engage with children more organically and turn their focus to the child rather their device.
Our technology stack includes both proprietary algorithms and third-party models to ensure robust performance and scalability. Alongside these AI components, our solution is equipped with a user-friendly front end developed with modern web technologies, providing a clean and responsive user experience. The intuitive interface ensures that users can easily navigate the application, engage children in natural and seamless conversation, and assess children’s curiosity in a format that is equally natural for adults and children alike. Leveraging AI-generated content backed by years of research combined with an intuitive and engaging interface will enable the Chat2Learn AI Suite to bridge inquiry-based classroom learning with parent engagement at scale.
Our proposed ER model provides a clearer picture of how various technology components will be structured to power our solution.
How do you know that this technology works?
In our recently completed Randomized Control Trial our measure of curiosity successfully engaged young children on a web-based platform. The academic paper summarizing these results is currently in progress. We successfully administered the assessment to over 700 preschool and kindergarten aged children remotely over Zoom. Children were highly engaged in the assessment and in many instances asked to continue the assessment after it was over. An example of our assessment trials (called the Animal Game for participants) is available here: https://biplab.uchicago.edu/bip-curiosity-assessment-2023. The assessor script for the demonstration (lion + wildebeest) is as follows to provide a more concrete idea of the assessment protocol: [Hover cursor over lion silhouette.] Do you know what this animal might be? Okay! Behind this box is another animal. [Hover cursor over box.] Which animal do you want to see – the one behind the box or this one [silhouette]? You chose a [animal name]! Do you want me to tell you something no one else knows about [animal name]? [Read fact if yes]. If you had chosen this one [box or silhouette] instead, I would have clicked here, like this. But for the rest of the game, you can only pick one animal to see – this one or this one. Are you ready to play? Okay, let’s play!
In a related academic field experiment, Alan & Mumcu (2024) tested a similar question about curiosity. These authors tested a pedagogical intervention in Turkish primary classrooms that aimed to boost children’s curiosity, which they conceptualized as novelty preference combined with an “urge to learn.” This intervention successfully boosted children’s urge to learn and this translated into better academic outcomes in multiple school subjects. This provides further proof that children’s curiosity is malleable and that boosting it has a causal impact on school achievement. However, these authors’ measure of curiosity does not extend beyond their unique research study. It also requires trained research assistants to go to classrooms to administer a series of standardized high-touch tasks to students in person. Our tool improves on both of these shortcomings.
What is your approach to ensuring equity and combating bias in your implementation of AI?
We developed Chat2Learn to foster learning among low-income children to narrow skills gaps between them and their advantaged peers. It is important to note why we embarked on this project and how it was designed from its inception to ensure equity and combat bias.
Children from low-income families enter kindergarten with weaker scores on both traditional academic skills like math and literacy as well and unconstrained or “non-cognitive” skills like curiosity. There have been successful programs to boost traditional academic skills in early childhood, but positive effects of even high quality programs fade out through the school years. It is non-cognitive skills, including curiosity, that shape social mobility over the life course for disadvantaged children, and curiosity specifically can be developed with intervention (Alan & Mumcu, 2024, Garcia & Heckman, 2023; Heckman et al., 2013; Lowenstein, 1994). Thus, we sought to experimentally boost and measure young children’s curiosity in a low-income sample for our recently completed randomized control trial.
Our experimental sample of 727 parents was 21% Hispanic, 32% Black, 39% White, 52% employed, and 52% with a college degree, with a median household income of $43,000. As a result, we have already demonstrated the feasibility and validity of our program content and measure of curiosity on a racially and economically diverse sample.
Chat2Learn contains the following 4 LLMs in a mixture of experts model: 1) inappropriate response LLM, 2) image generation prompt LLM, 3) question LLM, and 4) general app question LLM. We will fine-tune these LLMs by feeding them the content and parent feedback generated in our existing SMS-based Chat2Learn program offered in English and Spanish, which has enrolled 2,500 users since January and is expected to enroll 10,000 users by 2025. Nearly all current users are low-income families and we are training the algorithms on models based on community-informed design and feedback. We are continuously incorporating feedback into program revisions to include more topics and approaches for prompts (image generation and question LLMs) that align with the needs of the communities we serve. We will test the AI-integrations with the same and other similar communities with an eye toward combating algorithmic bias.
Our team has deep expertise with implementing programs in low-income communities in close partnership with institutions that serve families with young children. In fall 2020, in response to pandemic-related school closures and disruptions, we partnered with the Illinois State Board of Education (ISBE) to develop and deliver Chat2Learn by text messaging to parents in Illinois. Over the 9 months, the BIP Lab enrolled 2,000 low-income families with preschoolers in ISBE, Head Start, and Chicago Public Schools (CPS) networks and through local housing authorities. We improved upon and re-launched Chat2Learn in 2023-2024 with philanthropic support and are targeting the most economically depressed communities in Chicagoland. We expect to build upon the networks we have established to scale the Chat2Learn AI Suite widely in partnership with schools and districts across the country, using the program deployment strategies we have developed over years of community-informed work.
How many people work on your solution team?
The BIP Lab employs 9 full-time staff (including 2 faculty directors), 2 PhD students, and 30+ graduate and undergraduate student research assistants. MJ Ventures employs 5 full-time staff including product managers, designers, and developers.
How long have you been working on your solution?
4 years. Chat2Learn for parents: 2020. RCT: 2023-24. Chat2Learn AI Suite: launched March 2024.
If your solution has a website or an app, provide the links here:
https://biplab.uchicago.edu/chat2learn/
What is your plan for being pilot ready (if not already) within the next year, and what evidence can you provide that you are on track to meet your goals?
This is a full project timeline for the testing and development of the new program suite.
June - August 2024: Pilot test the next version of our curiosity measure through Booth School of Business Mindworks Museum partnership.
September – December 2024: Complete Chat2Learn AI agent prompt engineering and integrate assessment, API creation and modality configurations, full scope QA, and app/web deployment.
January – June 2025: Field test program and curiosity assessment with Pre-K-K teachers and n=300 students in classrooms in low-income communities (leveraging our network of school and district partners). Collect (via administrative or primary data collection) standardized math and literacy outcomes and measure correlations with curiosity constructs. Survey parents and teachers to get feedback about the program and assessment, barriers to usage, and how we might overcome them.
July – December 2025: Fine tune program and assessment. Write up and disseminate pilot findings and program usage guidelines. Establish district partnerships across the US to start scaling Chat2Learn AI Suite.
What are your plans to ensure your solution is available, accessible, and affordable to priority learners at scale?
We are offering four modalities of program delivery to ensure availability, accessibility, and affordability to a broad base of priority learners. We have a successful model of outreach for enrolling our target demographic of families in Illinois that we will replicate nationwide. We will build partnerships with state and local departments of education across the country as we have with the Illinois State Board of Education to obtain government-sponsored contracts. These state sponsorships will enable our subsidized early childhood center network and Title I school district partners to enroll low-income families in the program at a minimal cost or for free primarily via an opt-out model.
We will also offer the Chat2Learn AI Suite to schools serving higher income families for a higher cost and offer Chat2Learn (for parents) direct to consumers as a paid app on the Apple app store and Google play which will further enable us to subsidize the program cost for priority populations.
Why are you applying to the Learner//Meets//Future Challenge?
We hope Solve and the Gates Foundation can help us promote the Chat2Learn AI Suite to early childhood education stakeholders across the country, including practitioners and researchers, increasing the sense of urgency to promote and measure young children’s curiosity.
An important goal for this program and assessment is the creation of a novel, practical, and versatile learning engineering tool for researchers and practitioners alike. Researchers will be interested in using data on teacher and parent interactions with the program and on children’s curiosity outcomes via the assessment integration. Teachers and parents will be interested in tracking the development of children’s curiosity and discovering conversation topics of interest to children in the process. Collecting data from diverse populations allows the program LLMs to generate increasingly custom and novel conversation prompts for users and allows our team to fine tune the curiosity measure and generate a well-normed assessment pool over time.
In which of the following areas do you most need partners or support?
Solution Team
to Top
What is the name of your solution?
Chat2Learn AI Suite