Novel Measurement for Performance Improvement Challenge
Primary healthcare effective coverage monitor
What is the name of your solution?
Primary healthcare effective coverage monitor
Provide a one-line summary of your solution.
Integration, geocoding, analysis and visualization of administrative health records and population data for primary healthcare performance improvement
What specific problem are you solving?
Health Information systems, especially in low and middle-income countries, are fragmented, making performance evaluation of essential programs such as diabetes control very difficult.There is a need to develop techniques to integrate all relevant information to evaluate these program's performance.
Diabetes mellitus is one of the ten leading causes of death worldwide, increasing 70% since 2000. According to WHO, in 2019, this disease caused 71.8 deaths per 100 000 inhabitants. Furthermore, diabetes is a major cause of kidney disease, heart attack, stroke, blindness, and lower limb amputation, posing a heavy global burden on public health.
Diabetes mellitus prevalence has been rising quickly in low- and middle-income countries in the recent decades. Mexico is among the countries with the highest diabetes prevalence. In our country, 10.3% of people over 20 had diabetes diagnoses in 2018 (ENSANUT 2018). However, it is estimated that a significant number of cases are under-diagnosed. In 2019, 103,927 (82.4 per 100,000) people died of diabetes in Mexico, representing 14% of all deaths. Moreover, the Covid pandemic had a very high impact on the population living with diabetes, who experienced an excess mortality of more than 30%.
Micro and macrovascular complications of diabetes are consequences of poor metabolic control of people living with diabetes and reflect the quality of outpatient healthcare. These complications shorten life expectancy and represent significant financial challenges for those living with the disease and for the health systems, mainly in low- and middle-income countries. For instance, in 2017 in Mexico the total cost of hospitalizations attributable to diabetes was 500,334,908 US dollars, according to an analysis of the global, regional, and national burden of diabetes conducted by Lin et al. in 2020. Meanwhile, the total direct cost for diabetes complications care, such as retinopathy, nephropathy, cardiovascular disease and neuropathy, was estimated over 1.3 billion US dollars.
Diabetes and its complications may be avoided or delayed with proper and timely primary healthcare. However, the fragmentation of health systems and health information systems, may hinder the continuity of care. Also, the fragmentation by health programs within subsystems makes primary healthcare performance assessment even more challenging.
In this context, real-world data have increasingly been used to evaluate preventive programs at the population level. It is crucial to assess the effectiveness of interventions that are not properly evaluated through clinical trials, such as health service level interventions. Real-world data is defined as data relating to patient health status and the delivery of healthcare routinely collected from different sources including health and administrative records, surveys, and surveillance data, among others.
Data produced through different health information subsystems are key information pieces for characterizing diabetes patterns and healthcare services utilization and performance. Nevertheless, as various institutions create these data for different purposes, they must be optimized for decision-making. The amount and quality of data produced by the national health information systems have increased recently in Latin America. However, real-world evidence is not consistently used in healthcare performance assessment and decision-making because of limitations like the fragmentation mentioned above.
An integrated and interactive, population-based system for geocoded primary healthcare data, analysis, and visualization, could help decision makers to identify social, spatial, and demographic patterns in disease evolution and service utilization. Among other uses, it could help to assess primary healthcare performance by developing population-based quality indicators such as accessibility and effective coverage. This kind of system would facilitate the generation of useful, accessible, and timely information to support decision-making and optimize resource allocation and healthcare delivery at all health system levels. Locally, primary healthcare providers can identify catchment areas and sociodemographic and epidemiologic characteristics of the population they serve. It will also facilitate planning and evaluating the primary health system performance at the state and national levels.
What is your solution?
Our solution is a system for integrating, geocoding, analyzing and visualizing administrative health records and population data for primary healthcare performance assessment and improvement.
Through geospatial analysis, it offers different possibilities for estimating health systems coverage. Administrative data integration makes it possible to estimate the need, utilization, and quality of services, which are the necessary to assess effective coverage, plan health care infrastructure and evaluate primary care performance.
To build our system, we collect, clean up, standardize, and integrate different databases that would otherwise provide fragmented information because they are generated by various institutions for different purposes.
Our key information sources are:
A unique healthcare facility ID, in the case of Mexico the Catalog of Unique Health Establishments IDs (CLUES, acronym in Spanish). This data is collected and maintained by the MoH by a health facilities.
2. Data on equipment, human resources, and infrastructure for Health in the case of Mexico (SINERHIAS, acronym in Spanish). This data is collected and maintained by the MoH by health facility
3. Hospital Discharges Information. (SAEH, acronym in Spanish). This patient-level data is collected and maintained by the MoH
4. Service Provision Information Subsystems (SIS, acronym in Spanish). This health facility-level data is collected and maintained by the MoH.
5. General mortality vital statistics. The national statistics agency (INEGI) collects and manages this individual-level data.
6. Birth Information Subsystem (SINAC, acronym in Spanish). This individual-level data is collected and maintained by the MoH.
7. Population census, 2010 and 2020. This data is collected and maintained by the national statistics agency (INEGI in the case of Mexico) aggregated at different levels (block, census-tract, municipality, health jurisdiction, state and national.
8. National Geo-statistical Framework, 2010 and 2020 with Census cartography at different levels (block, census-tract, municipality, sanitary jurisdiction, state and national.
9. National Health and Nutrition Surveys (ENSANUT, acronym in Spanish), Information about health status and behavioral risks with different representativeness according to the year of the survey, ranging from state, region, and national.
The information available is being used to build a geographical information system (GIS) that integrates healthcare data from medical units, their installed capacity, their services, and their target population. This data is disaggregated at the individual, block or locality, medical unit, municipality, jurisdiction, and state levels.
Primary healthcare infrastructure is georeferenced using coordinates provided by the CLUES catalog and verified in Google Maps. We developed a derivative map was developed with population density for the whole country.
Three-step probabilistic gravitational models estimate the potential population for each medical unit.
The GIS allows us to:
· Define the catchment areas of the primary healthcare units based on their installed capacity, and for the hospitalization units based on the flow of patients based on hospital discharge information.
· Establish the spatial relationship between the catchment areas of the outpatient care units and hospitalization care.
· Analyze the patterns of utilization of primary health care services and their spatial relationship with the utilization of hospital services for the identification of empirical utilization networks for medical care services and the estimation of effective coverage.
· Build a control panel for interactive analysis and visualization of the geographical distribution of health care units, their installed capacity (infrastructure and resources), their productivity, and effective coverage in a comparative evaluation system (benchmarking).
· Estimate the effective coverage for primary healthcare programs.
Who does your solution serve, and in what ways will the solution impact their lives?
We will develop an interactive online Primary Healthcare Effective Coverage Monitor with information and dashboards for planning and evaluating the performance of primary healthcare facilities managed by MoH/IMSS-Bienestar across the entire country.
The Mexican health system comprises of a series of public and private institutions, and access to health care is subject to employment status. Employees in the formal sector and government workers and their families, about 50% of the population, receive health care at the social security institutions (insured population). The rest of the population, who work in the informal economy and include the self-employed and unemployed (uninsured population) access healthcare through medical facilities administered by the Ministry of Health (MoH) and IMSS-Bienestar. These are the most vulnerable people in Mexico. A small segment of the population has access to health care through private insurance schemes. Still, there is also an increasing use of private services with out-of-the-pocket payment at the point of service. Consequently, the health information system is fragmented by type of institution. Even within a single institution, the information system is fragmented by program in a silo-like structure.
The health subsystem for the uninsured population (MoH/IMSS-Bienestar) provides services to the most vulnerable people in Mexico. The information system requires innovative solutions to improve its capacity to provide useful information for monitoring and evaluating the health system.
Although healthcare providers make great efforts to generate data, besides providing health care, health information is not entirely useful for health institutions to make decisions on resource allocation, monitoring, and planning. The data has also not been helpful to primary health care and hospitalization facilities administrators and healthcare workers to assess sociodemographic characteristics or health needs of the population they serve. Neither to estimate population based performance assessments or to measure the impact of interventions to improve the quality of care on the health of the population they serve. The objective of the Primary Healthcare Effective Coverage Monitor is to put at the fingertips of primary healthcare physicians working at MoH/IMSS-Bienestar a tool to inform them about the sociodemographic and epidemiological characteristics of the population under their responsibility and to measure and evaluate their performance.
The most vulnerable people in Mexico are those in the informal economic sector, and they are the target population of the MoH/IMSS Bienestar health subsystems. The Primary Healthcare Effective Coverage Monitor is intended to assess key performance indicators to inform healthcare providers, facility managers, and decision-makers, at all levels in these health subsystems, as a way to improve the quality and efficiency of public healthcare for the uninsured population.
Our solution gives health system stakeholders, at all levels, the ability to access, analyze, visualize and interact with information related to the health and sociodemographic characteristics of the population they serve.
This information allows them to reach and compare key performance, population-based, health indicators to assess their performance at individual, medical unit, local, state, and national levels. This kind of system would facilitate the generation of practical, accessible, and timely information to support decision-making and optimize resource allocation and healthcare delivery for diabetes prevention and control programs and easily extend to other health programs.
Better knowledge about the needs, access, utilization patterns and services provided to prevent and control diabetes allows an approximation to effective coverage of primary healthcare services in benefit of more than 8 million people living with diabetes in Mexico.
Our solution would mainly benefit all those people directly under the responsibility of the Ministry of Health/ IMSS-Bienestar. Uninsured people living with diabetes have higher rates of mortality, complications, and disability. They would greatly benefit from a better planned, and improved performance of the primary health care services.
Moreover, our solution can potentially allow the assessment of many other primary health care programs besides diabetes prevention and control. The geographic information system and visualization tools we utilize are ready to be used as the basis for evaluating other priority health programs, such as prenatal control and adolescent pregnancy prevention.
How are you and your team well-positioned to deliver this solution?
The National Institute of Public Health (INSP, acronym in Spanish www.insp.mx) is an academic institution widely recognized, nationally and internationally, in the field of public health. In Latin America, it is a benchmark in research in different areas of public health and in higher level education and training.
Nested in the INSP, the Public Health Intelligence Unit (UISP) (https://uisp.insp.mx) is a multidisciplinary group with the mission to collect, curate, integrate and analyze health and health-related data in order to:
create a health data repository and data mart and make it available online to everyone
develop and publish a series of analytic dashboards with relevant and updated health information
The UISP’s primary objective is to generate evidence for health system evaluation and monitoring through public health surveillance, health information utilization research, and advanced statistical methods to identify the main health challenges and some of their social determinants.
The UISP has a wide network of collaborators at national and international levels. For instance, we collaborate with the Central American Network of Health Informatics at the international level. We have signed collaboration agreements with different universities in the region.
At the national level, the INSP/UISP has agreements for collaboration and information sharing with key institutions such as the General Directorate of Health Information, the Mexican Social Security Institute and the Mexican Council for Science and Technology.
Our team comprises a multidisciplinary group of researchers based at the INSP with extensive experience in health information systems, public health, epidemiology, demography, data science, business intelligence, and health economics. We have been working with health data and geographical information system projects since the year 2000. We have more than 20 years involved in research to improve health information systems in Mexico and the Latin American region. In 2010, we led a technical group to assess national health information systems in the Mesoamerica region. Through these projects, and our collaborations with key decision makers in the Mexican health system, our group is in deep knowledge of health information systems in Mexico and the Latin American region from data providers/users to high level decision makers, which has allowed us to understand their needs.
Our project has benefited from a seed funding of USD 40,000 from the Mexican Council of Science and Technology. This has been used to develop the system with a focus on reproductive health preliminarily. However, we are searching for additional funding to consolidate the system and extend its reach to diabetes prevention and control and other non-communicable diseases.
Which dimension of the Challenge does your solution most closely address?
Where our solution team is headquartered or located:
Cuernavaca, Mor., MéxicoOur solution's stage of development:
GrowthHow many people does your solution currently serve?
Our system informs healthcare providers, local clinic administrators, and decision makers, at all levels of the health system, mainly to those who work for the Ministry of Health. The Ministry of Health has a total of 15,699 medical units and a health staff of around 362,500 people including doctors, nurses, and other health professionals in contact with patients or in administrative tasks. Although for now, our system will focus on assessing performance of programs to prevent and control diabetes. According to ENSANUT, by 2018, in Mexico, there were 8.5 million adults diagnosed with diabetes. We believe that soon it may be extended to other primary healthcare programs, benefiting the population under the responsibility of the Ministry of Health, IMSS-Bienestar as a whole, roughly 50% of the Mexican population (65,604,407 people).
Why are you applying to Solve?
We hope this Challenge will help us overcome some financial and technical issues.
Consolidating a data science project is a time and labor-intensive undertaking. We need specialized human resources, constant training, and an adequate infrastructure to bring our system to the level that the Mexican health system demands. As researchers of a public institution in a middle-income country, external financing is a key piece for the development of our project.
Open source platforms are not suitable for the number of data points in our databases. For example, our mortality database (1990 - 2020) has more than 16 million records, and the hospital discharges database, more than 90 million records. The analysis of the catchment areas involves more than 2.6 million human settlements (blocks/locality centers) but it has to be expanded to the total number of health units to which every human settlement can access; this makes the number of data points in the analysis to go up to more than 56 million points. R, and Shiny can not handle this amount of data for online dashboards.
At this project stage, we plan to try alternative data management tools and business intelligence platforms, so that our data repositories and visualization dashboards are as friendly as possible for both developers and potential users. We are particularly interested in Microsoft Power BI as a versatile business analytics solution, but finding the best options for our system and our users will demand extra resources for premium licenses and training.
Who is the Team Lead for your solution?
Juan Eugenio Hernández-Avila, D.Sc.
What makes your solution innovative?
Assessing performance and long-term impact of primary health care programs is essential for decision-making. Usually, policymakers evaluate these programs with cross-sectional surveys, randomized controlled trials, and trials with comparison groups. These approaches are expensive and do not have the timing and geographic resolution needed to make accurate evaluations of the effectiveness and impact of these programs. Routine health information and other administrative records, considered real-world data sources, have been proven useful for health care assessment, but in Latin America, they are not consistently used in healthcare decision-making.
The amount of data produced in Mexico and other countries in the Latin American Region is increasing, but is not adequately exploited for improving the performance of the health system in benefit of the population. Mexico's health information system has been evaluated as fairly good in information products, but weak in diffusion and use of the information produced. This means that our health information system is focused on data collection, but has neglected the use that can be given to them. In this sense, our solution poses an innovative and affordable way of assessing effective coverage gaps in primary health care by leveraging the data already produced.
Effective coverage gathers population intervention characteristics and quality of care into a single metric and offers a direct method for assessing health system performance with a focus in universal coverage and reducing health inequities.
Appropriately collected, curated, standardized, and integrated data by our expert team are made available through attractive data visualizations that health providers and policymakers can interact with, regardless of their data management and statistics skills.
This way, our solution provides evidence for spatial analysis of health and disease trends, resources needed, services provision, medical unit management, and health system financing planning.
What are your impact goals for the next year and the next five years, and how will you achieve them?
For the next year we aim to develop a geospatial system for the automated analysis of the patterns of access and utilization of primary care services for diabetes prevention and control in units of the Ministry of Health and IMSS-Bienestar, at the medical unit, municipality, state and national level.
Our goal for the next five years is to consolidate an analysis and visualization system for primary care services performance assessment by estimating effective coverage of priority health programs. Evaluation of these services will be complemented by the analyses of avoidable hospitalizations susceptible to the quality of primary health care.
Currently, in Mexico, there is a health care reform and provision of services that were in responsibility of the MoH are being transferred, along with the corresponding healthcare infrastructure, to IMSS-Bienestar. Our systems are being used to help to plan the new integrated health system for the uninsured population in Mexico.
How are you measuring your progress toward your impact goals?
Some indicators we are already monitoring in relation Mexico’s primary health care performance are within targets 3.4, 3.7 and 3.8 from the 17 Sustainable development goals. These indicators provide us with information about health system needs and thus set our priorities. Indirectly, these same indicators will help us evaluate the performance of our own work.
Regarding target 3.4 (reduce by one third premature mortality from non-communicable diseases through prevention and treatment and promote mental health and well-being), our main indicator is the mortality rate attributed to diabetes. As our monitoring system grows, other indicators will be equally necessary, such as, cardiovascular, cancer and other chronic diseases mortality rates.
In relation to target 3.7 (ensure universal access to sexual and reproductive healthcare services) we are monitoring two indicators: the proportion of women aged 15-49 who have their family planning needs to be satisfied with modern methods and the adolescent birth rate.
With respect to target 3.8 (achieve universal health coverage) we will be monitoring effective coverage of some essential health services.
What is your theory of change?
Our theory of change is based on the better use of information to assess the health systems' performance, to improve it, and to positively impact the population's health.
We begin with the integration, curation, and standardization of databases with health and health related data, produced by different agencies in the country. Geocoding these data in different levels of spatial resolution will make it possible to derive catchment areas for primary health care units, based on their available resources and the distribution of the population by block. Using the 2010 and 2020 census microdata, it is possible to derive the sociodemographic characteristics of the user population, and to assign epidemiological characteristics based on survey data (ENSANUT), even if this is collected at larger representative units such as state or region. Then, given the sociodemographic and epidemiological characteristics of the population in the primary health care units, it is possible to estimate their health needs. Combining these data with health services provided by the primary healthcare facilities will make it possible to generate a measure of coverage. When combined with the quality of the services provided, we get an estimate of the effective coverage for each facility. As data for services provided is aggregated by facility, quality of services in primary health care can be evaluated using some indicators as a proxy; for example, the number or rate of hospitalization susceptible to the quality of primary health services (avoidable hospitalizations).
The estimations of effective coverage at the health facility level are used to develop a performance evaluation system. The system helps facility managers to prioritize actions to improve their service and the health of the people in their catchment area. State and national officials will be able to monitor the performance of primary health care services and the health of the population under their responsibility.
The impact of our project will be observed in three main areas. First, as integrated use of the information within dashboards helps health providers in their everyday practice, the perceived importance of generating quality data will grow to impact the overall performance of the health information system in, we hope, a virtuous cycle. A better monitored health system will lead to better performance. Finally, improved performance will positively impact population health, while reducing inequalities.
Describe the core technology that powers your solution.
Our solution uses data routinely collected by different agencies within the health system and other sectors. We integrate this data using available facility identification variables, as well as their geographical location with geographic information systems to estimate population-based indicators that could not be otherwise produced. Our solution makes use of unconventional and proxy data sources to inform primary health care performance improvement to provide improved measurement methods that are low cost, fit-for-purpose, shareable across information systems, and streamlined for data collectors. We leverage existing systems, networks, and workflows to streamline the collection and interpretation of data to support meaningful use of primary health care data and provide actionable, accountable, and accessible insights for health care providers, administrators, and/or funders that can be used to optimize the performance of primary health care.
Which of the following categories best describes your solution?
A new application of an existing technology
Please select the technologies currently used in your solution:
Which of the UN Sustainable Development Goals does your solution address?
In which countries do you currently operate?
In which countries will you be operating within the next year?
Who collects the primary health care data for your solution?
All the data used in this project is part of the routine health information system and is produced by healthcare providers and collected by the General Directorate of Health Information and shared with our group through a collaboration agreement.
Our key information sources are:
1. A unique healthcare facility ID, in the case of Mexico the Catalog of Unique Health Establishments ID’s (CLUES, acronym in Spanish). This data is collected and maintained by the MoH by a health facility.
2. Data on equipment, human resources, and infrastructure for health in the case of Mexico (SINERHIAS, acronym in Spanish). This data is collected and maintained by the MoH by health facility
3. Hospital Discharges Information. (SAEH, acronym in Spanish). This patient-level data is collected and maintained by the MoH
4. Service Provision Information Subsystems (SIS, acronym in Spanish). This data is collected and maintained by the MoH, aggregated by health facility
5. General mortality vital statistics, this data is collected and maintained by the national statistics agency (INEGI in the case of Mexico) at individual records.
6. Birth Information Subsystem (SINAC, acronym in Spanish). This individual-level data is collected and maintained by the MoH.
7. Population census, 2010 and 2020 this data is collected and maintained by the national statistics agency (INEGI in the case of Mexico) aggregated at different levels (block, census-tract, municipality, sanitary jurisdiction, state and national.
8. National Geo-statistical Framework, 2010 and 2020 Census cartography at different levels (block, census-tract, municipality, sanitary jurisdiction, state and national
9. National Health and Nutrition Surveys (ENSANUT, acronym in Spanish), Information about health status and behavioral risks with different representativeness according to the year of the survey, ranging from State, region and national.
What type of organization is your solution team?
Other, including part of a larger organization (please explain below)
How many people work on your solution team?
By now, we have eleven full-time researchers in our team, although we do not work full-time in this project. Depending on current projects and funding availability, we hire eventual collaborators for fees or scholarships.
How long have you been working on your solution?
Four years
What is your approach to incorporating diversity, equity, and inclusivity into your work?
Our team recognizes that health problems are multifactorial. So, we believe that the best way to face them is with a multidisciplinary and diverse work team. Our team includes men and women from different professional backgrounds, areas of experience and age groups. When we look for new collaborators, any applicant with the abilities that the job demands is welcome regardless of their age, gender, race, ethnic origin, place of residence and seniority in the field. We highly encourage pre and postgraduate students to collaborate with us and give them the opportunity to develop in the areas of their interest while they do so. We also acknowledge that, for equitable development opportunities, some people, for instance, students or women with young children, may have different needs regarding work schedules and flexibility, face-to-face vs. remote work, training, etc.
On the other hand, one of the central objectives of our project is to help decision makers identify the most disadvantaged population groups regarding their health status and the health care they receive. The Mexican health system is fragmented and access to healthcare and social services greatly depends on the employment situation. Our work focuses on the population without social security as a job benefit. In Mexico, this sector of the population is subject to great inequities. We believe that our efforts are valuable in identifying social, demographic, economic and geographic determinants of health in order to inform health decisions aiming at offering more equitable and inclusive services.
Solution Team
to Top
Our Organization
National Institute of Public Health of Mexico