Measurement Strategy & Capacity Building
Theories of change, measurement frameworks, and coaching that build lasting internal capacity, so data work outlives the engagement.
From Evaluation Overwhelm to Organizational Clarity for a Data-Focused Organization
The Challenge
Futures and Options runs career development programs across all five NYC boroughs that help students build professional skills, gain on-the-job experience, and establish a strong network of career professionals, all with the goal of increasing opportunities for young people. The team knew the programs worked, but they could not easily prove it in a language that funders, board members, and district partners recognized and trusted.
Their evaluation infrastructure was fragmented. Data lived in different systems with no story connecting them, and they lacked a current shared theory of change connecting activities to outcomes. Staff felt overwhelmed by evaluation expectations and unsure where to start.
As Executive Director Carolyn Silver described it: the organization was working to update their Theory of Change and overhaul their evaluation framework, but "we were generally apprehensive about taking on such a big project and wanting to ensure that staff voices were heard."
The Solution
TRAIL's approach started from the inside out, beginning with what the organization was actually trying to achieve and building the measurement infrastructure to match.
Phase 1: Building Shared Understanding. Through facilitated workshops with their staff, the team surfaced, debated, and aligned around a shared theory of change. Staff began seeing connections they had not articulated before: how confidence and skill-building were the mechanisms leading to career opportunities, and how their activities connected to specific measurable outcomes. The deliverables included a unified Theory of Change, logic models for each flagship program, and a comprehensive Measurement Framework defining what to measure, why, and how. Data sources were matched to the organization's actual capacity.
Phase 2: Creating Sustainable Tools. TRAIL worked with the team to build measurement tools where every survey question was aligned to a specific construct in the logic models, and data collection was built into normal program operations. TRAIL designed validated assessment instruments for the Career Essentials and Internship programs, developed data collection guidance, created analysis templates, and built funder-ready reporting formats.
Phase 3: Supporting Implementation. In Summer 2025, Futures and Options ran their Internship program using the new tools. TRAIL supported the team in analyzing the data, creating the foundation for ongoing continuous improvement cycles.
The Results
The team independently held their first-ever data review meetings using the data they collected. This was the moment evaluation transformed from accountability to learning. The data did not just confirm what was working; it surfaced insights the team had not been able to see before.
Rather than taking a single look at their data report, the team identified specific program improvements: more intensive networking workshops, clearer job descriptions from internship sites, and stronger support for students navigating workplace culture and belonging.
The organization now has credible student outcome data to share with funders and potential partners, and to use for its internal learning cycles.
"Anne-Marie and her team helped us think critically about our Theory of Change and develop Logic Models to capture our activities and outcomes. She helped us place our work within the larger landscape and used available research to both inform and confirm our outcomes and impact."
— Carolyn Silver, Executive Director, Futures and Options
Why This Matters
Futures and Options' transformation illustrates what happens when an organization invests in building evaluation capacity from the ground up. With staff voices centered, tools designed for their reality, and data that informs decisions, organizations do not just create better reports for funders; they build the foundation to be a true learning organization that understands its impact, makes smarter program decisions, and can articulate why its work matters with confidence and evidence. A long-term alumni outcomes study is now in development for 2026, which would have been impossible without the foundation built in 2024.
Why We Helped a Client Slow Down Before Building a Dashboard
The Challenge
A small youth-focused nonprofit came to TRAIL with what seemed like a straightforward request: they wanted a dashboard built. Their team was spending significant time pulling reports from multiple data systems, a process that was time-consuming, frustrating, and did not produce the insights they needed to improve their programs or tell their impact story.
A dashboard seemed like the obvious solution. But in early conversations, it became clear that they faced a more fundamental challenge. They had not yet defined what success looked like across their programs, what specific metrics mattered most, or even how to define key parameters like which time periods to examine, which student populations to track, or what counted as "enrollment" versus "completion."
Without these foundational decisions, building a dashboard would have been premature and likely would have produced a tool that looked impressive but did not actually answer the questions the team needed answered.
The Solution
Rather than proceeding directly to dashboard construction, TRAIL guided the organization through a strategic process to define its measurement framework before any technical build began. This involved mapping what success looks like across their programs, identifying the specific metrics that would demonstrate progress toward those outcomes, and making clear decisions about parameters that might seem technical but are actually strategic: What time frames matter? Which student populations should be tracked? How do you define a "completed" program participant versus someone who enrolled but did not finish?
TRAIL acted as a strategic thinking partner, asking the questions that needed to be answered before a single data visualization was designed. The process created not just a measurement framework, but shared organizational clarity about what the organization is measuring and why.
The Results
Dashboard build set up for success: When the organization begins to build their dashboard, they will start with a clear, well-defined measurement framework rather than guessing at the best data points to showcase.
Shared organizational language: The process created alignment across the team about what they are measuring and why, a prerequisite for any data tool to be useful.
Stronger foundation for future evaluation: The measurement framework positions the organization to not just display data but to use it for genuine program improvement and credible impact storytelling over time.
Costly rebuild avoided: By investing in strategic clarity first, the organization avoided a common and expensive pitfall: building a dashboard that does not serve the organization's needs and then having to start over.
Why This Matters
Organizations frequently approach data challenges as technology problems when they are actually strategy problems. The instinct to "just build a dashboard" is understandable. When your team is drowning in spreadsheets and manual reporting, a sleek new tool feels like the lifeline you need. But without clarity on what you need to learn and how you will measure it, even the best-designed tool will fall short. This organization's experience demonstrates the value of a partner who will ask the hard questions first and help an organization build the strategic foundation that makes technical tools actually work.
Designing a Cohesive Measurement & Evaluation Framework for a National Tutoring Organization Scaling with Evidence
The Challenge
A national tutoring program has established a strong evidence base for its high-impact tutoring model through sustained investment in rigorous research, including multiple randomized controlled trials. As they scale into new districts and navigate increasingly complex funding and implementation environments, the question has shifted from whether the model works to how the organization measures, monitors, and communicates its impact across a growing national footprint.
Like many organizations with strong research programs, their data strategy had been shaped by the demands of individual RCTs, district reporting requirements, and funder requests rather than by a unified organizational strategy. This created a disconnect between the data collected for research and the data that could be used to drive operational decision-making. The board needed different information than program staff. District partners had different questions than funders. And the data systems, platforms, and tools that had accumulated across the organization had no unified architecture connecting them.
The organization needed a partner who could take stock of its entire data ecosystem, from theory of change through data systems and reporting, and build a measurement infrastructure that was practical, sustainable, and designed to bring greater coherence to how it defines, measures, and communicates impact both internally and externally.
The Solution
TRAIL structured the engagement in three phases over twelve weeks, each building on the last.
Phase 1: Discovery and Theory of Change Refinement. TRAIL conducted structured interviews and working sessions with the organization's leadership and program staff to understand how impact is currently defined, measured, and communicated across the organization. Working collaboratively, the team refined the organizational theory of change and developed clear logic models for core program offerings, accounting for implementation variability across sites and making explicit the assumptions required for success.
Phase 2: M&E Framework Design and KPI Development. Building on the refined theory of change, TRAIL identified the organization's highest-priority outcomes for students, districts, and partners and defined the KPIs that best reflect those outcomes. This included distinguishing between formative indicators (used for real-time improvement) and summative indicators (used for impact reporting), and designing an audience-specific measurement architecture that serves internal teams, district partners, funders, and the board at the right grain size and cadence without creating duplicative reporting workflows.
Phase 3: Data Infrastructure and Systems Assessment. TRAIL conducted a comprehensive inventory of all data systems, platforms, and tools in use across the organization, from student tracking systems and district data integrations to internal program management tools, tutor tracking platforms, assessment instruments, and survey tools. The team traced the full data lifecycle to identify where the pipeline works well and where it breaks down, evaluated data governance practices, and produced a roadmap focused on high-priority steps to bring the measurement approach to life.
The Emerging Results
TRAIL delivered a refined theory of change, logic models, a comprehensive M&E framework, a KPI library with audience-specific reporting architecture, a complete data systems landscape map, and a prioritized implementation roadmap to support the organization's shift in reporting strategy. The framework complements their existing RCT research program, capturing the formative, operational, and implementation data that help the organization improve in real time, so it can tell the full story of its impact.
Why This Matters
Organizations with strong research programs often discover that their research infrastructure and their operational measurement infrastructure have grown in parallel rather than in concert. The data collected for RCTs serves a different purpose than the data needed for day-to-day program management, funder reporting, or board conversations. This engagement illustrates how an organization can build a unified measurement strategy that connects its theory of change to its data systems to its reporting, so that every audience receives the information they need, and the organization's impact story is coherent across contexts.
Coaching District Teams Through Disciplined Inquiry to Address an Urgent Equity Challenge
The Challenge
A large, urban school district's leadership wanted to enhance long-term stability and well-being outcomes for youth in foster care and temporary housing through data-driven, system-informed practices. Three network teams needed to investigate a pressing equity challenge: how to better serve students in temporary housing. These teams cared deeply about the issue and were eager to act, but they lacked a structured process to understand the problem before jumping to solutions.
This is a common pattern in education and social services. Teams identify an urgent need, feel the pressure to respond quickly, and move to action without first building a clear, evidence-based understanding of what is actually happening and why. The risk is well-intentioned interventions that do not address the root causes of the problem.
The Solution
TRAIL is coaching three network teams through the System Wise framework, a structured, disciplined inquiry process developed at Harvard Graduate School of Education. The framework guides teams through a sequence of steps that begins with examining data to understand the current reality before designing or implementing any interventions.
TRAIL's role is facilitation and coaching: rather than telling teams what to do, TRAIL guides them through the inquiry process: helping them identify the right data to examine, supporting them in analyzing what the data reveals, and coaching them to develop responses grounded in evidence rather than assumptions. The engagement is structured as targeted coaching support grounded in building the teams' capacity to use this inquiry approach independently in the future.
The Emerging Results
Teams shifted how they approach their work throughout this process:
From reacting to investigating: Teams are learning to resist the pull of immediate action and invest in understanding the problem through data before designing responses.
From assumptions to evidence: Conversations about students in temporary housing have shifted from "we think the problem is…" to "the data shows…", which is a fundamental change in how teams frame challenges.
From dependency to capacity: The coaching model is designed so that teams internalize the inquiry process, building the skills to lead similar investigations independently after the engagement ends.
Why This Matters
This engagement demonstrates the kind of targeted, short-term capacity-building support that can make a significant difference for teams facing specific challenges. Foundation funders increasingly recognize that their grantees need more than just funding; they need partners who can help build the internal skills and processes that make data-informed work sustainable. This engagement shows how focused coaching support can equip teams with an inquiry approach they will carry forward long after the engagement ends.