LEAP-ACE: Program Observation and Evaluation
Embedded instructional analysis for multilingual operational training environments.
Overview
The LEAP-ACE (Language Enabled Airman Program – Agile Combat Employment) program trains language-enabled airmen for multinational operational environments. The first cohort focused on Japan and Korea regional contexts, integrating consecutive interpretation, ACE doctrine, and trilateral crisis scenario exercises. This case study documents my work as the first Educational Support Staff (ESS) collaborator embedded in a LEAP-ACE cohort — conducting lesson observations, producing evaluation reports, and contributing findings that directly shaped the curriculum redesign for the next offering.
The Challenge
Observing a program where instruction and scholar discussion happen primarily in Japanese and Korean creates a fundamental access problem. Standard evaluation methods assume the observer can follow content, assess comprehension, and gauge participant engagement in real time. Without speaking the language of instruction, an observer is reduced to reading body language and counting hands raised, which can be useful but not the basis for a credible curriculum evaluation report.
Additionally, the program’s compressed timeline, rotating instructors, and mixed virtual and in-person formats meant evaluation had to be adaptive by design — capturing what was working, what wasn’t, and why, across sessions that didn’t always resemble each other.
The Workflow
For virtual sessions, I developed an AI-assisted multilingual observation workflow to process the auto-generated captions from the videoconferencing platform in real time. This gave me a working transcript I could reference during observation — not a perfect translation, but enough to follow the substance of instruction and scholar response, and to flag moments worth revisiting.
For in-person sessions, I recorded audio, took photos of boardwork, and uploaded them to NotebookLM along with all previous reports during each session. NotebookLM provided a real-time translated synthesis I could use to cross-check my real-time field notes and surface content I’d missed.
The result was a layered observation record: what I observed in the room, what the transcript confirmed, and where the two diverged. Leadership hadn’t seen this approach applied to LEAP-ACE before, and the real-time translation capability in particular came as a genuine surprise.
Why the Approach Worked
The most valuable findings were rarely visible in a single class period.
Patterns only emerged across days of observation: where scholars consistently lost cognitive bandwidth, which instructional techniques reliably restored engagement, and where operational complexity stopped being productive challenge and became avoidable friction.
The reporting process created a usable institutional memory for a fast-moving program that otherwise risked relying on anecdote and isolated impressions. It also provided leadership with a clearer view of how scholars actually experienced the transition from classroom language learning into operational interpretation under pressure.
Outcomes
The evaluation synthesis was delivered to program leadership at the end of the cohort and directly informed the curriculum redesign for the next LEAP-ACE offering. Findings on cognitive bandwidth, instructional pacing, and the transition from classroom language learning into operational interpretation were incorporated into the revised session structure. The framework also gave LEAP-ACE its first structured ESS evaluation record — and the AI-assisted multilingual observation workflow was flagged for use in future cohorts.
Tools & Methods
Custom observation protocols (designed for language-inaccessible instruction) • Chat GPT (AI-assisted real-time caption processing) • NotebookLM (cross-session multilingual synthesis) • Voice Notes • real-time field notes • evaluation report writing