Virtual Science Lab

Simulation-Based Assessment Environment

The Virtual Science Lab (VSL) was a research-driven project exploring how interactive simulations can be used to assess scientific inquiry skills. The environment allowed students to design and conduct experiments in a virtual lab using realistic tools and simulated phenomena, while capturing a detailed record of their actions and decisions.

Video demonstration of VSL presented at NCME 2018. Shows each major functional area of the lab: Lab Room, Lab Supply & Lab Cart, Workstation, Notepad, and Whiteboard for poster presentation.

By combining behavioral data (telemetry) with student-created artifacts like notes and data tables, the VSL produced a richer picture of student thinking than traditional assessments. Results showed that students could meaningfully engage in inquiry and that their behaviors could be used to distinguish levels of scientific reasoning—demonstrating the potential for more authentic, performance-based assessment at scale.

Context

The Virtual Science Lab (VSL) was developed within the NAEP SAIL (Survey Assessment Innovation Lab) program, an NCES initiative focused on advancing large-scale assessment through research-driven, technology-enabled approaches.

NAEP assessments are grounded in evidence-centered design (ECD), which provides a framework for linking observable student work to measurement claims. However, traditional assessment formats primarily rely on static artifacts (e.g., selected responses, written explanations, lab reports), which limit the ability to capture the process of scientific inquiry.

Within this context, VSL was designed to explore whether a simulation-based environment could generate a structured record of student behavior during experimentation, enabling the capture of process data—such as actions, decisions, and intermediate steps—that could be operationalized as evidence of scientific reasoning and inquiry skills.

Research Questions

Can a simulation-based virtual lab provide an authentic, open-ended scientific inquiry experience that enables students to plan and conduct investigations in ways comparable to real-world lab settings?
Can such an environment generate rich, structured process data capturing student behaviors during inquiry that provide meaningful evidence of scientific practices?
Can differences in student inquiry proficiency be identified and differentiated using behavioral and log data produced through interaction with the environment?

Diagram shows a valid path to conduct an investigation. The use had freedom to conduct their investigation in their own way and there is no one definitive order to interact with the VSL to conduct an investigation. Open-endedness was a critical innovation, compared to scripted cook-book experiences.

Scope

Led design and production of the NAEP Virtual Science Lab (VSL), a research-driven pilot under the NAEP SAIL program exploring simulation-based assessment.

The environment included multiple functional spaces (lab, supply cabinet, workbench, notepad, whiteboard) and multimodal student tools supporting experimentation, documentation, and communication.

Work included:

design of a semi-immersive, touch-first virtual lab environment
development of interactive mechanics and simulated scientific phenomena
definition of data logging schemas for process and behavioral analysis
coordination across research, engineering, vendor, and content teams
delivery across two development phases (prototype and extended implementation)

Screen capture of user interface allowing learner to precisely control the rate and volume of liquid poured from one container to another. Also shows the simulated phenomenon of diffusion of dissimilar liquids and a pH indicator reaction where an acid and base are mixed, where neutral pH is green.

Phase 1 of the VSL included 3 simulated phenomena, liquid pH indicator, precipitate formation, and effervescence. The visuals reflect early WebGL capabilities, but students were still able to identify the phenomena and engage productively with the simulation.

Approach / Skills:

Translated authentic scientific inquiry practices into structured interaction models, defining how student actions (e.g., manipulating variables, conducting experiments) map to system behavior and observable outcomes.
Led conceptual and technical design of a semi-immersive simulation environment, including real-time rendering of chemical interactions, visual/auditory feedback, and a custom pouring mechanic enabling precise, rate-controlled experimentation.
Designed workflows and UI/UX patterns for multi-space navigation and tool use, including experiment setup, data collection, graphing, and documentation within constrained platform environments.
Collaborated with science SMEs to define functional and visual requirements grounded in real-world phenomena, ensuring both authenticity and usability.
Co-designed data logging schemas and telemetry structures to capture fine-grained student behaviors (e.g., actions, sequences, variable manipulation), enabling reconstruction of inquiry processes for analysis and scoring.
Authored technical specifications, design documentation, and vendor RFP; led vendor selection and transition after initial implementation failure, ensuring successful delivery across both development phases.
Oversaw end-to-end production including SOW development, milestone reviews, usability testing, A/B trials, and QA, supporting iterative refinement and alignment between intended and implemented system behavior.

Results:

Authenticity of Inquiry Environment

Demonstrated that a simulation-based virtual lab can support authentic, open-ended scientific inquiry, with students able to design and conduct investigations using realistic tools, materials, and phenomena.
Observational and cognitive lab data showed that students could make sense of simulated phenomena, engage in experimental reasoning, and persist in investigation without step-by-step guidance.

Behavioral Data as Evidence of Inquiry

Established that the VSL could generate rich, structured process data (telemetry) capturing student actions, sequences, and interactions throughout an investigation.
Demonstrated that this process data, combined with student-generated artifacts (e.g., notepad entries), provides a more complete work sample than traditional lab reports alone.

Differentiation of Student Performance

Analysis of interaction data revealed clear behavioral distinctions between higher- and lower-performing students, including differences in experimental strategy, repetition of trials, and use of tools.
Findings showed that student performance could be meaningfully differentiated based on inquiry behaviors, supporting the feasibility of behavior-based scoring approaches.

Platform & Research Contribution

Delivered a functional prototype demonstrating that simulation, telemetry, and assessment design can be integrated within a large-scale digital assessment context.
Contributed to the research base through two peer-reviewed NCME conference papers, presenting both design and measurement findings.

Phase 1 of the Virtual Science Lab was completed as a successful proof‑of‑concept, demonstrating the feasibility and value of simulation‑based assessment for authentic scientific practices. A lot more research and development is required to bring a tool like this to the classroom or large-scale assessments, but it was a promising start.