Our research group is dedicated to facilitating scientific discovery by automating the multiple stages of the empirical research process, including model discovery, experimental design, data collection, and documentation for open science. To achieve this goal, we are developing broadly accessible research tools, starting with an open-source framework for autonomous empirical research . Our group actively collaborates with researchers across different domains, with the shared objective of creating and sharing tools that uphold rigorous scientific standards of reproducibility and transparency, thereby promoting scientific progress.
Autonomous Empirical Research (AER) is a paradigm centered around a dynamic interplay between two artificial agents. The first agent, a theorist, relies on existing data to construct computational models by linking experimental conditions to dependent measures. The second agent, an experimentalist, designs follow-up experiments to refine and validate models generated by the theorist. Together, these
agents enable a closed-loop scientific discovery process. In addition, AER
interfaces with platforms for automated data collection, such as Prolific or
Amazon Mechanical Turk, which enable the efficient acquisition of behavioral
data from human participants.
Scientific models are often expressed in the form of interpretable equations or programs that map experimental parameters (independent variables) to observable outcomes (dependent variables). Constructing such models in mathematical or computational forms poses two main challenges: (1) determining the architecture of the model, which requires composing and sequencing specific operations, and (2) fitting parameters of the model to available data. To address these challenges, we rely on a combination of techniques derived from machine learning and automated scientific discovery, which together constitute a theorist in the AER paradigm. Our goal is to automate model discovery, and thereby accelerate the rate at which researchers can uncover meaningful insights into the phenomena they study.
The primary goal of an experimentalist is to design experiments that yield scientific merit. Our framework offers various strategies for identifying informative new data points (e.g., by searching for experiment conditions that existing scientific models fail to explain, or by looking for novel conditions altogether). To complement these strategies, we integrate our framework with SweetPea–a newly developed programming language for automating experimental design.
Collecting scientific data is often expensive and time-consuming. To address these constraints, we develop tools for automating the collection of experimental data (e.g., behavioral data obtained from crowd-sourced online experiments) and for augmenting empirical data that cannot be obtained in an automated manner (e.g., EEG data).
A primary obstacle to open science is its demanding nature—that is, the time and effort required to standardize and document each step in the empirical research process. We seek to overcome this obstacle by developing methods that automate documentation of (automated) experiments. These methods build on community-supported and machine-readable standards for expressing scientific models in their respective domains. They are also designed to automate the translation of models and experiments into the English language, as well as facilitate uploads thereof to the Open Science Framework (OSF).