Justify Your Alpha

Sun, 09 Feb 2025 00:00:00 +0000

This project responds to the widely discussed proposal by Benjamin et al. (2018, Nature Human Behaviour) to redefine statistical significance by lowering the conventional alpha threshold to p ≤ .005. Rather than adopting a single fixed threshold, the collaborating authors argue that researchers should transparently report and justify the alpha level chosen for a given study, treating the significance threshold as a design decision that depends on context rather than a universal constant.

Sau-Chin Chen is a co-author of this large multi-author methodology project, which brought together statisticians and methodologists across many institutions to formulate a shared position on error control in study design.

The project’s significance lies in shifting the debate on statistical significance away from a search for a single “correct” threshold and toward a broader emphasis on transparency and justification in research design choices, a position that has since informed wider discussions of statistical practice.

Status: Stable — completed and archived on OSF.

Investigating Object Orientation Effects Across 14 Languages

Wed, 08 Jan 2025 00:00:00 +0000

This project, PSA 002 of the Psychological Science Accelerator, examined whether people mentally simulate the orientation of objects while reading sentences that describe them, extending decades of research on mental simulation in language comprehension. Earlier work using the sentence-picture verification task had found robust match advantages for shape, color, and size, but evidence for orientation was limited and confined mostly to Western samples. The study combined a large cross-linguistic replication with a mental rotation task to relate individual differences in mental imagery to the strength of the orientation effect.

Sau-Chin Chen proposed and led this project as a large-scale, multi-laboratory collaboration coordinated through the Psychological Science Accelerator network, bringing together dozens of contributing labs and languages. The resulting registered report, covering 18 languages and nearly 4,000 participants, was published as Investigating object orientation effects across 18 languages in Current Psychology (2025).

By testing the same paradigm across speakers of many different languages, the project offered one of the most direct tests to date of whether object-orientation simulation generalizes beyond the languages in which it was first documented, and it produced a well-powered dataset connecting the phenomenon to individual differences in mental imagery ability.

Status: Stable — completed and archived on OSF.

Research | Sau-Chin Chen's Website

Justify Your Alpha

Investigating Object Orientation Effects Across 14 Languages