The fourth edition of the Holy Grail of Research Practice took place on May 23. We would like to thank Iris van Rooij, Angelika Stefan, Fred Hasselman, Max Primbs, Farnaz Mosannanzadeh, Pierre Souren and Julian Quandt, as well as all attendees (in person and online) for their valuable contributions and input!
In case you want to revisit the materials or couldn’t join the workshop: On this page you can find all recordings of the research flashes and some extra materials/references. A link to all the materials of the Data simulation workshop can be found as well.
Research Flashes
Research Flash 1 - The Multiverse of Madness (by Max Primbs)
When preparing and analysing data, researchers make a multitude of decisions. In this flash, I highlight the consequences of this variability and show how seemingly small decisions in the pre-processing and analysis pipeline can have large and unwanted consequences for the interpretation of your findings.
Research Flash 2 - Beyond power: Research design in the Bayesian age (by Angelika Stefan)
The Bayesian statistical framework offers many practical benefits to researchers and has attracted more and more attention in the past decade. However, the added flexibility of the Bayesian framework is still rarely embraced in psychological research designs. Here, I will outline several ways in which Bayesian thinking can be integrated into the research workflow, and how it can make our research designs more efficient and informative.
Research Flash 3 - If and how to do a systematic review or meta-analysis (by Farnaz Mosannenzadeh)
Why do a systematic review or meta-analysis? What are its advantages and disadvantages compared to a narrative review? Which steps should you take to perform a systematic review or a meta-analysis? Which guidelines, and assisting tools/software are out there to facilitate the process? In this flash, I will provide answers to these questions, using my own ongoing systematic review project as an example.
Research Flash 4 - Mediation analyses: Am I really looking at mediation? (by Pierre Souren)
Although the idea of a mediation seems relatively easy to grasp at first, it is not. Indeed, a mediation analysis is often wrongly chosen. In this flash, the approach for conducting mediation analyses will be discussed, and particularly to what extent they actually investigate the mediation you are interested in.
Recording research flash talk pending
Research Flash 5 - Tools for thinking: Overcoming obstacles to theory in psychological science (by Iris van Rooij)
Psychology has been reforming its own practices over the last decade. Most of that reform has focused on experiments and statistics while neglecting theory building. The right tools for theory building are known and available. But psychologists are not picking them up. Why not? It seems that first some obstacles need to be overcome.
Additional sources:
Link to open-source textbook by Mark Blokpoel & Iris van Rooij: “Theoretical modeling for cognitive science and psychology”
Link to blog by Iris van Rooij “Can you give an example of a good theory?”
Research Flash 6 - When N=1 and t=many: ‘model-free’ analysis of multivariate time series data (by Fred Hasselman)
I will demonstrate approaches to multivariate time series analysis based on the quantification of (patterns of) recurring values (recurrence networks). These analyses are essentially descriptive of the observed dynamics and could be used in a clinical context in which several psychological and/or physiological variables are monitored in real-time within an individual. I will also briefly highlight the method of surrogate testing with constrained realizations of the data.
Data simulation workshop (by Julian Quandt)
Julian Quandt discusses the theoretical background of power simulation (why we need it, when do we need it and how to use it). He also explains how to perform a power simulation for data structures like mixed-effects models in R. The powerpoint slides of the workshop, data simulation exercises and answers can be found on the github page of Julian Quandt