Stat-ception II: How to fix statistics in psychology

Stat-ception Part II

I'm a star!

OK, my public speaking skills may not exactly have made me a star (yet!), but I AM on YouTube! I've included a link to my recent (Feb 2017) Cognition Forum presentations, as well as my current thinking about easily--and immediately implementable--solutions to ameliorate those weaknesses.

The first video goes into depth about the issues; the second describes my proposed solutions to those problems.

https://www.youtube.com/playlist?list=PLvPJKAgYsyoKcGOCKEYT2GyzK0yLVXvzN

For your viewing pleasure, I've also embedded the videos here:

 

Any feedback or advice is welcome!

I've also made the slideshows available on Google Drive. Here's the link to the first slideshow, so you can follow along: https://drive.google.com/file/d/0B4ZtXTwxIPrjTktiMGdoQ3JBSHM/view. And here's the link to the slideshow for the second video as well: https://drive.google.com/file/d/0B4ZtXTwxIPrjalZxdFJfUWNKTVU/view?usp=sharing

A draft of my manuscript on the topic (intended for eventual publication) is freely available for download at https://osf.io/preprints/psyarxiv/hp53k/. Since I'm an advocate of the open science movement, it's only right that I make my own work publicly available--hence why I uploaded these videos (and my manuscript) to public repositories.

You may not trust my own take on these issues, in which case I commend you for your skepticism! In the videos, I made numerous references to Ziliak & McCloskey (2009), Gigerenzer (2004), and Open Science Collaboration (2015)--all are worth reading, for anyone who cares about scientific integrity and the research process. All three works were highly influential in my thinking on this topic, though I cited a variety of other papers as well in my aforementioned manuscript.

You may disagree with my recommendations in the second video, and if so, that's okay! How to address the limitations of NHST and fix science is absolutely a discussion worth having; I advance my own ideas in the spirit of jump-starting such a discussion.

So, please put your thoughts in the comments, and share my work with colleagues who may be interested in the topic!

The Simple Life: Graphs to aid understanding of the results

In my first publication, my advisor and I created graphs according to the guidelines listed on the Judgment and Decision Making journal's webpage. My advisor and I both maintain a general preference to see results presented visually, in the form of easily-understood, properly-labeled graphs. However, in correspondence with JDM's editor, he felt that the results were simple enough to understand, and the graphs therefore weren't really necessary.

I think the editor's idea was that:
a) not including the graphs would save space, and
b) including the graphs would probably involve a bunch of difficult/time-consuming formatting. I certainly don't blame the editor for advising us to exclude the graphs; I understand perfectly where he's coming from!

So, to save time, space, and effort, our graphs were never published. Until now.

The graph for Experiment 1, which fits best with the information presented near the top of p. 305, presents the proportion of participants' decisions that were consistent with pre-exposure (bar on left) and with the recognition heuristic (as determined by participants' self-reported recognition; bar on right). Error bars represent 95% confidence intervals.



The graph for Experiment 2, below, illustrates the data presented on p. 307-308. The dots represent the mean proportion of choices that were consistent with the recognition heuristic, for participants in each of the three training conditions. Again, error bars indicate 95% confidence intervals, which I recommend reporting as part of a more complete picture of your data.

Here's a link to the full-size .jpg image for the first graph, and here's the link for the second graph.