title: "Pitching Hypothesis-Driven Data Investigations" pagetitle: "Pitching Investigations" description-meta: "Experiment checklist" description-title: "Experiment checklist" author: "Leon Yin" author-meta: "Leon Yin" date: "08-13-2023" bibliography: references.bib execute: enabled: false keywords: pitching stories, experiment planning twitter-card: title: "Pitching Hypothesis-Driven Data Investigations" description: "Answer these questions to bullet-proof your story" image: assets/inspect-element-logo.jpg open-graph: title: "Pitching Hypothesis-Driven Data Investigations" description: "Answer these questions to bullet-proof your story" locale: us_EN site-name: Inspect Element image: assets/inspect-element-logo.jpg href: checklist
This document asks essential questions to plan data experiments. Revisit these questions throughout your reporting, and use them to communicate your intentions and limitations with your editor. It will help determine if a story is worth pursuing by giving an estimation of time, complexity, and impact.
As a side benefit, these questions form the backbone of a methodology to get reviewed by experts, as well as the target of your investigation.
::: {.callout-note} Copy the checklist as text below, or as a public Google Doc. :::
1. What is the hypothesis of the story?
2. Who is being harmed and at what scale?
3. Who is causing the harm and what is the accountability angle?
4. What is the evidence (anecdotal or otherwise) you’ve gathered that leads you to think you have a viable hypothesis?
5. What is a viability study you can perform?
6. What data will you need to run an analysis? How will you gather the data?
7. How complicated is the data collection?
8. Will you need to filter out records from that data?
9. What are the limitations of the dataset(s) you are proposing to use? How will you test its accuracy?
10. Do you need to classify the data for your experiment? If so, please describe how you propose doing that. Are there outside classifications or experts you can lean on? What are the limitations of your classification method?
11. How will you analyze the data? What statistical tests will run? Please list any limitations to your proposed method and any alternatives.
12. What specific sentences will you be able to write based on your findings? What’s the lede? What’s the nutgraph?
13. Can you imagine the charts or other visualizations this data will produce?
This checklist is adapted from a checklist used by my editors Julia Angwin and Evelyn Larrubia at The Markup. Jeremy Singer-Vine provided feedback on the adapted list.