Assignments¶

Statistics 240, spring 2023¶

There will be approximately five problem sets, two coding projects involving open-source code contributions, and a longer term project involving data analysis.

The longer project involves re-analyzing data in a published paper that inappropriately used parametric methods or used inappropriate nonparametric methods, using appropriate nonparametric methods instead. Please start looking for a paper that interests you right away.

Schedule¶

All assignments are due at 11:59pm Pacific Time unless indicated otherwise.

1. Weeks 1-3: Review and inference about binary populations

   • If you have never logged into https://github.berkeley.edu, please do so asap so I can add you to the class organization.
   • If you do not already have an ORCID, sign up for one at https://orcid.org/.
   • review:
     • mathematical foundations
     • mathematical inequalities
     • discrete probability
   • Binomial and hypergeometric tests and confidence intervals
     • testing
     • duality between tests and confidence sets
     • confidence sets, tailoring the rejection region, bootstrap percentile CIs
     • binomial and hypergeometric simulations; comparison with the normal approximation
   • Randomized, controlled experiments
     • Fisher's exact test
     • Causal inference, Neyman's potential outcomes model, experiments with binary outcomes
     • Li, X. and P. Ding, 2016. Exact confidence intervals for the average causal effect on a binary outcome, Statistics in Medicine, 35, 6, 957-960. 10.1002/sim.6764
     • Aronow, P.M., H. Chang, and P. Lopatto, 2022?. Fast computation of exact confidence intervals for randomized experiments with binary outcomes. https://lopat.to/permutation.pdf
   • Inference about binary populations from stratified samples: Wright's method, Wendell & Schmee's method, and union-intersection tests using greedy optimization
   • The SPRT for Bernoulli trials

2. Weeks 4-6: Permutation tests

   • introduction to permutation tests
   • PRNGs and simulations
   • simulation and computational permutation tests
   • classical permutation tests based on ranks
   • tests of randomness and independence
   • Stark, P.B., and K. Ottoboni, 2018. Random Sampling: Practice Makes Imperfect.
   • Ivanova, A., S. Lederman, P.B. Stark, G. Sullivan, and B. Vaughn, 2022. Randomization tests in clinical trials with multiple imputation for handling missing data, Journal of Biopharmaceutical Statistics, 32, 441-449. https://doi.org/10.1080/10543406.2022.2080695
   • permutation tests based on the distance between the empirical and the null
   • combining tests
   • E-values

3. Weeks 7-8: Supermartingale-based tests

   • Reading:
     • martingales
     • nonnegative supermartingales and Ville's inequality
     • examples: additive, multiplicative, likelihood ratios, prior-posterior ratio, betting
     • the SPRT
   • Inference from stratified samples
     • combining tests using $E$-values: multiplication and averaging
     • stratified inference using union-intersection tests
     • product supermartingales: ALPHA and Sweeter than SUITE

4. Weeks 9-10: Inference about bounded populations. - Kaplan, H., 1987. A Method of One-Sided Nonparametric Inference for the Mean of a Nonnegative Population, The Amer. Statistician, 41, 157-158. https://www.tandfonline.com/doi/abs/10.1080/00031305.1987.10475470?journalCode=utas20 - Stark, P.B., 2023. ALPHA: Audit that Learns from Previously Hand-Audited ballots, Ann. Appl. Stat., https://www.e-publications.org/ims/submission/AOAS/user/submissionFile/54812?confirm=3a9dc0d4 - Vovk, V., and R. Wang, 2021. $E$-values: Calibration, combination, and applications, http://alrw.net/e/02.pdf - Waudby-Smith, I. and A. Ramdas, 2022. Estimating means of bounded random variables by betting, https://arxiv.org/abs/2010.09686

5. Weeks 11-13: Betting and $E$-values, more nonnegative martingales. Combining $E$-values. Multiple testing. The False Discovery Rate. Controlling FWER and FDR using $E$-values

6. Weeks 14-15: Conformal prediction