Inside Our ProgramProgram Events
Events
-
Jul31
EVENT DETAILS
Data analysis processes are regularly employed to inform high-stakes decisions. However, typical workflows for implementing data analysis overlook the considerable subjectivity that is baked into the analysis process and how that might impact the results. This subjectivity reflects ontological uncertainty---implicit, qualitative uncertainty regarding how data should be analysed and modelled. While statisticians have proposed techniques such as multiverse analysis to surface implicit ontological uncertainty, analysts currently do not possess the tools to implement, evaluate and communicate the results of such analyses. This dissertation bridges that gap by providing a pipeline for systematically reasoning about ontological uncertainty that is implicit in data analysis. In the first study, I developed and evaluated multiverse, an R library, which lowers the barrier to implementing multiverse analysis. The library provides flexible and expressive syntax to allow analysts to declare any alternative data analysis step through local changes in code. The library is designed to integrate into both computational notebook and scripting data analysis workflows, and optimises execution by pruning redundant computations. . I evaluate how the multiverse R library supports programming multiverse analyses using (a) principles of cognitive ergonomics, and (b) case studies based on semi-structured interviews with researchers who have successfully implemented an end-to-end analysis using multiverse. I identified design trade-offs (e.g., increased flexibility versus learnability), and suggested future directions for supporting analysts in adopting multiverse analyses (e.g., how to evaluate a multiverse analysis?). In the second study, I address the issues of evaluation by first identifying principles for validating the composition of, and interpreting the uncertainty in, the results of a multiverse analysis. I designed Milliways, a novel interactive visualisation system, to support the principled validation and interpretation of multiverse analyses. Milliways provides interlinked panels presenting result distributions, individual analysis composition, multiverse code specification, and data summaries. In the third study, I compare the two different approaches for depicting ontological uncertainty---ensembles and p-boxes---by conducting experiments to investigate the impact of the visual representation on how the multiple uncertainty distributions are interpreted. Based on these results, I identified how the results of multiverse analyses should be visualised so that viewers adopt the desired (possibilistic) interpretation of ontological uncertainty. Together, these three studies outline a systematic approach for surfacing, reasoning about, and communicating ontological uncertainty that is often implicit in data analysis processes.
TIME Thursday, July 31, 2025 at 2:00 PM - 4:00 PM
LOCATION ITW, Ford Motor Company Engineering Design Center map it
CONTACT Wynante R Charles wynante.charles@northwestern.edu EMAIL
CALENDAR Department of Computer Science (CS)
-
Sep25
EVENT DETAILS
TBA
TIME Thursday, September 25, 2025 at 9:00 AM - 11:00 AM
CONTACT Wynante R Charles wynante.charles@northwestern.edu EMAIL
CALENDAR Department of Computer Science (CS)
