Artificial intelligence (AI) is increasingly important to decision-making across various domains, e.g., financial planning, disease diagnosis, etc. While applied methods excel at identifying specific phenomena and challenges within tested environments, theoretical methods provide rigorous guarantees across all possible scenarios, including those that may be rare or even adversarial. This worst-case guarantee is even more important in the modern AI workflow, where upstream AIs are provided as a service to downstream decision-making instead of being designed for an application.

This thesis evaluates and designs AIs out of the context of a particular application, through mechanism design and statistical decision theory that models rational decisions under uncertainty. At the core of the theoretical framework is proper scoring rules, the class of functions that evaluate a probabilistic prediction by the decision payoff it leads to. In mechanism design, proper scoring rules are mechanisms that elicit truthful predictions from a strategic agent who optimizes for expected score (McCarthy, 1956; Savage, 1971). In statistical decision theory and machine learning, proper scoring rules (a.k.a. proper losses) evaluate and reward predictions by their decision payoff (Gneiting and Raftery, 2007), which can be equivalently viewed as eliciting predictions from AIs or learning algorithms.

This thesis defense will be divided into two parts. Part I evaluates and improves the trustworthiness of predictions for downstream decision-making, from a statistical decision-theoretic perspective and an information elicitation perspective. Part II demonstrates the applications of the theoretical framework, including the design of a provably truthful text elicitation mechanism and a statistical decision-theoretic benchmark for human-AI interaction.