Information systems have become the cornerstone of how decisions are made in nearly every domain of human activity, constituting the infrastructure through which knowledge is accessed and acted upon. At their core, an information system is any integrated combination of hardware, software, databases, networks, and human expertise organized to collect, process, store, and analyze information in support of operations and decision-making. The usability of such systems in practice is governed by two properties that have historically stood in tension: accuracy and fluency. Symbolic, query-based systems offer high accuracy but require specialized query languages or code to interact with effectively, while large language model (LLM)-based systems are fluent and accessible through natural language but remain prone to hallucination and factual error. The goal of this dissertation is twofold: to build information systems that answer questions and generate reports from data both accurately and fluently, and to generate training data that improves the accuracy and fluency of such systems, in both cases by using structured knowledge and rules as a source of ground truth for language generation.

To this end, this dissertation introduces a neurosymbolic framework, grounded in knowledge representations of data and analytics, that first produces language outputs that are accurate but not necessarily fluent, and then uses these outputs to condition LLM generation, yielding final outputs that are both accurate and fluent. These outputs can serve two roles: as the final product delivered to a user, or as training data for the models that later serve users. This framework is first demonstrated through Satyrn, a report generation system in which domain-independent report blueprints produce accurate but coarse information that is then used to condition LLMs, jointly producing reports that are more factual, fluent, and coherent than those generated by existing neural methods. The framework is then applied to training data generation through RingSQL, a platform that combines these knowledge representations with LLM-based paraphrasing to produce text-to-SQL training data that is as diverse and fluent as LLM-generated data while remaining more accurate and faithful to the underlying query. Data generated with RingSQL provides a stronger training signal than data produced with prior LLM-based methods, template-based methods, and even human-annotated data. Finally, this dissertation introduces methods for grounding the generation of reasoning traces in verifiable, rule-based evaluation, and uses these traces in a mixed-policy reinforcement learning setting, offering an additional path toward improving the accuracy of LLM-based information systems. Together, these contributions demonstrate that structured knowledge and language are complementary, and that unifying them can reconcile the accuracy-fluency trade-off in information systems.