Visualizations are highly effective at communicating data-relevant information. However, they can also exacerbate misinformation if not appropriately designed, making visualization literacy (i.e., the ability to both accurately interpret and create well-formed visualizations) essential for informed decision-making in today's society. Existing studies of visualization literacy largely focus on low-level readings of well-formed visualizations, which are not representative of what it means to be literate with visualizations in a world where not all information is well formed. I advance the understanding of visualization literacy by developing enhanced measurements of higher-level skills (e.g., critical thinking, visual mapping), examining how visualization literacy measurements are designed and used, and grounding educational interventions in this broadened understanding. To address the specific challenges, such as how to reliably capture complex cognitive abilities, how to evaluate open-ended tasks with vast solution spaces, and how to scale assessment beyond manual expert judgment, I (1) develop a Critical Thinking Assessment for Literacy in Visualizations (CALVI), a validated measurement that tests people's susceptibility to misleading visualizations, (2) co-design an educational game with high school

students for teaching critical thinking in visualization interpretation, (3) develop an Assessment for the Visual Encoding Ability in Visualization Construction (AVEC), a validated measurement that tests people's ability to visually encode data for creating well-formed visualizations, and (4) investigate factors that make visualization literacy hard to operationalize through conducting an autoethnography with users and designers of assessments. My thesis clarifies what constitutes visualization literacy and outlines concrete steps for the research community to systematically advance this understanding.