Friday / CS Seminar

October 9 / 12:00 PM

In Person / Tech L361

Speaker

Yusuf Pisan, University of Washington Bothell

Talk Title

Teaching an LLM Tutor to Withhold the Answer

Abstract

LLMs are good at answering homework questions, and for a tutor that is the problem. Learning requires a struggle. A tutor that hands the student the answer is not doing its job. Tutor's job is to withhold information and guide the student towards the answer. We call this refusal-under-knowledge: the tutor knows the answer, the student can see that it knows and asks directly, and the tutor still has to hold back while staying warm and useful. Standard LLM safety refuses content a model should not produce. A tutor has to refuse content it can produce.

The tutor's response to the student has to be warm, Socratic, factually grounded and not reveal the answer. When students push back, the tutor's job becomes harder. What information the tutor withholds cannot be checked by reading its prompt. It has to be measured against real student behavior, including deliberate attempts to extract the answer, and adjusted until it holds. So rather than writing a better prompt, we enforce withholding outside the model that generates the response and calibrate that enforcement against evidence.

Biography

PhD in Artificial Intelligence from Northwestern University (1998). Teaching Professor at the Computing & Software Systems division of University of Washington Bothell, where I have taught since 2017. Before Seattle: 20 years in Sydney at UTS, and visiting appointments at Harvey Mudd, USC, WPI, and ITU Copenhagen.

My research group T4G (Tech for Good) works on CS education, AI systems, and video games. I founded the UTS Games Studio and the Australasian Conference on Interactive Entertainment. I received the UWB Distinguished Teaching Award in 2023.

Research Interests: Artificial Intelligence, Education, Qualitative Reasoning, LLMs, Tutoring Systems