Professor Shvartzshnaider presented at the recent
In LLMs We Trust? A Contextual Integrity Perspective
Yan Shvartzshnaider (91亚色), Vasisht Duddu (University of Waterloo)
The talk broadly discuss recent results in
"Large language models (LLMs), while memorizing parts of their training data scraped from the Internet, may also inadvertently encode societal preferences and norms. As these models are integrated into sociotechnical systems, it is crucial that the norms they encode align with societal expectations. These norms could vary across models, hyperparameters, optimization techniques, and datasets. This is especially challenging due to聽prompt sensitivity鈥搒mall variations in prompts yield different responses, rendering existing assessment methodologies unreliable. There is a need for a comprehensive framework covering various models, optimization, and datasets, along with a reliable methodology to assess encoded norms."