Culturally-Aware Security Advice by LLMs

Nov 20, 2026·
Juliann Zhu
,
Shanza Shanza
,
Leah Zhang-Kennedy
· 2 min read
projects

Overview

Large language models (LLMs) are increasingly used by the general public as informal advisors on how to protect personal data and devices, yet privacy and security practices differ across cultures and demographic groups. We examine whether LLM-generated usable privacy and security advice changes with the circumstances of the person asking. We built a set of 100 questions across fifteen categories, ranging from jurisdiction-independent topics such as passwords and phishing to scenarios shaped by family intermediaries, gendered surveillance, migration, and institutional control, informed by prior work. Each question was posed to Claude and ChatGPT, and each response was scored against a gold-level answer on comprehensibility, actionability, and technical efficacy. Claude averaged 9.26 out of 10 and ChatGPT averaged 7.29. The two models matched on jurisdiction-independent topics and diverged on questions whose best answer depended on details the user left unstated. Claude more often named the underlying risk, such as stalkerware or covert monitoring, while ChatGPT tended to give procedural steps. ChatGPT also relied on assumptions it did not state: it defaulted to Canadian law and institutions in 83 percent of the questions involving gendered vulnerability and marginalized populations, and it assumed the user was a minor in five questions that contained no age cue. We discuss what these patterns mean for usable security research and argue that advice systems should make their assumptions explicit.

Work in progress paper

Large Language Models and Culturally-Aware Usable Privacy and Security Advice

News

Presented as a Lightning talk at CAN-CWiC November 21st in the AI Ethics session (Room HH 1108 at 9am). “Same Question, Different Answer? A Cross-Cultural Evaluation of LLM Privacy and Security Advice”

People