Welcome to the Symposium on Usable Security and Privacy happening in Hannover, Germany and being hosted by CISPA. The Symposium is a community that studies how people use security and privacy technologies.

Blog posts:

• Day 1
• Day 2
• Day 3

Session: Usable Security Mechanisms & Emerging Threats

IdentitySign: Design and User Study of a Prototype Application for Digitally Signing Documents Using an Identity Wallet

Presenter: Yorick Last, Paderborn University

You can sign digital documents via electronic or digital signatures. Handwritten signatures work well but provide no guarantees of the authenticity or the integrity. Often because they are typed, or could be easily copied.

One way to make this easier and better for users is Digital Identity Wallets applications that collect various identity attributes, presented to the user in a card-based interface. These are decentralized user-centric means of identiy management. EU is rolling out a Digital Identity Wallet next year.

The project proposes such a prototype wallet: IdentitySign. Free and open-source web applicatoin, uses the Yivi identity wallet and lends cryptographic support from PostGuard.

Had a user study including an unmoderated usability test with 99 participants recruited accross the EU.

IdentitySign was generally found to be useable. But some users did not understand why such a signature was needed over just signing their name with a (digital) pen. Only 20% could detect a valid signature.

So the researchers tried to address the issue of misplaced trust. Also added some friction to the UI for verification to slow the user down.

Ran the study again with 74 participants. Again from across the EU and an unmoderated remote user study. The scenarios included 1 signing and 2 verification tasks.

Usability was still considered to be good. 82% could now correctly find broken signatures. 95% correctly identified a signature as correct.

A key lesson was that the UI needs to balance being helpful without seeming to imply trustworthyness. It is important that users be able to judge if a signature is correct themselves.

Is It Real? Exploiting Virtual-Physical Discrimination Vulnerability in Mixed Reality

Presenter: Xueyang Wang

The visual-physical discrimination vulnerability - the point at which humans cannot differentiate between real and faked. This is also known as the uncanny valley.

This research project looks at ways that an attacker can inject virtual objects onto the user’s vision.

• Endogenous injection
  • invisible duplicates of existing objects in the user vision
  • users have trouble detecting these
• Exogenous Injection
• Type-deceptive overlay
  • Modifying a part of the user vision. For example adding flowers to the top of a trash can so that it looks like a flower pot. The user can still see it but cannot figure out where the trash can is.
• Attribute-overlay
  • Adding elements like putting logos on a white t-shirt can impact user decisions. Even when a user knows that the image was fake and had been added. This is well in alignment with marketing research.

Attacks with the highest behavioral success rates received the highest plausibility, lowest workload, and highest usability ratings. In other words, if people thought the fake thing was real, they were happy to engage with it and were happy about it.

Research outcome: even if the user detects the fake, it still impacts their decision making and physical behavior.

Clear, Actionable and Confidence-Inspiring Recommendations? Comparative Study of AI-Generated and Human-Written PT Reports

Presenter: Katarina Galanska, Masaryk University

Interested in how recommendations were acted on. Security recommendations tend to be high-level and general so many possible actions could be taken based on them.

Suggestions for how to address a security vulnerability were written by humans and various of AI. Did a survey-based study asking users about the following three aspects of each suggestion.

• Clarity
• Actonability
• Confidence

Lightning talks

• Florin Martius, Registered Reports at SOUPS: Call for Discussion
  • Registered reports: comprehensive research plan is submitted to the reviewers rather than the final research paper. The idea is to not waste research time, and create more high-quality research.
  • Challenges exist though, it now shifts reviewing effort.
  • Kami’s opinion: Contextualization of results is important in a final publication. Because it is not just about the methods. It is also about the interpretation of the results which cannot be done up-front.
• Simon Althaus, PIONEER - A PrIvacy companion for mOtivatioN and knowlEdge transfER
  • Rough motivation is to make it easier for users to handle privacy
  • App involves privacy and lessons and motivations
  • Can compare apps to each other to make decisions
• Christina Detsika and Julia Justen, Age Groups for Security and Privacy Studies with Children
  • Deciding what age ranges of children to do research with is a bit complex because developmental stages can drastically impact cognitive understanding. The work aims to determine suitable age ranges for research in the USEC space as well as how to best talk about the population choices in research papers.
• Camille Cobb, How do we encourage students (and ourselves) to take meaningful action against dystopian technologies?
  • Advocating action-oriented tech ethics education
  • Current ethics materials focus on how to make judgments. But not much about how to take action after the judgment is made. Especially when the action is professionally risky or it is unclear how to do.
• Nicole Dircksen - What should security security researchers do in an ethical situation.
  • Research project is aimed at understanding where researchers agree and disagree on what is ethical in the field. Aim is to better understand our current understanding to advise new rules.
  • For example: researchers find a vulnerability in a medical device where the company is now out of business. Should they disclose so that hospitals know and can remove or better protect such machines BUT also inadvertently tell attackers about the issue. Or should they stay silent and hope that attackers do not also find the vulnerability.

Session: Scams, Fraud & Misinformation

“You Have Been Selected as the Winner”: Characterizing User-Reported Scams on TikTok

Presenter: Smirity Kaushik, The George Washington University

$3 billion is lost to social media fraud in 2024 (FTC, Top scams of 2024) and short form videos are popular with the general population. This also leads to a whole ecosystem of influencers.

RQ’s

• What types of scams?
• How do users report scams?
• How do TikTok users identify these steps?

The researchers manually searched user-reported videos for scams. Found 150 videos (after reviewing them).

Types of scams:

• Financial
• Influencer targeting
  • asking influencers to do things like buy a product and review it, but the influencer has to buy first before getting compensated.
  • cloneing an influencer and then asking for things. Causes the influencers to have to post about such scams
  • Deepfake of an influencer to promote a different product than the influencer initially promoted.
• Shopping
  • Fake or poor-quality products
  • Goods never delivered
• Account
• Romance
• Miscellaneous

How do scammers entice people?

• Lots of use of direct message and “for your page”
• Scammers used “need and greed” or the “distraction” principles to lure victims

How were scams identified?

• Found the same as the FTC common approaches like request to pay right away
• Also found that scammer communication patterns, account discrepancy, unauthorized transactions, product and price mismatch

Kami’s view: this is reminding me of our paper on what people say when reporting phishing, I think that we need more research that looks at what people say after falling for a scam organically, that is what they write immediately after they fall for the scam, as opposed to what they say to a researcher which may be biased by the presence of the researcher.

[Source-Level Disengagement: A Usable Security Defense Against Misinformation](Source-Level Disengagement: A Usable Security Defense Against Misinformation)

Presenter: Bogdan Carbunar, Florida International University

Misinformation is a serious issue and even if users are aware it can still influence thinking as shown by the earlier paper today on MR and changing what users see.

This research targets the issue that the user is seeing repeat misinformation from a person they are following and are not unfollowing.

Three interventions

• Inline Warning
  • Warning below main post
• Action Controls
  • Warning followed by buttons like block and filter
• Blur Overlay
  • post cannot be seen without clicking
• Control
  • no intervention, just the post content

29 participants in a within-subjects study. Processed 4 timelines each of which had 12 posts (3 with misinformation).

The control condition had only one user do something about the misinformation. Bu tfar more with the other options, particularly the action controls option where controls were made readily available.

Users sometimes chose the blur or filter options over other actions like unfollowing because unfollowing leads to sending social signals and breaks a relationship.

B luring still caused participants to view the content anyway, and then took an action.

• Kami question: I wonder if this finding would persist in the real world. Earlier comment is about filtering and blurring to avoid content without breaking a relationship. It may be that in the real world users get to know sources and therefore do not open the content because they know its likely shape already.

Design Implications:

• It should be easy to act when misinformation is encountered, not just warned, users should have a clear path to act
• Preserve relationships should be possible. Users should have silent blocking ability where the followed person does not know their content has been hidden. Users are more likely to take this option
• Design for limits - platforms also need to take actions themselves. Users get overwhelmed, so the platform itself needs to reduce user burden.

A Penny for Your Prompts: Experiments Detecting and Mitigating LLM Usage by Survey Respondents

Presenter: Nathan Malkin, New Jersey Institute of Technology on behalf of Zane Xu

Survey research is starting to have problems with LLMs giving answers instead of real users. This is super bad for science. Attention check questions like: *“please answer `strongly agree’ to this question” are irronically easier for LLM than humans are are more likely to be correctly answered. One option is questions that LLMs answer poorly, but well those are vanishing. We could also just ask participants “please do not use AI”, but well that doesn’t really work.

This research

• Can we detect LLM
• How often
• What can we do

So a survey was designed… [scarcasum]

• Collected metadata like keyboard speeds, page timing, lots of things
• Collected 250 responses across both Prolific and MTurk, including a “Please don’t use AI” situation.
• Asked an AI agent to also fill out the survey to provide a baseline.

Recommendations:

• Do not use MTurk.
  • 84% of MTurk responses were LLM
  • vs 8% on Prolific
• They tracked what participant copied and pasted
  • If they copied text from the page, and if they pasted text that was not already present. Also if they pasted text that they did not type.
  • This is the main way to differentiate ….
• Could we we disabled copy/paste?
  • MTurk users re-enabled it or typed in gibberish
• Asking users on Prolific to not use AI actually worked
  • 0% used AI
  • Agents were also less willing to fill out the survey if it said not to use AI on the survey.
• Asking for self-report AI usage
  • Agent is happy to state that it took the survey
• Keep reading the answers you get
  • Detecting AI by hand is possible. It is not the content, it is the style
• Consider using open-ended attention checks
  • Distinctive answers by AI. For example “what is the shape of a ball?” Humans say “round”, AI says “spherical”

These methods work now, but are unlikely to last. AI is changing quickly.

It is important that as a field we work with other researchers to solve these problems. Individuals cannot solve this alone. Data quality checks need to be discussed in paper, because we cannot assume good data for online surveys any longer.

SoK: Mapping Threats to Defenses in Online Survey Fraud

Presenter: Wellington Esposito Barbosa, The George Washington University

Online surveys are prevalent in data collection. But detecting fraud in such studies is very hard. Researchers have to know what real data looks like and what fraud looks like.

Research papers surveyed are coming from non-security communities and is not necessarily being framed as “fraud” the way security would frame it. Researchers often discover fraud while doing other data processing or review. This can be very expensive if the study must be terminated or abandoned.

RQs

• What is research fraud?

  • No consistent definition among papers.
  • Some definitions:
    • Intentional data fabrication
    • Careless and Insufficient-effort
    • Automated, AI-mediated
  • Fraud behaviors found
    • Multiple accounts, duplicate responses,

• At what stage in research are researchers trying to stop fraud?

  • Mitigation strategies are being implemented late stage in cleaning data or were very fragile like asking attention checks.
  • For example if a participant fills it out twice: could be blocked based on IP address.

High level point: similar to other aspects of security, we see researchers selecting protections that may not align well with the threats.

Q&A

• How can we setup an early warning system for when survey detection methods for bots start being less effective. The publication system is far too slow, and many people need to know these.
  • It is a concern that if a “how to avoid survey fraud” was put together, it would likely only be good for a few months.
  • We are trying to build a survey system that has security features built into it. And then make it easier to put things that work into it directly.
• Double problem: participants are using AI to fill out surveys AND researchers are using LLMs to simulate users in some research. How have we already reflected on the ethics of the means of identification of fraud?
  • There were no real discussions of ethics. There were some concerns about how methods were detrimental to reaching specific communities, but it sounds like this was discussed more in terms of research integrity and getting data from a range of users being important.
• Should we all go back to in-person surveys? This will slow research. But given the speed of paper submissions, maybe slowing down is a good thing.
  • There are pros and cons to that. But we were surprisingly pleased by how effective current mitigation are.

Session: GenAI Privacy & Risks

AI’ve Got a Bad Feeling About This: A Privacy Threat Modeling Framework for GenAI

Presenter: Jonah Bellemans, DistriNet, KU Leuven

Kami was busy and missed most of the talk.

Understanding U.S. Users’ Security and Privacy Transparency Needs for Consumer-Facing Generative AI

Presenter: Jiaxun (Messie) Cao, Duke University

Users have poorly informed mental models about how GenAI is using their data. This includes things like company chat bots.

Most transparency information is expert-facing and aimed at explaining model capabilities and how it uses data.

RQ’s

• Do users consider S&P concerns when picking a model?
• Current practices and challenges
• User preferences

21 interviews with users and 20 designs.

Initial adoption was initially driven by non-S&P factors. Instead popularity, utility, and price were most considered. S&P was a desired adoption feature, but hard to get good data about so not considered.

S&P uncertainty became a larger concern based on the content of the planned discussions with the AI.

When people did read privacy statements, they see it as lacking credibility (8/21) which means they did not trust it. This is a very similar result to what general privacy policy research on how people read them. From my memory: “privacy policies are designed by lawyers to make it hard for me to understand and beneficial to the company.”

User wanted independent evaluations of the privacy policies. Somewhere they could go to compare privacy practices and that had the skills to properly read the practices.

Kami’s thoughts: Consumer Reports in the USA does now include commentary on the privacy practices of some products. Not all, sadly, but their comments on baby monitors regarding privacy are rather amazing.

Maybe… I Don’t Really Wanna Clone: Attitudes and Anticipated Harms of (Consensual) After-Death Cloning of One’s Own and Voices of Entrusted Others

Presenter: Jennifer Vander Loop

When a loved one dies, those left behind may want to be able to talk to the deceased as part of their digital legacy. Some older people are starting to consider their wills and considering how they want their data used after death, including how they do or do not want to be emulated by a Generative AI.

People were interested in digital resurrection, but less ok with the idea of immortality where such things were used indefinitely.

Ran a survey on 900ish people on Prolific. Looked at what people wanted for themselves and what they wanted for others that had passed. Many people also thought about how creepy this is. Disconnect between teh voice, person, and after-death content. It may sound like them but the content might not match because it did not think like them.

Participants came up with many possible harms. Scams were a large one, they did not want a loved ones voice used to scam others. Also harms to the person’s reputation, and social standing. There is a risk the company might use this for their own befit, such as suddenly getting a loved one to market to you. There is also the risk of non-aligned models where there are two versions that were made with different models. The model itself could be held ransom from the family, especially if an emotional attachment had been made.

Session: Security Advice & Education

Who Can Actually Follow IT-Security Advice? Exploring the Usability of IT-Security Advice

Advice given to users can be straight forward, but lead to lots of extra questions around how to opperationalize and understand the advice. There is a difference between being able to find the advice, and the ability to follow it because it can be too complex and make them overwhelmed.

Experts view themselves as a good source of advice, which is partially true, but they are not necessarily good at providing advice for users that is easy to use.

Usability survey used with about 350 participants.

Designing an IT security advice usabilty scale. Started with the System Usabilty Scale (SUS), mildy modified to Information Security Policy Usability Scale then resulted in a Adaptation for consumer Advice. The final scale is similar to the SUS in that it is a short 10-question scale.

Older people perceived the advice as more usable than younger people did.

Normal advice like sentence length had no positive advice on usability. But there was some improvement from having clear steps, low tech words, and context alignment.

Consider adjusting advice based on the audience.

CyQured: Design, Development, and Empirical Evaluation of a Tabletop Game for Personal Cybersecurity Education

Presenter: Farida Chowdhury, BRAC University on behalf of Utsho Das and Argha Pratim Sha

CyQured board game was developed for cybersecurity education.

Security at home is a human decision problem. Users have many devices, no security team at home, and traditional training does not provide practice. So how are home users going about learning about security.

Related work

• CyberThreatDefender
• Many others already on my games list

The game was modeled off of Monopoly. Used STRIDE in a simplified format. There are 16 device cells such as laptops, router, NAS. Placement is also not guesswork.

Conducted a pre-post evaluation with a security knowledge test before and after an hour of group game play. Some participants were also interviewed.

1 hour of play did produce substantial knowledge gains. People with pror security experience did gain more, but everyone learned.

Nice to Know You’re Not Alone: Co-designing Community-centered Online Safety and Privacy Education with Librarians

Presenter: Florian Schaub, University of Michigan on behalf of Tanisha Afnan

People struggle to stay safe online:

• Lacking privacy and security literacy
• Do not seek help of security experts

Pror work at PETS: How We Define Privacy Literacy: Teaching Experiences & Challenges of Community-Engaged Privacy Educators

• Found that people who are trying to help others, such as librarians, have challenges moving past intro sessions to get people to implement help

In this work, focus was on helping librarians in doing community engagement.

Conducted 4 participatory design workshops with 14 librarians. Participants had 5-20+ years of experience teaching a range of workshops.

Privacy and safety are sometimes taught specifically, but they are often part of a broader digital literacy. They struggled a bit with how to balance conceptual issues with practical skills. This group also has limited resources, such trainings are only one part of their job.

There are many existing educational resources out there that are often diffused and hard to find. Paper has long list of existing educational materials. Existing resources were not easy to just adopt. Many were aimed at experts. Many had loads of jargon. There are also limited resources in non-English. They also wanted to print resources and many are not print-friendly. So most of the time these materials were used to help inform the teacher rather than the learner.

Co-designed solutions for learners:

• Interactive games came up
• Take-home worksheets and practice exercises
• Printed handouts and low-tech materials
• Non-class based educational offerings
  • Like a monthly community challenge
• More need for professional development
  • Kami: reminds me a bit about how CS4All tackles the issue of teaching Computer Science in schools and taking the view that educators are good at their job, what they need is assistance in making a plan and organizing resources.

Kami note: reminds me a bit of the Connected Canadians project that supports older adults in Canada.

Security versus Productivity: A Case Study of Email Management Practices with Job Task Analysis

Presenter: Philip Shumway, University of Tulsa

Cybersecurity policy is written at the higher levels of an organization. That policy then becomes tasks for lower level employees.

Goals:

• Re conceptualize security as embedded work
• Identify and map task-based tension
• Understand how work done

Task Analysis is derived from Job Analysis. Jobs are broken into tasks, and tasks are broken into attributes.

Looked into a case study of a medium sized publisher that had fallen for a phishing attack. So they hired a full time security person to help them be more secure and do security training. Researchers identified a large number of tasks that employees do, broke them down to a smaller set, and then had the employees rank the tasks.

Employees do not see security and productivity as the same. Employees saw getting it wrong more serious for security.

People who self-report having more external emails also spend more time on security.

One interesting thing about this work is the division of employees based on their security context. People have different security needs if they get lots of external emails, than if they did not.

Kami note: Reminds me a bit of the paper Introducing the Cybersurvival Task: Assessing and Addressing Staff Beliefs about Effective Cyber Protection which also asked employees to rank security tasks as a way of communicating culture with others.