Smart speakers in multi-user spaces, such as Amazon Echos, introduce risks to both owners and anyone sharing the space. They store voice recordings of user requests, and anyone in range can potentially interact with the device. As smart speakers are usually bound to a single account, despite being shareable by design, it introduces potential tensions between users. We systematically searched the literature for findings on concerns and scenarios in which problems may arise and synthesised the resulting 20 papers in a narrative review. Owners were concerned about other users’, potentially malicious, interactions, device faults, and third party sharing. In contrast, bystanders worried about “being listened” to and a lack of awareness and protections. Our findings show a clear gap in literature on the privacy concerns of regular and incidental secondary users of smart speakers.