Membership Inference Attacks on LLM-based Recommender Systems

Abstract

Large language models (LLMs) based recommender systems (RecSys) can adapt to different domains flexibly. It utilizes in-context learning (ICL), i.e., prompts, to customize the recommendation functions, which include sensitive historical user-specific item interactions, including implicit feedback like clicked items or explicit product reviews. Such private information may be exposed by novel privacy attacks. However, no study has been done on this important issue. We design several membership inference attacks (MIAs) aimed to revealing whether system prompts include victims’ historical interactions. The attacks are direct inquiry, contrast, hallucination, and poisoning attacks, each utilizes some unique features of LLMs or RecSys. We have carefully evaluated them on four of the latest open-source LLMs and three well-known RecSys benchmark datasets. The results confirm that the MIA threat on LLM RecSys is realistic: direct inquiry, contrast, and poisoning attacks show significantly high attack advantages. We also discussed possible methods to mitigate such MIA threats.