An Analysis of Privacy-Aware Personalization Signals by Using Online Evaluation Methods

Personalization despite being an effective solution to the problem information overload remains tricky on account of multiple dimensions to consider. Furthermore, the challenge of avoiding overdoing personalization involves estimation of a user's preferences in relation to different queries. This work is an attempt to make inferences about when personalization would be beneficial by relating observable user behavior to his/her social network usage patterns and user-generated content. User behavior on a search system is observed by means of team-draft interleaving whereby results from two retrieval functions are presented in an interleaved manner, and user clicks are utilised to infer preference for a certain retrieval function. This improves upon earlier work which had limited usefulness due to reliance on user survey results; our findings may aid real-time personalization in search systems by detecting a user-related and query-related personalization signals.