Abstract

There has been a significant interest in applications that require algorithms that can keep track of the users’ state in the recent years. Examples of such systems include conversational systems and intelligent assistants, contextual information retrieval and advertising systems, intelligent tutoring systems, expertise finding systems or systems that model player expertise in gaming, among others. This workshop aims at bringing together researchers from academia and industry that focus on different aspects of state-based user modelling to exchange ideas and build state-aware user-centric systems. We encourage submissions to this workshop in a variety of topics aiming to discuss the challenges associated with capturing and effectively utilising user state in different web search and data mining applications.

The Workshop

Inferring and utilising user states and goals is becoming a timely challenge for successfully leveraging intelligent user-centric systems in the real-world [7]. Examples of such systems are conversational agents [1, 9], intelligent assistants [8], educational [5] and contextual information retrieval systems [6], recommender/matchmaking systems [4] and advertising systems [3], which all rely on identifying the user state, in order to provide the most relevant information to the user and assist them in achieving their goals. For example, due to their interactive nature, search and recommender systems are notoriously hard to develop and evaluate, since they involve a multi-step decision-making process, where a stream of interactions occurs between the user and the system. Traditionally, to make the problem tractable, the interactions are often viewed as independent, but in order to improve such systems further, it becomes important to consider and leverage user states. In the recent years, various interactive machine learning techniques (including bandit based and reinforcement learning based techniques) have been applied for interaction tasks as these approaches allow to learn and continuously change the strategy based on user feedback and other type of rewards [10].

Even though many such systems would significantly benefit from having a representation of the user’s state, there has been very limited work towards building intelligent learning mechanisms that can be used to identify, represent and update the state of the user. Hence, devising information retrieval systems that can keep track of the user’s state and make recommendations based on this has been listed as one of the grand challenges of information retrieval that needs to be tackled during the next few years [2]. In this context,we believe it is timely to organize aworkshop that re-visits the problem of designing and evaluating state-aware usercentric systems and makes sure the community, spanning academic and industrial backgrounds, is working on the right problem.

Goals and Objectives

This workshop aims to facilitate a forum for leveraging intelligent methods for modelling user’s state and its transition in an effective manner. Modelling user’s state is a challenging task with many currently open questions:

  • User representation and recommender systems: How to design accurate state-aware representations of the user and embed that information in predictive models?
  • Task understanding and supporting user tasks: How to identify the specific sub-task a user is trying to accomplish and design state-aware intelligent systems that assist users in each step of their goals?
  • State-aware evaluation: How to develop metrics and evaluation techniques that model and understand user states, so as to provide more sensitive and enhanced evaluation of the user-centric systems?
  • Human in the loop: How to include the human in the learning "loop" in an interactive fashion, making feedback an integral part of the modelling?
  • User-aware systems: How to incorporate user state information in systems to achieve better human-machine intelligence?
  • Cognitive/contextual user understanding: How to understand from a cognitive point of view how actions taken by users and their context can influence their state?
  • State-aware ML algorithms: How do the different ML paradigms which enable a system to identify and leverage user states compare for different application scenarios?

To progress and answer these questions, it is imperative for research communities from cognitive science, intelligent tutoring systems, information retrieval, human computer interaction, psychology and other diverse fields to bring forward their inputs, which is one of the primary goals of this workshop. We encourage sharing the ongoing work that is geared towards addressing these challenges, thus aiming to improve the research landscape and understanding of this emerging topic.We aim to provide a platform for budding ideas from different fields to unite together giving inspiration to more powerful user modelling approaches. WSDM, with its attendance by a broad spectrum of cross-disciplinary researchers offers the ideal venue for this exchange of ideas. Topics of interest include areas concerning the above mentioned highlighted points.

Invited Speakers

We plan to invite established researchers for talks from different domains for which representation of users’ state is significant.

Programme Committee

  • Emine Yilmaz, University College London
  • John Shawe-Taylor, University College London, UNESCO Chair in Artificial Intelligence
  • Sahan Bulathwela, University College London
  • Maria Perez-Ortiz, University College London
  • Rishabh Mehrotra, Spotify Research
  • Colin de la Higuera, Université de Nantes, UNESCO Chair in teacher training technologies with OER
  • Davor Orlic, COO, Knowledge 4 All Foundation

References

[1] Konstantina Christakopoulou, Filip Radlinski, and Katja Hofmann. 2016. Towards Conversational Recommender Systems. In Proceedings of the 22Nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD ’16). ACM, New York, NY, USA, 815–824. https://doi.org/10.1145/2939672.2939746
[2] J Shane Culpepper, Fernando Diaz, and Mark D Smucker. 2018. Research frontiers in information retrieval: Report from the third strategic workshop on information retrieval in lorne (swirl 2018). In ACM SIGIR Forum, Vol. 52. ACM, 34–90.
[3] Thore Graepel, Joaquin Quiñonero Candela, Thomas Borchert, and Ralf Herbrich. 2010. Web-scale Bayesian Click-through Rate Prediction for Sponsored Search Advertising in Microsoft’s Bing Search Engine. In Proceedings of the 27th International Conference on International Conference on Machine Learning (ICML’10). Omnipress, USA, 13–20. http://dl.acm.org/citation.cfm?id=3104322.3104326
[4] Tom Minka, Ryan Cleven, and Yordan Zaykov. 2018. TrueSkill 2: An improved Bayesian skill rating system. Technical Report MSR-TR-2018-8. Microsoft.
[5] Chris Piech, Jonathan Bassen, Jonathan Huang, Surya Ganguli, Mehran Sahami, Leonidas J Guibas, and Jascha Sohl-Dickstein. 2015. Deep Knowledge Tracing. In Advances in Neural Information Processing Systems 28, C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett (Eds.). Curran Associates, Inc., 505–513. http://papers.nips.cc/paper/5654-deep-knowledge-tracing.pdf
[6] Rohail Syed and Kevyn Collins-Thompson. 2017. Retrieval Algorithms Optimized for Human Learning. In Proceedings of the 40th International ACM SIGIR
Conference on Research and Development in Information Retrieval (SIGIR ’17). 555–564.
[7] Johanne R. Trippas, Damiano Spina, Falk Scholer, Ahmed Hassan Awadallah, Peter Bailey, Paul N. Bennett, RyenW. White, Jonathan Liono, Yongli Ren, Flora D. Salim, and Mark Sanderson. 2019. Learning About Work Tasks to Inform Intelligent Assistant Design. In Proceedings of the 2019 Conference on Human Information Interaction and Retrieval (CHIIR ’19).
[8] Alexandra Vtyurina, Denis Savenkov, Eugene Agichtein, and Charles L. A. Clarke. 2017. Exploring Conversational Search With Humans, Assistants, and Wizards.
In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA ’17). ACM, New York, NY, USA, 2187–2193. https: //doi.org/10.1145/3027063.3053175
[9] Rui Yan, Yiping Song, and Hua Wu. 2016. Learning to Respond with Deep Neural Networks for Retrieval-Based Human-Computer Conversation System.
In Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’16). ACM, New York, NY, USA, 55–64. https://doi.org/10.1145/2911451.2911542
[10] Xiangyu Zhao, Long Xia, Jiliang Tang, and Dawei Yin. 2019. "Deep Reinforcement Learning for Search, Recommendation, and Online Advertising: A Survey" by
Xiangyu Zhao, Long Xia, Jiliang Tang, and Dawei Yin with Martin Vesely As Coordinator. SIGWEB Newsl. Spring, Article 4 (2019), 4:1–4:15 pages. https://doi.org/10.1145/3320496.3320500