About The Forum
Calendar 2017 Poster Sessions : World of Bits: An Open-Domain Platform for Web-Based Agents
Abstract:
While simulated game environments have greatly accelerated research in reinforcement learning, existing environments lack the open-domain realism of tasks in computer vision or natural language processing, which operate on artifacts created by humans in natural, organic settings. To foster reinforcement learning research in such settings, we introduce the World of Bits (WoB), a platform in which agents complete tasks on the Internet by performing low-level keyboard and mouse actions. The two main challenges are: (i) to curate a large, diverse set of interesting web-based tasks, and (ii) to ensure that these tasks have a well-defined reward structure and are reproducible despite the transience of the web. To do this, we develop a methodology in which crowdworkers create tasks defined by natural language questions and provide demonstrations of how to answer the question on real websites using keyboard and mouse; HTTP traffic is cached to create a reproducible offline approximation of the web site. Finally, we show that agents trained via behavioral cloning and reinforcement learning can successfully complete a range of our web-based tasks.
Bio:
Tim Shi is a PhD student with Prof. Percy Liang at Stanford. His current research interest focuses on reinforcement learning, in particular with application to building general personal assistant. He worked with OpenAI in launching World of Bits as part of Universe platform. He is a also co-founder of AI+ Club (http://aiplus.club), and organizes a series of events on the impact of AI on society.
While simulated game environments have greatly accelerated research in reinforcement learning, existing environments lack the open-domain realism of tasks in computer vision or natural language processing, which operate on artifacts created by humans in natural, organic settings. To foster reinforcement learning research in such settings, we introduce the World of Bits (WoB), a platform in which agents complete tasks on the Internet by performing low-level keyboard and mouse actions. The two main challenges are: (i) to curate a large, diverse set of interesting web-based tasks, and (ii) to ensure that these tasks have a well-defined reward structure and are reproducible despite the transience of the web. To do this, we develop a methodology in which crowdworkers create tasks defined by natural language questions and provide demonstrations of how to answer the question on real websites using keyboard and mouse; HTTP traffic is cached to create a reproducible offline approximation of the web site. Finally, we show that agents trained via behavioral cloning and reinforcement learning can successfully complete a range of our web-based tasks.
Bio:
Tim Shi is a PhD student with Prof. Percy Liang at Stanford. His current research interest focuses on reinforcement learning, in particular with application to building general personal assistant. He worked with OpenAI in launching World of Bits as part of Universe platform. He is a also co-founder of AI+ Club (http://aiplus.club), and organizes a series of events on the impact of AI on society.