Seamless interaction between AI agents and humans using natural language remains a key goal in AI research. This paper addresses the challenges of developing interactive agents capable of understanding and executing grounded natural language instructions through the IGLU competition at NeurIPS. Despite advancements, challenges such as a scarcity of appropriate datasets and the need for effective evaluation platforms persist. We introduce a scalable data collection tool for gathering interactive grounded language instructions within a Minecraft-like environment, resulting in a Multi-Modal dataset with around 9,000 utterances and over 1,000 clarification questions. Additionally, we present a Human-in-the-Loop interactive evaluation platform for qualitative analysis and comparison of agent performance through multi-turn communication with human annotators. We offer to the community these assets referred to as IDAT (IGLU Dataset And Toolkit) which aim to advance the development of intelligent, interactive AI agents and provide essential resources for further research.

Visualisation makes the system easier to use, and easier to improve. Whether it's an app, a consumer service or part of an internal process, the end goal is to use AI technology to power a product. One of the biggest challenges is understanding and addressing the system's error profile. Your system is almost certainly going to make mistakes. When it does, you want to fail gracefully.

Data collection is the single most important step in solving any machine learning problem. As such, teams that dive head first into projects without considering the right data collection process often don't get the results they want. Fortunately, there are many data collection tools to help prepare training datasets quickly and at scale. The best data collection tools are easy to use, support a range of functionalities and file types, and preserve the overall integrity of data. In this article, we outline the best data collection tools for machine learning projects.

One of the first things we do after launching a website nowadays is connect to Google Analytics. A little bit down the road we'll connect more "out-of-box" analytics tools to calculate funnels, retention, A/B tests, and more. These tools are great and work fine until a company gets bigger and analytics requirements get more sophisticated. It's time to set up a data infrastructure, which means selecting a data collection tool, ETL tool, data warehouse, and BI tool on top of that. In the startup world this usually happens when a company has raised Series A and has around 25-50 employees.

Visualisation makes the system easier to use, and easier to improve. Whether it's an app, a consumer service or part of an internal process, the end goal is to use AI technology to power a product. One of the biggest challenges is understanding and addressing the system's error profile. Your system is almost certainly going to make mistakes. When it does, you want to fail gracefully.

Players begin games at different skill levels and develop their skill at different rates—so that even the best-designed games are uninterestingly easy for some players and frustratingly difficult for others. A proposed answer to this challenge is Dynamic Difficulty Adjustment (DDA), a general category of approaches that alter games during play, in response to player performance. However, nearly all these techniques are focused on basic parameter tweaking, while the difficulty of many games is connected to aspects that are more challenging to adjust dynamically, such as level design. Further, most DDA techniques are based on designer intuition, which may not reflect actual play patterns. Responding to these challenges, we have created Polymorph, which employs techniques from level generation and machine learning to understand level difficulty and player skill, dynamically constructing levels for a 2D platformer game with continually-appropriate challenge. We present the results of the user study on which Polymorph's model of level difficulty is based, as well as a discussion of the unique features of the model. We believe Polymorph creates a play experience that is unique because the changes are both personalized and structural, while also providing an example of a new application of machine learning to aid game design.