About Plus One Robotics
Plus One Robotics builds vision software for logistics robots. Their software helps power robotic arms to automate a range of tasks in warehouses and distribution centers.
Plus One’s PickOne product relies on a system of 2D and 3D cameras paired with AI algorithms to detect packages and determine the ideal place to pick them up. This enables a system of robot arms to handle the manual work of sorting packages. Meanwhile, a team of human Crew Chiefs can monitor groups of robots and intervene to handle certain difficult scenarios.
Plus One partners with companies like FedEx, Yaskawa, and FANUC to sort thousands of packages an hour. By playing to the strengths of robots and humans, Plus One’s technology helps alleviate the severe labor shortage in the logistics industry and helps get packages where they need to go.
Goals
Plus One’s software relies on having accurate predictions from their AI models. In some cases, the Plus One AI team works to enable new tasks or new customer deployments by developing new models. In other cases, they work to improve the performance of models in use with existing customers to address failure cases or handle new types of packages.
Plus One wanted to improve the picking performance of their robot system for their customers. To do so, they focused on the following areas:
- Continuous model deployment: Frequently and consistently ship improved models to customers.
- Catching issues early: Find and fix critical model failures before shipping new software, preventing regressions from making their way to customers.
- Reducing load on the Machine Learning team: Scale the impact of the machine learning team to more tasks across more customers.
Using Aquarium
Plus One Robotics partnered with Aquarium to manage their machine learning iteration process. The Plus One team wanted to do model validation, comparison, and exploration in order to iterate on their datasets and their models. Aquarium provided a range of functionality to speed up this workflow.
- Intuitive user interface: Basic tasks of visualizing datasets and model inferences, querying for interesting scenarios, and plotting dataset distributions could be done in a matter of clicks from the Aquarium web UI.
- Collaboration with stakeholders: Aquarium’s web UI and collaboration features made it easy to communicate model results with team members, managers, and customers.
- Advanced surfacing of failure patterns: Embedding analysis and visualization quickly exposed areas in the dataset where the model struggled, allowing the Plus One team to prioritize data collection on important object types and adjust learning parameters based on the errors they saw.
- Model performance comparison: Aquarium made it easy to compare the performance of different models through overviews of model performance metrics and drill-down visualizations of the exact scenarios where the model outputs differed.
- The entire machine learning workflow in one place: The Plus One team was able to find and diagnose model failures, address failures with proper data curation, and ensure that retrained models resolved those failures, all in Aquarium.
“With Aquarium, we are more confident that we’re consistently delivering good models.”
Dr. Dan Grollman, Senior Engineer
Impact
Aquarium significantly streamlined Plus One’s AI workflow by providing a fast and objective framework for iterating on their models. At the end of the day, Plus One saw:
- 25% absolute improvement in F1 score for a model used for container decanting, in addition to performance gains to a variety of other models. This produced impressive demos for new clients and assisted in retaining existing customers.
- Hundreds of hours saved for the machine learning team as compared to the previous spreadsheet-based workflow.
- Systems integration engineers empowered to take on the day-to-day heavy lifting of data curation and model improvement, freeing up machine learning engineers to focus on deep model research.
