About Zesty.ai
Zesty.ai offers insurers and real estate companies access to precise intelligence about every property in North America. The company uses AI, including computer vision, to build a digital twin for every building in North America, encompassing 200B property insights accounting for all details that could impact a property’s value and associated risks, including the potential impact of natural disasters.
Goals
Zesty’s AI system trains on massive datasets that consist of aerial imagery, weather, and real estate data to generate accurate assessments of climate risk.
In particular, Zesty develops a suite of machine learning models that operate on multispectral imagery and lidar data collected from airborne and satellite platforms. Zesty’s models can segment out individual properties and identify attributes of properties like roof geometry, building material, and presence of overhanging vegetation.
Improvements in the accuracy of Zesty’s machine learning models deliver better data streams and risk assessments to their insurance customers. Zesty’s machine learning team chose to partner with Aquarium to accelerate their development cycle and optimize performance of their models.
Using Aquarium
Zesty’s machine learning team understood that the key to improving their models was to iterate on the datasets the models were trained on, and Aquarium provided key features that accelerated Zesty’s machine learning workflows:
- Model performance analysis: Aquarium made it easy to examine disagreements between model inferences and human labels. This allowed team members to diagnose failures, categorize them into different segments, and monitor performance improvement over time.
- Dataset visualization with embeddings: With Aquarium’s embedding visualizations of the dataset, the Zesty team was able to quickly spot outliers in their dataset without having to manually scroll through thousands upon thousands of datapoints.
- Dataset similarity search: Aquarium’s embedding-based similarity search made it easy to find more examples of difficult edge cases where the model struggled. Zesty’s team used similarity search to trace errors in their test sets back to issues in the training datasets. Additionally, Aquarium also enabled Zesty to search through their unlabeled datasets and find more examples of these edge cases to label and retrain their model on.
- Collaboration with team members: Aquarium’s cloud-based interface made it easy for Zesty’s scientists to collaborate with other teammates on machine learning projects. The team could easily share datapoints, comment on issues, and tag one another within Aquarium.
- Labeling integration: After identifying potential opportunities to increase accuracy of their datasets and models, Aquarium made it easy for Zesty’s team to get the right data labeled to improve their model performance.
"With Aquarium, we've been able to easily identify flaws in our data and correct them to enhance model performance"
Megil Gallant, ML Data Scientist
Impact
Using Aquarium, Zesty identified opportunities to improve model performance, curated their datasets, and trained new models that performed significantly better. Aquarium helped increase Zesty’s return on time investment by improving their ability to analyze and improve the most influential factor on a model's performance - the data.
- The team’s speed of development improved considerably. Zesty saved hundreds of hours of human and machine time compared to their previous workflow. Using Aquarium, a new hire was able to ship an improved roof material classification model within a month of starting.
- Significant model performance improvement for a skylight detection model, in addition to performance improvements on multiple other tasks and models.
- Aquarium’s efficient data curation workflow freed up ML engineers to focus on higher value tasks. Instead of building in-house tools and writing custom code in Jupyter notebooks for data curation, the ML engineering team can now focus on improving model code and infrastructure.
“Aquarium helps our team quickly review our datasets, find edge cases and issues, then take the right steps to address them. Aquarium has been a huge time saver for our team.”
Michael Ulin, Vice President, Data Science & Machine Learning
