Amazon

The Associate-in-the-Loop subsystem ("AITL") uses human expertise to solve visual product queries on which the automated visual search subsystems have failed. AITL is invoked only with the explicit permission of the customer.

An Amazon customer uses the mobile app to submit a query image. The automated system fails to return a result set in which it has high confidence. The customer (optionally) forwards the request to an associate. The associate uses the Amazon product search and other web-based tools to return a result set to AITL, which then renders the result set to the customer.

Main goals of AITL are to earn and maintain Amazon customers' trust that their image queries are safe and to deliver reliably relevant results. Working closely with the Program Managers, Product Managers, and engineering team, I created UX flows and a working prototype for the initial product launch.