The Associate-in-the-Loop subsystem ("AITL") uses human expertise to solve visual product queries on which the automated visual search subsystems have failed. AITL is invoked only with the explicit permission of the customer.
An Amazon customer uses the mobile app to submit a query image. The automated system fails to return a result set in which it has high confidence, the customer (optionally) forwards the request to an associate. The associate uses the Amazon product search and other web-based tools to return a result set to AITL, which then renders the result set to the customer.