Image for post
Image for post
Gerardo Sánchez | AI Consultant

Intelligent Retail Checkout With an Android App

Ever wanted an easier way to check out at the store? Read this article in order to look at an intelligent self-checkout system embedded in a mobile app.

Architecture Description

In the next figure, we provide a high-level functional description overview.

Image for post
Image for post
  1. Object detection with a neural network (YOLO V.3) to detect bottles, boxes, etc.
  2. The images of each object are extracted from the photo.
  3. Transfer learning is used from SqueezeNet to ConvNet to classify the product in each sub-image and to build an ID products list.
  4. Finally, the cost of the purchase is retrieved through a product prices list and all of the information in the Android mobile app referring the detected products is deployed.

Image Detection Through YOLO V3

You Only Look Once (YOLO) is an artificial neural network used for object detection. It is trained with the ImageNet 1000 class classification dataset in 160 epochs.

Image for post
Image for post

Transfer Learning for Classification of Objects

Transfer learning is a deep learning technique, which allows you to use pre-trained ConvNet models either as an initialization or as a fixed feature extractor for the task of interest. These models are trained with a very large dataset (e.g. ImageNet, which contains 1.2 million images with 1000 categories).

Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post

Backend on Python

The backend code is separated into two Python scripts (Detection.py and SmartRetail.py). Detection.py has all the predictions from each model (YOLO, SqueezeNet, and ConvNet) and SmartRetail.py is the service to construct the purchase ticket.

Mobile App

The mobile app is developed in Cordova. Cordova camera plugin is used to take a picture from the smartphone and the Cordova file transfer plugin to send the image to the server. To deploy the price list sent by the server, JQuery Mobile is used. The next image shows the mobile APP views.

Image for post
Image for post

Exponential intelligence for exponential companies

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store