RetailNet: A deep learning approach for people
counting and hot spots detection in retail stores

Valerio Nogueira Jr., Hugo Oliveira, Jose Augusto Silva, Thales Vieira and Krerley Oliveira
Federal University of Alagoas (UFAL), Maceio, AL, Brazil

This project was carried out by researchers and students from the Institute of Computing and the Institute of Mathematics of Federal University of Alagoas, Brazil.

Abstract: Customer behavior analysis is an essential issue for retailers, allowing for optimized store performance, enhanced customer experience, reduced operational costs, and consequently higher profitability. Nevertheless, not much attention has been given to computer vision approaches to automatically extract relevant information from images that could be of great value to retailers. In this paper, we present a low-cost deep learning approach to estimate the number of people in retail stores in real-time and to detect and visualize hot spots. For this purpose, only an inexpensive RGB camera, such as a surveillance camera, is required. To solve the people counting problem, we employ a supervised learning approach based on a Convolutional Neural Network (CNN) regression model. We also present a four channel image representation named RGBP image, composed of the conventional RGB image and an extra binary image P representing whether there is a visible person in each pixel of the image. To extract the latter information, we developed a foreground/background detection method that considers the peculiarities of people behavior in retail stores. The P image is also exploited to detect the hot spots of the store, which can later be visually analyzed. Several experiments were conducted to validate, evaluate and compare our approach using a dataset comprised of videos that were collected from a surveillance camera placed in a real shoe retail store. Results revealed that our approach is sufficiently robust to be used in real world situations and outperforms straightforward CNN approaches.

Our database and annotation tool is free to use for research purposes. Please cite our paper in your work:

@inproceedings{Nogueira2019,
author={Valerio Nogueira Jr. and Hugo Oliveira and Jose Augusto Silva and Thales Vieira and Krerley Oliveira},
title={RetailNet: A deep learning approach for people counting and hot spots detection in retail stores},
booktitle={2019 32nd SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI)},
year={2019},
pages={155-162},
ISSN={1530-1834},
organization={IEEE}
}

Acknowledgements: Alagoas Research Foundation – FAPEAL (Grant #60030 000421/2017), PIBITI/UFAL and PRMB Comercio e Distribuidora de Calcados LTDA.

For questions, please contact Prof. Thales Vieira.