DesignSpark Electrical Logolinkedin
Menu Search
Ask a Question

Student Innovation - Object recognition in shops

My name is Papa Valentin and I will soon finish my first year studying Computer Science and Electronics. My project is not something new, I'm just surprised that this doesn't already exist in shops, so I tried to do it myself to figure out why.

Parts List

  • 1 × Any webcam with HD resolution

The aim is to reduce the time staying in queues and the staff needed in order to manage a shop as well. Before I went to college, I was living in a small town, and time lost in queues was never a problem, but since I have temporarily moved to a bigger city, this has turned around. Sometimes I even had to wait 30 mins and I am sure that many others can relate.

This project was done using Tensorflow and as I had very little experience using it, I used this video to get started with the recognition part.

The next step was to make a simulation of the summary window that would be presented to you after the items are processed. Finally, I connected the program to a database that contains a few products with the associated prices that should represent a shop's inventory. I used the database in order to come closer to how a shop would actually use this program, as changing many variables in the program when adding new products or making changes to the existing ones is inconvenient. The aspect of the interface is not polished yet because I was much more interested in getting it to work properly. The next step would be to improve it and try to simulate a bigger inventory to see if there could appear memory problems while training the classes. 

If you want to try out the program, here is a bitbucket link to all the required files. Firstly, watch the youtube video from above to see how you can choose what objects you want to be able to detect. Next, you have to run the from models\research\object_detection. The laptop camera or a USB camera attached to the laptop should pop on the screen. Place the objects in front of the camera and press SPACE. It will take a photo for you named "opencv_frame_0" which will be saved in the same folder and then it will be processed. The interactive menu should be easy to handle. Close the program by pressing ESC, after the menu disappears. If the program freezes, just restart it. It could happen if there is nothing detected in the picture. This is just an imperfection of the code that I will take care of if the project is extended.

I think that this kind of system will work properly in a real environment, as I only used a few photos (around 30) for the training of each of the classes, which were taken by the phone. In contrast, when the program runs, the photos that will be processed are taken with the help of the laptop camera, which has lower quality and still recognizes the objects. Therefore, if for each class there would be more training photos allocated and these photos would be taken by the same camera that will be used to run the program, a very reliable system would be created.

This video provides an example:

This screenshot shows how the system should look:



I'm a Computer Science and Electronics student and I like making projects in these fields.