Image Classification for Magic Poi images using Machine Learning

The title says it all. Here are a couple of video’s to demonstrate the process:

Testing the version 5 ML model accuracy:

Testing the image recognition model

Deployed using temporary database:

I added a filter for different categories found by the machine learing model

Todo:

  • New images uploaded are not categorized – because I did the classification at home and just uploaded the database. I still need to get the image recognition to work on the server.
  • Scrolling is really not great, gets stuck sometimes and also new images loaded aren’t filtered.
  • Re-categorizing support – and support for more than one category per image
  • Remove bad images (or just hide them).
  • Refine the model to do better (“text-img” recognition isn’t great, for example).
  • Add more categories.

The categories are live! Check out http://magicpoi.circusscientist.com (and sign up for the good stuff).

Sign up for updates:

UPDATES:

Sign up for our update alerts:

bin files and Magic Poi

I recently received a question via email which I thought might need a bit of answering – “Hello, may I ask how to convert the image into a bin file? I couldn’t find the source code on the website and look forward to your reply. Thank you very much.” Here is my response:

(tl:dr I didn’t share source yet but there are two ways to do it)

Great question! I haven’t been working on the file access much yet for the new MagicPoi. The hardware and getting the binary files is coming second to having an easy way to manage images – which I am spending most of my time on right now.

The Android app had a bug with creating .bin files which I never managed to fix, so I did update the “Smartpoi Android App” page to link to two PC programs which you can download – see the page here: https://www.circusscientist.com/smart-poi-android-app/ – just scroll to the bottom to see the (Linux Only) versions. These are easy-to-use drag-and-drop apps which will convert the image files to .bin format and also upload wirelessly to any connected poi. The .bin files are stored temporarily in a folder in the program files, I am sure you can find them there once you try it. I didn’t share the source because frankly the code is a terrible mess which even I am embarrassed about #messyprogrammer – I am getting better at this by the way, look forward to some documentation coming soon!

Currently the magicpoi website api is limited to returning 10 image names. You can see this by typing this into the browser url bar:
http://magicpoi.circusscientist.com/api/get-filenames

To download one of the .bin files listed, you can use the filename(xxx.jpg.bin), also in the browser bar – eg:
http://magicpoi.circusscientist.com/api/output/ajzs2w1aczeq.jpg.bin

Incidentally the example PlatformIO sketch linked from the “api” section here https://www.circusscientist.com/magic-poi-api/ does exactly this, getting the list of 10 files and then using the /api/output endpoint to download .bin files – to the D1 Mini in this case – and then displaying them in 72px. I would have to check but I think maybe all .bin files are compressed to 72px.. yes I checked – currently that’s the default, with option to support any size (thank goodness).

All files which are uploaded using the magicpoi.circusscientist.com site are automatically converted to .bin files. The .bin files are simply compressed image files – with one byte representing one pixel in R3G3B2 format (in the case of MagicPoi I think I rotate them 90 degrees also for the line by line scanning).

To get any image .bin file from the MagicPoi site at the moment you can use a quick hack – if it is an image you uploaded yourself you have to “share” it for this to work. Then scroll down to where the original image is visible (on the “profile” page) – not the rotated version! Right click and select “Open image in new tab” or just “copy url” – then go to the new tab and copy the image name. Now use the name with “.bin” added on for the download api:
http://magicpoi.circusscientist.com/api/output/ajzs2w1aczeq.jpg.bin

I’m sorry I haven’t really been working on the poi hardware and that side of the project. Like I said, I’m focusing right now on front end usability for the web interface. Soon there will be categories for the images, for example, and sorting.

Once the interface is working I will return to hardware and firmware – and also documentation and new features as well as proper api accessibility.

I hope my response answered your question. Everything is a work in progress, but there is progress.

Regards

Tom