Category: SmartPoi

The deal with PlatformIO and ESP32 Arduino

I recently purchased a new board, the ESP32S3 Super Mini. That’s an S3 dual core version of ESP32 on a tiny yet powerful board.

The problem

The issue was that this board wasn’t working right – I had my code all set up (the new “Alpha” version of Magic Poi firmware – not published) and parts of it ran great for testing in Arduino IDE but as soon as I ported to PlatformIO it wasn’t compiling.

The real problem

It turns out that ESP32 S3 Super Mini and many newer ESP32 boards are simply not supported any more in PlatformIO. In fact, anything that relies on Espressif Arduino framework > version 4 is out of luck – as far as I can tell the owner of PlatformIO had a falling out with Espressif (business? personal? no reason at all?) and now they don’t support the new versions which work great on Arduino by the way (now version 6.something) Here is the issue on Espressif arduino-esp32 github: https://github.com/espressif/arduino-esp32/pull/8606#issuecomment-1805781410 for reference.

PlatformIO has their own version of arduino-esp32 called platform-espressif32. Here is a long discussion about the dropping of support on their github: https://github.com/platformio/platform-espressif32/issues/1225 for reference.

After reading more discussions on the PlatformIO github, Reddit and other places, I finally stumbled on this thread https://github.com/platformio/platform-espressif32/issues/1435 which contained:

The Solution

A very smart and intrepid PlatformIO user, Jason2866 made a fork of the PlatformIO Arduino Espressif base and put it here, with instructions on how to use it – and he updated it on his own to use the latest arduino-esp32. After some light editing and moving stuff around (including accidentally putting GND into +5v and vice versa – thank you Super Mini board for not blowing up) everything is now working. The Magic Poi project moves forward!

Magic Poi Lite Firmware (Definitive Version)

Ever had your computer crash and had to re-install the OS, then restore all of your projects one by one, only to forget about an important file and re-do all of the work?

I did the test firmware for Magic Poi Lite 7 months and one Laptop crash ago. Today I went to upload the firmware I worked on all last week and it turns out I already have a GitHub repo set up – for the same project – which I forgot about.

Turns out the new version is better though, so here it is: https://devsoft-co-za.github.io/magic-poi-lite/ – made in PlatformIO, now with bonus Documentation!

In case you didn’t already know, Magic Poi Lite is the single colour version of Magic Poi – WiFi enabled, internet connected flashy lights poi.

SmartPoi js utilities update

Hi Smart Poi enthusiasts!

I have been working on more stability and functionality for the offline web utilities for Smart Poi. (see the code here: https://github.com/tomjuggler/SmartPoi-js-utilities)

Check out the video walkthrough here:

If you want to try it out, make sure to get the latest Smart Poi firmware in order for this to work: https://github.com/tomjuggler/SmartPoi-Firmware

SmartPoi File List

Just a quick one to let you know that SmartPoi has a server (running on the lowly ESP8266) which means I can do stuff like this:

The html to do this is now on GitHub here: https://github.com/tomjuggler/SmartPoi-js-utilities – usage is in the README.

Basically it fetches a list of files from the poi and then fetches each compressed binary file in the list, decompresses them and displays the resulting images. Up until now the only way to see what was actually on the poi was to spin them, so this is definitely an improvement. This is all done using the SmartPoi AP – no internet access required!

Any day now* I will be upgrading my poi to work with ESP32 but until then I’m still trying to make the ones I use in my shows better!

*I am aware I have been saying this for a while now 🙂

SmartPoi – new Web Software to upload images.

I wrote some new code, so now you can upload offline images from a web browser! Finally the (old) poi can take advantage of the hack I did to have up to 52 offline images displaying.

First, install the latest SmartPoi firmware (otherwise the web upload won’t work) https://github.com/tomjuggler/SmartPoi-Firmware

Then clone this repository here https://github.com/tomjuggler/SmartPoi-js-utilities to your computer and follow the instructions in the README. It all runs in a web browser, nothing to install or download (currently tested working on 36px poi using Firefox browser).

Thanks to Andreas for giving me the reason to make this – look out for updates to the SmartPoi project, more control from a web browser coming soon. I think I already mentioned this in my previous blog post – I am a bit tired of fighting with Android!

SmartPoi have a hidden feature I didn’t tell anyone about yet

So far I haven’t gotten my new ESP32 based version of SmartPoi hardware working (re-named MagicPoi) – make no mistake, it’s coming but due to various reasons this keeps getting delayed. *ESP32 dual core is a game changer, that’s all I’m going to say here.

My ESP8266 based SmartPoi 36 and 72px poi are still going strong, on their third battery (now Lithium). So I got bored recently and instead of working on the new poi I had another look at SmartPoi Arduino code and remembered the “Hidden Feature” – Router Mode!

Activating Router Mode

First of all, make sure you have the latest firmware. This is because the older versions may not include the fail-safe I built in. Remember that “Main” and “Auxillary” poi – and number of pixels – need to be set in code, before uploading. So in the SmartPoi app there is a menu entry called “Settings” – with Pattern and Channel fields as well as Router settings. Fill these in, using your own router settings to guide you. My address is 192.168.8.78 with Address2 79 but your router may be 192.168.1.xx or something completely different – the last numbers are for the fixed IP on your network, please consult the router options for available addresses on your own network.

Once you press “Send Request”, nothing happens. That is for a reason, it was an experimental feature and I didn’t want to brick anyone’s poi if they accidentally put the wrong Router or Password.. In order to activate the setting you have to go to a browser (on a device connected to the poi Access Point) and put in http://192.168.1.78/router?router=1 and then http://192.168.1.1/router?router=1

First the “Auxillary” poi (78), then the “Main” poi (1) please or it won’t work! At your own risk! You may need to try again if it doesn’t work.

What this does is set the poi to start up 50% of the time in STA mode – connected to your router with the static IP address you configured with “Settings”. Every time you switch the poi off and on again it changes, AP mode then STA mode and back again. Make sure to keep the 2 poi in sync, that is on the same startup mode as each other!

Both Poi will show a green . . . . . indicator when attempting to connect and fast green ……. when connected. If you see green dots going up slowly for ages it means it’s not connecting and the Router and Password need to be checked (in AP mode with the app)

What is this for?

AP mode is simple – your phone connects to the poi direct, no router needed. I have noticed, however that the AP signal is pretty weak on the ESP-01 so using a router could help reliability.

Also, ESP-01 AP mode can only connect 4 devices at a time – including the phone, so that’s 2 pairs of poi and controller max. With a router connection you could have a lot more.

Finally the Router connects to the internet. This is helpful because I am moving all of the functionality to the cloud (sorry) mainly due to being sick of Google changing things for Android developers all of the time – breaking things really. It’s too frustrating so I’m moving everything to the cloud. The Offline mode remains the same.

My SmartPoi demo app doesn’t work anymore!?

Yes I haven’t had a chance to do anything with the Router Mode yet – the Android code doesn’t know about your new STA mode IP address. You can use the C code from the circusscientist site here with the new IP address for example: https://www.circusscientist.com/pov-wifi-streaming-with-c/

To switch off the Router option just do the following in a browser connected to the poi AP (poi in AP mode so not connected to your router) http://192.168.1.78/router?router=0 and then http://192.168.1.1/router?router=0

The SmartPoi Router option will soon be accessible from the main “MagicPoi” site, with downloads of your offline images (72px only) direct from the browser, control of functions, and more!

Image Classification for Magic Poi images using Machine Learning

The title says it all. Here are a couple of video’s to demonstrate the process:

Testing the version 5 ML model accuracy:

Testing the image recognition model

Deployed using temporary database:

I added a filter for different categories found by the machine learing model

Todo:

  • New images uploaded are not categorized – because I did the classification at home and just uploaded the database. I still need to get the image recognition to work on the server.
  • Scrolling is really not great, gets stuck sometimes and also new images loaded aren’t filtered.
  • Re-categorizing support – and support for more than one category per image
  • Remove bad images (or just hide them).
  • Refine the model to do better (“text-img” recognition isn’t great, for example).
  • Add more categories.

The categories are live! Check out http://magicpoi.circusscientist.com (and sign up for the good stuff).

Sign up for updates:

UPDATES:

Sign up for our update alerts:

bin files and Magic Poi

I recently received a question via email which I thought might need a bit of answering – “Hello, may I ask how to convert the image into a bin file? I couldn’t find the source code on the website and look forward to your reply. Thank you very much.” Here is my response:

(tl:dr I didn’t share source yet but there are two ways to do it)

Great question! I haven’t been working on the file access much yet for the new MagicPoi. The hardware and getting the binary files is coming second to having an easy way to manage images – which I am spending most of my time on right now.

The Android app had a bug with creating .bin files which I never managed to fix, so I did update the “Smartpoi Android App” page to link to two PC programs which you can download – see the page here: https://www.circusscientist.com/smart-poi-android-app/ – just scroll to the bottom to see the (Linux Only) versions. These are easy-to-use drag-and-drop apps which will convert the image files to .bin format and also upload wirelessly to any connected poi. The .bin files are stored temporarily in a folder in the program files, I am sure you can find them there once you try it. I didn’t share the source because frankly the code is a terrible mess which even I am embarrassed about #messyprogrammer – I am getting better at this by the way, look forward to some documentation coming soon!

Currently the magicpoi website api is limited to returning 10 image names. You can see this by typing this into the browser url bar:
http://magicpoi.circusscientist.com/api/get-filenames

To download one of the .bin files listed, you can use the filename(xxx.jpg.bin), also in the browser bar – eg:
http://magicpoi.circusscientist.com/api/output/ajzs2w1aczeq.jpg.bin

Incidentally the example PlatformIO sketch linked from the “api” section here https://www.circusscientist.com/magic-poi-api/ does exactly this, getting the list of 10 files and then using the /api/output endpoint to download .bin files – to the D1 Mini in this case – and then displaying them in 72px. I would have to check but I think maybe all .bin files are compressed to 72px.. yes I checked – currently that’s the default, with option to support any size (thank goodness).

All files which are uploaded using the magicpoi.circusscientist.com site are automatically converted to .bin files. The .bin files are simply compressed image files – with one byte representing one pixel in R3G3B2 format (in the case of MagicPoi I think I rotate them 90 degrees also for the line by line scanning).

To get any image .bin file from the MagicPoi site at the moment you can use a quick hack – if it is an image you uploaded yourself you have to “share” it for this to work. Then scroll down to where the original image is visible (on the “profile” page) – not the rotated version! Right click and select “Open image in new tab” or just “copy url” – then go to the new tab and copy the image name. Now use the name with “.bin” added on for the download api:
http://magicpoi.circusscientist.com/api/output/ajzs2w1aczeq.jpg.bin

I’m sorry I haven’t really been working on the poi hardware and that side of the project. Like I said, I’m focusing right now on front end usability for the web interface. Soon there will be categories for the images, for example, and sorting.

Once the interface is working I will return to hardware and firmware – and also documentation and new features as well as proper api accessibility.

I hope my response answered your question. Everything is a work in progress, but there is progress.

Regards

Tom