Goodbye Fritz.ai, goodbye proprietary online services

Fritz.ai “sunsetting services”:

Fritz.ai is (was) a cool service for, in particular, making machine learning apps easily. The marketing hype was slick, and they roped in a lot of developers to help them push their service to many devs, particularly in the Android space, with paid articles on medium, and on their website.

First of all, I have to say, I take full responsibility for this, and I don’t blame anyone at Fritz.ai for my problems, it’s their service, they can do what they want – but I wasted a lot of time learning how to use their api, publishing my own (unpaid, no affiliation) posts about the service, and using it in apps published to the play store. Their business model seemed clear – free to use but once you reach a threshold you have to pay for the service. Seems legit, probably was legit.

My Android apps didn’t take off, so I never reached that threshold, never ended up paying for anything at all – but I did waste a lot of time making apps using a service which is getting switched off at the end of the month (August 2021). In other words, my apps are going to be switched off, for the (very) few users who did install them. If I want to continue with these apps, they will need a substantial re-write. In many ways I’m lucky that the apps were not successful!

So I have learned a valuable lesson here:

The lesson is: don’t trust your work to “flashy” or “new” products and services. I could have spent a week back then just learning the ins and outs of tensorflow on android, and done it myself. Just because I’m lazy doesn’t mean I’m stupid. Lesson learned.

*here is the full unedited email for those who are interested – I still don’t see any mention of this on their site or blog:

We’re Sunsetting Fritz AI

Service will end August 30, 2021

Hello,

This email is to let you know that we will be sunsetting the Fritz AI mobile machine learning platform, effective August 30, 2021.

What does this mean for my account?

As always, any custom trained models, datasets and annotations are exportable. Please see this documentation for how to export datasets and models.

After the sunsetting date, access to the Fritz AI SDK, webapp, API, and hosted services will be discontinued. All models, datasets, and resources stored in Fritz AI will be removed.

For mobile apps, the Fritz SDK will lose API connectivity, but on-device model inference should continue to function. Regardless, we advise you to update your apps, removing the Fritz SDK and models.

Snapchat lenses do not depend on the Fritz SDK nor API, and lenses should continue to function as before.

Why are we doing this?

Fritz AI’s mission has been to make the power of machine learning solutions available to mobile developers. When we started in 2017 there were few options available. In the following years there have been many great entries to the market such as ML Kit, Create ML, MediaPipe, Lobe.ai, MakeML, Firebase ML, and more. Now there is a mature ecosystem of tools, including free and open source options.

We understand that this decision will have significant consequences for our community, but we truly believe the wide range of incredible tools and resources out there will empower you to continue on without Fritz AI.

Sincerely,

Dan & Jameson
Founders, Fritz AI

Magic Poi Magic

SmartPoi is re-branding as MagicPoi

New stuff:

Check out http://magicpoi.circusscientist.com to sign in to the new experimental online portal. A place to upload and share poi images. I am working on a way to get the images directly onto your poi, straight from the server. If you have access to a linux terminal, you can test this using curl: curl http://magicpoi.circusscientist.com/api/get-filenames for a list of files in .bin format. Or just visit the link here to see all the files listed in a browser: http://magicpoi.circusscientist.com/api/get-filenames

To download a specific file you need to send a get request to the api like so: curl -X GET http://smartpoi.circusscientist.com/api/output/e5d2dd82008b343354a0f37ab435c32aab9dc132.jpg.bin -O -J -L $url – which will download the file and save in the current directory. Now that the api is set up, I have working poi code to get and display the images directly from the poi, so will be sharing this on GitHub once it is cleaned up a bit.

Oh and I finally managed to get the images to work right-side-up! So no more need to rotate 90 degrees before uploading.

coming soon:

There is a lot to do still, both on the server side and in the poi code. I recently switched over to PlatformIO, as the huge code base was getting a bit unwieldy for Arduino IDE. This is going to mean a huge re-write of everything, but will give me an opportunity to enhance things along the way, such as:

Built in web server on the poi, able to switch between Online and Offline modes, including access to the magicpoi.circusscientist portal (so you will be able to control and manage the poi operation from any browser) – I tried this already, the magicpoi site works in an <iframe>, and there are many open source libraries for the access point management.
Many utilities such as creating timelines, image generation, text to image etc on the magicpoi portal. Also a better user interface (looking at javascript gallery plugins, but first getting the basic functionality up and running)
Much easier poi sync and multi-poi control using mesh networking
Support for more poi sizes (72px is currently the default on the portal – or was it 80px?)
ESP32 support – this is inevitable, unfortunately my only ESP32 device broke, so this is on hold for now.
Many other hardware improvements being worked on (thanks Caleb!) such as accelerometer, ESP32, external flash, smaller embedded LED’s, battery management and more.

some notes:

The online portal is “alpha” quality, so please expect things to break. I have limited the image uploads to 10 per account, and the .bin files (for imaging on the poi) are all kept in one public folder for now, so please don’t upload anything personal or proprietary at this point. This is just for ease of use, as I haven’t set up security end-points for the poi – server connection. Passwords are hashed, however, so at least that much is secure (this means even I can’t see your password), and the server is kept up-to-date with security updates.

So what do you think about the Magic Poi project? So many people have emailed with stories and photo’s of their test builds, it’s hugely motivating, so thanks everyone, you know who you are! I need to do a blog post with some of the images I have received, many different versions of the poi being built and spun all over the world – will have to get permission from the individuals concerned first though.

It’s slow but steady progress, and I am getting more certain over time that Magic Poi could be a serious contender in the LED poi space eventually. At least in kit form, on Indigogo!

UPDATES:

Loading…

SmartPoi going online

Recently due to Covid my focus has shifted from performance to full time programming. I now code for a living, and have done for the past six months.

Since my work project involves server architecture I have come up with a new idea for SmartPoi – what if I could use the full power of the internet on the poi? The idea is to have an online server which the Android app can connect to and retrieve images, timelines, and visualisations. Each Android app could be associated with an account, with private and sharing functionality (so we could share images – a bit like on the visualpoi site).

The first part of this new functionality I am working on is a virtual poi imager (see https://circusscientist.com/Axel_Belhache_Demo/ for an idea of how this could look). This will have a way to upload your own images, and eventually you will be able to create an account and save images there. The long term goal is to connect the online server with the Android app, and make it easier to get new images, and share them. *UPDATE: The test page is now live! Try it out by uploading your own poi images here: magicpoicircusscientist.com. Currently limited to 10 images, and they need to be rotated 90 degrees before uploading.. I’m working on it!

As an added bonus, it is possible to send images to the poi straight from a browser (someone asked me about that recently on GitHub), so there could be a web app which would make SmartPoi usable without the Android app. Some functionality is not possible, though, with web alone – such as streaming images straight to the poi. For security reasons (DDOS) web browsers don’t allow UDP sending from the browser. Luckily SmartPoi backup images are uploaded using http requests. This means that standard poi things like offline images will be able to be loaded straight from the website onto the poi soon. Possibly timelines as well. I also hope to add re-sizing of images and text image generation to the list at some point.

Just as a side note – due to personal circumstances I am not able to do any hardware work for the rest of the year. The SmartPoi kit Indigogo launch will thus not be possible this year. Coding work is still very much in progress (I recently updated the app to work on Android version >9. )

As always, if you want to be first to hear the latest news about SmartPoi, you can sign up using the form below. Many thanks to those of you who emailed me with images of your own SmartPoi in action, it’s great to know that my contributions are inspiring people to make their own. We are just getting started. Loads more to come!

UPDATES:

Loading…

SmartPoi Update

There have been a number of changes to the SmartPoi project in the last year. Due to issues I have had while spinning them at gigs I decided to make some much needed changes. This is a major upgrade

Changed to binary format to save images (in LittleFS file system) – this is so much smaller than the previous text based format, I can now fit 100 offline images onto the small poi and 50 images onto the large poi. The maximum image size is larger. Also, all images can be displayed one after the other. And the timing can be changed for offline images as well.
Added setup code to the Android app – no longer do I need to run a separate program to set up the test images, the Smart Poi Demo app now creates the correct files and folders*
*unfortunately, Google in their wisdom are removing file system access slowly in newer versions of Android. This means that my code is going to have to be updated once again (angry face emoji)
Uploading offline images to the poi is simple and available straight from the Android app. An upload image button uploads all of the images at once (one folder at a time only for now)

The Future of Smart Poi

I am planning the launch of a kit version, hopefully this will be possible some time during 2021(*Update: more like 2022). The design will be modular, with the battery, circuit, and LED’s all coming separate. I have a friend who is a qualified designer who will be helping with this part. The idea is to be able to have a ‘snap-on’ LED strip, so that more pixels can be added at a much lower cost than if I was making a whole new set of poi. The same goes for the battery, and even the main circuit (ESP32 anyone?). If I can do it right, it will mean that upgrades will be cheaper and also easier.

The new poi will be known as “Magic Poi”. Future Magic Poi Website here (It’s a fun Visual Poi Comparison site for now..)

Try out the current code:

– Smart Poi Overview page… may be a bit outdated now. Things are changing fast. I think you will agree that it’s improving with time. The future of Visual Poi is here, and it’s modular, open source, and streams over WiFi!

Sign up to hear news and updates:

Loading…

Monkey Detector Circuit and App

It’s been a while since I was so excited about a project. After much struggle (to do with a very hot voltage regulator) I have come up with a working circuit for the hosepipe tap trigger.

I was hoping to power the whole thing from 12 volts but the voltage regulator was getting a bit hot, and my main directive for this whole project is to not spend any money (otherwise I would have a relay shield like this one instead of the bare 5v relay which I had to trigger with a transistor)…

The other exciting thing was the Android app. I decided to create a stand-alone app which recognizes monkeys to put on the play store. Right now it plays a loud siren sound when a monkey – or baboon – is detected. I can’t even count how many times it would have been useful to have a simple alert which would warn me about vervets attempting to break in on the other side of the house.

Don’t get me wrong, I love the monkeys. They are really interesting to watch, their kids play in our yard, they do tight-rope on the telephone wires, and the troupes have all-out warfare sometimes on the road. The other day we found out that our neighbor’s dog has a pact with the monkeys: they open the bin for him and he helps them eat the contents. Just some monkeys and a dog getting along just fine!

So the Android app is pending review, it’s called Monkey Detector, I will add a link to it when available. My third play store app!

Update: Monkey Detector on Google Play Store – unfortunately some devices are having a problem where the app isn’t playing the alert sound. I am looking into this (it works on my phone…)

Mobile Monkey Detector

It’s amazing how machine learning has improved over the last few years. I was dabbling with TensorFlow for a while, then moved on to look at pyCharm, but ultimately I was going to go with the easy-to-use Google image recognition API for the mobile version of my monkey detector.

Then a few days ago I stumbled upon fritz.ai and their even easier-to-use pre-trained mobile classifier. Unfortunately the example was written in Kotlin, but anyone developing an Android app today should know that, so I gave it a go. Here is the tutorial on Medium (with link to Kotlin code on GitHub) https://heartbeat.fritz.ai/image-labeling-on-android-in-kotlin-using-fritz-ai-and-camerax-1466089b2e34

The current version of the app (detects monkeys but doesn’t send a signal to my WiFi connected sprinkler system yet) is here: circusscientist.com/seeMonkey.apk – now available on the Google Play Store!
In order to use it you need to give camera permissions before first run, otherwise the app will force close. Just point it at a monkey and get alerted* (pictures of monkeys work fine)

*If you are wondering what happens, my 6 year old son shouts “Hey you, get away from my lunch!” when a monkey is detected.

The next step is to adapt my Arduino controlled tap valve (more on this to come soon) to receive WiFi (using ESP8266 of course) and then send the signal from the app (in Kotlin…!)

I can’t wait to make the demo video, those monkeys are trying to get into the bin (and our house) every day, and they hate getting squirted.

SmartPoi Code Dump!

2020 is the future! In the future I want to have many projects complete and be working on something new. So here is the everything I have been working on for the past few years, all in one go. Is this complete? You tell me!

I have uploaded the code to github, and some programs to my site, all is explained, although possibly in a rather rushed way.

Significant things which I have finally open-sourced (as promised) are the Android App (for streaming images and patterns to the poi), the full firmware for ESP8266, the circuit and PCB I used, CAD files for 3d printing and more.

Still to come is the code for uploading the backup images to the poi (an old windows .exe is available for now).

To read about how all of these programs work together check out the SmartPoi menu option above, which include the following 3 new pages:

https://www.circusscientist.com/smart-poi-android-app/
– description and code for android controller app
https://www.circusscientist.com/smartpoi-full-circuit/
– PCB, breadboard, parts and CAD files to make the hardware
https://www.circusscientist.com/smartpoi-firmware/
– ESP8266 code and description

If anyone is interested in making their own streaming poi all of the parts are linked.

Affiliate links are included on this website. Please support my future projects! Another way to support my work, and help yourself to free stuff at the same time, is to check out DigitalOcean with their promotional 2 months free hosting. If you haven’t yet heard of them, they are the best and cheapest virtual server on the market right now, use the link!

If you are trying to make the poi please consider signing up to the mailing list, I am emailing updates to quite a few interested parties right now. No spam, just updates. In the last two years there have been only four emails, so not a whole lot…

UPDATES:

Loading…

Stripboard Sucks

I have been trying to make my poi easier for anyone to duplicate by eliminating the costly custom circuit board. So I tried to make it on stripboard, which was a disaster. Here is the idea:

And here is how it ends up looking on the stripboard (with tilt switch):

So that looks great and all but in practice, having to cut the stripboard and solder and everything else, there are too many possibilities for shorts and troubleshooting it was turning out to be a nightmare. There are 18 solder points on the board above, not to mention cutting the metal on the stripboard (under the esp-01)

Here is another idea, I am trying to make this simple, the D1 mini costs just $1 more and can run on 5v which is the right voltage for the LED strip anyway. This is the new circuit:

Now we have only 9 points to solder (not including the LED Strip, I didn’t include it above either).

I also re-imagined my poi outer shell, to give access to the usb programming port on the D1 Mini, so this will help with development as well.

Looking forward to putting this all together, and updating my tutorial series (if it all works as planned)

Smart Poi Upgrade

So, my Smart Poi are battery powered, of course. I chose AAA NIMH batteries in a pack of 4 for this originally as this provides around 5 volts of power for the LED strip, and I have a voltage regulator to turn that into 3.3 volts for the ESP8266. Originally I got the batteries with tags on and had them soldered together into a battery pack (by a pro) and added a jack input for the charger. Nothing lasts forever, and after almost two years of use I find my battery packs are degraded to the point that they don’t power the poi anymore.

The simple solution would be to get some more packs made and simply replace the old ones, however I have found myself in situations where the one hour (big poi) or two hours (small poi) is simply not enough time. While they are charging the poi are not earning me money, so the plan now is to take the same batteries and make them replaceable. I found a cheap spring loaded plastic battery holder which fits inside the poi housing, so a simple jtag connector (molex whatever you want to call it, I am getting the cheapest one) will give me hours more poi spinning time, which means more money of course!

In addition, I am going to be adding a tilt switch to the LED strip power line, so while the poi are held upside down the strip will be off, but the controller will still be on, and connected to the Android app. This will save power significantly during down time (walking from one area to another for example).

Simple improvements, but with a major impact on my performance.

Now to get onto improving the code I promised to release as open source! A hard task but it’s coming.

Progress with Monkey Detector

So I have been working on the vervet monkey detector idea for a little while now. So far I have had some success with the back end. Here is an update on how it works so far. (tldr I used Google Cloud Vision api and some bash scripts)

1. I take a photo of the monkey (I still need to automate this, with motion detection)
2. Upload to my server using scp
3. Some moving around on the server to make the image public for Google Cloud Vision to see it – using inotifywait and mv
4. Google cloud vision api call with image url using curl
5. To do: parse json response from google to check if a monkey is actually there.
6. Send a signal to wireless valve (switches on the sprinkler and hopefully gives the monkeys a fright) – I have built one already, just need to modify the code slightly for wireless. Thinking of using esp8266 for this, since I have so many lying around, and a 12v battery since the tap valve solenoid runs off of 12v. I have a nice circuit board I built for my wireless poi which incorporates an ESP-01 and a voltage regulator. Add in a relay to switch 12v and it’s done.

So there is much to do. There is also the question of accuracy, obviously I don’t want any false positives wetting any unwary visitors, so I need to test this a lot before using it. The first version will trigger an alarm only, which in itself could be useful as a phone app, for example. We might be able to leave the windows open on occasion, with a monkey detector to alert us when one tries to get in and steal our food (this happened twice in the last week)

At some point I will have to make my own machine learning classifier for monkey images, as Google charges $1.50 currently per 1000 images classified. The first 1000 are free, but they have a habit of changing the pricing on their products without notice.