Category: Flask

Moving On

TLDR:

Some old Circus Scientist services have to go, in order to move forward with new projects. This is due to Ubuntu Server upgrade (not intentional)

Outline:

There are two ways to look at things in software:

  1. Keep things compatible and working no matter what. I think of this as the “hoarder” mentality (think Microsoft)
  2. Move fast and break things (aka Google)


I guess I try to keep a happy medium, leaning towards the “hoarder” side. Right now though, something is going on that I don’t have any power to change. The Ubuntu Server version 20 which I have most of my “experimental” services on is going EOL in a few months.
I don’t have a choice – I need to upgrade all of my online projects. Sometimes this is as simple as upgrading Ubuntu in-place but due to Python version updates this time it’s impossible. Each service has to be updated and put on a new version of the server – I’m using the latest LTS version, Ubuntu Server 2024.

What is going away

  1. LED Website Indicator. I really love this project, my first fully integrated Web and IOT project. Basically nobody cared except me. I’m going to open source the Arduino code at some point, some parts are already are available on GitHub.
  2. MagicPoi old website. There is already a new version with most of the same functionality (my patreon subscribers have had access to this for a while). This was my first really large Flask website with an api accessible from ESP8266 and is a bit old and insecure now. I have a backup if anyone has data from there that they need to get back, just send me an email and I will sort you out.
  3. AI Site Generation. A fun project that was just for learning purposes really.
  4. Monkey Detector (web version with api). Another learning project, not being used.
  5. K8 Juggling. Very old fun service.
  6. Invisible Deck Practiser. Just a JavaScript version of an Android app (magic).
  7. Other personal sites not related to this blog.

Temporary disruptions

Several other services need to be migrated, this may take up to a week depending on the complexity. Expect disruptions.

  • SmartPoi Downloader
  • The new alpha version of magicpoi site (with api)
  • devsoft.co.za
  • show.circusscientist.com
  • Other personal sites not related to this blog.

Conclusion

Sometimes you have to break things in order to move forward.

Everything that is moving to the new server is going to have proper Python virtual environments with up-to-date libraries. More secure, faster, future proof.

Boost Your Online Reach: How Multiple Related Websites Can Grow Your Business

Use your industry specific knowledge to create a related website and fill it with your own ADS. Create your own referrals with much less cost than paying for ads. I could go on, but let me start with

An Example

My own business is in the Entertainment Industry. We do high value entertainment which can run to thousands of rands invoiced for a single booking, and our customers often convert to returning customers (we get 50% returning customers on average) – so a single lead can be very valuable, long term. That is why we advertise on Google.

Many years ago we set up a page on our main site, which was in response to all of the enquiries regarding Party Venues. This was a simple service to our customers with our recommendations regarding our favorite venues in the Durban area. This quickly became the most popular page on our site, according to Analytics. Soon after I decided to branch out and create a full website using the initial list – and now this site durbanpartyvenues.co.za is getting up to 2000 visitors per month. We get 10-20 referrals from this site which converts to an estimated extra R1000-R10000 per month (guesstimate according to converstion statistics).

Since we were already paying for a web server the actual extra cost is R100 per year for the domain fee (!). Of course I spent a lot of time on the website development – first in WordPress and later Flask (Python) which made it easier to add new venues using a simple spreadsheet which makes it easier for my non-technical partner to do it.

Expanding

Last year we added capetownpartyvenues.co.za since we occasionally do entertainment in that city. This year I am working on another related services website, durbanpartyhub.co.za which will cover related entertainment services. This time we are planning on allowing our friends in the entertainment industry to pay a small fee for preferential placement (once the site gets traction).

Does this apply to your industry?

I am sure you can think of related products and services that you have inside knowledge of which can

  1. Provide informative content for potential customers
  2. Be made into a website which you create and control with your own brand front and foremost

Shameless plug

The years I spent refining this “Related Services” template has given me an unique insight into efficiently building out an effective marketing vehicle for my company – while at the same time offering a valuable service for our customers. At DevSoft, we use AI development tools to speed up, refine and enhance the website creation process. Perhaps the ideas in this blog make sense to you? If so, get in touch!

Changing times

I recently read an article on ZDNet which was talking about the new ChatGPT search and how Google Ads is going to be less relevant in future. The takeaway from that is that your organic web presence is going to be even more relevant, if paying for search placement becomes less effective. I did a test with the new ChatGPT search and “Magician for Kids parties in Durban” and “Top 10 Outdoor Party Venues in Durban” both return sites that I own at the top, organically – thankfully due to our SEO.

The article didn’t mention it in but I think social media pages (Facebook, Instagram, etc) are going to be even more important in the future – but you don’t have control over those. Owning your own content and related content is the most important thing in my opinion.

PS: check out the AI Enhanced super commercial version of this blog post here: https://devsoft.co.za/blog

MQTT port 1883 – access denied!

Well things are going really well with the LED Website Indicator project. I re-wrote the firmware, this time using the amazing Autoconnect library (UI for managing MQTT broker and WiFi) and PubSubClient.

The new D1 mini and shield have arrived at the factory – along with a fancy new 3d print cover, so we uploaded the firmware and reached out to friends to test the WordPress plugin – in order to see some flashing lights. No lights. One friend even had his managed WordPress install locked (he wasn’t allowed to install new plugins, some sort of restricted mode – luckily a call to customer support resolved the issue quickly).

After a lot of troubleshooting I finally worked out the issue (on two different servers so far): the admins have locked down outgoing messages and closed most ports – including the one I’m using in the LED-SITE-INDICATOR WordPress plugin, port 1883. Long story short, we can’t have outgoing MQTT messages on all servers (it works on mine though).

So I had to re-write the PHP code as well, removing the MQTT and replacing with http call to an api – which then does the MQTT stuff, on my own server which is allowed.

Conclusion

Check out the LED Website Indicator project – launching any day now!

I got banned from Ngrok after 10 requests!

Reverse Proxy

So in order to test a new payment provider for a site I was looking around for a quick and easy reverse proxy. I didn’t have to look far as Ngrok is clearly the market leader, with a free tier to boot. They also have a convenient Python library called pyngrok for integration with Flask.

Secure Payment

So the payment provider has an easy system where you generate a form and fill it with details (customer, order details, etc) which once posted redirects to their payment processing page – so your standard “Pay Now” procedure. My error in conception was I ignored the docs and thought that this could be handled through the back end (for “security”) so I sent the form from Flask and returned the “Pay Now” page to the front end – also through Flask.

What went wrong?

In hindsight it seems fairly obvious to me that this is exactly how a bad guy would behave – only changing the html along the way. Ngrok service, which my requests and responses was routing through – from my laptop to the payment provider and back – thought the same. They monitor for exactly this type of behaviour I found out.

What happened?

I was banned! The routing stopped working and an error message appeared instead of my self-hosted site. I did contact support and after explaining myself the free account was actually re-instated, but not before two days had passed.

Reverse proxy using WireGuard

So I learned something about what not to do when processing payments. I wanted to keep learning more (and hopefully get to the point where I can process payments for the service I’m working on) so I needed a reverse proxy to receive messages direct from the payment provider sandbox on my laptop. So I found this site (by ducking the term “alternatives to Ngrok”) here: https://github.com/anderspitman/awesome-tunneling and from there I got to this free, open source service built with WireGuard: https://tunnel.pyjam.as/

Now I can host my own reverse proxy (using DigitalOcean droplet) and get back to learning about payment providers, phew!

K8 Virtual Juggling with working remote

K8 are my favourite type of LED juggling equipment. Recently I updated my virtual juggling web app to include a working remote control – just like the real thing.

The App

So the app consists of an animated juggler, a remote control and “Change your pattern” button.

While the juggler is juggling, you can press buttons on the remote to change the juggling ball colours (accurately emulating the actual K8 equipment settings)

The “Change your pattern” menu is a large list of different juggling patterns, which when selected will change the animation displayed.

How it works: back end

The back end is based on Flask. I am using the beautiful soup library to fetch the menu of juggling patterns from the awesome library of juggling website. Once selected, the gif of the pattern is fetched. It is then processed (a script inverts the colours and makes the juggling balls transparent) – if that hasn’t already been done for the particular pattern.

How it works: front end

The juggling animation and remote control are written with P5.js. The juggling ball colours are implemented as a background which shows through the transparent balls. The button co-ordinates are relative, so work on any size screen (looks best on Desktop)

Magic Poi 2022 update

Current state of Magic Poi – and some ideas for the future.

First of all, an announcement: Magic Poi is now available for ESP32, as well as ESP8266 architecture. This will bring improvements in performance. I plan on continuing support for both, and in the near future a combined code base will be provided.

I am going to list current features here, and improvements I plan to implement.

On-board images:

  • I have partnered with EnterAction, an awesome Sydney based fabrication company who are taking over the hardware development from now on. Improvements will include an SD card add-on for limitless on-board storage. This will require changes to the code, as currently the maximum is 52 images supported.

UDP streaming:

  • this is a defining feature of Magic Poi. The images are generated off-device, and “streamed” via UDP pixel by pixel. I plan to keep improving this functionality but change it to not be the default mode. Due to WiFi interference the UDP stream is sometimes interrupted, making the LED’s stutter, so work is being done to mitigate that.

“Timeline” – images changing in time to music:

  • currently there is a desktop app to generate the timeline (and associated images) and save as a zip file, which needs to be uploaded to the Android app in order to be “streamed” to the poi. I plan on changing this functionality to rather happen in the poi code, thus avoiding the WiFi interference problem. The timeline editor will be made into a web app, with the option to download directly to the poi.

Station mode:

  • poi connected to a router provides more stable WiFi than the current AP mode. I have made a start on providing a way to use this mode.

Online account:

  • like a PlayStation or Kindle, there is a benefit to having a cloud aspect to any product that consumes media. The Magic Poi website is going to be a place where you can upload and share images and timelines, as well as interact with other poi owners. All uploaded images will be private of course, unless shared. I have made a start on this cloud aspect, with an option in testing to download images directly from your cloud account to the poi. The ultimate goal is to be able to sync any two pairs of poi with two clicks!

Android app:

  • Still not working: text to image (stream words directly to the poi).
  • Once the online portal is finished, this will be added to the app, so shared images and timelines can be viewed without need for a web browser.

The above is a small part of the list – thanks to EnterAction taking over the hardware development side, I will have more time to devote to the software improvements. We also plan on adding a battery level indicator, and a higher power battery for more play time.

Thanks for reading!

Keep an eye on this blog, and sign up to the newsletter (if you haven’t already) for more updates as Magic Poi moves forward towards it’s inevitable crowd funder launch!

UPDATES:

Sign up for our update alerts:

I made an LED indicator for my portfolio site

Job hunting is tough – I’m busy until February 2022 but already feeling anxious about finding the next gig. That’s why I wanted to give myself a bit of an incentive – a visible indicator of success. I decided that whenever someone visits my portfolio site, I wanted an LED to light up. Read on to find out how I did it (using Flask, http requests, and an ESP8266)

Self Hosting

I recently had a bad experience with an online service shutting down on me – had a bit of a rant about it, although in the end it wasn’t too serious – but I am now determined to do self-hosting wherever possible *disclaimer: my current work project is hosted on Google Cloud, and I’m using Firebase and push services for some Android apps also.

The Flask api

The first step was to create a simple Flask api to facilitate tracking of site visits. This is based on this minimal flask api template on GitHub. I used a simple global variable to keep track of website visits because I’m doing this in my spare time and because it works fine – and I love boolean switches. Here is how it works in one simple gif:

Think of the hand as a visitor to my site, and the “switcher offer” is the ESP8266 at my home checking the api (the switch)

HTTP request from resume site

Since I am learning React my portfolio site was a good way to have another look at the framework. I used this single-page React resume site template as a base, adding my own details and an http request to the Flask api endpoint on load.

ESP8266 code

I used the basic http requests example with my own api details, and added in EEPROM code to record the incrementing number of visitors to persistent memory. The ESP8266 module checks once per second with the api whether there has been a new visitor to my site. If there has, the built in LED on my D1 Mini switches on. Although I have mostly moved over to using PlatformIO, for this very simple sketch I used the Arduino IDE.

Deployment

Like I said, this one is self-hosted. I’m using Digital Ocean droplets, which are a fixed cost of 5 dollars per month, for as many sites and services you can cram on there (trust me, it’s a lot). The React site was surprisingly simple to deploy, just build, copy the build folder and point Nginx at it. Flask is a little bit more complicated, compared to how easy it would be on Google Cloud, for example, but a few config files are really not too much to handle.

The result

Whenever someone visits my website, the LED lights up. Simple as that. And I can plug in and check how many visitors I have had. I’m hoping that one of those visitors will like what I do enough to hire me next year!

Visit my site https://devsoft.co.za to light up my visitor tracking LED.

Potential improvements

If I was making this into a product, I would certainly upgrade the Flask API to include a database to keep track of the number of visits, rather than doing this on the ESP8266 EEPROM – which maxes out at 255!* Obviously this could include a web interface for accessing the information, I could log the times… But most of this tracking stuff has been done already – analytics for websites. Perhaps the ESP8266 could pick up some of this information and display it on an LCD screen. A flask service for accessing Google Analytics from Arduino perhaps? Let me know if this is something you are interested in!

Also, proper authentication – if this wasn’t just for myself… JWT, rate limiting, CRUD endpoints and a web interface to change LED behaviour.

And maybe an RGB LED would be nice, then I could add in some of my other websites, in different colours!

*apparently the Arduino EEPROM library works differently on ESP8266 – ignore that part of the code, I need to update it (the counter still increments while the module is plugged in, though)