Use scheduled shutdowns to save money and energy

Hans Blaauw — Tue, 26 Sep 2023 07:26:53 +0000

What is the problem?

Cloud prices are surging and the promise that public cloud would save you money has not been fulfilled. The hyperscalers have locked you in with their services and there aren’t easy ways out. One easily overseen but hard to manage option is to run your VM’s on a schedule to save on cloud costs. Hyperscalers charge you based on usage so it makes sense to do some kind of orchestration.

On top of that, our planet is at risk and it makes sense to cut energy consumption.

Global energy consumption data centers Estimated global data centre electricity consumption in 2022 was 240-340 TWh1, or around 1-1.3% of global final electricity demand. This excludes energy used for cryptocurrency mining, which was estimated to be around 110 TWh in 2022, accounting for 0.4% of annual global electricity demand.

How do you start?

I think it is really important to create awareness that running VM’s is not a free ride. It’s also not very sustainable. The world eats data but this all comes with a costs for our planet. Using serverless architecture is the best way to be more sustainable and better for our planet.

You need to make a plan for scheduling some of your services. First candidates to save money on are developer machines. Developers tend to leave public cloud facilities on, not being aware of the fact that this is expensive. So the first thing you can do is to make a list of developers and the VM’s they use and when they need them. Probably most of the time they need these machines during working hours.

Second candidates are solutions that don’t need to run 24 hours a day. Think of machines that run logistical processes during the day or do document processing.

What are the drawbacks?

When it comes to running virtual servers you need to make sure machines get back into their original state when started again. Sometimes this can be done in the software solutions itself and sometimes it needs to be done at the server level.

Do you run nightly backups? Make sure to schedule carefully.

Office hours

It’s clear that most people work between 09:00 and 17:00 in the US and Europe. In Asia people tend to work slightly longer.

The easiest step to save money is to shutdown your machines outside office hours.

Which tools do you need?

Pick any tool you like as long as it can support scheduling and API’s. Below is a list of the tools that can work:

Zapier
Make.com
N8N
cron
Activepieces
Directus
Pipedream
Power automate
Postman

Besides the tools you need to have the possibility to manage your cloud services with an API. The following cloud providers have some API’s in place:

Vultr
Digital Ocean
whitesky.cloud
AWS
Azure

We run parts of our services at Digital Ocean, and yes we schedule some of our VM’s.

Digital Ocean is not our sponsor but we use their affiliate system. We use their products on a daily basis for customers and internal projects.

A sample

The sample we provide is based on Digital Ocean and their API. We first tested the API with Postman and then we hooked it up with Pipedream (just in this case).

The first thing you need is the URL of the API. The base URL is:

https://api.digitalocean.com/v2/droplets//actions

The JSON payload that you need to send for a power off is:

{“type”: “power_off”}

Authorization is done by using a Bearer token which can be created in the Digital Ocean management environment.

To make it work you also need a Droplet ID which can also be found in the Digital Ocean Dashboard.

When using the power off you also need to send a shutdown after 1-2 minutes. This can be done by using a different payload which contains shutdown.

Below you can see the workflow done in Pipedream.

It makes a lot of sense to use schedules and even notify user about the status of their machines. And yes, using cloud services to schedule things also costs money and energy but compared to machines running at night it is still a reasonable thing to do.

Interesting reads

Is your organisation ready to cut cloud costs?

Let us know!

The post Use scheduled shutdowns to save money and energy first appeared on THINGS IO | Low-code data koppelen vanuit Groningen.

Can you run your automations on your own servers?

Hans Blaauw — Thu, 14 Sep 2023 06:40:58 +0000

Why automations?

Don’t we all love automations? You set and forget something and it does it’s work.

When I started in Information Technology way back in 1985 we all talked about having automations to do our work. If you ask me, the tools to do this in a good way just exist for 10-15 years. Zapier for example was founded in 2011.

For me this is what automation should have been all the time. Employees click together their solution to streamline their work.

DIY vs Public cloud We all know public cloud comes with a price. The surge in costs at hyperscalers and the simplicity of products like Digital Ocean makes it much easier to host your own solutions in a safe way. When your database, traffic or storage grows on hyperscalers it can become expensive very fast.

Which solutions can be hosted on your own infrastructure?

The big names in the automation business don’t allow you to host your own automations. They make money by charging you for operations/tasks. This is perfectly fine and in many cases the easiest and fastest way to start automating pieces of your work.

However, there might be situations, more data and bigger workflows, where it becomes too expensive to run public cloud solutions. Take for example IOT, what if you want to use these automations for every sensor event that happens?

For these type of solutions you need to host your own automation solution. We have selected the following solutions that are capable of doing some automations:

N8N: full visual automation solution;
Activepieces: full visual automation solution;
Directus: data connections and visual flows;
Budibase: data connections, apps and flows.

To be honest we have not tested all solutions but N8N, Directus and Budibase are the solutions we have tested.

What infrastructure do you need?

The easiest way to get started is running a machine on Digital Ocean. I’m not an infrastructure guy but we have been able to setup machines in safe environments over and over again.

Digital Ocean is not our sponsor but we use their affiliate system. We use their products on a daily basis for customers and internal projects.

Easy docker management

We prefer to run our automations inside a docker container. However, to avoid manual configurations etc. we use a control panel which can setup docker containers in an easy way. A few months ago we discovered Easypanel.

When you create your server at Digital Ocean you can pick the Easypanel template from the Marketplace.

After that you are ready to install some of the mentioned automation solutions.

What can you create?

The sky is the limit but you need to understand these tools are mainly used to automate things between data and cloud applications. For desktop applications you need RPA, Robot Process Automation.

We have been able to create ETL tools for customers and hookup systems like: Visma.net, Televend (vending), Digital Ocean, MySQL, WooCommerce, Prestashop and may more.

What we have notices on day to day usage is that a combination of some of these tools works best. Better make sure you keep an eye on the automations!

Do you think you need help to set it up?