Skip to main content

Scheduling Google Compute Instances with Cloud Scheduler

Our company needs to save some cost for the running staging instances. Since the instance and any related resources are used only when we are at work, we need the instance to be available only at working hours, or from Monday to Friday, 9 to 5. It approximately reduces our cost to roughly 45%. A huge diff.

diary on keyboard


The Ingredients

We need the following components from Google Cloud:
  • Google Cloud Scheduler: to run a function at an exact day/time.
  • Google Cloud Functions: to start and stop our compute instances.
  • Google Cloud Pub/Sub: as a bridge or messenger from the Cloud Scheduler to the  Cloud Functions.
  • Lastly, the Google Cloud Compute Engine Instances.
Each of those will cost us money, but the good news is we will work under the fair usage policy so it can be said that all of those will be free of charge. For example, now Google Cloud gives us 93x each month for Cloud Functions. That means if we only use it twice a day (to start and stop the instance), it will cost us nothing.

We also can use the Google Cloud HTTP Functions as an alternative for the Google Cloud Pub/Sub, but notice that there is no authentication so basically it is less secure than the Pub/Sub component.

Prepare the Compute Engine Instances

Create our compute engine instance, or edit the existing ones. You can work at any zone location, specs or anything. The most important things for this tutorial is the label of our instances should be "env=dev". You can use any value but make sure it is unique enough because we need the label as an identifier.

labels on google compute engine

Setup Google Cloud Functions with Cloud Pub/Sub

Go to your Google Cloud Functions page, and create a new function. This function will be used to start the instance.
  • Set the name to "startInstancePubSub"
  • Set the 'Trigger" to "Cloud Pub/Sub"
  • Set the Topic to "start-instance-event"
  • Set "Runtime" to "Node.js 8"
IMPORTANT. Set the starter code with the following code. The current code used as an example at the official page is obsolete for node.js 8.  This way, we will start all instance with "env=dev" label. We don't care about the zone, etc.

Here is the content of our index.js.


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
/**
 * Copyright 2018, Google, Inc.
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *    http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

// [START functions_start_instance_pubsub]
// [START functions_stop_instance_pubsub]
const Buffer = require('safe-buffer').Buffer;
const Compute = require('@google-cloud/compute');
const compute = new Compute();
// [END functions_stop_instance_pubsub]

/**
 * Starts a Compute Engine instance.
 *
 * Expects a PubSub message with JSON-formatted event data containing the
 * following attributes:
 *  zone - the GCP zone the instances are located in.
 *  label - the label of instances to start.
 *
 * @param {!object} event Cloud Function PubSub message event.
 * @param {!object} callback Cloud Function PubSub callback indicating
 *  completion.
 */
exports.startInstancePubSub = (event, context, callback) => {
  try {
    const payload = _validatePayload(
      JSON.parse(Buffer.from(event.data, 'base64').toString())
    );
    const options = {filter: `labels.${payload.label}`};
    compute.getVMs(options).then(vms => {
      vms[0].forEach(instance => {
          compute
            .zone(instance.zone.id)
            .vm(instance.name)
            .start()
            .then(data => {
              // Operation pending.
              const operation = data[0];
              return operation.promise();
            })
            .then(() => {
              // Operation complete. Instance successfully started.
              const message = 'Successfully started instance ' + instance.name;
              console.log(message);
              callback(null, message);
            })
            .catch(err => {
              console.log(err);
              callback(err);
            });
      });
    });
  } catch (err) {
    console.log(err);
    callback(err);
  }
};
// [END functions_start_instance_pubsub]

// [START functions_start_instance_pubsub]

/**
 * Validates that a request payload contains the expected fields.
 *
 * @param {!object} payload the request payload to validate.
 * @return {!object} the payload object.
 */
function _validatePayload(payload) {
//   if (!payload.zone) {
//     throw new Error(`Attribute 'zone' missing from payload`);
//   } else 
  if (!payload.label) {
    throw new Error(`Attribute 'label' missing from payload`);
  }
  return payload;
}
// [END functions_start_instance_pubsub]
// [END functions_stop_instance_pubsub]


At the package.json tab, copy and paste the following code:


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
{
  "name": "cloud-functions-schedule-instance",
  "version": "0.0.2",
  "private": true,
  "license": "Apache-2.0",
  "author": "Google Inc.",
  "repository": {
    "type": "git",
    "url": "https://github.com/GoogleCloudPlatform/nodejs-docs-samples.git"
  },
  "engines": {
    "node": ">=8.13.0"
  },
  "scripts": {
    "test": "mocha test/*.test.js --timeout=20000"
  },
  "devDependencies": {
    "@google-cloud/nodejs-repo-tools": "^3.3.0",
    "mocha": "^6.0.0",
    "proxyquire": "^2.0.0",
    "sinon": "^7.0.0"
  },
  "dependencies": {
    "@google-cloud/compute": "^0.12.0",
    "safe-buffer": "^5.1.2"
  }
}

Theoretically, the code will
  1. Search for any instances by label (our label is "env=dev"). It is not limited to 1 instance.
  2. Start the instances.
I think it doesn't matter if we have no experience using node.js before.

For "Function to execute", enter "startInstancePubSub"

Next, we are going to set the "stop instance function"
  • Create another function named "stopInstancePubSub"
  • Set everything else the same way we set up the start instance function above.
  • For pub-sub topic, set it to "stop-instance-event"
Here is our index.js code.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
/**
 * Copyright 2018, Google, Inc.
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *    http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

// [START functions_start_instance_pubsub]
// [START functions_stop_instance_pubsub]
const Buffer = require('safe-buffer').Buffer;
const Compute = require('@google-cloud/compute');
const compute = new Compute();
// [END functions_stop_instance_pubsub]

/**
 * Starts a Compute Engine instance.
 *
 * Expects a PubSub message with JSON-formatted event data containing the
 * following attributes:
 *  zone - the GCP zone the instances are located in.
 *  label - the label of instances to start.
 *
 * @param {!object} event Cloud Function PubSub message event.
 * @param {!object} callback Cloud Function PubSub callback indicating
 *  completion.
 */
exports.startInstancePubSub = (event, context, callback) => {
  try {
    const payload = _validatePayload(
      JSON.parse(Buffer.from(event.data, 'base64').toString())
    );
    const options = {filter: `labels.${payload.label}`};
    compute.getVMs(options).then(vms => {
      vms[0].forEach(instance => {
          compute
            .zone(instance.zone.id)
            .vm(instance.name)
            .start()
            .then(data => {
              // Operation pending.
              const operation = data[0];
              return operation.promise();
            })
            .then(() => {
              // Operation complete. Instance successfully started.
              const message = 'Successfully started instance ' + instance.name;
              console.log(message);
              callback(null, message);
            })
            .catch(err => {
              console.log(err);
              callback(err);
            });
      });
    });
  } catch (err) {
    console.log(err);
    callback(err);
  }
};
// [END functions_start_instance_pubsub]
// [START functions_stop_instance_pubsub]

/**
 * Stops a Compute Engine instance.
 *
 * Expects a PubSub message with JSON-formatted event data containing the
 * following attributes:
 *  zone - the GCP zone the instances are located in.
 *  instance - the name of a single instance.
 *  label - the label of instances to start.
 *
 * Exactly one of instance or label must be specified.
 *
 * @param {!object} event Cloud Function PubSub message event.
 * @param {!object} callback Cloud Function PubSub callback indicating completion.
 */
exports.stopInstancePubSub = (event, context, callback) => {
  try {
    const payload = _validatePayload(
      JSON.parse(Buffer.from(event.data, 'base64').toString())
    );
    const options = {filter: `labels.${payload.label}`};
    compute.getVMs(options).then(vms => {
      vms[0].forEach(instance => {
          compute
            .zone(instance.zone.id)
            .vm(instance.name)
            .stop()
            .then(data => {
              // Operation pending.
              const operation = data[0];
              return operation.promise();
            })
            .then(() => {
              // Operation complete. Instance successfully stopped.
              const message = 'Successfully stopped instance ' + instance.name;
              console.log(message);
              callback(null, message);
            })
            .catch(err => {
              console.log(err);
              callback(err);
            });
      });
    });
  } catch (err) {
    console.log(err);
    callback(err);
  }
};
// [START functions_start_instance_pubsub]

/**
 * Validates that a request payload contains the expected fields.
 *
 * @param {!object} payload the request payload to validate.
 * @return {!object} the payload object.
 */
function _validatePayload(payload) {
  // ignore the zone
  //  if (!payload.zone) {
  //    throw new Error(`Attribute 'zone' missing from payload`);
  //  } else 
  if (!payload.label) {
    throw new Error(`Attribute 'label' missing from payload`);
  }
  return payload;
}
// [END functions_start_instance_pubsub]
// [END functions_stop_instance_pubsub]


Then, replace the package.json tab to the following code:


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
{
  "name": "cloud-functions-schedule-instance",
  "version": "0.0.2",
  "private": true,
  "license": "Apache-2.0",
  "author": "Google Inc.",
  "repository": {
    "type": "git",
    "url": "https://github.com/GoogleCloudPlatform/nodejs-docs-samples.git"
  },
  "engines": {
    "node": ">=8.13.0"
  },
  "scripts": {
    "test": "mocha test/*.test.js --timeout=20000"
  },
  "devDependencies": {
    "@google-cloud/nodejs-repo-tools": "^3.3.0",
    "mocha": "^6.0.0",
    "proxyquire": "^2.0.0",
    "sinon": "^7.0.0"
  },
  "dependencies": {
    "@google-cloud/compute": "^0.12.0",
    "safe-buffer": "^5.1.2"
  }
}


Then, we need to set "Function to Execute" to "stopInstancePubSub".

We can optionally test the function using the following Triggering Event:

We can use this command on our console to make the base64-encoding.

echo '{"label":"env=dev"}' | base64
eyJsYWJlbCI6ImVudj1kZXYifQo=


Input the following string into the "Triggering Event" textarea : 

{"data":"eyJsYWJlbCI6ImVudj1kZXYifQo="}

Setup the Google Cloud Scheduler

Our functions will not work if we don't trigger them. That's why we need the Google Cloud Scheduler component: to trigger our start/stop functions at an exact day/time.

  • Go to the Google Scheduler page and create a new job named "startup-dev-instances".
  • Set the Frequency to your preference. 0 9 * * 1-5. That means we will start the instance from the 1st day of the week to 5th day of the week (1-5), at 9 AM (0 9).
  • Select our related PubSub topic as the Target
  • For payload, enter the following:
    {"label":"env=dev"}

Then, create the exact job as the start one, except for the name of the scheduler and Frequency.

  • Set the Frequency to 0 17 * * 1-5. That's it, we just create a scheduler to call the PubSub topic from Monday to Friday, at 17:00 or 5 PM.
  • That's it. We are almost done.

Bear in mind that the functions will run at the exact time, but with a random order if we have multiple instances to be started/stopped. In our case, we need to start the A instance first, before the B and C running. That way, I create another Google Scheduler which starts 15 minutes earlier to the B and C instances.

We can modify the flow or the starter code to be more suitable to our needs. I hope you enjoy the article. Don't hesitate to ask a question and give me your comment below. Thank you!

Comments

Popular posts from this blog

About Prabowo Murti

Assalaamu'alaykum.

Hello. My name is Prabowo Murti. This website is made to write all things about programming, business, religion (especially Islam), my family, minimalism, etc, basically anything that related to my journey in life. I will do my best to write all articles in English, not because I am good at it, but just for practice.



About Me I obtained my bachelor's degree in Computer Science from Gadjah Mada University. I love to read and write. I was born in a small town named Kubu Raya, in West Kalimantan. Now I live in Tangerang, Banten. Let's say I am proud to be a muslim and an Indonesian.

I believe that we can make this world a better place to live by our writings.

If you have any question, feel free to ask by writing an email to prabowo.murti at Gmail or submit your comment (at the end of each article). I also have some digital footprints elsewhere.

Muhajirin Dotnet It is a small company I built from 2008. It also has a subdomain named Blog Belajar Bisnis, captur…

DIY Table Legs with PVC for Under $5

I now have two pieces of IKEA Lerberg table legs. It's a great product and has a minimalist design. But the price was about $20 each. My pocket is not deep enough, so I decided to make a similar structure but with cheaper materials. Half inch PVC pipes come in rescue. The total cost for the entire project is about IDR 65,000 or still under 5 dollars.


For this DIY project, we need to make two table legs. Each leg needs:

1 x 21.5 cm: middle leg (bottom)
1 x 18.3 cm: middle segment
2 x 17.8 cm: lower legs (right and left)
4 x 10.9 cm: this depends on your top table's width
2 x 47 cm: upper legs (right and left)
1 x 43.9 cm: middle leg (upper)
5 x T-shape fittings
2 x L-shape fittings


Each leg needs about 2.6 metres so in total we need ~5.2 metres. That means 2 x 4 metres half inch pipes. Each 4 m pipe costs about IDR 18,000. Each fitting costs about IDR 2,000. We also need an iron saw. I get one for about IDR 5,000, but I am a little bit disappointed about the quality. The IDR 20…

DIY Tab Stand Using PVC

I think this will be the simplest tablet stand for my Samsung Galaxy Tab A with a 10-inch display. The main idea (at the end) is to make a super-duper tiny and minimalist home office consists of a desk and chair, a tab, a tablet stand, a Bluetooth keyboard, and a Bluetooth (and silent) mouse. You may already read how I made my desk using a pair of PVC table legs, and a folded top table.

Each week, I will try to post about each item needed for our Zen-wannabe home-office. This article will focus on how we make a tablet stand using a half-inch of PVC pipes, and some fittings. We use PVC pipes for almost all of our project since they are pretty cheap and flexible enough. Combined with some fittings, we can easily make any kind of shapes.

What we need:

- 2 x T-shape fittings
- 1 x straight fittings
- 2 x L-shape fittings
- 30 cm half-inch PVC pipes
- An iron saw to cut the pipe
- Some iron glue, or any similar one.



First. Cut the pipes to your preference. For example, you may cut longer if…

Create a Minimalist Office at Home

This will be the last article of my series on making a home-office workstation wanna-be. The minimalist office is a trend as people tend to work at home instead of spending hours commuting to office nowadays. The reason behind this is that I don't wanna bring my heavy MacBook Pro back and forth from home to office. Besides, I only need a light-computation like typing, editing an image using Canva, or just watching some screencast on YouTube.



We already have a small desk (using dirt-cheap PVC pipes and foldable table) and a DIY tablet stand. For the peripherals, I also have a 10-inch Samsung Galaxy Tab A, a Bluetooth (and silent) keyboard, and a Bluetooth mouse. I often use some keyboard shortcuts, so we can ignore the needs of the mouse.

Let's break them down.
Samsung Galaxy Tab A with 10-inch Display This device is more than enough for typing. If you type, you type with your heart. The 'thinking' part comes after that.

For the editing, change some typos, or replace so…