Tutorial: User Interface Testing with Jest and Puppeteer

I started to consider testing with Jest and Puppeteer right after the library came out. Puppeteer has quite an interesting API.

Testing with Jest and Puppeteer

In the following post I’ll introduce you to a basic UI test for a contact form.

We will testing with Jest and Puppeteer. Even if it’s still under development and the API could be subject to changes, Puppeteer is here to stay.

Continue reading “Tutorial: User Interface Testing with Jest and Puppeteer”

4 + 1 ways for making HTTP requests with Node.js: async/await edition

HTTP requests with Node.js are a means for fetching data from a remote source. It could be an API, a website, or something else: at one point you will need some code to get meaningful data from one of those remote sources.

4 + 1 ways for making HTTP requests with Node.js: async/await edition

During your career as a Web Developer you will work with HTTP requests all the time. This is why in the following post I want to introduce you to some different ways for making HTTP requests in Node.js.

Starting from the easier one we will explore the “classic way” for doing HTTP requests all the way through libraries which support Promises.

Continue reading “4 + 1 ways for making HTTP requests with Node.js: async/await edition”

Socket.IO, React and Node.js: Going Real-Time with WebSockets

Learn how to create a simple real-time application with Socket.IO, React and Node.js!

Socket.IO, React and Node.js: Going Real-Time with WebSockets

Looks like everybody is building chat apps with Socket.IO these days yet messaging applications are only the tip of the iceberg. Think a moment about it: there are a million of other things you can build within the real-time domain.

In the following post I would like to explore an interesting use case and hopefully give you ideas about what to build next with Socket.IO. We will start with some basic concepts all the way through exploring what Socket.IO and React can do for us when paired together. By the end of the article you will build a super simple real-time application.

Continue reading “Socket.IO, React and Node.js: Going Real-Time with WebSockets”

How to inspect the memory usage of a process in Node.Js

While everyone seems to agree about the fact that premature optimization could be detrimental, you must care about performances either way: in the most simplest case you may want to know how much memory a given Node.js process uses during its execution.

In this post we will see how to use a Node.js builtin method in order to gain knowledge about the memory usage of any given process.

Disclaimer: This is not intended to be a thorough explanation of the Node.js memory management mechanism, in fact I have some suggestions for you at the end of the article.

What you will learn

  1. How to get information about the memory of a Node.js process
  2. How Node.js organizes the memory

Requirements

In order to follow along you need a basic understanding of Javascript and Node.js

Node.js can be downloaded from the official website, installed via a package manager, or even better via Nodesource if your Linux distribution doesn’t ship with the latest version of Node.

How does Node.js organizes the memory?

Well, a single blog post won’t be enough to explain all the machinery: you should just be concerned about the heap for now so let’s start by breaking down the basics.

Did I say heap?? Yes sir: the first thing you will get in touch with by querying the Node.js memory usage is the heap.

The heap is a memory segment used to store objects, strings and closures. This is where all the magic happens.

The heap is part of something bigger though: a running Node.js process store all its memory inside a Resident Set. You can think of it as of a big box which contains some more boxes. The Resident Set contains also the actual Javascript code (inside the Code segment) and the Stack, where all the variables live. Hopefully the following illustration will clarify the concept:

The Node.js memory organization
An illustration of the Node.js memory organization

Sounds complicated? We need to focus only on the heap for now, so don’t worry if you can’t grasp all the terminology right now!

Querying the Node.js heap

We know what the Resident Set is and we also know that the Heap is what we’re looking for: now we need to find out how much heap is used by a given Node.js process. But how do we access the heap?

With process! process is a global Node.js object which contains information about the current Node.js process, and it provides exactly what we were searching for: the memoryUsage() method.

memoryUsage returns an object with various information: rss, heapTotal, heapUsed (and external):

  • rss stands for Resident Set Size, it is the total memory allocated for the process execution
  • heapTotal is the total size of the allocated heap
  • heapUsed is the actual memory used during the execution of our process
  • external is, according to the documentation, the memory used by “C++ objects bound to JavaScript objects managed by V8”

Armed with the knowledge we are ready to see some examples.

Testing reverse() memory usage for a small array

In our first example we will see how much memory is used by a very basic Javascript method: reverse(). reverse is an array method: it takes an array and reverses its content.

Create a new file inside named reverse_node.js:

const arr = [1, 2, 3, 4, 5, 6, 9, 7, 8, 9, 10];
arr.reverse();
const used = process.memoryUsage().heapUsed / 1024 / 1024;
console.log(`The script uses approximately ${Math.round(used * 100) / 100} MB`);

Now if you run this script you should see the following output:

This script uses approximately 3.6 MB of memory

In the example above I picked the property heapUsed from process.memoryUsage() but if you run process.memoryUsage() by itself, you will have access to the other properties as listed in the previous section. For instance by running:

const arr = [1, 2, 3, 4, 5, 6, 9, 7, 8, 9, 10];
arr.reverse();
const used = process.memoryUsage();
for (let key in used) {
  console.log(`${key} ${Math.round(used[key] / 1024 / 1024 * 100) / 100} MB`);
}

You’ll see the following output:

rss 25.63 MB
heapTotal 5.49 MB
heapUsed 3.6 MB
external 0.01 MB

Now, this is not the most accurate way for measuring the memory consumption in Node.js but at least you’ll get the job done. There is more to take care about when doing instrumenting, in particular you should account for the Node.js garbage collector. Besides that, with memoryUsage you have at least an indication about how much memory a given script/process takes.

Let’s continue with another example!

Testing reverse() memory usage for a big array

How about testing a very big array now?

Create a new file inside your playground directory named reverse_node_bigArray.js:

let arr = Array(1e6).fill("some string");
arr.reverse();
const used = process.memoryUsage().heapUsed / 1024 / 1024;
console.log(`The script uses approximately ${used} MB`);

Array(1e6) creates an empty array with a length property of 1 million. Then the fill() method fills every position of the array with a string. At this point we have an array with 1 million of elements into it.

By running the new script you should see the following output:

This script used approximately 11.23 MB

Things will start to get more interesting if we increase the size of our array. In the script change the following line from:

let arr = Array(1e6).fill("some string")

to

let arr = Array(1e8).fill("some string")

Array(1e8) creates an empty array with a length property of 100 millions. Again, the fill() method fills every position of the array with a string. At this point we have an array with 100 million of elements into it, and by running the script again you will see how the memory usage increases:

This script uses approximately 1278.83 MB

That’s really interesting right? From now on you’ll be able to measure how much memory a Node.js process uses.  Go check your code and see if you can optimize some functions!

Where to go from here

We’ve seen how to measure the memory usage of a Node.js process. Keep in mind that for the most common needs the memoryUsage() method will suffice but if you were to investigate a memory leak in an Node.js application you would need more than that.

To start with, I suggest taking a look at how the v8 garbage collection works. Here’s a great writeup: http://jayconrod.com/posts/55/a-tour-of-v8-garbage-collection

Troubleshooting a Node.js memory leak is not simple but fear not, there are many libraries available: memwatch-next is a great starting point and it is worth a look.

Finally, if you want to step up there are also some commercial tools available but either way, understanding how the Node.js garbage collection works is really important. Start with the basics and then switch to a monitoring tool when you feel more confident.

Thanks for reading!

Psst.. you might like 4 + 1 ways for making HTTP requests with Node.js: async/await edition