4 + 1 ways for making HTTP requests with Node.js: async/await edition

HTTP requests with Node.js are a means for fetching data from a remote source. It could be an API, a website, or something else: at one point you will need some code to get meaningful data from one of those remote sources.

4 + 1 ways for making HTTP requests with Node.js: async/await edition

During your career as a Web Developer you will work with HTTP requests all the time. This is why in the following post I want to introduce you to some different ways for making HTTP requests in Node.js.

Starting from the easier one we will explore the “classic way” for doing HTTP requests all the way through libraries which support Promises.

I will focus mostly on GET requests in order to keep things simple and understandable.

What you will learn

  1. How to make HTTP requests in Node.js with various modules
  2. The prons and cons of every module

Requirements

To follow along you should have a basic understanding of Javascript and ES6.

Also, make sure to have one of the latest versions of Node. In the following post I’ll make use of the async/await pattern, introduced into Node 7.6.0.

Making HTTP requests with Node.js: why?

At this point you might be asking yourself “Why would I ever do a HTTP request?”.

The answer is simple: as a Javascript developer you will interact every day with remote APIs and webservers. Almost everything today is available in the form of an API: weather forecasts, geolocation services and so on.

Node.js can be used to serve a vaste range of purposes: you can build a command line tool, a proxy, a webserver, and in its simplest form it can be used just for querying a remote API and returning the output to the user.

That’s exactly what we’re going to do in the next examples, looking at some possible ways for making HTTP requests with Node.js by querying the Google Maps API.

Making HTTP requests with Node.js: old school HTTP requests with callbacks

To start, create an empty directory named making-http-requests-node-jsand run:

npm init -y

to initialize package.json. The -y flag configures package.json with the default values rather than asking us any questions.

There are two simple ways for making HTTP requests with Node.js as of today: by using a library which follows the classic callback schema or even better by using a library which supports Promises.

Working with Promises means you could also use the async/await keywords.

Let’s start with callbacks!

Making HTTP requests with Node.js: http.get and https.get

Making HTTP requests with Node.js: http.get and https.get
The documentation for http.get

http.get or https.get (for HTTPS requests), are the first choices for making requests in Node.js. If you just need to GET something from an API, stick with them.

PROS:

  1. native API, there is no need to install third party modules
  2. the response is a stream*

CONS:

  1. a bit verbose
  2. the response is a stream*
  3. no support for promises

To test things out create a new file named https-native.js:

const https = require("https");

const url =
  "https://maps.googleapis.com/maps/api/geocode/json?address=Florence";

https.get(url, res => {
  res.setEncoding("utf8");
  let body = "";
  res.on("data", data => {
    body += data;
  });
  res.on("end", () => {
    body = JSON.parse(body);
    console.log(
      `City: ${body.results[0].formatted_address} -`,
      `Latitude: ${body.results[0].geometry.location.lat} -`,
      `Longitude: ${body.results[0].geometry.location.lng}`
    );
  });
});

Now if you run this code with:

node https-native.js

you should be able to see the following output:

City: Florence, Italy - Latitude: 43.7695604 - Longitude: 11.2558136

https.get expects an url as a first argument and a callback as a second argument. The returned response is an http.ClientRequest object. That means, in order to manipulate the body of the response you have to listen for events: notice the res.on() in the above example.

*The http.ClientRequest object emits some events that you can listen to. And that’s both good and “bad”: good because you will be tempted to dig further into the Node.js internals to learn more and “bad” because you are forced to do a lot of manipulation if you want to extract the JSON response.

In the end, working with http.get could be slightly more verbose compared to other libraries but that shouldn’t be necessarily considered a drawback because in the process you will learn more about Node.js internals.

Making HTTP requests with Node.js: the request module

Making HTTP requests with Node.js: the request module
The request NPM module

request is one of the most popular NPM module for making HTTP requests with Node.js. It supports both HTTP and HTTPS and follows redirects by default.

PROS:

  1. ease of use

CONS:

  1. no support for promises
  2. too many dependencies

To install the module run:

npm i request

inside your project folder.

To test out the example create a new file named request-module.js:

const request = require("request");

const url =
  "https://maps.googleapis.com/maps/api/geocode/json?address=Florence";

request.get(url, (error, response, body) => {
  let json = JSON.parse(body);
  console.log(
    `City: ${json.results[0].formatted_address} -`,
    `Latitude: ${json.results[0].geometry.location.lat} -`,
    `Longitude: ${json.results[0].geometry.location.lng}`
  );
});

By running the code with:

node request-module.js

you should be able to see the same output as in the previous example:

City: Florence, Metropolitan City of Florence, Italy - Latitude: 43.7695604 - Longitude: 11.2558136

request.get expects an url as a first argument and a callback as a second argument.

Working with the request module is pleasing. As you can see in the example, it is much more concise than http.get.

There’s a drawback though: request carries 22 dependencies. Now, I wouldn’t consider this a real problem but if your goal is to make just an HTTP GET request, sticking with http.get will be enough to get the job done.

The request module does not support promises. It could be promisified with util.promisify or even better you could use request-promise, a request version which returns promises (and has less dependencies).

Making HTTP requests with Node.js: I Promise I’ll be async

So far we’ve seen how to make HTTP requests in the most basic way with callbacks.

But there is a better (sometimes) way to handle async code: by using Promises alongside with the new async/await keywords.

In the next examples we’ll see how to use a bunch of Node.js modules which support Promises out of the box.

Making HTTP requests with Node.js: the node-fetch module

Making HTTP requests with Node.js: the node-fetch module
The node-fetch module

node-fetch is an implementation of the native Fetch API for Node.js. It’s basically the same as window.fetch so if you’re accustomed to use the original it won’t be difficult to pick the Node.js implementation.

PROS:

  1. support for promises
  2. same API as window.fetch
  3. few dependencies

CONS:

  1. same API as window.fetch

To install the module run:

npm i node-fetch

To test out the example create a new file named node-fetch.js:

const fetch = require("node-fetch");

const url = "https://maps.googleapis.com/maps/api/geocode/json?address=Florence";

const getLocation = async url => {
  try {
    const response = await fetch(url);
    const json = await response.json();
    console.log(
      `City: ${json.results[0].formatted_address} -`,
      `Latitude: ${json.results[0].geometry.location.lat} -`,
      `Longitude: ${json.results[0].geometry.location.lng}`
    );
  } catch (error) {
    console.log(error);
  }
};

getLocation(url);

By running the code with:

node node-fetch.js

you should be able to see the same output again:

City: Florence, Metropolitan City of Florence, Italy - Latitude: 43.7695604 - Longitude: 11.2558136

If you paid attention I’ve listed “same API as window.fetch” both in Pros and Cons.

That’s because not everybody likes the Fetch API. Some people can’t stand out the fact that in order to manipulate the response you have to call json() (and some other quirks).

But in the end it’s a matter of getting the job done: use whichever library you prefer.

Making HTTP requests with Node.js: the r2 module

The request module for Node.js was written by Mikeal Rogers back in 2010. In 2017 he is back with the r2 module.

The r2 module uses Promises and is another implementation of the browser’s Fetch API. That means r2 depends on node-fetch.

At first it wasn’t clear to me why would I consider using r2 over node-fetch. But I thought the module is worth a mention.

PROS:

  1. support for promises
  2. same API as window.fetch
  3. few dependencies

CONS:

  1. it depends on node-fetch

To install the module run:

npm i r2

To test things out create a new file named r2-module.js:

const r2 = require("r2");

const url =
  "https://maps.googleapis.com/maps/api/geocode/json?address=Florence";

const getLocation = async url => {
  try {
    const response = await r2(url).json;
    console.log(
      `City: ${response.results[0].formatted_address} -`,
      `Latitude: ${response.results[0].geometry.location.lat} -`,
      `Longitude: ${response.results[0].geometry.location.lng}`
    );
  } catch (error) {
    console.log(error);
  }
};

getLocation(url);

Run the code with:

node r2-module.js

and you should be able to see (again):

City: Florence, Metropolitan City of Florence, Italy - Latitude: 43.7695604 - Longitude: 11.2558136

By comparing r2 to node-fetch I can see it is less verbose. I saved 1 line of code.

In all honesty I didn’t take the time to look at r2 in details. Maybe because it is almost the same as node-fetch. But I’m sure it has more to offer.

Making HTTP requests with Node.js: the axios module

Making HTTP requests with Node.js: the axios module
The axios NPM module

Axios is another super popular NPM module for making HTTP requests. It supports Promises by default.

Axios can be used both for the front-end and the back-end and one of its core feature is the ability to transform both the request and the response.

Plus you don’t need to explicitly process the response in order to get the JSON as you did with node-fetch: axios will do it automagically.

PROS:

  1. support for promises
  2. ease of use
  3. 2 dependencies

CONS:

  1. ??

Run:

npm i axios

to install axios inside your project folder.

To test out the example create a new file named axios-module.js:

const axios = require("axios");

const url =
  "https://maps.googleapis.com/maps/api/geocode/json?address=Florence";

const getLocation = async url => {
  try {
    const response = await axios.get(url);
    const data = response.data;
    console.log(
      `City: ${data.results[0].formatted_address} -`,
      `Latitude: ${data.results[0].geometry.location.lat} -`,
      `Longitude: ${data.results[0].geometry.location.lng}`
    );
  } catch (error) {
    console.log(error);
  }
};

getLocation(url);

Again, by running the above code you should be able to see the same output of the previous examples.

I really can’t find any drawback in axios. It’s super simple to use, highly configurable and intuitive. Last but not least it has only 2 dependencies.

Making HTTP requests with Node.js: in conclusion

All the above libraries share one common denominator: each of them produces the same result. As with almost everything with Javascript sometimes picking one module over another it’s a matter of personal preferences.

In the end, I feel that by using the native API methods you can learn more about the Node.js internals, and that’s both great and educational.

My rule of thumb is to pick the smallest library with the fewer dependencies possible based on what I want to do.

If I were to make a super simple GET request I wouldn’t install neither axios nor node-fetch.

In contrast, using a third party module could save you a lot of coding when in need for more complex functions: i.e. request and response manipulation.

A small note about “Making HTTP requests with Node.js”

A lot has changed since the first version of this blog post. I’ve received a lot of useful comments plus some recommendations.

First time I wrote about making HTTP requests with Node.js I put the article on Medium. The response so far has been crazy. It got 21K views as of today:

Making HTTP requests with Node.js: stats

In the meantime async/await became a thing so I thought it would be nice to update the article with modern examples. Plus, since I care about my own blog, better to bring the article back where it belongs: here.

Thanks for reading!

The code

The examples are available on Github: making-http-requests-node-js

But! … I want more!

Looking for more articles about Node.js? Fear not, I’ve got you covered: take a look at How to inspect the memory usage of a process in Node.Js and Going real time with Socket.IO, Node.Js and React

Valentino Gagliardi

Valentino Gagliardi

Consultant, Developer Coach. Are you stuck on a project? Let's talk!
Valentino Gagliardi

7 Replies to “4 + 1 ways for making HTTP requests with Node.js: async/await edition”

  1. If you’re using ES6 and counting lines, you might as well do destructuring object assignment, e.g.:

    const { data } = await axios.get(url);

    Though in the real world you’d want to check the response first of course…

  2. I’ve always been an advocate for using native code over third-party libraries whenever feasible. I typically reach for http.get() as a first option. (Also, not all third party mini-modules are actively supported)

    Considering now that, as of version 8, Node.js has it’s own promisify function included in the util module. You can add promise support for http.get() with a single line of code if promise support is necessary.

    Granted, you’ll still have the following cons:
    1. a bit verbose
    2. the response is a stream*
    But promise support is readily available in Node.js, and you can make a promise-based http request without adding additional dependencies.

    Here’s an example = https://github.com/Multishifties/async_code_review/blob/master/examples/09_compatibility/10_promisify.js

  3. Another great module to use for HTTP requests is Snekfetch by Devsnek. It’s super simple to use and relies on native promises (no bluebird or such). It might be worth adding a section for Snekfetch to this article.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.