Simple HTTP Requests on Arduino Microcontrollers

For one of my current projects I had a closer look into the HTTP Protocol. Usually you might use a separate library for doing http calls from Arduino (e.g. with an ESP32) but if you are low on memory, you might get away without this by just using the WiFiClient:

Setting up a WiFi connection

On an ESP32 you would e.g. do the following: First you need to connect to WiFI and then you can connect to the port 80 of your destination with the WiFiClient class:

#include <WiFi.h>
#include <WiFiClient.h>

WiFi.begin("network name", "pwd");
while (WiFi.status() != WL_CONNECTED) {        
  delay(500);
   Serial.print(".");
}

WiFiClient client;
const char* host = "www.pschatzmann.ch";
int port = 80;

while (!client.connect(host, port)) {
  delay(500);
  Serial.print(".");
}

HTTP Versions

During the years http made quite some evolutions each increasing the complexity.
For the embedded devices the HTTP/1.0 is still your friend because it is simple!

HTTP/1.0

In order to make a http request you can just enter the following line, followed by an empty line. The official line endings are CR LF!

// Make a HTTP request:
client.print("GET /home/ HTTP/1.0\r\n");
// empty line -> end of header
client.print("\r\n");

Though officially the lines should end with CR LF – usually a simple linefeed is also working and therefore you find that many other examples just use a simple client.println(“GET http:://www.pschatzmann.ch/home/ HTTP/1.0”)!

After doing this request you can read the answer back. It consists of a the header information, followed by a empty line, followed by the data:

if (client.connected()) {
    // read header
    while (client.available()) {
        String line = client.readStringUntil('\n');
        if (line == "\r") {
            break;
        }
        Serial.writeln(line);
    }    
    // the remaining bytes is the data
    while (client.available()) {
      char c = client.read();
      Serial.write(c);
    }
}

I also noticed that it’s not good if you use the connected() method while you are still in the process of receiving data!

HTTP/1.1

HTTP/1.1 introduced the following functionality:

• persistent and pipelined connections,
• chunked transfers,
• compression/decompression,
• content negotiations,
• virtual hosting
• cache support.

So if you decide to use HTTP/1.1, you might need to add some additional logic or make sure that the additional functionality is not hurting you!

From my point of view the most relevant functionality is the ‘virtual hosting’ which means that you must now add a Host parameter to the request.

A valid request might look as follows:

client.print("GET /home/ HTTP/1.1\r\n");
client.print("Host: www.pschatzmann.ch\r\n");
client.print("Connection: close\r\n");
// empty line -> end of header
client.print("\r\n");

Now you might receive chunked data, but if you don’t really want to deal with this, you can deactivate it by requesting the Accept-Encoding: identity__!

HTTPS

It is almost as easy to connect to Https: You just need to replace the WiFiClient with WiFiClientSecure and connect to the 443 port!

#include <WiFiClientSecure.h>

WiFiClientSecure client;

const char* host = "www.pschatzmann.ch";
int port = 443;
while (!client.connect(host, port)) {
   delay(500);
   Serial.print(".");
}

Conclusion

It is very easy to implement some web requests using HTTP 1.0 (and if you are careful with HTTP 1.1 as well)

The good thing is, that for HTTP/1.0 you can add the host parameter as well which might be relevant if you deal with a server that hosts multiple sites at the same address.

All later standards (HTTP/2 HTTP/3) however are too complex for the described approach and you will definitely need a separate library that deals with the additional complexity – if you really need to use this!