WordPress Plugin Development: Talking to APIs

Introduction

For some WordPress Plugins, its better to not implement functionality and rather use an external service through an API. These use cases may include data-heavy or computation-heavy work.

For our WordPress Search Plugin CurrySearch we decided to implement core functionality as a separate service which is written in Rust.

This post will walk you through our findings, give you some general advice and talks about solutions to common pitfalls.

General Advice

Build a wrapper

Calling an API usually takes an URL that is made up from following components:

But instead of combining them ad hoc, you should use constants for the individual components and define methods which accept parameters and do the actual call to the API.

This is a trimmed down example which we will use throughout this post.

<?php
class API {
    const BASE_URL = "https://api.example.com/";
    const GET_DATA_METHOD = "get/data/";
    const POST_DATA_METHOD = "post/data/";

    static function get_data($id) {
        $url = BASE_URL.GET_DATA_METHOD.urlencode($id);

        // Do the actual and return the decoded result
    }

    static function post_data($data) {
        $url = BASE_URL.POST_DATA_METHOD;

        // Do the actual call and return the decoded result
    }
}
?>

Be aware of user input

As you may have noticed from the previous example we didn’t just append the id to the url. We called urlencode before. This is to make sure that no illegal characters reach the api and minimize risk.

WordPress Sites are generally a hostile environment and no one and nothing should be trusted.

It also makes sense to check the input for validity (e.g. if it should be an int, make sure it is).

Check you API provider and understand its limitations

Blindly using any API on find on the internet may result in a malfunctioning plugin or even you violate the terms of this API. Therefor its important to check the API and its provider before using it.

Some things to watch out for:

Often APIs are limited in the number of request per day or require authentication through an ApiKey to be able to talk to them. Make sure you understand these limitations.

Using the WordPress HTTP API

Using the example from the first section we will now implement the details using the WordPress HTTP API. Recommended by the official WordPress Developer Handbook, there are good reasons to use it.

The main reasons are that it is easy to use, you can be sure it is supported and it is harder to shoot yourself in the foot.

Let’s take it for a spin:

<?php
class API {
    const BASE_URL = "https://api.example.com/";
    const GET_DATA_METHOD = "get/data/";
    const POST_DATA_METHOD = "post/data/";

    static function get_data($id) {
        $url = BASE_URL.GET_DATA_METHOD.urlencode($id);
        $response = wp_remote_get($url)

        // Decode the response into an associative array and return it
        return json_decode(wp_remote_retrieve_body($response), true);
    }

    static function post_data($data) {
        $url = BASE_URL.POST_DATA_METHOD;

        // Set the content type to application/json and add a body
        $response = wp_remote_post($url, array(
            'headers' => array(
                'Content-Type' => 'application/json'
            ),
            'body' => json_encode($data)));

        return json_decode(wp_remote_retrieve_body($response), true);
    }
}
?>

Simple. This approach should cover most use cases.

PHP and HTTPS Connections

Even though the WordPress HTTP API is a bit slower than using curl directly, the main advantage when using curl is, that connections to an API can be kept open.

Let me expand a bit on this: Modern APIs usually use the HTTPS protocol (if not, they should). HTTPS usually has no impact on performance for browsers and servers but for WordPress plugins it may matter. Why, you ask?

HTTPS connections are relatively expensive. Not compute-wise but network-wise. To open a HTTPS connection it takes about 3x to 4x the latency of opening a HTTP connection. Most browser and web servers allow keeping connections open. So this penalty is payed once and all subsequent requests use an already open connection which does not entail the latency increase.

PHP’s design ,on the other hand, makes it very hard to implement keep-alive HTTPS connections that can survive multiple requests. The very related problem of database connections is solved internally. Alas, for HTTPS connections there is no easy solution.

When frequently talking to APIs building a new HTTPS connection for each request will cost precious time.

Improving Performance by using curl

While it is not possible to keep connections open throughout multiple (client) requests, it is possible to reuse connections during a single request.

Alas, the WordPress HTTP API does not support this. We are forced to use more blunt tools: curl.

Before you throw yourself into curl make sure you understand the trade offs:

Make sure you really have a performance problem before turning to curl.


With all the warnings let me now show you how to use curl to keep connections alive during a single request:

<?php
class API {
    static $curl_handle = NULL;


    const BASE_URL = "https://api.example.com/";
    const GET_DATA_METHOD = "get/data/";
    const POST_DATA_METHOD = "post/data/";

    static function get_data($id) {
        $url = BASE_URL.GET_DATA_METHOD.urlencode($id);

        if (curl_installed() {
            //curl is installed and we can use it

            //Initialize the curl handle if it is not initialized yet
            if (!isset($curl_handle)) {
			    $curl_handle = curl_init();
			}

            // Copy it and fill it with your parameters
            $ch = curl_copy_handle($curl_handle);
            curl_setopt($ch, CURLOPT_URL, $url);
            curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "GET");


            // Enable keep alive
            $header = array(
				'Connection: keep-alive',
				'Keep-Alive: 300'
			);

            // Set the user agent which tells you the wordpress and php version and that curl is used
            // This will help you when debugging problems
            curl_setopt($ch, CURLOPT_USERAGENT, 'Curl/WordPress/'.$wp_version
                       .'/PHP'.phpversion().'; ' . home_url());

            // Do not echo result
            curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
            // Set header
			curl_setopt($ch, CURLOPT_HTTPHEADER, $header);
            // Set HTTP version to 1.1 to allow keepalive connections
			curl_setopt($ch, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_1);

            $response = curl_exec($ch);
			$http_status = curl_getinfo($ch, CURLINFO_HTTP_CODE);
			if ($http_status != 200) {
                //TODO: Handle error
			}
            curl_clone($ch);
            return json_decode($response, true);
        } else {
            //Curl is not installed. fallback to WP HTTP API

            $response = wp_remote_get($url)

            // Decode the response into an associative array and return it
            return json_decode(wp_remote_retrieve_body($response), true);
        }
    }

    static function post_data($data) {
        $url = BASE_URL.POST_DATA_METHOD;

        if (curl_installed() {
            //curl is installed and we can use it

            //Initialize the curl handle if it is not initialized yet
            if (!isset($curl_handle)) {
			    $curl_handle = curl_init();
			}

            // Copy it and fill it with your parameters
            $ch = curl_copy_handle($curl_handle);
            curl_setopt($ch, CURLOPT_URL, $url);

            // Encode payload and set post body
            $data_string = json_encode($data);
			curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
		   	curl_setopt($ch, CURLOPT_POSTFIELDS, $data_string);
	   		array_push($header, 'Content-Type: application/json');
   			array_push($header, 'Content-Length: ' . strlen($data_string));

            // Enable keep alive
            $header = array(
				'Connection: keep-alive',
				'Keep-Alive: 300'
			);

            // Set the user agent which tells you the wordpress and php version and that curl is used
            // This will help you when debugging problems
            curl_setopt($ch, CURLOPT_USERAGENT, 'Curl/WordPress/'.$wp_version.'/PHP'
                       .phpversion().'; ' . home_url());

            // Do not echo result
            curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
            // Set header
			curl_setopt($ch, CURLOPT_HTTPHEADER, $header);
            // Set HTTP version to 1.1 to allow keepalive connections
			curl_setopt($ch, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_1);

            $response = curl_exec($ch);
			$http_status = curl_getinfo($ch, CURLINFO_HTTP_CODE);
			if ($http_status != 200) {
                //TODO: Handle error
			}
            curl_clone($ch);
            return json_decode($response, true);
        } else {
            //Curl is not installed. fallback to WP HTTP API


            // Set the content type to application/json and add a body
            $response = wp_remote_post($url, array(
                'headers' => array(
                    'Content-Type' => 'application/json'
                 ),
                 'body' => json_encode($data)));

            return json_decode(wp_remote_retrieve_body($response), true);
        }
    }

    private static function curl_installed(){
		return function_exists('curl_version');
	}
}
?>

As you see, that’s much more code. And we didn’t even get rid of the old code. Nevertheless, with this you are now able to call the API multiple times using a single connection during a single client request.

The difference will not only be measurable but your users will feel it.

Improving Performance by taking a detour through the browser

Even though we are able to improve performance using curl, in some cases this might not be enough. Imagine a autocompletion API: The user types some letters and expects the autocomplete to respond instantly.

Using capabilities in WordPress you will not be able to achieve this: WordPress Ajax calls are slow , and even if you improve the ajax performance by cutting down on initializsation time, its most likely not going to be satisfactory for your users.

What you can do instead is to let the users browser talk directly to the API instead of routing the request through WordPress. But be aware: this makes your users IP visible to an external API and if you use ApiKeys as authentication it might not be possible.

Conclusions

You now know how to use the WordPress HTTP API to build wrappers around other APIs and are aware of problems caused by the way PHP handles HTTPS connections. In extreme cases you can fallback to a better curl solution.

If you are unhappy with the default WordPress Search be sure to check out CurrySearch!