Connecting to an API: Working with Data and APIs

Data is presented in different formats. This guide contains information, tips and suggestions on how to use some of the formats presented in the Open Data Portal as well as how to work with some of the application programming interfaces (APIs).

In general, APIs are intended for developer use. This section will cover examples using various tools and programming languages. Consider searching the Open Data Portal for datasets if you do not have any programming experience.

On this page

Using cURL

cURL is a command-line tool used to transfer data from or to a server over various TCP/IP protocols. In this case, we will review how to use cURL to request data from an HTTP RESTful API. cURL can be installed on Windows, OSX or Linux distributions.

Windows

OSX

  • via Macports;
  • via Homebrew; or
  • Download cURL

Linux

  • A package manager such as YUM, APT-GET or the GUI software package installer; or
  • Download cURL.

Once installed, cURL should be accessible from a command-line prompt by simply typing curl. Typing curl will not yield anything useful other than confirm ou have it installed. At a minimum, you need to specify the destination URL.

You may also need to provide some parameters including HTTP headers to successfully query an API. In our example, the API recognizes the Accept and Accept-Language headers.

curl -H "Accept: application/json" -H "Accept-Language: en" \
  "http://www.earthquakescanada.nrcan.gc.ca/api/v2/locations/" > data-raw.json

By default, cURL will output the response from the destination onto the console screen (STDOUT). The above example redirects the output to a file named data-raw.json which will contain the English dataset in JSON format.

Figure 1 - A raw JSON response
Figure 1
Accessible description of Figure 1

A screenshot of a JSON response file in its raw minified format.

In most cases, the response will be optimized (minified) to reduce the data transfer size (Figure 6). You can use another command-line tool such as jsonpp to format the data into a human-readable output (Figure 7).

Figure 2 - A "pretty print" formatted JSON response
Figure 2
Accessible description of Figure 2

A screenshot of a JSON response file prettified for human readability.

Using Browser Plugins

The majority of browsers have plugins available which are tools to interact with RESTful APIs. These tools allow you to set parameters, headers, fetch a response and visualize the data. These can be very useful tools to quickly explore API data. Search your browser’s plugin repository to see what plugins are available for you.

Using Programming Languages

Most modern programming languages have the capabilities of accessing network-based resources such as HTTP-based services either natively or through libraries. We will cover some simple examples of how to get started using some common languages used for web and mobile development to access a RESTful API.

The sample programs below will each demonstrate how to query an API and parse the JSON response. The response is parsed to display the name of the result set and iterate over one of the result groups.

PHP

The following PHP example uses the cURL:

<?php
/**
 * Simple example of a PHP script used to query an API.
 *
 * NOTE: This is a simple example without any error handling.
 *
 * @license http://data.gc.ca/eng/open-government-licence-canada
 */

// Request URL
$url = 'http://www.earthquakescanada.nrcan.gc.ca/api/earthquakes/';

// Request HTTP headers
$headers = array(
    'Accept: application/json',
    'Accept-Language: en'
);

// Initialize the cURL object
$cu = curl_init($url);

// Set the cURL options
curl_setopt($cu, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($cu, CURLOPT_HTTPHEADER, $headers);

// Run the cURL request
$response = curl_exec($cu);

// Properly close the cURL object
curl_close($cu);

// Display response data
if ($response) {
    $json = json_decode($response);
    echo "{$json->metadata->request->name->en}\n";
    foreach($json->latest as $key=>$value) {
        echo "{$key} -> {$value}\n";
    } 
}

Output:

Earthquake listings available 
7d -> /api/v2/locations/7d.json 
30d -> /api/v2/locations/30d.json 
365d -> /api/v2/locations/365d.json 

Python

In Python, two libraries are required; the json and requests libraries which are used in the example below:

#!/usr/local/bin/python
# coding=utf-8
"""
 Simple example of a Python script used to query an API.

 @note:    This is a simple example without any error handling.
 @license: http://data.gc.ca/eng/open-government-licence-canada
"""
import json
import requests

url      = 'http://www.earthquakescanada.nrcan.gc.ca/api/earthquakes/'
options  = { "Accept":"application/json", "Accept-Language":"en" }
response = requests.get(url, headers=options)
jdata    = response.json()

print jdata['metadata']['request']['name']['en']

for (key, value) in jdata['latest'].items():
    print key, "->", value

Output:

Earthquake listings available 
365d -> /api/v2/locations/365d.json 
7d -> /api/v2/locations/7d.json 
30d -> /api/v2/locations/30d.json 

JavaScript jQuery

JavaScript executed within a browser is bound by security restrictions around connecting to 3rd party web sites. By default, remote queries are denied by browsers as it is considered a Cross-Site Scripting (XSS). Remote queries are permitted if the API response contains an appropriate Access-Control-Allow-Origin HTTP header allowing foreign queries. The following example connects to an API using the jQuery library.

<!DOCTYPE html>
<html lang="en">
<head>
    <title>Demo API call using jQuery</title>
</head>
<body>

<div id="short"></div>
---
<div id="long"></div>

<script src="//ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script>
/**
 * Simple example of a jQuery JavaScript used to query an API.
 *
 * NOTE: This is a simple example with very little error handling.
 * @license: http://data.gc.ca/eng/open-government-licence-canada
 */
(function( $ ) {
    $(document).ready(function() {

        // Shorthand method
        $.getJSON('http://www.earthquakescanada.nrcan.gc.ca/api/earthquakes/', 
            function(json) {
                $('#short').append(json.metadata.request.name.en + '<br />');
                $.each(json.latest, function(idx, item) {
                    $('#short').append(idx + ' -> ' + item + '<br />');
                });
                return;
            }
        );

        // Expanded method
        $.ajax({
            url: 'http://www.earthquakescanada.nrcan.gc.ca/api/earthquakes/',
            dataType: 'json',
            headers: { 
                'Accept'          : 'application/json', 
                'Accept-Language' : 'fr' 
            },
            crossDomain: true,
            type: 'GET',
            success: function(json) {                        
                $('#long').append(json.metadata.request.name.fr + '<br />');
                $.each(json.latest, function(idx, item) {
                    $('#long').append(idx + ' -> ' + item + '<br />');
                });
                return;
            },
            error: function(req, status, error) {
                $('#long').append(
                    status.toString() + ': ' + 
                    error.toString()
                );
                return;
            },
        });
    });
})( jQuery );
</script>
</body>
</html>

Output:

Earthquake listings available 
7d -> /api/v2/locations/latest/7d.json 
30d -> /api/v2/locations/latest/30d.json 
365d -> /api/v2/locations/latest/365d.json 
--- 
Annonces de séismes disponibles 
7d -> /api/v2/locations/latest/7d.json 
30d -> /api/v2/locations/latest/30d.json 
365d -> /api/v2/locations/latest/365d.json 

Java

There are several 3rd party libraries for Java SE which allow you to easily work with JSON data. For this example, we are using the JSON.org library which is also available via Maven.

package com.tbs.devcorner.simple;
/**
 * Simple example of a Java program used to query an API.
 * @license http://data.gc.ca/eng/open-government-licence-canada
 *
 * Maven Dependency:
 *   <dependency>
 *     <groupId>org.json</groupId>
 *     <artifactId>json</artifactId>
 *     <version>20131018</version>
 *   </dependency>
 */
import java.io.*;
import java.net.*;
import org.json.*;

public class App 
{
    public static void main( String[] args )
    {
        try {
            // Build Connection
            URL api_url = new URL("http://www.earthquakescanada.nrcan.gc.ca/api/earthquakes/");
            URLConnection api = api_url.openConnection();

            // Set HTTP Headers
            api.setRequestProperty("Accept", "application/json");
            api.setRequestProperty("Accept-Language", "en");

            // Get Response
            JSONTokener tokener = new JSONTokener(api.getInputStream());
            JSONObject jsondata = new JSONObject(tokener);

            // Display API name
            System.out.println(jsondata.getJSONObject("metadata")
                                       .getJSONObject("request")
                                       .getJSONObject("name")
                                       .get("en").toString());

            // Iterate over latest links
            JSONObject latest = jsondata.getJSONObject("latest");
            for (Object item : latest.keySet()) {
                System.out.println(item.toString() + " -> " + latest.get(item.toString()));
            }

        } catch (MalformedURLException e) {
            System.out.println("Malformed URL");
        } catch (IOException e) {
            System.out.println("IO Error");
        }
    }
}

Output:

Earthquake listings available 
7d -> /api/v2/locations/7d.json 
30d -> /api/v2/locations/30d.json 
365d -> /api/v2/locations/365d.json
Date modified: