JRC REST Services Directory

Service tools

Language

JOINT RESEARCH CENTRE

REST Services Directory

Navigation path

REST Services Directory Help

Background

This document contains information for developers and software engineers on the JRC REST Services Directory. The REST Services Directory is a catalog of information services available from JRC that can be used within information systems such as websites, desktop software and client applications. The services provide a wide range of information produced by JRC in the public domain and within specific thematic areas, e.g. biodiversity, land resource management etc. All of these services are available over the internet as simple URLs.

This help document provides information for both providers and consumers of the REST Services. For details on how to publish your data through REST services, see here. For details on how to access and use the services see the following section.

Understanding the REST Services Directory

There are many ways in which the REST Services can be consumed and used from applications, websites and tools etc. REST, short for Representational State Transfer, provides a simple, open Web interface to services hosted by JRC. All services exposed by the REST Services Directory are accessible through a hierarchy of endpoints or Uniform Resource Locators (URLs) for each service published at JRC.

When using the REST Services, you typically start a the root level which shows all of the databases that have REST Services available. Clicking on one of these databases then shows you the next level down in the REST Services Directory which are the schemas for that database. These describe the thematic areas within that database where the services are located. Clicking on the schema then shows you all of the available REST Services that are available within that particular schema.

The final level in the REST Services hierarchy is the service level itself which includes detailed information on each individual service in the directory. This page includes a description of the service, the input and output parameters and some example calls. The url for the service page is the endpoint for the service and this is the address that you will use to consume the service. In order to consume the service you will also need to pass in a number of parameters which are described in the next section.

REST request

REST requests can be created using the information on the service page. The list of parameters for the service includes the input parameters that you must pass to the service and the output parameters that are returned from the server (if specified in the fields parameter). For the input parameters, a service can have mandatory parameters and optional parameters (the parameter name is in italics for optional parameters). For each of the parameters the following information is given:

Name	The name of the parameter (optional parameters are shown in italics)
Description	A full description of the parameter
Type	The data type of the parameter. See the section on input parameter data types below.
Default value	(input only) If a parameter is optional then a default value will be specified in this column. Default values are values that will be included in the REST request if not explicitly specified.

Input parameter data types

All services contain input parameters with specific data types which must be complied with in order to use the services succesfully. The following data types are supported for input parameters:

Data type	Represents	Encoding	Example
integer	Positive 4 byte integer value	None	wdpa_id=785
string	Variable length characer data	None	rl_status=EN
double	Double precision number	None	percent=20.62
array	Variable length array of strings or integers	Using commas to separate the items	rl_status=EN,CR
date	Date only	Encoded as a string using a mm/dd/yyyy date format	startdate=03/31/2012
datetime	Date and time	Encoded as a string using a mm/dd/yyyy hh:mm:ss timedate format	startdate=03/31/2012 10:15:25
enumeration	Discrete set of valid values	String	format=json

For example, the following REST Service call includes two parameters (one a mandatory integer and one an optional array of numbers) and returns a list of species for a particular protected area:

http://dopa-services.jrc.ec.europa.eu/services/ibex/especies/get_pa_species_list?wdpa_id=785&category=EN,CR

Standard optional parameters

For all REST services there are a number of standard optional parameters that can also be specified in the REST service call:

Name	Description	Default value
format	The format parameter is used to specify the response format and can be one of the following values: json - This format is a standard Javascript based format widely used in REST services where the information is encoded within objects and arrays. array - Identical to the json format except that the field names are not repeated for each record thus keeping the size of the return data to a minimum. The field names can be retrieved using the metadata section of the response. xml - A standard format type for web services and includes the information in elements and attributes. Currently, the xml format does not include a metadata section. xmlverbose - A standard format type for web services which in this case includes all data in nested elements. Currently, the xml format does not include a metadata section. pdf - Adobe Acrobat format which can be used for printing html - web page format csv - Simple comma-delimited format for easy import into databases and spreadsheets If an invalid value is used then an 'Invalid response format' error is returned.	json
fields	The fields parameter is used to determine the fields that are returned and is encoded as a comma-separated list of field names. These field names must match one or more of the output parameters listed in the service description. The fields parameter also determines the order in which the output fields are returned. If an invalid field is requested then an 'Invalid output fields' error is returned.	all fields
sortfield	The sortfield parameter specifies the field that will be used to sort the returned records. If no sortfield is specified the results are returned in the default sort order as specified in the database function. If an invalid sortfield is requested then an 'Invalid sortfield' error is returned.
dplimit	The dplimit parameter specifies the number of decimal places that any floating point values are rounded to in the resulting data. This parameters only applies to fields of type='float' and does not apply to fields with the names 'lat' or 'lng'. These are assumed to represent latitude and longitude values.	2
includemetadata	The includemetadata parameter sets whether to include a metadata section in the response which includes summary information about the results. See the metadata section for more information.	true
parseparams	The parseparams parameter sets whether to parse the input parameters in the service call. If true then input parameter names are validated against the database and invalid parameters will raise errors. If false then parameters with invalid names will be ignored by the server.	true
metadataname	The metadataname parameter determines the name of the metadata object that is returned. For extJS clients this should be set to 'metaData'.	metadata
rootname	The rootname parameter determines the name of the root section that contains all of the actual records that are returned.	records

For example, the following REST Service call includes values for all optional parameters :

http://dopa-services.jrc.ec.europa.eu/services/ibex/especies/get_pa_species_list ?wdpa_id=785&category=EN,CR&format=json&fields=iucn_species_id&sortfield=taxon &dplimit=3&includemetadata=true&parseparams=true&metadataname=md&rootname=r

Specifying parameter values

Parameter values for the REST Service calls can be done by simply appending the parameter values to the end of the service end point as a set of key/value pairs preceeded by a question mark. There are example calls in each of the service pages that show the syntax for calling a particular service (both using mandatory and optional parameters). All mandatory parameter values must be specified, and all data types must be adhered to otherwise the service call will fail. Parameters can be specified in any order (both mandatory and optional) and any parameters whose names do not match those of the input/output parameters will return an error (unless the parseparams flag is set to false). If mandatory parameters are missing or values are invalid then the server will return an error. Finally, parameter values are case sensitive, e.g. category=VU and not category=vu.

REST response

The REST response from the server will include all of the data and also an optional metadata section (specified by the includemetadata parameter). The sample below shows a typical json response:

{
"records": [
  {
   "iucn_species_id": 150726,
   "taxon": "Abroscopus superciliaris",
   "min_presence_id": 2,
   "kingdom": "ANIMALIA",
   "phylum": "CHORDATA",
   "class": "AVES",
   "order": "Passeriformes",
   "family": "Sylviidae",
   "status": "LC",
   "assessed": "2009/01/01"
  },
..
..
  {
   "iucn_species_id": 148255,
   "taxon": "Zosterops everetti",
   "min_presence_id": 2,
   "kingdom": "ANIMALIA",
   "phylum": "CHORDATA",
   "class": "AVES",
   "order": "Passeriformes",
   "family": "Zosteropidae",
   "status": "LC",
   "assessed": "2009/01/01"
  }
],
"metaData": {
  "duration": "0:00:00.034309",
  "error": null,
  "idProperty": "iucn_species_id",
  "successProperty": "success",
  "totalProperty": "recordCount",
  "success": true,
  "recordCount": 705,
  "root": "records",
  "fields": [
   {
    "type": "integer",
    "name": "iucn_species_id"
   },
   {
    "type": "string",
    "name": "taxon"
   },
   {
    "type": "integer",
    "name": "min_presence_id"
   },
   {
    "type": "string",
    "name": "kingdom"
   },
   {
    "type": "string",
    "name": "phylum"
   },
   {
    "type": "string",
    "name": "class"
   },
   {
    "type": "string",
    "name": "order"
   },
   {
    "type": "string",
    "name": "family"
   },
   {
    "type": "string",
    "name": "status"
   },
   {
    "type": "string",
    "name": "assessed"
   }
  ]
}
}

Metadata section

At the end of the response there is an optional metadata section (this section will only be present if the includemetadata parameter is true). The metadata section includes information about the actual REST service request and includes a fields section which describes all the returned fields and their data types together with a number of other parameters (shown below). The metadata section can be used by developers for working with the schema of the response.

Name	Description
duration	The duration of the query on the database server (NOTE: this is not the duration of the web response which may be considerably longer due to network issues)
error	A detailed description of any errors returned from the server if the call did not succeed (i.e. "success": false).
idProperty	The name of the field to use as the unique identifier for the returned data. This field can be used in data aware controls.
successProperty	The name of the property in the metaData section that includes if the request was successful or not. This parameter is to support extJS javascript library clients. This value is 'success' by default.
totalProperty	The name of the property in the metaData section that includes a count of all of the records returned. This parameter is to support extJS javascript library clients. This value is 'recordCount' by default.
success	true if the call was succesful, false if the server returned an error. If an error is returned then the error details will be included in the error section
recordCount	The total number of records returned within the records section
root	The name of the root element or container in the response that contains all of the actual records.
fields	A list of all of the fields that are returned for each record. Each field shows the name of the field and its data type. For more information on return data types see the section below.

Output parameter data types

The following data types are supported for output parameters:

Data type	Represents
boolean	true or false
integer	Positive 4 byte integer value
number	<TODO>
float	A floating point number currently return to 4 decimal places
string	Variable length characer data
date	Date only

Records section

The records section of the response includes all of the data requested by the REST service call as an array of rows. Each row contains the information requested in the fields parameter, or all fields if the fields parameter is not specified. All data will be encoded using the standard encoding for that format.

Consuming REST Services

Consuming REST services in websites

REST services can be accessed and used in websites using Javascript by using native Javascript HTTP GET methods or by using Javascript Frameworks, like jQuery or Dojo. The following examples show how to use the REST services from a number of commonly used libraries.

Using Dojo

This simple example HTML page shows you how to request a list of species for a protected area and show the number of species found together with the list. The following example can be copy and pasted into JSBin to see it work:

Using extJS

The following example HTML page shows you how to request a information from a REST service and put that into an extJS chart. The following example can be copy and pasted into JSBin to see it work:

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en">
<head>
  <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
  <title>DOPA REST Client</title>
  <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.7.1/jquery.min.js" type="text/javascript"></script>
  <script src="http://extjs.cachefly.net/ext-3.4.0/adapter/ext/ext-base.js" type="text/javascript"></script>
  <script src="http://extjs.cachefly.net/ext-3.4.0/ext-all.js" type="text/javascript"></script>
  <script src="http://code.highcharts.com/2.3.2/highcharts.js" type="text/javascript"></script>
  <script language="JavaScript" type="text/javascript">
   Ext.onReady(function() {
    var chartOptions = {
     chart : {
      renderTo : 'container',
      type : 'column'
     },
     yAxis : {
      title : {
       text : 'Agricultural pressure'
      }
     },
     xAxis : {
      title : {
       text : 'Ranking for Tanzania'
      }
     },
     series : [{
      data : [],
     }]
    };
    var datastore = new Ext.data.Store({
     autoDestroy : true,
     autoLoad : {
      params : {
       'countryid' : 834,
       'parseparams' : false,
       'metadataname' : 'metaData'
      },
      callback : function(records) {
       Ext.each(records, function(item) {
        chartOptions.series[0].data.push([String(item.get('wdpa_id')), item.get('normalised_ap')]);
       }, this);
       var chart = new Highcharts.Chart(chartOptions);
      }
     },
     proxy : new Ext.data.ScriptTagProxy({
      method : 'GET',
      url : 'http://dopa-services.jrc.ec.europa.eu/services/ibex/ehabitat/get_agri_pressure_country',
     }),
     reader : new Ext.data.JsonReader()
    });
   });
  </script>
</head>
<body>
  <div id="container" style="min-width: 400px; height: 400px; margin: 0 auto"></div>
</body>
</html>

Consuming REST services in desktop tools

The REST services can also be used in desktop applications, such as Microsoft Office, by using the service endpoint. For many desktop applications the format parameter should specified as xml.

Importing data into Microsoft Excel

The following walkthrough shows you how to use the REST services in Microsoft Excel 2010 by retrieving a list of species for a protected area:

Open Microsoft Excel 2010 and click on the Data tab
Click on 'From Other Sources' and select 'From XML Data Import'
In the File name text box, enter the endpoint of the REST service that contains the data you require - this is shown in the Example Calls section of the Service Page (e.g. http://dopa-services.jrc.ec.europa.eu/services/ibex/especies/get_pa_species_list?wdpa_id=785&format=xml)
Click Open and when prompted select where you want to put the data
The data is imported into Excel as a spreadsheet.

Refreshing data into Microsoft Excel

One of the advantages of importing data into Desktop tools is that it can be simply refreshed when the data from the service changes. To update the data in Excel:

Open Microsoft Excel 2010 and click on the tab that contains the data you want to update
Click on 'Table Tools | Design' and click on 'Refresh'. This will update the data in the spreadsheet using the new data from the server.
To update any pivot tables, charts or reports, click on the appropriate tab and click on Refresh

Publishing REST Services

Background

The DOPA REST Services Directory makes available in HTML format, functions from a Postgresql database. It supports the following features to expose your data as a REST service:

Creates HTML pages from schemas and functions in a Postgresql database
Publishes the descriptions as HTML for functions and function parameters based on COMMENTS in the database
Creates a service page for each function in a postgresql schema including information on the parameters, descriptions, types and default values
Creates sample REST service calls for each service

To take advantage of these features you simply need to create a function in Postgresql with a description. Optionally you can add additional information to the function such as parameter descriptions.

Step-by-step service publishing

Create the Function to publish

In Postgresql, create a normal database function to retrieve data, optionally with a number of input parameters. The name of the function will determine its visiblity within the REST Services Directory. Currently functions that begin with 'get', 'set' or 'insert' are shown in the REST Services Directory. In addition, if a function begins with an underscore it will only be shown internally and not on the publically visible REST Services Directory. This can be useful for debugging/testing purposes. In addition, the function must be accessible using the restuser account.

Define the input parameters

Any input parameters in your function will automatically be shown in the REST Services Directory. Parameter names cannot be Postgresql reserved words, e.g. user, value. If you want to include default parameters in your REST service then you need to implement them in your function. For example:

CREATE OR REPLACE FUNCTION getcountryspeciescount(country_id integer, presence_id integer[] DEFAULT ARRAY[1,2,3]) RETURNS bigint AS ..

NOTE: There is a bug in pgAdmin that prevents you from recreating a function with a default value of ARRAY[1,2,3]. You will need to edit the CREATE script that is generated by pgAdmin to correctly create a function with a default value of an ARRAY.

Define the return parameters

Scalar type

If you want to return simple scalar data then define the output as a simple data type. For example:

CREATE OR REPLACE FUNCTION getcountryspeciescount(country_id integer, presence_id integer[]) RETURNS bigint AS ..

This will return a single value with the name of the function by default (e.g. in this case {"getcountryspeciescount": 425}). If you want to specify a name for the return value then you will need to name it as an OUT parameter:

CREATE OR REPLACE FUNCTION getcountryspeciescount(IN country_id integer, IN presence_id integer[], OUT spcount bigint) RETURNS bigint AS ..

Table type

To return data with a specific schema, define a TABLE return data type (see here for more information). For example:

CREATE OR REPLACE FUNCTION getirreplacibilityindices(IN wdpa_id integer) RETURNS TABLE(taxonomic_group character varying, irreplacibility_score double precision) AS ..

If your function returns all fields from a particular table (for example 'select * from <tablename>) then if you modify your table definition by adding fields then service call will fail because the new schema of the table will not match the functions declared return type. To prevent this, specify the names of the fields in the sql rather than using a wildcard *. If you want your service to be accessible using data-aware controls you will also need to supply a unique identifier with your response. By default the idProperty in the Response metadata lists the first field that is returned as the id field. So in this case you will need to ensure that the id field is returned first.

Write a description for the function

By default if a function is not commented then it will not be shown in the Services Directory. Create comments for functions using the usual COMMENT statement. For example:

COMMENT ON FUNCTION getspeciespalist(bigint) IS 'Returns a list of protected areas that a species range intersects with.';

Specify the field that contains a unique identifier (optional)

If your data contains a field which is a unique identifier then this should be the first field in the return data. The name of this field will also be returned in the metadata sections 'idProperty' value. Many Javascript client libraries need a unique identifier to be present in the returned data.

Write a description for the function parameters (optional)

There is currently no mechanism to provide comments or descriptions for function parameters in Postgresql so a workaround has had to be developed which involves writing the comments for the function parameters in the comment for the function. The syntax for doing this is:

COMMENT ON FUNCTION <functionname> IS '<function description> {<param description 1>$<param description 2>$<param description 3> etc. }';

Parameter descriptions are given inside braces {} and each parameter description is separated using a dollar sign $. These descriptions are optional and if they are not given then 'No Description' appears alongside the parameter in the REST Services Directory. Parameter descriptions can also include HTML markup and so, for example, you can include links in your descriptions for functions or function parameters. However, all apostrophes will need to be escaped using the Postgresql escaping of a single quote. So for example, the following comment includes full escaping to include hyperlinks in the COMMENT for a function:

COMMENT ON FUNCTION getspeciespalist(bigint) IS 'Returns a list of protected areas that a species range intersects with.{IUCN Species Identifier from the Red List website. To find the IUCN Species Identifier, search for a species and then look for the number in the URL, e.g. Orang Utan is 17975$The unique identifier for the protected area as shown in the protectedplanet.net website. See the <a href=''http://protectedplanet.net'' target=''_protectedplanet''/>Protected Planet website</a> for more information$The country for the protected area$The name of the protected area$The designation for the protected area$The IUCN Category for the protected area taken from the following values: <br/>Ia: Strict Nature Reserve, Ib: Wilderness Area, II: National Park, III: Natural Monument or Feature, IV: Habitat/Species Management Area, V: Protected Landscape/Seascape, VI: Protected area with sustainable use of natural resources (see <a href=''http://www.unep-wcmc.org/iucn-protected-area-management-categories_591.html'' target=''wcmc''>here</a> for more information)}';

If only some parameters have a description available then all of the rest of the parameters will appear with 'No Description' beside the parameter name in the Services Directory.

Licensing and Copyright

When publishing services into the REST Services Directory, particular care must be taken to abide by any license or copyright restrictions, particularly where data has been derived directly from an external partner. Currently JRC are not licensed to redeliver any spatial data from external partners including IUCN and UNEP-WCMC.

Support

For support/comments/amendments please email: ec-dl-dopa@ec.europa.eu

Use of these web services is subject to the terms and conditions of use governing the original datasets. These restrict the use of the data for commercial purposes and require users to acknowledge the source of the data. JRC would like to be informed should the web services be used regularly.
For full terms and conditions see here.