api_tutorial.html

---
layout: tutorials
title: API Tutorial
permalink: /api_tutorial/
---
<div class="container mtb">
    <div class="row">
        <div class="col-lg-12">
            <div id="toc"></div>
            
            <!--add content below this comment-->
            <!--headers need ids in order to render automatically in the table of contents-->
            
            <h2 id="topheader"> Tutorial: APIs for Open Government Data</h2>
            <p>
            <p>Collecting data is essential for data curators. In order to learn how to collect data, this tutorial introduces the concept of an API, and will show you examples of and practical guidelines for accessing data and metadata from public sources using APIs. Specifically, this tutorial first teaches you how to work with SODA API, which is provided by an open government data portal provider, <a href="https://socrata.com" target="_blank">Socrata</a>. Once the tutorial is complete, you will know how to retrieve open government datasets and their metadata through the <a href="https://dev.socrata.com/consumers/getting-started.html" target="_blank">SODA API</a>, <a href="https://socratadiscovery.docs.apiary.io/#" target="_blank">Discovery API</a>, and <a href="https://socratametadataapi.docs.apiary.io/#" target="_blank">Metadata API</a>.
            <p>
            <h3 id="nextheaderone"> Learning Objectives </h3>
            <p> <ul> 
            <li>Understand the concept of an API.</li>
            <li>Use an API to collect data from public data sources such as open government data portals and open research data repositories.</li></ul>
            <h3 id="nextheadertwo"> Prerequisites </h3>
            <p> <ul> 
            <li>Computer and internet access
            <li>Basic understanding of open data and data repositories
            <li>Basic knowledge of popular open data portal software providers (<a href="https://ckan.org" target="_blank">CKAN</a>, <a href="https://socrata.com" target="_blank">Socrata</a>, <a href="https://dataverse.org" target="_blank">Dataverse</a>, etc.)</li>
            <li>Basic knowledge of common web exchange file formats, such as JSON, XML, and CSV
            <li>Basic knowledge of any programming language is helpful, but not required</li></ul>

            
            <h2 id="myHeader">Introduction to API</h2>
            <p>API stands for Application Programming Interface. An API defines how different applications or different software components talk to each other, providing a way for developers to simplify programming. Just as a graphical user interface makes it easier for people to use programs, APIs make it easier for developers to use functions or data from another application (or another component of an application) without needing to know the implementation details [<a href="https://www.mulesoft.com/cn/resources/api/what-is-an-api" target="_blank">1</a>][<a href="https://en.wikipedia.org/wiki/Application_programming_interface#Web_APIs" target="_blank">2</a>].</p>
            <p>In this tutorial, we will focus on web APIs. Applications such as Facebook, Twitter, and Google Maps provide access to some of their services and data through web APIs, which receive requests and send responses. For example, if you’re an iPhone user and you use Yelp to search for a restaurant, you will see an Apple Map embedded in Yelp showing you the location of the restaurant. Yelp doesn't provide map services; instead, it uses Apple’s API to integrate maps into its own functions. Another example of API usage is the ability to use Google or Facebook to log into other platforms. When you sign up for an account on a non-Google, non-Facebook website, you may be asked to sign in with your Facebook or Google account. When you do so, the website uses a Facebook or Google API to retrieve your account information.</p>
            <p>The APIs used in this tutorial adhere to an architectural style called REST, which stands for Representational State Transfer. As blogger Sarah Maddox explains, “REST services tend to offer an easy-to-parse URL structure consisting of nouns that reflect the logical hierarchical categories of the data on offer” [<a href="https://ffeathers.wordpress.com/2014/02/16/api-types/" target="_blank">3</a>]. An example of a REST API web URL is: <a href="http://api.us.socrata.com/api/catalog/v1/domains" target="_blank">http://api.us.socrata.com/api/catalog/v1/domains</a>. When using a REST API, you will need to know how to structure your query according to rules, and these rules can usually be found in the API documentation for the web application.</p>
            <p>Why should we learn about API? Software developers use APIs to create well-structured applications, and data scientists and data curators use APIs to collect large amounts of data automatically. Thus, learning to access data and metadata using an API will be the main objective of this tutorial.</p>
                
            <h2 id="secondheading">Example Overview</h2>
            <p>We will walk through two examples of retrieving datasets from <a href="https://data.seattle.gov" target="_blank">the municipal open data portal of City of Seattle</a> using an API:
                <p> <ul> 
            <li>Example 1: Retrieve open data from open government data platforms using an API</li>
            <li>Example 2: Retrieve open government data metadata using API</li></ul>
            <p>The underlying software, Socrata, provides APIs for accessing, filtering, and downloading data and metadata. To learn more about Socrata's API, visit their <a href="https://dev.socrata.com/consumers/getting-started.html" target="_blank">API starter guide</a>.</p>
            
            <h3 id="thirdheading">Example 1: Retrieve data from Socrata Platforms</h3>
            
            <p><b>Step 1:</b> First, take some time to explore Seattle Open Data Portal datasets and choose a dataset that interests you. For example, <a href="https://data.seattle.gov/Transportation/Road-Weather-Information-Stations/egc4-d24i" target="_blank">Road Weather Information Stations</a>. API documentation is in the upper right corner of the data landing page, as shown in the image below: 
            <p><img src="http://odl.ischool.uw.edu/assets/img/apitutorial-i1-roadweatherinfostations.png">
            <p>Click “API” and copy the “API Endpoint” using the default JSON data format. Then, open a new tab or window and paste the link into the browser. Then, try it again, selecting a different data format, such as CSV. Finally, click “API” again, but this time, click “API Docs," instead of using an endpoint. 
            <p><b>Step 2:</b> On the “API Docs” page, scroll down to “Fields." Read through this section, and then try different fields, and their subsequent filters. To do so, click the “+” next to a field.  For example, when you expand “stationname” (see image below), you can see a sample filter by clicking “try it.” This shows you what type of response (data) you will get.
            <p><img src="http://odl.ischool.uw.edu/assets/img/apitutorial-i2-tryit.png">
            <p><b>Step 3:</b> Now, go to <a href="https://dev.socrata.com/docs/endpoints.html" target="_blank">https://dev.socrata.com/docs/endpoints.html</a>. Review the API endpoint concept, and try the examples on the page.
                
            <h3 id="fourthheading">Example 2: Retrieve catalog information and metadata using Socrata's API</h3>
            
            <p><h4 id="fourthheadingsubone"><b>Example 2a: Explore catalog-level metadata using Discovery API</b></h4>
            
            <p><b>Step 1:</b> Go to <a href="https://socratadiscovery.docs.apiary.io/" target="_blank">Discovery API</a>, read the introduction, and scroll through the rest of the content. Try a few examples by clicking on the text boxes that begin with “Search by . . .” You can also view different code examples by choosing from the drop-down menu under “Request” that is set by default to “Raw.” Take a look at a few code examples, especially if you are familiar with any of the provided languages.
            <p><img src="http://odl.ischool.uw.edu/assets/img/apitutorial-i3-findbyid.png">
            <p><b>Step 2:</b> After trying some of the examples in the documentation, try customizing your own API call in a browser window. Here are some examples:
            <p><ul>
            <li><b>1)</b> Find the list of domains and the resource count for each Socrata-hosted domain using the following API call: <a href="http://api.us.socrata.com/api/catalog/v1/domains" target="_blank">http://api.us.socrata.com/api/catalog/v1/domains</a>. (The link should open in a new tab; however, you can also copy and paste it into a new tab or window.)
            <p><img src="http://odl.ischool.uw.edu/assets/img/apitutorial-i4-thingcodeportal.png">
            <p>In the image above (and in the link you followed for the API call above), “thing” represents portals hosted by Socrata, and “count” represents the number of resources each portal possesses. You can then search within a particular domain. 
            <p>
            <p>Here is an example: <a href="http://api.us.socrata.com/api/catalog/v1domains=data.seattle.gov" target="_blank">http://api.us.socrata.com/api/catalog/v1domains=data.seattle.gov</a>. Results for this API call are displayed below. 
            <p><img src="http://odl.ischool.uw.edu/assets/img/apitutorial-i5-domain.png">
            <p>Try it yourself with a different domain, such as data.nasa.gov.</li>
           
            <li><b>2)</b> Get all metadata of resources that belong to category “public safety” using the following API call: <a href="http://api.us.socrata.com/api/catalog/v1?categories=public%20safety" target="_blank">http://api.us.socrata.com/api/catalog/v1?categories=public%20safety</a>. In this call, the results contain metadata records that belong to category “public safety” of all Socrata-hosted portals (see image below).
            <p><img src="http://odl.ischool.uw.edu/assets/img/apitutorial-i6-results.png">
            <p>Try the search again with a different category. You can find a list of all possible categories with this API call: <a href="http://api.us.socrata.com/api/catalog/v1/categories" target="_blank">http://api.us.socrata.com/api/catalog/v1/categories.</a></li>
            
            <p><h4 id="fourthheadingsubtwo"><b>Example 2b: Retrieve asset-level metadata of datasets using Metadata API</b></h4>
            
            <p><b>Step 1:</b> Go to Socrata’s <a href="https://socratametadataapi.docs.apiary.io/#" target="_blank">Metadata API</a> page. Read the introduction and try a few examples. For example, you could choose to retrieve all metadata on a domain (see image below). 
            <p><img src="http://odl.ischool.uw.edu/assets/img/apitutorial-i7-retrieveallmetadata.png">
            <p>
            <p><b>Step 2:</b> Customize your API calls to retrieve metadata of interest to you. Add different parameters. For example, you can start by working with existing examples provided by Socrata. 
            <p>
            <p><img src="http://odl.ischool.uw.edu/assets/img/apitutorial-i8-pagingthroughmetadata.png">
            <p>The API call, <a href="http://evergreen.data.socrata.com/api/views/metadata/v1?limit=10&page=1" target="_blank">http://evergreen.data.socrata.com/api/views/metadata/v1?limit=10&page=1</a> will return the 10 metadata records of page one from <a href="https://evergreen.data.socrata.com" target="_blank">https://evergreen.data.socrata.com</a>. 
            <p><img src="http://odl.ischool.uw.edu/assets/img/apitutorial-i9-americaincrisis.png">
            <p>Try these queries with a different domain, such as one you discovered in Example 2a, Step 1. For example, we can see the metadata from the Seattle domain with the following API call: <a href="http://data.seattle.gov/api/views/metadata/v1/" target="_blank">http://data.seattle.gov/api/views/metadata/v1/</a>. Notice how we replaced evergreen.data.socrata.com in the URL with data.seattle.gov (see image below).
            <p><img src="http://odl.ischool.uw.edu/assets/img/apitutorial-i10-dataseattlesocrata.png">
            
            
            <h2 id="fifthheading">Resources</h2>
            <p>1. REST API: <a href="https://en.wikipedia.org/wiki/Representational_state_transfer " target="_blank">https://en.wikipedia.org/wiki/Representational_state_transfer</a>
            <p>2. Other useful APIs: 
            <ul>
            <li><a href="https://developer.twitter.com/" target="_blank">Twitter API</a></li>
            <li><a href="https://www.ncbi.nlm.nih.gov/home/develop/api/" target="_blank">PubMed API</a></li>
            <li><a href="http://wiki.datadryad.org/Data_Access" target="_blank">Dryad API</a></li>
            <li><a href="http://docs.ckan.org/en/latest/api/" target="_blank">CKAN API</a></li></ul>
            <p>3. Data Science project using Soda API: <a href="https://github.com/ViDA-NYU/urban-data-study" target="_blank">https://github.com/ViDA-NYU/urban-data-study </a>
            <p>
    
            <h2 id="fifthheading">Exercises</h2>
            <p>1. Explore Soda API and discovery API documents. 
            <p>2. Try to use Soda API to access datasets of interest, filter data records according to your selected criteria. Submit your API calls. 
            <p>e.g. Get all records about “Animal Complaints” after April 4th, 2018: <a href="https://data.seattle.gov/resource/pu5n-trf4.json?$where=event_clearance_date >'2018-04-04T12:00:00.000'&initial_type_group=ANIMAL COMPLAINTS " target="_blank">https://data.seattle.gov/resource/pu5n-trf4.json?$where=event_clearance_date >'2018-04-04T12:00:00.000'&initial_type_group=ANIMAL COMPLAINTS</a>  
            <p>3. Try to use Soda discovery API or metadata API to get catalog information or asset-level metadata information about open government data portals. Use filters and parameters to refine your queries. Submit your API calls. 
            <p>e.g. <a href="http://api.us.socrata.com/api/catalog/v1?categories=public%20safety" target="_blank">http://api.us.socrata.com/api/catalog/v1?categories=public%20safety</a>  
            <p>e.g. <a href="http://data.seattle.gov/api/views/metadata/v1?limit=10&page=1" target="_blank">http://data.seattle.gov/api/views/metadata/v1?limit=10&page=1</a>
            <p>

            <h2 id="fifthheading">Creators</h2>
            <p>This material is part of <a href="http://odl.ischool.uw.edu/about/" target="_blank">Open Data Literacy Project</a> funded by IMLS grant. 
            <p>An Yan: Information School, University of Washington
            <p>Bree Norlander: Information School, University of Washington
            <p>Carole Palmer: Information School, University of Washington
            <p>Kaitlin Throgmorton: Information School, University of Washington
            <p>

            <h2 id="fifthheading">References & Credits</h2>
            <p>1. Mulesoft. (n.d.). What is an API? (Application Programming Interface). Retreived March 7, 2019, from <a href="https://www.mulesoft.com/cn/resources/api/what-is-an-api" target="_blank">https://www.mulesoft.com/cn/resources/api/what-is-an-api</a>
            <p>2. Wikipedia. (2019, March 5). Application programming interface. Retrieved March 7, 2019, from <a href="https://en.wikipedia.org/wiki/Application_programming_interface#Web_APIs" target="_blank">https://en.wikipedia.org/wiki/Application_programming_interface#Web_APIs</a> 
            <p>3. Maddox, S. (2014, February 16). API Types. Retrieved March 7, 2019, from <a href="https://ffeathers.wordpress.com/2014/02/16/api-types/" target="_blank">https://ffeathers.wordpress.com/2014/02/16/api-types/</a>

           
            
            <p></p>
            
            <!--end content above this comment-->
            
        </div>
    </div><! --/row -->
</div><! --/container -->