Cambridge University Library

Getting started with Web Services

web

Web Services (or APIs) are a way of interacting with databases and systems via the web. They are commonly used to embed system functionality into widgets and web pages, or to "mash up" data from different systems. Many systems come with their own set of documented web services (we will refer to these as the system's proprietary web services). If your system doesn't have proprietary services, or if they don't fulfil all of your requirements, you can write your own local web services.

This short introduction is intended to tell you a little about what web services are and how they work. It gives tips and example templates on writing your own local web services and writing "proxies" to proprietary web services. It is very much aimed at librarians and library users, and is by no means intended as an exhaustive or authoritative guide (of which there are plenty on the web).

For more general information on web services and widgets see our project blog here. For the full range of tools, services and gizmos provided by Cambridge University Library see our Library Toolbox.


Basic Principles

Some basic principles apply to local and proprietary web services.

  • Every Web Service has a separate web address or URI, e.g.
    http://www.lib.cam.ac.uk/api/local/libraries_data_elements.cgi?type_code=D
  • Every Web Service performs one distinct Function - for instance Search or Renew
  • Web Services can receive and process Parameters, additional instructions which allow the Service to perform its Function. For a renewal service, the parameters might be a book identifier and a person identifier. For a search service parameters might be a search term and a search type. In the example above, the parameter name is "type_code" and the parameter value is "D"
  • Web Services send back Return Messages to whoever calls them. These might be content, i.e. for a search service the return message would be the search results. Or they might be informative, i.e. for a renewal service the return message would indicate the success or failure of the renewal.

Writing your own

If your system doesn't have proprietary web services, or it doesn't provide a service for the particular piece of functionality you need, you might consider writing your own local web service. To do this you will need to be able to query the database which underlies your system. More details on writing your own web services can be found on our blog here

General considerations:

  • You will need some familiarity with a programming language. We will use Perl for our examples - other possibles are PHP, Python and Ruby.
  • You will need to know at least some SQL in order to query the underlying database
  • You will need to know some things about your system's database - where it is, what type it is (Oracle, MySQL etc.) and how to log into it (username and password). Your system administrator, system documentation or help desk should be able to help with this.
  • Web Services are web-accessible programs (or scripts). As such they will need to live somewhere accessible to the web. Usually this means a directory on your web server. If you are using Perl to write your web services, make sure your system administrator CGI-enables the directory

Data and Permissions

When you have all the necessary details to log into the database, you will need to establish what kind of permissions you have over the data. Broadly speaking, you will either have:

  • Read Only Access - which means you can retrieve data from the database but not add, delete or alter data in the database
  • Write Access - which means you can retrieve data from the database and also add, delete or alter data in the database

Your permissions will affect what kind of web services you can write. If you have read only access you will only be able to write services which retrieve data from the system (i.e. SQL SELECT statements). If you have write access you will also be able to write services which alter data in the system (i.e. SQL INSERT, UPDATE, DELETE etc. statements). Again, your system documentation, administrator or helpdesk should help you find out what rights you have over the data.

NB The majority of library systems only allow their customers Read Only Access to the data, which means you may have to rely on the system's own proprietary web services (if any) for functional services like renew, borrow, request etc.

Naming, Parameters, Return Messages

Try to give each of your web services a meaningful name - i.e. renew.cgi, search.cgi - it makes life a lot easier for anyone using the services

Identify what information your web service will need in order to fulfil its function. In the simplest sense, these will be whatever needs to be passed through as arguments in the SQL query to be sent to the database. So for renew.cgi you will need identifiers for the item to be renewed and the library user.

Identify what needs to come back from the web service so that it can be used properly. This will either be content (i.e. search results) or an informative message (i.e. "Renewal succeeded", "Renewal failed")

Example Template for a local Web Service

You are now ready to write your own web service. Open a file in the directory on your webserver. Give it a sensible name, i.e. loans.cgi. Then follow the example template here

Now make sure that the file has the appropriate permissions (it needs to be executable by the web server). "chmod 755 [filename]" on the command line in your Web Services directory should do the trick.

Now type the address of your Web Service, including any parameters, into your browser's address bar to test. Your Web Service's return message should appear in the main browser window, as in this example

Proxy Web Services

A Proxy Web Service intermediates between a user or interface and a proprietary web service. There are a number of reasons why you might want to use a proxy web service to access your system's proprietary web services

  • Security Proprietary web services may alter or delete data in the system, or expose sensitive data. Using a proxy web service you add an extra layer of security, controlling and mediating access to the basic services. Your system provider may, in any case, be unhappy if you provide unmoderated access to the system's native web services and insist that you use a proxy.
  • Convenience Proprietary web services are not always written to be human readable, and can be very complex in terms of the parameters they take in and the messages they return. A proxy allows you to mask some of this complexity by asking for simple input and returning simple output
  • Functionality If your interfaces are calling the proprietary web services using javascript and the web services are in a different domain to the web page which is calling them, the browser may refuse to process the call. This is because of security concerns. There are ways round this if the web services can return json rather than XML (see below), but the simplest solution is to write a proxy to your proprietary web services which is in the same domain as the interfaces calling the services.

Proxy Web Services - considerations

Find the address of the proprietary web service and the parameters it needs to perform its function. These should be in your system's documentation. You will have to pass these parameters into the proxy web service so it can pass them on to the proprietary web service

Look at what the proprietary web service returns. You will have to decide what you want the proxy to return - either exactly what the proprietary service returns or a processed, simplified version

Example Template for a Proxy Web Service

If you are following our Perl template, you will need to ensure that the following Perl modules are installed on your server (ask your system administrator)

  • LWP::Simple
  • LWP::UserAgent
  • HTTP::Request::Common

You are now ready to write your proxy. As with local web services, open a file in the directory on your webserver. Give it a sensible name related to the proprietary service - i.e. renewal_proxy.cgi. Then follow the example template here

As before, make sure that the file has the appropriate permissions (it needs to be executable by the web server). "chmod 755 [filename]" on the command line in your Web Services directory should do the trick.

Now type the address of your proxy Web Service, including any parameters, into your browser's address bar to test.

Further considerations

This is just a quick start-up guide. If you want to start developing around web services, you might want to consider

  • Security Encoding any sensitive identifiers or information before passing them around to web services. One possibility is MD5 encoding. You may also want to put services which reveal personal information behind your institution's authentication system (your system administrator should be able to help with this), ensuring that users can only access their own information.
  • JSON You may want to write local services which return json as well/instead of XML. This can help get round the cross-domain security issues mentioned above. There are a number of existing modules which will convert XML to json.