Book Sources Widget

The book sources widget is nearly done now. We're asking some other people to try it out, and it turns out the service it uses is a bit slow (which we're trying to track down), but works splendidly if you're a patient type. As soon as we find out which machine we need to beef up, we'll add it to our catalogue.

"Final Post"

I'm in a slightly odd position writing a final post for this project. There's two reasons for that.

The first is that we're still working on future versions of all of the things we produced in this project, and will for some time. This blog won't be stopping, please don't unsubscribe! You'll miss out on all our future widgets.

The second is that I'm used to writing formal final reports, where we detail what the project has achieved, this time it's already on the blog. It's also very difficult to get the tone right in such a different environment.

I don't think it's worthwhile duplicating all that detail, but I'll summarize it here, and you can find the detail if you're interested.

First, you'll have seen the financial statement. The project management side of things seemed to work well, and we've also forged some good relationships within the University, for example getting a feed from our Student Information System.

The widgets we're proudest of, probably because they're the most advanced, are the "Library Widget" which provides the functionality you'd expect from a library (catalogue, books on loan, etc), in a single widget, embeddable in our VLE, in Facebook and in iGoogle. A related widget provides the same functionality on a mobile. These were half-done at the start of the project, through institutional commitment, but it was great to spend the time and effort to put them into production as a proper University service. The uptake was dramatic, and a little scary (see previous posts). If you want hard numbers for impact, that's where you'll find some pretty impressive ones!

Whilst Huw was working on productionising more developed widgets, like the general purpose library widget, and the mobile interface, I was responsible for getting the new widgets off the ground and into alpha. You can see inside the laboratory here, to see what's coming. But a laboratory can be dangerous, these widgets really are all in "alpha", and will almost certainly break at some point in the future, on their path to production. So have a play, but don't be surprised if something odd happens!

More advanced widgets are here on our demo page, and our forthcoming "storefront" is here. With three more widgets due for release in the next couple of weeks, we'll do a big announcement of the storefront to the University at that point.

I'm personally particularly excited about "sputnik" which isn't so much a widget in the traditional sense, but a Firefox plugin (Chrome to follow) which advertises how your library (or University more widely) can cope whilst you surf the web. In some ways it resembles Libx, and was inspired by the iCite life sciences plugin, but it's much less shy than either of them and, we think, has massive potential. For example, if you can't use the OPAC, find it in Amazon, and we'll tell you where it is in the library. Commercial vendors are not going to be shy about their offerings, they're not going to hide their information one or two clicks away, so why should we be reticent about our alternatives on commercial pages, particularly as the user will have to actively elect to install the plugin. The large area set aside also gives room for us to go beyond holdings in the future.

The exam paper widget (and associated uploader) helps students find exams and knows their subjects from our student information system. That's pretty much the first time we've integrated with our SIS, and we now have a feed which can help contextualise information. We have to work to the natural student year for this widget, of course, we can't move exams, and also the natural round of meetings of committees, but we've already got many papers from one of our subjects and approval to include more, so watch out for future posts on this subject.

Cambridge has many libraries and which library a student is a member of is not trivial to work out. We did it in the end, though, and now you'll see a widget which allows a student only to search libraries which they actually have access to. We think this is the first time this has been possible!

We've widgetised a reading list for a course (data entered via google spreadsheet) and are working on genericising it to other data sources including the possibility of linking it to Zotero or some other reading list manager.

We mustn't forget what might be the most valuable output of the project, the services for others to use, and our advice pages for widget writers which summarize much better than I am doing here the advantages of the approach.

Thanks to JISC for their support, and keep on looking for future updates. I've probably missed something out which I'll remember over the weekend, so I'll continue this shortly. In the meantime, here's a screenshot of my (personal) favourite "Sputnik" plugin, which I've already personally found invaluable (and see also Huw's usage graphs in other posts!). All the code for these widgets, even the experimental ones, should be in our googlecode svn (seach for culwidgets), and we'll make sure they're all complete at release of each widget.

In the spirit of the brave new world of blog, if you'd like further information on this project, or would like to steer any updates as to outcomes, just comment below and I'll happily answer.

CamLib Mobile interface Report

The CamLib mobile interface is entirely a product of the project. We decided that as well as pursuing students into social networking sites, we also wanted to inveigle ourselves onto their phones. There is no escape from the Library!

We initially set out to simply recreate the library widget for smartphones, but as we went along we did some thinking about what people would want to do on their phones (as opposed to waiting until they were at a 'proper' computer)

So the focus was on finding out if something existed, where it was, and how to get your hands on it. We did quite a lot of integration with maps and request functionality, and added a new "bookbag" feature to allow users to build lists of books from the catalogue and email it to themselves. Plus we kept in all the renewal, loans, libraries stuff as well.

How is it doing so far? Let's take a look (vertical axis figs are 35 and 70)

Well, just a bit better than expected. We knew that only a minority of students have smartphones, and only a minority of them use smartphones for work. So 540 unique users in a month isn't bad.

But we're expecting a tipping point sometime in this academic year, much as happened with laptops a few years ago. And when they come we will be ready!

Verdict: B-
Good effort, great things expected

Library Widget Report

This is the first of a series of end-of-term reports on each of our widgets. Who made the grade? Who must try harder? Who are the troublemakers? Who are the swots?

We'll start with the Library Widget. We've done a fair amount of development on what was already a fairly established platform. We added in Library pics to help with navigation, we beefed up security, we integrated it into our new resource discovery layer, and we did some behind-the-scenes jiggery-pokery to make everything work more smoothly.

For the Library Widget, the project was more about launching it and seeing if it floated. So let's take a look at this month's Google Analytics and see how it's doing (vertical axis figs 200 and 400)


Not bad at all! Plenty of users, plenty coming back for more, and with 2,168 unique users we are already exceeding our end of year target of 2,000 users per month. Obviously, there is more we can do with the Library Widget, but it's essentially a stable product with an established and growing user base. The future looks bright!

Verdict: B+
Steady progress, will go far.

Financial Report

Here's our project's final financial report. I hope the format is ok. If you want any more info, just ask me. It's fun to work on a project so inline with our aims, as it takes no effort at all to get people involved. The difficulty can be getting them to do their other work!

Cost	Apr 10 - Jul 10	Aug 10 - Oct 10	Total Budget	Total Spend	Variance (+ve=over)
Total Directly Incurred Staff (A)	£12,725	£9,620	£22,345	£24,988a	£2,643
Total Directly Incurred Non-Staff (B)	£1,200	£300	£1,500	£800b	-£700
Directly Incurred Total (A+B=C)	£13,925	£9,920	£23,845	£25,788	£1,943
Directly Allocated Total (D)	£5,477	£3,305	£8,782	£8,782	£0
Indirect Costs (E)	£12,657	£9,250	£21,907	£21,907	£0
Total Project Cost (C+D+E)	£32,059	£22,475	£54,534	£56,477	£1,943
Amount Requested From JISC	£26,400	£13,600	£40,000	£40,000	£0
Institutional Contributions	£5,659	£8,875	£14,534	£16,477	£1,943

a. 195hr, 643hr, 57hr, 64hr. Spine points redacted for DPA.

b. £300 travel (-£700 variance), £500 contingency.

Contributions over life of the project:

	Budget	Actual
JISC	73%	71%
Partners	27%	29%

Those Advice Documents Now Online

The documents which started out as our final post and kind of grew, are now online.

So tomorrow probably marks the formal end of the project, with our actual final post, followed by the financial report, and so on. But don't discard this blog: there should be a fair few "going live" announcements over the next couple of weeks. I'm hoping to get the Library Companion, which is a reimagining of LibX, available in alpha within the next few hours.

Update

Sorry for the delay in updating, everyone. We're in the process of putting together our closing remarks and have decided that there would be enough value in putting what we have into a number of permanent pages on our website, to guide future widget authors. We're just in the process of tidying and CSS-ing them all, and they should be up in the next couple of days.

In the meantime, we'd welcome comments. Please forgive the formatting.

We will create three pages: a top level page for high-level comments, and one each for writing services and widgets.

Here's the Services page, already formatted and complete: http://www.lib.cam.ac.uk/api/web_services.html

And below is what we have for the top level page, and widget writing page, awaiting that formatting goodness. Comments greatfully received: though these are static pages, they're intended to be living documents.

We're building a little app store for all our widgets, which we can reveal in the next couple of days (well, we can reveal it now, but you'll find it confusing until we fix a couple of things). I'll announce that ASAP, along with the locations of these pages when they're up (ETA Tuesday). We're also finalising the project accounts.

INTRODUCTION
============

This document describes why you might want to write widgets and services to solve some of the problems in your library.

The other documents in this collection are aimed at software developers. However, this introduction is for a more general audience of technically literate people, and describes motivation and general shape.

Whilst the remainder of these pages require software development skills, they do not require expensive engineers: a new graduate or a librarian with a sideline in software development should easily be able to get going with them to produce something useful.

Why bother with widgets?
========================

Often, the biggest obstacle to overcome when explaining widgets is to overcome the frivolity associated with the concept: the silly name; their ubiquitous use on social networking sites for producing pointless distractions, and so on. We encourage you to set such preconceptions aside, at least temporarily.

Reason 1: Widgets reduce the risk associated with experimentation
-----------------------------------------------------------------

Widgets are small (we're talking person-hours or -days) and easily rewritten. This allows you to take risks in creating them. If some users have indicated that a widget might be useful, you can spend half a day creating one: much less time than it would take to perform a formal survey.

Think of how many projects there have been at your institution concerning integration of various computer systems. And think, nationally, how many projects there have been only to integrate catalogue and circulation information between VLEs with LMSes. Consider, particularly, their budgets. We have such functionality already, in production, against our enterprise sites, in use by academics and students, as a result of a small project, created by Huw Jones, a librarian. And that's just one widget.

Particularly at larger HE institutions, we are used to inter-departmental collaborations involving endless meetings, vague strategy, and nobody to arbitrate or direct the project below vice-chancellor level. So, what we offer to students simply ends up being disjointed.

Developing software with such a low overhead allows integrations to occur "below the radar", and "under the budget" of large, difficult projects.

Reason 2: No more "you'll need to login to another system"
----------------------------------------------------------

Remembering URLs, usernames and passwords is difficult, and we seem to spend so much of our time logging into systems and finding our way from the front page to where we want to be.

As a user, widgets allow you to do things "from where you are". There's no "that's in another system" issues. A user can, effectively, design their own screens to fit the way they work. There is no need to login again (auth permitting), to remember a URL, or to navigate to the page you actually want through a maze of initial pages. A screen of widgets might contain some widgets derived from the LMS, some from the VLE, some from teacher-provided data (such as reading lists), and so on.

Many of these containers (for example, in a VLE) have rich knowledge of the current user, too. That means that you can present "zero-click" search results, for example for exam papers: there is no need for a student even to search for the papers they are taking because the results displayed by default are usually what the user is after. (However the opportunity is presented to change this, if the user wishes).

Both of these mean that the user spends more time interacting with the real content, and less time navigating.

Reason 3: Moving development towards specialists
------------------------------------------------

Writing widgets, and particularly services, does require some development expertise, but it doesn't require much, and certainly less than most other approaches. This makes it much easier to keep control of the development. You might have a librarian who is handy with software development. Your library might have a website maintainer who can do it. You might be able to hire someone directly into your library on a short term basis. There's certainly no need to engage a big external organisation.

Recommendation
==============

We'd particularly recommend widget writing to institutions which have struggled, technically or socially, with integration and development in the past, and where the students are faced with a confusing array of different systems to use.

Despite their silly name.

Widgets document:

Introduction
============

Widgets are small packages of code which put information from external sources onto regions of webpages. Typically they comprise small collections of HTML, CSS, Javascript and (sometimes) XML.

In their simplest form widgets contain only an "iframe" which is a window onto a website hosted on another machine. More complex widgets contain code which is to be embedded and run actually within the outer page.

Widgets can usually be written in a day in a text editor, including the time spent to learn a new widget technology, if you have the appropriate services feeding you data. Widgets use services, detailed here to perform queries on underlying systems. A widget may, or may not, have access to further information from the container such as the current user, their permissions and so on.

Basic Principles
================

* Widgets sit inside a widget container (aka portal). This can be as simple as a static webpage or as complex as Facebook, iGoogle, or a VLE. For most of the different containers which you will target, you will need to write a separate widget. However, writing a widget should be quick, and you can reuse a lot of your code.

* Widgets execute entirely in the browser of the user. Any interactions which they need to make with your own systems takes place through the use of services.

* Widgets can either use iframes or not (not using them is usually called using HTML fragments). You need to decide how yours are to be implemented (see below). But changing your mind later on isn't painful.

* If you eschew iframes and use HTML Fragments, you write HTML and CSS to lay the widget out as you intend. Then use AJAX in your javascript to call services and then update the webpage.

* Using iframes is simpler. You write a small fragment of code to be hosted on the container, containing the iframe "window", and then write the rest of your code on your own server.

* If using HTML Fragments, a container will typically help you with the javascript by providing you with libraries to perform common tasks (such as calling a remote URL for services). These may well include the jQuery or trimpath libraries of useful javascript methods. Do check particularly if jQuery is installed as, if so, jQuery can save a lot of bother. If using iframes, we strongly recommend using jQuery in your pages.

* A container may provide you with a means of storing a user's preferences, or you could use client-side cookies. This is useful for things like preferred layout and usernames.

* Because widgets run entirely in the user's browser, you cannot rely on what is returned for security purposes, as a user can intercept and change messages between widgets and services. (see below for help on security).

To iframe or not to iframe?
===========================

As described above, an iframe tag is like a window in one web page onto another, often on another site altogether. One use of iframes is for widgets. By placing only an iframe into the widget code for the container, the content of the widget can come from your own server, and will be put inside the iframe by the browser.

Some widget containers require the use of iframes. Others don't allow iframes, your widget must hosted on the site of the widget container as HTML fragments. Many containers allow both.

If your container only allows one approach or the other, then you have no choice.

If both are available, I recommend choosing HTML fragments first, and move to iframes if you find something easier to achieve using them. The exception is Facebook, where iframes are generally easier. This blog post is a good discussion of an alternative perspective on the iframe vs fragments issue.

Typically, access to services will become the deciding factor in whether or not to use iframes.

An iframe widget is running on your server, and so can be given complete access to whatever is running on your server with little difficulty through your own programming. However, the widget is at some remove from the container, and so you will find it harder to access facilities provided by the container (for example, it may be harder to determine which user is using the widget).

The opposite is true of HTML fragments: they are closely embedded in the container, but further from your own systems.

Trying, and being prepared to change if you run into trouble, is probably the best approach, until you get an instinct for this decision.

Writing Your Own
================

If you are interested in widgets in general, but have no specific target in mind, try iGoogle Gadgets. These will help clarify the issues discussed in this document ready for when you write widgets in anger.

To write an iGoogle Gadget start here. This describes writing an iGoogle gadget as an HTML fragment. The "Include Gadget" on this page is an example of an iframe-based iGoogle gadget.

A Camtools start page widget is like an iGoogle Gadget, and can be written and tested as such. Extra facilities are available within camtools, however. For details contact the Camtools team.

To write a Facebook application start here. See here for a discussion of iframe vs fragment issues in relation to Facebook in particular.

To embed a widget into a static HTML page, all you need to understand is how the iframe tag works. Write your widget as for any other iframe widget and then write the relevant iframe tag and encourage others to embed it on their page. One way to encourage this is to make the code copy-and-pastable from your site.

If you want more info, talks.cam views could be valuable, as they show how such a technique is used in a live site.

If at all possible, use a library like jQuery to help you write your javascript. This is always possible with iframe widgets, and often provided with HTML fragments. jQuery helps you write javascript by providing common methods for manipulating a webpage which have been extensively tested and debugged. Whilst the javascript language itself is quite simple (adding two numbers, calling a function, and so on), interacting with browsers (making something green, implementing tabs), whilst superficially simple, is very subtle if you want it to work on most browsers, and best left to the experts.

Integrating With Services
=========================

As described in our document on services, services are presented at a particular URL. To consume a service you need to connect to that service as if a web client. You can do this from javascript running on a user's web browser. This is known (formally erroneously) as AJAX.

However, AJAX can only issue a request to the page's originating server. In the case of HTML Fragment widgets this will be the widget container's domain (eg google for an iGoogle widget). This would make HTML Fragment widgets almost worthless were it not for the additional ability to "proxy" your request through the container: a javascript function is provided by the container to allow it to issue any requests on behalf of the widget. For iGoogle these methods are detailed here.

For iframe widgets, it will be the server from which the content of the frame was served, which may be more useful. Also, with an iframe widget you can, of course, implement server-side code to issue HTTP requests to any site of your choice.

Security
========

Computer security is very difficult. The difficulty is not only in the coding, but in reasoning correctly. Almost everyone believes their concepts are watertight, only to be surprised by an unanticipated attack. But you will probably need to think about security when writing widgets.

There are three main approaches to say safe.

First, don't use security, unless you have to, and then use it as little as possible. This may sound counterintuitive at first, but the fewer things there are to go wrong, the less vulnerable your system is. Do you really need to protect this data? Also, do you really need to be so fine-grained in your control? Keeping a simple undifferentiated circle of trusted individuals is likely to lead to better security than complex fine-grained restriction in many cases. The chances of one of the trusted team betraying the others is usually less likely, or has less of an impact than, the risk of a faulty complex system revealing details to the whole world.

Secondly, if you do use security, use a known technique. The best are those you copy from APIs at well-regarded, popular sites. Discover the techniques they use and copy them.

Thirdly, apply strength in depth. Implement as many techniques as make sense, even if they seem redundant. However, beware of unnecessarily complex systems (see first point).

Useful toolkits in your armoury include: shared secrets, salts, hashes, HMACs, OAuth, Raven, Shibboleth, IP-range restriction. Google for these for details.

Remember, a user can change anything which is sent from their browser.

Guidelines
==========

* Keep the widget as simple as possible: make it do one thing well. Don't dump lots of functionality into a single widget, users can install multiple widgets matching functions they choose to use.

* Let users customise their widget. Your widget will be a part of their page. Issues as simple as title bars, multiple branding, and different colour schemes can be irritating in a page containing many widgets and, more importantly, needlessly consume a lot of screen space.

* Consider Context. Where will your widget go? Make sure you include enough information for the user to understand what the widget does but to reduce the amount that they will already know from surrounding context. If it is to be installed in Facebook don't say "Library Widget" say which library. On your libraries internet, however, say nothing.

This site has excellent general advice about designing widgets.

What a difference a day makes

The leaves are turning and the students are back in Cambridge. We always saw the start of this term as the real launch target for the library widget as a fully-fledged service. And taking a look at the Google analytics

our little bird is ready to fly! Guess when the start of term was ...

We've been working hard on getting the widget out there - blogging, inveigling it into library inductions, promoting at the Freshers' Fair, tweeting and generally making a nuisance of ourselves.

The embeddable version of the widget is now springing up all over the place, from our new unified Cambridge Libraries Gateway, to forming the account functionality for our (also new) LibrarySearch resource discovery platform. Everywhere you look the widget is gluing services and systems together, offering consistent functionality in a diverse online world.

We're hoping for 2,000 unique users a month by Xmas (c. a fifth of our undergraduates) and are well on the way. That kind of usage certainly makes the "real estate" at the bottom of the widget valuable property for messages and info for our users.

So it was nice to see the usage figures take off on Tuesday - 24 little hours in Cambridge can make quite a difference!

JISC LMS meeting in Glasgow

Huw and I (Dan) went to Glasgow to the JISC LMS meeting.

So, if the joys of two nights spent catching sleep on the Caledonian Sleeper are the downside, what were the benefits?

Many actually. In three important ways.

First, we heard a lot of ideas. We plan to implement a fair few of them in our project. For example, another JISC-funded project, ConnectedWorks, has led to us investigating a browser plugin called iCite. When someone mentioned the idea of adding a "find in my library" button to amazon searches, we realised we had just the tool for the job. So we're going to add a web-assistant browse widget to the project. There were many other examples, some we will have time to do, others will have to wait for future projects.

Secondly, we learnt about other projects in the programme, including projects we could work with. The Wolfie project at UEA is a standout there, as is us using some of LSE's earlier work. Generally, lots of good networking went on.

Finally, we were helped to develop our arguments and pitches for our projects, to help us describe them to each other, and the various people who might be able to help us in their execution.

There was also some excellent discussion around UX, and how we might be better at pooling the outputs of our user research (to avoid duplicating effort) and be better critical consumers of UX research (we all know how much poor-quality UX research is there: how do we wind the diamonds amongst the glass?).

One Minute Pitch

One of the requests which came out of the JISC LMS meeting was that we prepare a one minute pitch for our projects. (Commonly known as "elevator pitches". The idiosyncratic lift at the venue was very much the ghost at the feast on the first day of the meeting, having a tendency to steal the limelight by appearing on our floor unbidden and aimlessly chomping away with its doors throughout proceedings But, anyway,...).

Often, the biggest obstacle to overcome when explaining widgets is to overcome the frivolity associated with the concept: the silly name; their ubiquitous use on social networking sites for producing pointless distractions, and so on. We encourage you to set such preconceptions aside, at least temporarily.

There are two reasons for using widgets for library services.

Number One: widgets make things happen in institutions with complex organisation and a tendency for long meetings and endless debate. That's one of their main advantages. Development is in three parts. The information provider provides an API (as they see fit). The consumer provides a widget container (again, as they consider appropriate). Neither of these two parts require extensive consultation. The final part, the widget itself, -- which is the only part which ties the two parts together, -- can be written in a day.

Widgets are small (we're talking person-hours or -days) and easily rewritten. This allows you to take risks in creating them. If some users have indicated that a widget might be useful, you can spend half a day creating one: much less time than it would take to perform a formal survey.

Think of how many projects there have been at your institution concerning integration of various computer systems. Think, nationally, how many projects there have been to integrate just catalogue and circulation information between VLEs with LMSes. Consider their budgets. We have such functionality already, in production, against our enterprise sites, in use by academics and students, as a result of a small project. And that's just one widget.

Number Two: as a user, widgets allow you to do things "from where you are". There's no "that's in another system" issues. A user can, effectively, design their own screens to fit the way they work. No need to login again (auth permitting), or remember the URL or navigating to the page you want from the front page of a particular site. A screen of widgets might contain some parts derived from the LMS, some from the VLE, some from teacher-provided data (such as reading lists), and so on.

Many of these containers (for example, in a VLE) have rich knowledge of the current user, too. That means that you can present "zero-click" search results, for example for exam papers: no need even to search for them because the results displayed by default are usually what the user is after. Both of these mean that they spend more time interacting with the real content, less time navigating.

Both of these things excite us, and present an opportunity to improve the way we do things and make what we offer to students more compelling.

So we're producing example APIs and Widgets as might be used at other institutions, along with guidance of how to do so. We're putting our money where our mouth is by developing at least half a dozen widgets we anticipate being useful to students and academics (some of them have been repeatedly explicitly requested, some others are widget-ising of existing useful functionality), and writing up how you can go about it at other institutions.

So, we'd particularly recommend this approach to institutions which have struggled, technically or socially, with integration in the past, and where the students are faced with a confusing array of different systems to use, as widgets can help with both of these.

Despite their silly name.

Back to school, off to Glasgow

After a typically fragmented August (if you're not on holiday yourself then everyone you need to speak to is) it's back to school for the CULWidgets team, starting with a trip to Glasgow for the JISC LMS Programme Meeting. As we draw slowly towards the end of the project it's a great opportunity to focus on what we've achieved and what we still need to achieve as well as a chance to see what everyone else is doing!

One thing we have made progress with over the Summer is getting the student data we need to give our widget services relevance. We have one minor technical hitch to get over, and then we should be able to get a test version of our Exam Papers Widget up and running.

This really is a key strand of the project for us. It involves institutional collaboration - the University Library, CARET, DSpace@Cambridge (institutional repository) and the Management Information Services Division (who run the student registry) are all integral to the project. It involves a new kind of material - the Library has traditionally only dealt with exam papers in paper form. It involves the provision of relevant services in relevant places. And it involves direct engagement with the student and academic community. Finally, we hope it will be a blueprint for the course-based delivery of teaching and learning materials across the University.

Reading List Hackathon - a Summary

On Thursday and Friday of last week, I attended a Reading List Hackathon run by DevCSI, at the Møller Centre, Cambridge. The event was designed to encourage the hacking of library information systems and reading list software.

We looked at two major pieces of reading list software:

• Telstar

and

• List8D

with talks from Mendeley (who provide citation software), Talis (who provide library software and reading list software, amongst other things) and Emerald (a publisher who are creating peer-reviewed reading lists as a free service).

After these talks, we got together into teams, and started hacking, creating, amongst other things:

• a way of pushing Emerald RSS feeds through Yahoo pipes, allowing filtered searching
• some extensions to List8D
• Six degrees of Harry Potter, which used Talis API data to generate a graph of degrees of separation
• Compare the Citation: a teaching tool to educate undergraduates about citation styles
• an extension to a library catalogue, showing availability of items on a reading list
• an extension to List8D, allowing users to display citations in different styles
• using the Talis APIs in List8D
• Llikes, a tool to reorder a reading list based on social metrics
  

I'll write a couple of blog posts detailing what people said in their talks, and more about what was created during the event.

But basically, I had a really good time; I discovered lots of new things; some really cool and interesting things were created, and I'd love to go to a similar event again.

Reading List Hackathon

One of the things we're interested in doing is innovating ways to disseminate and manage Reading Lists.

Our latest widget, the Reference List widget I mentioned last week, covers a related but slightly different use case to Reading Lists. Reading Lists are much more broad, and could include published books, journals, local (e.g. departmental) reading material etc; whereas the Reference List widget caters for lists of citations in a very well defined format.

Right now one of our team members, Verity, is at a DevCSI workshop exploring the interoperability between reading list software and other software systems - we'll let you know of any useful things she picks up at the Hackathon.

A new widget and a new gallery!

Over the past few weeks we've started a "production line" of widgets where widgets start off life as ideas, become prototypes, then emerge as demos and gradually develop into more finished products. Watch this space for more news about some of the other widgets we have in "incubation" in the production line.

The first new widget to emerge from this production line as a "beta" release is a Reference List widget - this displays a list of journal citations and allows the user to filter them by various criteria like keyword and author. This could be used as a handy way for a course lecturer to disseminate the References relevant to various lectures within a course. It is powered by public JSON feed from a Google Spreadsheet which makes it easy for non-developers to maintain the source data. I'll blog a bit more about how this widget works in a later post.

In the meantime, I've set up a quick and dirty Gallery Page to show off our widgets as they emerge from the production line - you can see the Reference List widget in action on there, along with the two other widgets that are in a ready-to-demo state (see earlier posts for more about them). The widgets and the Gallery Page itself have some rough-edges which we'll tidy-up in the coming weeks but for now you get a flavour of what we've been up to. We also hope to supplement this demo page with a more "directory-like" page, a bit like an "app store", which will tell you more about each widget, and allow you to add the widgets to your own portals (e.g. Facebook, iGoogle).

Widgets, mobile and atomized services

Interesting thoughts from Lorcan Dempsey:

http://orweblog.oclc.org/archives/002115.html

Talking about mobile devices, but one point in particular is very relevant to widget design

"Atomization: get to relevance quickly. Mobile encourages designers to think of atomic services rather than complicated workflows or rich multilayered experiences. And to think about services that are immediately relevant and convenient. Room or equipment booking or bus time tables may become more visible, for example."

The simplest widget just takes an existing web page and puts it in a box - but this really isn't a very effective use of the medium.

When we're discussing ideas, a question we ask ourselves is "what makes you want to turn this into a widget?" Just because you can doesn't mean you ought to.

A few possible reasons:

• You want to make services/information available outside traditional interfaces i.e. in social networking sites
  
• You want to isolate a particular, commonly used service and provide a streamlined way to access it
• You want to bundle a few related services/bits of information together around a particular set of workflows

I suppose the "widgetisation" of an interface is to do with how and where you want access to happen. But the restrictions (i.e. space) on widget development make you think very hard about use.

In a nutshell, it seems to me that widget development is about providing relevant services in a context. You could probably say this about all development, but there's something about widgets which concentrates the mind!

Another Hello

Hello my name is Anne and I am a CARET-based developer who will be working on Library Widgets with Raymond.

I am getting up to speed with this project and took part in a Widgets Brainstorming session last week. I have also been benefiting from reading everyone's postings on this Blog.

I have been tasked with writing PHP mediator scripts to automatically run a series of searches using both the Aquabrowser and Voyager APIs. The results from these searches will be merged together to allow Search Widgets to do their stuff with the data.

I'm looking forward to meeting and working with our colleagues at the UL.

transitions

Dan has now returned to CARET from the University Library, and will be returning to the helm of this project over the next few weeks. So there will be less from me, and more from him!

Google Code Site

Software Developers - have you checked out our Google Code site yet? This is where we're keeping documentation and code, and where we'll track any issues like bugs and feature requests.

In particular, you can browse our source code - all covered by the Apache License 2.0. We will be augmenting the code with full documentation of how to host, configure and install your own version of each widget in due course, but for now there is a snapshot of the code that currently powers the original Library Widget, and most of the code used in the prototype of the Exam Papers Widget.

If you simply want to use or try our widgets as-is, there's no need to delve around in the source code - find out more about how to use the Library Widget and the mobile version, CamLib.

Cambridge Library APIs

I've blogged a few times about how our widgets and interfaces are underpinned by web service APIs to our data sources. We've now made the decision to publish all of our public APIs - documentation can be found at http://www.lib.cam.ac.uk/api/

This means that any developer can access and build functionality around Cambridge Library data. It's going to be interesting to see what they come up with!

CamLib mobile interface and Aquabrowser

One of our aims in this project was to integrate our widgets with the new (forthcoming) Aquabrowser discovery layer, which will be launched later this summer. Cambridge library resources come in from a huge variety of data sources, including eight (yes, eight) Voyager databases. Searching across these collections presents a huge challenge for our users. Look at the previous search options:

• UL & Dependents
• Departments and Faculties A-E
• Departments and Faculties F-M
• Departments and Faculties O-Z
• Colleges A-N
• Colleges P-W
• Affiliated Institutions
• Manuscripts

So it was difficult to even search across all colleges, let alone the whole of Cambridge.

Aquabrowser takes feeds from all these data sources and rearranges and tidies up the data (through a mixture of straight deduplication and FRBR) to make it all more manageable - and more importantly provides a single point from which to search all of our collections.

We've now managed to integrate Camlib, our mobile interface, with the Aquabrowser APIs to provide a much improved search service. Our new search options are:

• Search All
• UL & Dependents (incl. Manuscripts)
• Departments and Faculties
• Colleges
• Affiliated Institutions
• Electronic Resources

The big difference is that you can now search across the whole of Cambridge. But you can also search across the different types of collection more sensibly. As an extra we've separated out electronic resources as a distinct section for those only interested in the online.

The search results are very differently presented - one result for each title, followed by a list of the libraries who have a copy.

In short, if you are looking for all copies of a particular title in Cambridge, instead of having to make eight separate searches and plough through multiple results in each, you make one search with all the relevant information returned under one result.

We're hoping that a by-product of this kind of searching will be increasing integration of library services in Cambridge - if people see us and our collections as an integrated whole, then we might move faster to meet their expectations.

And we've even got some funky book covers from Syndetics to spice up our results list ...

Proprietary web services and proxies

A couple of weeks ago I blogged about writing local web services to extract information and embed functionality. However, many data sources come with their own range of ready-made proprietary web services. The big advantage of using proprietary web services is that you don't have to write your own. The disadvantage is that (usually) you can't alter them. Unless you have a very good relationship with your system supplier, you're stuck with what you're given.

Most data sources will allow you to write web services which extract data from the system, but the majority won't allow you to write web services which alter data in the system. If you want to use functionality in your widgets rather than simply displaying information (i.e. renew books, rather than just say what you have on loan) then you may find yourself having to use proprietary web services.

Basically, if the existing proprietary web services do exactly what you want, use them. If not, you might want to consider writing your own to extract data. But as far as altering data in the system goes, you may well be stuck with the proprietary ones.

[In practice, we find ourselves using a mixture of local and proprietary web services for data retrieval, and proprietary web services for data alteration]

Somewhere in your system's documentation you should find a list of existing web services and some notes on how to use them. This should include the service address, input parameters and return values. Which translates as how to find the service, how to ask it to do something, and how it will respond. Pretty much the same as using locally written web services.

But there is one potential complication with proprietary web services. If you make a request to a web service from a browser and then try to read the response, the browser will first establish what the response is and where it comes from. If the response is in XML and the domain of the web service is different from the domain of the page calling it, the browser gets worried about security and the whole enterprise collapses.

There are two ways round this. If the proprietary web service has the capability to return its response in json then you don't have worry about where it's coming from. If not, you will have to write a proxy.

A proxy is an intermediate web service which can sit alongside your local web services, in your own domain. You access it just like any local web service, and your browser will treat the xml it returns as if it had come from a local web service.

When you send a request to a proxy it passes it on to the proprietary web service, reads the response and then passes it back to you. Because the proxy is not a browser it doesn't care where the proprietary web service is, and because the proxy is in your domain, your browser is cool with it too.

An additional advantage of going through a proxy is that if your system's proprietary web services return responses which are over-long, oddly structured or confusing, then you can talior that response on the way back so it gives you what you actually need.

We use one proxy web service to connect to all of our library management system's proprietary web services. So we pass the service name, and the request to the proxy. The proxy calls the appropriate service with the request. Then it processes it and returns it to the browser.

Opening Hours and Library Pics

We've now integrated library information (address, opening hours) into various parts of CamLib

Opening hours are pretty important. When a user finds a book online, the next thing they want to know is usually "when can I get this book" - though in Cambridge they may also be asking "can I borrow this book" or even "can I use this library". All of this information is available in CamLib, but not in a particularly joined-up way.

The next challenge is not just to display this kind of information but to use it in the provision of relevant services - allowing users to filter searches to only bring back items in libraries they can use, or items in libraries they can use where they can borrow, or items in libraries which are open and which they can use.

We've also put some pictures of libraries up. Not all libraries have pics yet - we're adding them as they get sent in. The pics make the interface look nicer, but also have a practical purpose. The bigger sites in Cambridge have large numbers of buildings, often crowded together. A map reference only gets you into the general vicinity.

We asked libraries to include photos of the building as a whole and the entrance to the library so that when users are there or thereabouts they can use them to find the particular library they need.

Developing for mobile devices

Good article in UKOLN's Ariadne on developing for mobile devices http://www.ariadne.ac.uk/issue63/massam-et-al/#16 Lots of useful links and tips, and some nice quotes from students

Basically reaffirms that students will use mobile devices for the web when it's cheap enough, quick enough, and when screens are big enough to let them see what's going on ... and popular wisdom suggests that we are reaching this point.

Students also seemed to be saying that they would use the mobile internet to check (and possibly discover) rather than to consume (or possibly create). My hunch is that users will want to be able to duck into mobile interfaces to perform a distinct task - checking a reference, renewing a book, finding a library - and then duck out again.

The minimal startup time for mobile devices compared to laptops/desktops means that I often check something on a mobile even when I have access to a "proper" computer. For fact checking, the advantages of a proper keyboard/big screen don't outweigh that interminable wait for the whole thing to get going.

I use a laptop/desktop if I need to write anything longer than a tweet, or read anything longer than a (short) article - which I may have discovered on my mobile.

Providing library content on mobile devices is another matter, particularly as most of our electronic content is accessed via third party interfaces.

Integrating our own service

We've talked a little about collaboration and integration of services between institutions - but sometimes the need for integration is closer to home. At the UL we have two sources of library data - our Voyager library management system, and a local database holding specific information about each library in Cambridge. This ranges from opening hours to location, staff details to system information. The two databases are linked by common identifiers so they can "talk to each other".

This local database is used to drive interfaces such as the Cambridge Libraries Directory and Libraries Map. It's also the foundation of a number of services we provide for Cambridge librarians - web based reporting from Voyager, course booking, record enrichment, staff management and stockchecking being just a few. Essentially, we have used development based around this local database to fill in the gaps where Voyager either has no slot for the information we want to store, or doesn't provide the service we need.

Widgets provide the perfect opportunity to start tying this data together more closely. We're already doing this to a certain extent in the CamLib mobile interface - Voyager knows which library a book is in, our local database knows where that library is, allowing us to link out to a map.

But there are all sorts of other possibilities - from simple ones like including library opening times in searches, to more complex functionality like "show me the nearest copy of this book", or "only show me books which are held in libraries which are open now". Another example of how bringing data together to get an idea of context allows you to provide relevant services for your users.

Collaboration with DSpace@Cambridge

This is a quick hello from the CARET-based developers on the project, Raymond and Bert!

So far in our portfolio of widgets we have Huw's "Library widget", we have CamLib, a mobile version of that widget currently in beta release - and a prototype version of an Exam Papers widget written by Harriet, a previous Arcadia fellow.

As Huw explained in an earlier post, there are two main types of widget - those that retrieve data, and those that alter data. All of our widgets so far are primarily about the retrieval of information - the next new widget will be one focussed on alteration: a DSpace Upload Widget. This will make it easier for users to contribute content to our institutional repository: DSpace@Cambridge.

To facilitate this, we'll be collaborating with the DSpace@Cambridge team, who are based at the UL. Bert and I met with the DSpace team last week to kick this off.

Writing Web Services for widgets

In a previous post I outlined the importance of web services to widget development, and pointed to two kinds of web service - proprietary ones which you can't alter, and locally written ones which you can. Today, I'm going to concentrate on locally written web services, and talk about some of the issues involved in writing your own web services. We're going to be making our code available towards the end of the project, so you should be able to pick up some complete examples soon. In the meantime, here are some of the basic ingredients of a home-baked web service.

1. Getting started

• Programming language: we use Perl (and occasionally PHP).
• Data source: the database or dataset you want to interact with (i.e. Library Management System).
• Query language: the language you will use to retrieve or alter the data in this system. Usually SQL.

2. Finding a home

• Your web services will need to live somewhere on your web server. In theory they can go anywhere, but it's best to have a separate directory for web services in general, and then distinct sub-directories for each system or dataset you want to interact with.
  
• Putting all your web services into one overall directory has the added benefit of making it easier for your system administrator (or you) to keep on top of the permissions and settings needed for web service files. Basically, if you're writing web services in Perl, the directory needs to be CGI enabled, and the files need to be readable and executable by anybody (usual octal permissions 755).

3. Do a bit of thinking

• If you're doing a lot of interaction with a dataset you'll probably have a range of questions/alterations you want to throw at it. Most "complete" transactions with datasets are made up of multiple smaller transactions (i.e. to book a train ticket, you might need to find the current time, find the next train, and book the ticket.)
• Instead of trying to write one huge web service to do everything, break each activity into its smallest distinct parts. This will leave you with lots of small, simple web services, each of which does only one thing.
• Now you can chain these simple services together into more complex transactions - but you can also reuse them in other transactions. This saves you asking the same question (i.e. "what's the time?") again and again in multiple web services.
  

4. What input do you need?

• Web services are triggered by requests. These requests (like web services themselves) can be of two types - questions for services that retrieve data or instructions for services that alter data.
• Requests are made up of the location of the web service, plus as many parameters as are necessary to either frame the question or perform the instruction. These parameters make up the input for the web service.
• The smallest amount of parameters needed is none. Think about a question like "what's the time?". If you had a web service which just gave back the current time when triggered, no further parameters would be needed.
• Most web services take at least one parameter. Taking the "what's the time?" service, if you introduced a parameter of "city" the web service could alter the query depending on the value of the parameter. So the service could ask "what's the time - in London?", "what's the time - in New York?", depending on the value of the parameter.
• The rule of thumb with parameters is that you should use as few of them as possible in order to ask the question/perform the alteration. You can always add more if required. Again, you should be thinking of small, simple services, each related to a specific task.

5. Checking input

• There are two things you need to check your parameters for - existence and content.
• For each parameter, you need to decide if it is compulsory or optional. Compulsory parameters are necessary for the web service to perform its function. For example, a service to book a train will need the train time in order to make the booking.
• But you might make the "city" parameter of "what's the time?" optional - if it exists, return the time in that city - if not, just return the current time in the UK.
• All parameters, compulsory or optional, need to be checked for content. Either the value will have to be in a limited range (i.e. if the parameter value isn't "Y" or "N" then it is an invalid value). Or it might need to be a type of value (i.e. a number, or a string). Even in the case of parameters which allow "free text" values, you will still have to check for "dangerous" characters. Web services (and all programs executable by the web server) are vulnerable to security attacks, and you will need to make sure that incoming parameters don't contain anything dodgy.
• A good rule is to decide what values or kind of values you will allow in parameter values and build from there, rather than trying to disallow all the possibly dangerous characters (you may miss one). So a rule like "the value must be a mixture of letters, numbers, commas and whitespace" is better (or more secure) than "the value must not contain semicolons).
  

6. Building the query

• Having gathered and checked parameters (if any) the web service now has to perform its function. To do this, it needs to talk to the dataset - and the usual language for doing this is SQL.
• Web services which retrieve data will use SQL to ask the dataset a question - such as "What's the time in New York?"
• Web services which alter data will use SQL to form an instruction. For instance "Book a seat on the 9.15 from Cambridge to London"
• If your web services use parameters for input, these will (usually) be interpolated into the SQL. So, for the train booking example above, "9.15" would have been the value of the parameter "time", "Cambridge" the value of the parameter "from" and "London" the value of the parameter "to". Parameters are a means of altering the question or instruction which gets sent to the dataset
  

7. Returning the response

• All web services should send a response back to whoever made the request.
  
• For web services which retrieve data, this will be the answer to the question - i.e. for question "What is the time in New York" the response might be "The time in New York is 10.30".
• For web services which alter data, the response will tell you what the web service has done. So for the instruction "Book me a seat on the 9.15 from Cambridge to London" the response might be "Seat booked" or even the more verbose "Seat no. XX booked on the 9.15 from Cambridge to London".
  
• The response should be tailored to how it's going to be used. Sometimes, you'll just want to know if the request was processed ok, sometimes you'll need much more detailed information.
• Your web services should always send a response, even if the request fails. A response of "your request has failed" is far more useful than nothing at all!

8. Chaining web services together

• A common way of using web services is to chain them together - which means using the response from one web service as part of the input for the next.
• Taking the (very simplified) example of booking a seat on the next train, the chain might go something like this:
• Request to "what's the time?" returns value TIME
• Request to "what's the next train?" with parameter TIME returns next train after time as value TRAIN
• Request to "book seat" with parameter TRAIN returns value SEAT BOOKED

Et in Arcadia ego

Avid followers of this blog will have spotted references to the Arcadia programme, a three-year programme funded by a generous grant from the Arcadia Fund to Cambridge University Library. The grant will enable us to explore the role of academic libraries in a digital age, create new programmes and services, particularly for undergraduates -- and also to improve the external environment of the library.

If this sounds like your cup of tea, you might also be interested in the Arcadia project blog, written by many of those involved in the project, as Arcadia Fellows or supporters. This includes a surprisingly large number of the CULwidgets team!

Project Plan Post 7 of 7: Budget

Here's our project budget! (and this is the last of the "project plan" posts, so, dear reader, you can relax as we return to our normally scheduled blogging about progress and things we've learned.)

We have some travel and other expenses included, to enable us to meet up with other jiscLMS projects and to share what we learn on the project, and to pay for food and incentives for users who are involved with our user testing.

Project Plan Post 6 of 7: Projected Timeline, Workplan & Overall Project Methodology

Our project has a number of overlapping stages. Given that we're developing a portfolio of widgets, with some common elements, some of these activities will be repeated for the different widgets; and some early activities will be "infrastructure" creation, such as developing links with the institutional repository, which will be needed for most of the widgets later on.

1. Setting up of a project blog, Google Code site etc. Allocation of staff time and resources.
2. Identification of a sample set of library materials for a usable and useful prototype.
3. Identify target student groups in liaison with faculties and departments.
4. Investigation of course and exam codes as used by the student registry. Adoption of file naming and storage strategy for course materials to allow retrieval by course code. Link up with student registry data to tie course codes in with student authentication in a widget environment.
5. Creation of routine to automatically retrieve library materials relating to a student's course into a widget. Design of clear and logical display for library/course materials retrieved.
6. Integration of course materials delivery with existing library widget.
7. Production of prototypes
8. User testing of interface.
9. Rollout of live version. Launch and marketing within and outside university.

Here's our project timeline - we're already well underway!



We will follow a lightweight, agile development model, aiming to deliver frequent rough prototypes for user testing with students where possible, and staff outside of term. This approach has delivered successful JISC projects before, and is closely linked to the JISC Users and Innovation Development Model. In addition, we will apply our expertise in user-centric design which derives from the JISC Academic Social Networking project.

Project Plan Post 5 of 7: Project Team Relationships and End User Engagement

Dr. Dan Sheppard will manage the project from CARET, and brings extensive software development expertise and also project management experience of cross-departmental projects, particularly with the University Library.

A project board, comprising representatives of Cambridge University Library and the University of Cambridge Centre for Applied Research in Educational Technologies will take responsibility for overseeing the project:

• Patricia Killiard, Head of Electronic Services and Systems, Cambridge University Library
• John Norman, Director, Centre for Applied Research in Educational Technologies
• Dr. Laura James, Chief Operating Officer, Centre for Applied Research in Educational Technologies

News flash!
Dan is this term's Arcadia Fellow at the University Library, so Laura is stepping into the breach as interim project manager. We look forward to having Dan back for the bulk of the project...


We will follow a lightweight, agile development model, aiming to deliver frequent rough prototypes for user testing with students where possible, and staff outside of term. This approach has delivered successful JISC projects before, and is closely linked to the JISC Users and Innovation Development Model. In addition, we will apply our expertise in user-centric design which derives from the JISC Academic Social Networking project - this has shown us how to carry out really good user testing.

Project Plan Post 4 of 7: IPR (Creative Commons Use & Open Source Software Licence)

All software outputs will be released under an Apache2 licence, and all documents under a Creative Commons "BY" (Attribution) licence. This means that people can reuse our outputs, even commercially, which can support the creation of business models for more sustainable systems, including collaborative development across both nonprofit and commercial organizations.

You can see the CC licence at the bottom of this blog!

(Note I'm sticking with the British English grammar forms of licence/license :)

Project Plan Post 3 of 7: Risk Analysis and Success Plan

As all the resources we will produce (code and documentation) are hosted on servers elsewhere (primarily Google's) we are confident that success in terms of hits can be managed.

If the widgets we create turn out to be huge successes with users within Cambridge, supporting them will form part of the "day job" of some of the project team members. Support for a popular codebase within other institutions would be provided through the community and by the project team.

Here's our risk analysis:

Project Plan Post 2 of 7: Wider Benefits to Sector & Achievements for Host Institution

The integration of library resources and services with the University's VLE and social networking sites, particularly iGoogle and Facebook, is an institutional priority for Cambridge University Library. The Arcadia Fellowship Programme, based at the library, has already explored student information needs and identified the potential value of delivering library materials and resources to students in their workspaces in addition to conventional library-managed interfaces.

Through a partnership between the library and the Centre for Applied Research in Educational Technologies (CARET), which develops and manages the VLE Cambridge has made significant progress in this area, having developed the Cambridge Libraries iGoogle Widget. This already provides most of the example functionality outlined in the call, enabling students to view their library profile, view and renew loans, and view and cancel requests (it lacks only the ability to pay library charges). It is fully integrated with the University's authentication system, and is currently live in Facebook, iGoogle and CamTools, the University's Sakai-based VLE. One strand in this proposal would be to document and share the code for the widget in its current state.

The Library and CARET will build on this foundation, to provide a joined-up and relevant combination of library services and resource delivery outside the traditional library environment. We will extend the use of widget functionality to the delivery of other library materials - such as exam papers, reading lists, handouts, content from the institutional repository, and generic subject lists. It will make use of university identifiers to retrieve student course details from CAMSiS, the student registry system and this course information will be used to provide relevance in the delivery of library materials. The result will be a potentially powerful model of evaluating student profiles to provide relevant library services alongside appropriate teaching materials.


A further stage of development will focus on mining these course materials for links to library resources. This will enable us to start delivering relevant library content to students beyond the immediate library web environment. We envisage that one deliverable of the project will be a clear outline of the next steps in this area, and possible implications.



This project is a partnership between the University Library and CARET, a strong combination in terms of both skills and institutional roles. The University Library is the main provider of content for the support of teaching and learning in the University, and is repositioning itself to support the processes of teaching and learning more directly. CARET are leaders in the development of innovative educational technology, and are also the developers and operators of CamTools, the institutional VLE.

This project will produce a number of widgets delivering access to library content within social networking sites and VLEs in partnership with the Centre for Applied Research in Educational Technologies. It will explore opportunities for making content delivered through widgets relevant to the user by linking to course-codes held by the student registry. The project will assist the wider take-up of widgets for library-related content by making the code for the existing widget and those developed through the project openly available along with documentation and guidance.

Project Plan Post 1 of 7: Aims, Objectives and Final Outputs of the project

The project will build on the success of the iGoogle widget developed by Cambridge University Library to produce a number of widgets delivering access to library content within social networking sites and VLEs in partnership with the Centre for Applied Research in Educational Technologies. It will explore opportunities for making content delivered through widgets relevant to the user by linking to course-codes held by the student registry. The project will assist the wider take-up of widgets for library-related content by making the code for the existing widget and those developed through the project openly available along with documentation and guidance.

We will deliver a portfolio of fully documented widgets released under an appropriate open source licence, which will work across iGoogle, Facebook, and Sakai (our institutional VLE) as far as technically possible. We will aim to deliver as many widgets as possible which are found useful in user testing, and to open up as much appropriate and useful library content through the widgets as we can.



Our project outputs will be:

1. Widgets for: reading lists, past exam papers, lecture handouts and other course teaching content, and connections to other library content, delivered to students for their own courses using the student information system to create relevance information

2. Software code with full and clear documentation for the Cambridge University Library iGoogle widget
3. Guidance for libraries on code development for library widgets
4. Recommendations for future development path for library widgets
5. Project blog (which you are reading!)
   

Widgets and institutional collaboration

One of the reasons we became interested in widgets was that they seemed like a quick and easy way to start weaving together some of the disparate parts of University life. One of the truisms of working in large organisations is that service providers tend to focus on the boundaries between themselves and other groups (i.e. this is what we do) - despite the fact that for users these boundaries are either invisible or annoying.

The University Library has traditionally been seen as a provider of content. That content is then whisked off and used in the core processes of the University - research and teaching. The UL is now moving to support these core processes more directly - which means getting more closely involved with other bodies in the University.

Institutional collaboration, between the University Library and CARET, is at the heart of this project. Another important connection is with the Management and Information Services Division (MISD). They hold the student data (which courses students are on, which exams they are taking) which is essential to providing relevant services. We are hoping that our exam papers widget will provide a blueprint for using this data to retrieve learning objects - and also cement the working relationship between our institutions.

Our other initial collaboration is closer to home -with DSpace@Cambridge (the University's institutional repository) which forms part of the University Library. We're not just interested in getting valuable information out of DSpace (exam papers, theses, reading lists etc.), but also at getting material in. Part of the exam papers widget project will be to provide a widget for the deposit of material into the repository.

Collaborations tend to work best if there is something concrete to work on together, and widgets are a fast and lightweight means of providing that collaborative basis. Which makes widgets important for strategic reasons as well as for functionality.

library widget goes mobile

One of our project objectives was to take what we had learned when designing the original library widget - particularly about providing useful services in limited space - and apply it to mobile interfaces.

Today sees the Beta launch of CamLib - basically the library widget for mobile devices, with extra bells and whistles. Available here:

http://www.lib.cam.ac.uk/mob/camlib.cgi

Test login (for library card) is:

Barcode: VABCD
Surname: Bloggs

[do have a look around, try out the bookbag etc., but please don't use the request functionality if you can help it - not fatal, but will save some confusion at our end!]

It's been tested for iPhone, iPod Touch and Android devices - though not exhaustively, so please comment on this post and let us know if it's misbehaving.

We used jQTouch, a jQuery library for mobile development, which lets you write web interfaces which behave (to a certain extent) like native apps. There is a big debate in library circles about native apps vs web interfaces ... we decided on a middle path to get the best of both worlds.

We did some hard thinking about what people would want from a library mobile device, and apart from all the usual renewing/requesting/searching activities, we plumped for bookbag functionality - essentially the ability build up a list of items to look at later.

Viewing content on a smartphone is possible but awkward. And nobody wants to be finding books on their phone and then writing down classmarks on a piece of paper. The bookbag lets you build up a list of items as you search and then email it to yourself or others.

There's plenty of other functionality to explore, including links out to full text, Google Books, maps, floorplans etc.

For items in libraries other than the University Library (which locates onto a floorplan), the link to Google Maps will even use your phone's location system and tell you how long it will take you to walk/drive/bus to the book in question (click on Directions). Playing with this functionality over the weekend produced the surprising titbit that it would take 1 day and 10 hours of solid walking to get from my Dad's house in Bromsgrove to the issue desk of the English Faculty Library in Cambridge - quite a challenge for even the most committed scholar!

Widgets and web services

Behind (most) widgets you’ll find one or more data sources – whether a relational database, a bunch of linked data or even a text file. The widget is basically an interface to this data. The relationships between widgets and their data sources fall into two main types:

• Retrieval: the widget asks the data source a question and uses the answer for some purpose (usually display)
  
• Alteration: the widget causes a change to be made in the data source

So, in our Library widget, an example of retrieval would be to show a list of the items a user has on loan (no change to underlying data). And an example of alteration would be renewing one of these items (change made in underlying data).

Which means that widgets need a way of talking to their data sources.

Most widgets talk to their data sources through web services (also known as APIs) - little scripts which sit on your web server and mediate between your widget and your data.

They work something like this - your widget makes a request to a web service - either that some data be retrieved, or that a change be made in the data. The web service translates the request into data-speak and sends it on to the data source. The data source processes the request, makes any alterations, and sends back a response. The web service translates the response into widget-speak and sends it back to the widget.

So, why can’t widgets talk to their data sources directly? Why the need for the middle-man? There are four main reasons:

• Security – if your widget is talking to the data source directly, it is likely to contain passwords. Because widgets are usually written in html/javascript, pretty much anyone can access the source code and so access your passwords.
• Efficiency – web services are reusable. Say you have three widgets that need the same bit of data. You can write one web service to make the request and do any processing on the response, and all three widgets can use it.
• Ease of use – most widgets are written in html and javascript. Basically, it’s pretty easy to make calls to web services from javascript. Making calls to data sources directly from widgets is a bit of a nightmare.
• Innovation – if you have a bunch of web services sitting on your server, you can choose to publish them. This means that other developers can build systems and functionality around your data. Which is not only pretty exciting, but also saves you having to do everything yourself.

Many systems now come with their own set of proprietary web services for talking to their data source. These should be written up in your system’s documentation. If the system has no web services or it doesn’t have a web service for what you want to do, it’s pretty simple to write your own.

EXCEPT - reflecting the two types of request you can make to your data source, there are two types of web service - those that retrieve data and those that make alterations to your data. Most systems will allow you to write your own web services to retrieve data BUT most systems will insist that you use their own proprietary web services to alter data in the system.

So, going back to our original example, we have a locally written web service which retrieves a list of a user’s loans, but we use a proprietary system web service to renew one of those loans.

Our widgets use a mixture of locally-written and proprietary web services to relate to data sources, and (unless your system’s proprietary web services are truly excellent) you’ll probably find yourself doing the same. I’ll stop here before this post turns into an essay, but in future posts I’ll outline how to write web services, and how to use them.

Where we are

The University Library (UL) and CARET have already dipped a mutual toe into widgets with the development of the Cambridge Libraries Widget. Before we immerse ourselves further, here’s an update on where we are and how we got there.

The UL became interested in widgets (portable web interfaces) as a quick and easy way of making library functionality and content available outside the traditional library catalogue. We were particularly interested in CamTools, the University’s VLE.

CamTools is written and administered by CARET, who have a wealth of experience in developing web interfaces. The UL and CARET were casting around for opportunities to collaborate, and the development of a library widget seemed like the perfect opportunity.

We found we were able to work very well together, and after various struggles (mainly to do with authentication), the Cambridge Library Widget was launched in CamTools, iGoogle and Facebook in January 2010, with an embeddable version following soon afterwards. The Widget is deployed in our production VLE, with usage climbing steadily.

It allows library users to search library collections, view their library profile (which can be fairly complex in Cambridge), and manage loans and requests. Widget functionality will underlie the user interfaces for our new resource discovery platform to be launched at the start of the next academic year.

So, what have we achieved so far? We have taken library functions out of the library catalogue and freed them to wander in our VLE and social networking sites. We have taken control of the presentation of library services and functionality in a way we haven’t been able to do before. And we’ve written a few snazzy extra features. But essentially we have been in the (admittedly important) business of recreating existing functionality in a new environment.

Which is all well and good, but we were left feeling “there must be more to widgets than this”. Hence this project, in which we aim to unlock the potential of widgets to do entirely new things, and to bring together some of the disconnected parts of academic life for our users.

This means building widgets which talk to more than one system at a time, linking people to integrated services and content. We want our interfaces to connect not just to our library management system, but also to our institutional repository and student registry.

Of course, we are still interested in new environments, and one strand of the project will tackle the challenges of deploying widget-like interfaces on mobile devices. And we’re also hoping to have some fun with a few one-off widgets which explore the odd and innovative possibilities of tying disparate data together.

Over the coming weeks, as well as keeping you up-to-date with new developments, we’ll be delving into the past to illuminate what widgets are, how they work, and how to get them working for you.