This blog post outlines the whys and hows of measuring outcomes for public libraries and has been developed from the introductory presentation made at a recent workshop for The Libraries Taskforce in London by Dr Darren Smart, Strategic Manager for Operations with Kent County Council’s Libraries, Registration & Archives Service.

Check out his insights below.

Why Measure Outcomes?

Most organisations are outcome focused, including local and national government. Your Authority will have a vision and a strategy defined as outcomes to deliver, and your Service will be expected to contribute to these.

By measuring outcomes rather than simple inputs/outputs you will be able to:

1. Demonstrate more effectively how your library contributes to the delivery of your Authority’s evidence-based plan for addressing local needs which will be defined in terms of desired outcomes.

2. Be a more effective advocate for your services, using the stories you have collected to challenge traditional perceptions by illustrating the impact across a wide range of key outcomes.

3. Make better bids for funding through commissions or grants, as awarding bodies are outcome focused.

4. Deliver more effective partnership working as it is easier to identify common objectives/outcomes and so remain focussed on the win-win.

Put simply, in practical terms measuring outcomes is vital for demonstrating the impact, and thus value, of the services you deliver.

Kent Outcomes Evaluation Framework

The Principles of Measuring Outcomes

The key principle is that to measure outcomes you need both qualitative and quantitative data. At its heart, qualitative data is simply customer feedback that describes the impact a service or activity has on people’s lives. The base unit for this is the individual case study which tells the story of what happened and how that made a difference to the person.

In isolation, these are simply nice stories, at best suitable for a soundbite. However, collectively they become a powerful body of evidence which demonstrate the effects are specific and reproducible, thus making the data trustworthy and convincing.

Quantitative data is all about the numbers and include the metrics most librarians are used to measuring, e.g. loans or visits. This type of data is more empirical and thus seems more trustworthy. However, to be useful the data must be predictive of the outcome and, unfortunately, the causality is often assumed rather than proven. For example, despite the prevalent myth, overall loans are only a transactional measure and not an indication of improving literacy in the community.

Nevertheless, where causality has been proven in large scale, well controlled published studies, e.g. the benefits of Books on Prescription, the quantitative data can be used as a surrogate measure to evidence impact on an outcome. This is then a very efficient way of measuring outcomes.

Of course, it is possible to quantify qualitative data by collecting and collating enough case studies, and this is often the best method to employ if trying to measure a broad spectrum of outcomes.

In practical terms this means that any approach must:

1.Be simple & easy for all frontline staff to implement.

2.Systematic, to quickly gather case studies to build an effective database.

3.Engage staff to embed an outcomes-focused culture to ensure sustainability.

Putting it into Practice

Build an Outcomes Focused Culture

1. Start at the top
   • It is essential to get political and senior management buy-in
   • Focus on the strategic value and how it is a proven model
2. Spread the word
   • Use presentations, newsletters, meetings, webinars etc to ensure all stakeholders are thoroughly briefed re the benefits
   • Periodically reinforce through regular updates including examples
3. Involve the staff
   • Hearts & minds is key, they are a critical stakeholder
   • Involve staff in designing the process e.g. the capture form
   • Good communication is vital, use lots of positive reinforcement,
   • Train staff in relevant skills e.g. active listening & asking open questions

Gather case studies

1. Broad spectrum approach
   • Capturing stories on an opportunistic basis, using active listening & asking open questions to get the full details
   • Make this ‘Business as Usual’ for frontline staff
   • Use a standard capture form & reporting process
   • Make sure the capture forms are at every staff point in every library
2. Targeted approaches
   • Short term, focused on a specific activity, event or demographic
   • Use observation, structured questionnaires, 1-1 interviews and/or group chats to capture as many stories about the focus as possible
3. Surrogate measures
   • Build a database of internal & published evidence proving links between specific outcomes and quantitative data that you can measure
   • The level of proof will vary considerably depending on the audience, e.g. Public Health teams will expect a clinical level of proof

Develop your outcomes database

1. Developing your framework
   • This is just like cataloging
   • Build a tiered framework around your chosen outcomes
   • Ensure all your Authority’s key outcomes are covered
   • In addition, include tags for key demographic groups e.g. refugees
2. Quality control
   • Is vital for the credibility of your data
   • Set up a ‘So What’ panel to review every story submitted to ensure it demonstrates clear outcomes and is not simply a comment/compliment
   • The panel should consist of an odd number to avoid split votes
   • Must not over-interpret the stories, generally, no individual story should be assigned to more than 3 outcomes

Use your outcome data

1. This is important, otherwise, why are you doing it?
2. Ensure outcomes data is included in your KPIs
3. Periodically share case studies with your staff & tell them how using the data. This will help motivate them and reinforce the outcomes culture
4. Include outcome data in all reports, papers and publications
5. Use routinely in your external communications, especially social media
   • ensure customers and wider community are aware of the good work you do
   • it will also encourage other customers to tell you their stories.
6. Use it to bid for & report back on commissions and grants

Does It Really Work?

The successful use of this approach in Kent has been described here too and is now being explored/instigated in 35 other UK Library Authorities, as well as in 2 school library services and 1 health and leisure service.

In conclusion

I outlined the why & how of measuring outcomes for libraries and hopefully will encourage more library services across the UK and beyond to adopt an outcome-focused approach.

Want more insights from libraries across the world ? Stay tuned for our weekly posts and read the latest developments in libraries from around the world. Find us on Facebook and Twitter and sign up to our blog to receive new library insights directly to your e-mail.