This blog post outlines the whys and hows of measuring outcomes for public libraries and has been developed from the introductory presentation made at a recent workshop for The Libraries Taskforce in London by Dr Darren Smart, Strategic Manager for Operations with Kent County Council’s Libraries, Registration & Archives Service.

Check out his insights below.

Why Measure Outcomes?

Most organisations are outcome focused, including local and national government. Your Authority will have a vision and a strategy defined as outcomes to deliver, and your Service will be expected to contribute to these.

Why measure outcomes? Your #Authority will have a vision and a strategy defined as outcomes to deliver, and your Service will be expected to contribute to these.#libraries Share on X

By measuring outcomes rather than simple inputs/outputs you will be able to:

1. Demonstrate more effectively how your library contributes to the delivery of your Authority’s evidence-based plan for addressing local needs which will be defined in terms of desired outcomes.

2. Be a more effective advocate for your services, using the stories you have collected to challenge traditional perceptions by illustrating the impact across a wide range of key outcomes.

3. Make better bids for funding through commissions or grants, as awarding bodies are outcome focused.

4. Deliver more effective partnership working as it is easier to identify common objectives/outcomes and so remain focussed on the win-win.

Put simply, in practical terms measuring outcomes is vital for demonstrating the impact, and thus value, of the services you deliver.

Kent Outcomes Evaluation Framework
Kent Outcomes Evaluation Framework

The Principles of Measuring Outcomes

The key principle is that to measure outcomes you need both qualitative and quantitative data. At its heart, qualitative data is simply customer feedback that describes the impact a service or activity has on people’s lives. The base unit for this is the individual case study which tells the story of what happened and how that made a difference to the person.

To measure outcomes you need both qualitative and quantitative data, collectively they become a powerful body of evidence. #libraries #libraryresult #librariesmatter Share on X

In isolation, these are simply nice stories, at best suitable for a soundbite. However, collectively they become a powerful body of evidence which demonstrate the effects are specific and reproducible, thus making the data trustworthy and convincing.

Quantitative data is all about the numbers and include the metrics most librarians are used to measuring, e.g. loans or visits. This type of data is more empirical and thus seems more trustworthy. However, to be useful the data must be predictive of the outcome and, unfortunately, the causality is often assumed rather than proven. For example, despite the prevalent myth, overall loans are only a transactional measure and not an indication of improving literacy in the community.

Nevertheless, where causality has been proven in large scale, well controlled published studies, e.g. the benefits of Books on Prescription, the quantitative data can be used as a surrogate measure to evidence impact on an outcome. This is then a very efficient way of measuring outcomes.

Of course, it is possible to quantify qualitative data by collecting and collating enough case studies, and this is often the best method to employ if trying to measure a broad spectrum of outcomes.

It is possible to quantify #qualitative data by collecting and collating enough case studies, and this is often the best method to employ if trying to measure a broad spectrum of outcomes.#libraries Share on X

In practical terms this means that any approach must:

1.Be simple & easy for all frontline staff to implement.

2.Systematic, to quickly gather case studies to build an effective database.

3.Engage staff to embed an outcomes-focused culture to ensure sustainability.

Putting it into Practice

Build an Outcomes Focused Culture

  1. Start at the top
    • It is essential to get political and senior management buy-in
    • Focus on the strategic value and how it is a proven model
  2. Spread the word
    • Use presentations, newsletters, meetings, webinars etc to ensure all stakeholders are thoroughly briefed re the benefits
    • Periodically reinforce through regular updates including examples
  3. Involve the staff
    • Hearts & minds is key, they are a critical stakeholder
    • Involve staff in designing the process e.g. the capture form
    • Good communication is vital, use lots of positive reinforcement,
    • Train staff in relevant skills e.g. active listening & asking open questions

Gather case studies

  1. Broad spectrum approach
    • Capturing stories on an opportunistic basis, using active listening & asking open questions to get the full details
    • Make this ‘Business as Usual’ for frontline staff
    • Use a standard capture form & reporting process
    • Make sure the capture forms are at every staff point in every library
  2. Targeted approaches
    • Short term, focused on a specific activity, event or demographic
    • Use observation, structured questionnaires, 1-1 interviews and/or group chats to capture as many stories about the focus as possible
  3. Surrogate measures
    • Build a database of internal & published evidence proving links between specific outcomes and quantitative data that you can measure
    • The level of proof will vary considerably depending on the audience, e.g. Public Health teams will expect a clinical level of proof

Develop your outcomes database

  1. Developing your framework
    • This is just like cataloging
    • Build a tiered framework around your chosen outcomes
    • Ensure all your Authority’s key outcomes are covered
    • In addition, include tags for key demographic groups e.g. refugees
  2. Quality control
    • Is vital for the credibility of your data
    • Set up a ‘So What’ panel to review every story submitted to ensure it demonstrates clear outcomes and is not simply a comment/compliment
    • The panel should consist of an odd number to avoid split votes
    • Must not over-interpret the stories, generally, no individual story should be assigned to more than 3 outcomes

Use your outcome data

  1. This is important, otherwise, why are you doing it?
  2. Ensure outcomes data is included in your KPIs
  3. Periodically share case studies with your staff & tell them how using the data. This will help motivate them and reinforce the outcomes culture
  4. Include outcome data in all re