A brief introduction to ‘outcomes data’

Outcomes data show the result, or impact of something

Outcomes data show the impact of an activity (e.g. impact of attending a day centre on wellbeing). The terms "outcome(s) data" and "impact" are sometimes used interchangeably. Outcomes are not the same as outputs which refer to measurable results (e.g. numbers of day centre places filled).

Outcome, or impact, data are any data (information) that evidence what happens (benefits and changes) as a result of a particular input, service or intervention – in this case the day centre. Outcomes/impact may include changes and benefits for individuals, changes for organisations or systems (e.g. NHS, social care), communities, or financial changes, for example. These changes may be positive (good), negative (unwanted), inconsequential (neutral) or may be completely unexpected. Outcomes data may reflect potential, slow and small progress, or maintenance (i.e. delaying deterioration). Or they can focus on the ‘here and now’, for example, people being happy and having a good day rather than on achieving set goals. They may aim to discover whether people got what they hoped for from attending the day centre that day and what was missing if they didn't.

Data can be quantitative (involving numbers) or qualitative (rich, informative text). Both are useful in different ways. These types of data are explained in more detail further down this page. 

Data may be collected by regular monitoring or in evaluations. Monitoring may be undertaken at set times, for example, before starting to attend a day centre, one month after starting to attend, 3 months later, 6 months later and after a year. Regular monitoring shows long-term impact on individuals. Evaluations may take place during a project that is unfolding to inform its future and further development. Or they may take place after an event to discover what has been learnt. One-off measurement of outcomes for individuals can also take place.

Outcomes can be:

  • ‘hard’ – obvious and very likely quantifiable, for example, number of GP appointments attended
  • ‘soft’ – for example, feelings of wellbeing, quality of life or safety
  • ‘maintenance’ – aiming to retain a certain level of quality of life, for example, rather than changing it
  • ‘process’ – about the way a service is delivered and experienced
  • change’ – about any changes experienced due to the service.

Are outcomes data routinely gathered in English day centres?

No standardised ways of gathering outcomes data about day centres are in place in England. This is the case even in other countries where there are day service umbrella bodies, for example the United States of America.

Experts in the United States (US) have called for the development of a uniform set of outcomes measures for use in US day services. One reason for this is to help the leverage of additional funding streams. As in England, US day services are varied, having developed in a piecemeal and inconsistent way, and they may lack resources. Unlike English day centres, US day services are part of the health care system’s home and community services, they cater for larger numbers (average 58 per day) and, as some are social and some address their attenders' medical or physical needs, staff bodies often include social workers, nurses and physiotherapists. Measures proposed, therefore, may not all necessarily be suitable for English day centres, although some have been included here.

In England, some commissioners want a better, more systematic way of gathering data across directly commissioned services (for example, a framework). However, because of the lack of systematised outcomes data collection, individual commissioners and providers need to develop their own local approach. Without a strong, systemised evidence base, local authority (LA) and NHS commissioners struggle to justify continuing to fund or to expand services.

Challenges involved in gathering and making use of outcomes data are discussed later in this page.

What are outcomes data useful for?

Within a day centre context, outcomes data can serve various purposes and may be of interest to many people in different roles, including people who take decisions related to day centres. They may be useful for:

Tracking individual progress

Outcomes data can track the progress of individuals who attend the day centre. People may wish to see how they’ve progressed since starting to attend their day centre (especially if feeling tired or having ‘a wobble’ about going). Carers may appreciate knowing how their relative/friend is doing, too.

Outcomes data can show (evidence) the usefulness (or impact) of the day centre to its users for funders, commissioners and others who make service-related decisions. Outcomes data can be helpful for funding applications/tender bids and service monitoring. They can inform service reviews. They can also contribute to economic evaluations or social value/social return on investment exercises. Business analysts who can help construct strong justification for commissioning decisions are less common in LAs now, and, therefore, easy-to-use data/evidence is appreciated.

Informing staff who are new or new in their role within LA or NHS services

Outcomes data inform new staff and staff moving roles within LA and NHS services where there is high turnover, reorganisations and resulting changing roles are common. Background information will be necessary for new or re-assigned staff and is particularly important within an environment that encourages joint commissioning.

Outcomes data can also help map service impact across to current policy goals, thus furthering the understanding of day centres’ relevance to policy. Policy goals include ensuring good quality of life for people with care and support needs, delaying or reducing the need for care and support, ensuring people have a positive experience of care and support, safeguarding adults whose circumstances make them vulnerable and protecting them from avoidable harm. Some contracts may specify policy-related outcomes for service providers to aim for.

Evidencing impact for people who make referrals or who signpost to day centres

Outcomes data can evidence the usefulness (impact) of the day centre to social care and NHS professionals and others who may consider making referrals, or signposting, to day centres (e.g. social workers, occupational therapists, social prescribing link workers, information and advice services). Any referrals made should be appropriate, and outcomes data may help with this.

Raising awareness of how day centres can support people

Demonstrating how day centres make a difference to those who attend them supports marketing of the service to potential service users and to the general public. This is especially important because day centres will increasingly need to make their services attractive (marketable) to people who will pay for services themselves or their families. These ‘self-funders’ include people who do not meet a local authority’s criteria for services but who have other needs which could be met by a day centre (e.g. functional mental health needs such as depression). These people and their carers will need to understand exactly what they would get for their money. This will be even more important for services not subsidised by contracts and which, therefore, may make higher charges to users. The Resources Hub includes a section on marketing communications.

For service improvement

Outcomes data can support day centre providers to self-audit and undertake service improvement.

Supporting recruitment and retention of staff and volunteers

Monitoring staff and volunteers’ feelings/role satisfaction/views may improve staff/volunteer retention as it helps identify where changes need to be made. Publicising satisfaction data may also help market a day centre as a good place to work or volunteer.

Recruitment and retention in social care are often problematic. Volunteering at a day centre is not only an opportunity to help others and can also be beneficial to volunteers’ health and wellbeing, especially if they are older themselves.

Informing topic-based local initiatives

Localities may wish to address particular problems that have been identified, such as high levels of loneliness. Outcomes data can inform themed briefings for staff or local strategies – such as how to reduce loneliness locally.

Demonstrating value for money

Outcomes data may also help demonstrate value for money (economic evaluation).

Supporting care plan reviews and enable people to know more about the service

A participant in this research also explained how central registration of outcomes data for individuals on statutory databases used by other professionals, for example, can support reviews of an individual’s overall care and support package. At the same time, data support other staff within a directorate to better understand the service and raise the profile of the day centre.

What are qualitative data (rich information)?

Qualitative data are ‘rich’ information. They help to gain a better understanding of people’s perspectives, views, experiences or emotions, the ‘whys’ and ‘hows’.

They are detailed feedback (or descriptions). They are usually provided in response to open questions (i.e. ones that do not require yes/no or a 1-5 scaled response answer). They may also be provided not on request (e.g. compliments or complaints). Qualitative data are individual and, therefore, subjective.

Gaining such insights and understanding can help a provider better understand their users (or staff/volunteers) which can help improve service provision (and recruitment and retention). Data can be practically useful. For example they can:

  • inform, for example, development or direction of services, for understanding what goes well and what goes less well, or for understanding satisfaction with various aspects of service provision or working/volunteering at a day centre.
  • help (potential) funders, commissioners, decision-makers and other supporters (e.g. local businesses, ‘Friends of’ groups) to understand the service offer better and its value.
  • help the wider public and potential future users know more about what a ‘day centre’ is.
  • provide evidence that service users have been consulted about, for example, organised activities, meals, service developments.

Qualitative data can be collected in various ways - in surveys, in conversation, by email, or in group discussions, for example. They may be collected alongside quantitative data, for example in a survey that asks people to rate how happy they were with x, and which also asks what they particularly liked about x.

It is crucial to record and process the data collected so that they are of practical use. They can be collated and reported in different ways – narratively (in text), as case studies, or a mix that includes descriptive statistics (e.g. percentages). Short reports can be useful for internal purposes, as records, and for funders or marketing (e.g. on a website). Short reports can also be circulated to service users, carers or 'Friends of', and to motivate staff/volunteers.

Individual case studies (e.g. short overview of the person and the benefits they have experienced as a result of using the service) can also be written up and used for this purpose. Information about ways to do this appear further down this page, under ‘Some ‘tools’ (questionnaires) or approaches to gathering data that could be used by day centres’. For example, the Most Significant Change approach uses a person’s own words to explain the most important change they have experienced as a result of using a service and why this change is important to them, and Twelve principles for effective personal outcomes-focused recording are suggested by Social Care Wales.

Examples of ways to present qualitative data

People reported enjoying the meals and valued eating in company. It was important to them that the meals were served to everybody at one table at the same time. Their suggestions for improvement included….

Most people reported enjoying the organised activities. Although most attenders enjoyed bingo – saying it was fun, engaging and mentally taxing – a small number did not. 

Most carers reported that their relative seemed happier after attending the day centre, which also helps their own wellbeing. Some carers made some suggestions about x. 

Staff satisfaction was influenced by x.

The volunteering survey found that an important factor contributing to people choosing to stay was x. This was because y. All volunteers said volunteering at the day centre improved their wellbeing and was satisfying.

Case study of an individual that demonstrates the value of a service to that person. 

Case study demonstrating the ‘added value’ of a service (what is provided in addition to what is expected or contracted for), for example:

●  staff making a GP appointment for someone they suspect may have a urinary tract infection, or liaising with the District Nurse so that someone can have their insulin injection/bandages changed at the day centre instead of waiting at home and missing the companionship they enjoy at the day centre

●  how the provider promotes staff wellbeing

●  someone taking some flowers from their garden into the day centre for others to enjoy gave the person purpose in life and made them feel like they have actively contributed to this community.

Some things to bear in mind when planning to gather qualitative data:

  • Permission may need to be gained to report individual 'stories' (case studies) on websites, social media or in funding bids.
  • Although it may be time-consuming to organise, collate and summarise qualitative data, this is usually time well-spent.

Resources

The Institute for Research and Innovation in Social Services (IRISS) developed a guide about collecting information from people supported by health and social services and their unpaid carers about what’s important to them in their lives and the ways in which they would like to be supported . It covers why qualitative data is important and what it can help achieve, the practical steps involved in the process and case study examples developed by people using qualitative data about outcomes for the first time.

The National Institute for Health and Care Research (NIHR) Applied Research Collaboration, Kent, Surrey and Sussex (ARC KSS) developed CAVEAT (The Community And Voluntary Organisation EvaluAtion Toolkit) which is a free resource designed to help Voluntary, Community and Social Enterprise (VCSE) organisations demonstrate their impact by helping organisations to describe, measure and report key information.

What are quantitative data and standardised questionnaires / scales / tools?

Quantitative data relate to numbers; they are countable and measurable. These data help us to understand matters concerning quantity (how much, many or often?) or people's characteristics.

Data can be objective (e.g. how many days someone attends the day centre, how many times has someone been absent) or subjective (e.g. how much did you enjoy x on a scale of 1-5?). People often refer to this type of evidence as 'hard' (rather than 'soft' which is a term often used to describe qualitative data).

Some examples of objective data include overall attendance levels; how often people attend; do people return after their first attendance; how many people do not return; numbers of carers staying in paid work, the personal or demographic details of those who attend collected in a standardised way (ethnicity, date of birth, health conditions etc).

Such data may be presented statistically (using descriptive statistics). For example, x% of people reported enjoying Y; x% of carers reported that Y; x% of volunteers said Z; x% of people reported improved quality of life since starting to attend their day centre. Presenting them alongside qualitative data gives a broader picture of the service and its outcomes/impact.

Quantitative data can also be gathered about individuals’ experiences (subjective data).Standardised’ questionnaires (i.e. validated scales, tools or measures) may be used to do this. These are questionnaires developed to measure a certain thing in a specific group of people. If a questionnaire (tool, scale or measure) has been ‘validated’, it has been tested to check how well it measures what it is intended to measure. This will have involved using it with a representative group of people and checking that the results did actually represent what they are supposed to measure (i.e. were valid) and if the results could be reproduced when used again (i.e. were reliable). Because standardised questionnaires have been carefully tested, they usually have questions with pre-determined wording and structured answer options. Some questionnaires are objective (e.g. health status) and some are subjective (e.g. perspective on feelings of wellbeing or how/if a service has helped someone). Some questionnaires are very short (e.g. five questions) and some are longer (e.g. 20 questions). Their length does not necessarily indicate how long they may take to use; some are quick, and some are not. Many standardised questionnaires are free to use. Some require registration (i.e. informing their issuer that they are being used, with whom and in what context). Some require payment. Many tools that are likely to be useful in day centres are available free of charge. Some of these appear further down this page, under ‘Some ‘tools’ (questionnaires) or approaches to gathering data that could be used by day centres.

Collecting data in a standardised, systematic way like this can be helpful for reporting - both about individuals and about the whole service user group – because it means that the resulting scores are comparable and standard across the group.

As with qualitative data, it is crucial to record and process the data collected so that it is of practical use. Standardised questionnaires are usually accompanied by guidance on their use, scoring and interpreting the results. You do not necessarily need to be 'an expert' to use these questionnaires.  

Traditionally, 'hard' data, like those resulting from standardised questionnaires, have not been collected by day centres about their clients. In the current environment, in which evidence-based commissioning is encouraged, it may become increasingly important to consider this option. Gathering such standardised evidential data may contribute towards day centres beginning to be considered more as health and wellbeing 'interventions' and be taken more seriously. This is particularly important as LAs are continuously under pressure to save money and justify spending, and social prescribing is becoming increasingly widespread in primary care.

In summary, standardised questionnaires can be useful for:

  • monitoring individual outcomes and experiences – at one point or at several points over time (e.g. when starting to attend a day centre, one month later, 3 months later, 6 months later and after a year). Scores may indicate individual improvement (i.e. the service has made a positive impact) or maintenance (i.e. that deterioration has been delayed)
  • feeding into care plan reviews - as evidence of outcomes/impact for older people attending day centre (and their carers, if also using tools with them)
  • enabling a person to see how far they've progressed since starting to attend their day centre. This may be useful for people who may be feeling tired and need encouragement to go to a day centre or are having 'a wobble' about going
  • carers may also appreciate knowing how the person they care for is progressing
  • reporting about whole groups (e.g. all the older people attending a day centre) – as averages, for example. Such data may be reported to funders (and other decision-makers). 'Hard' evidence, like the scores resulting from standardised questionnaires (validated tools), are often very appealing to people working in the NHS.

Challenges involved in gathering and making use of outcomes data, and things to bear in mind

Background knowledge, expertise and time are significant barriers to gathering and using outcomes data. Measuring outcomes can be complex and involves time and resources.

Individual outcomes can cover a very broad span. For example, safety is an outcome (feeling safe, lack of falls, being out of a risky situation), as is improved wellbeing; feeling significant as a person is an outcome; and enjoying companionship is an outcome.

It is important to remember that service or organisational outcomes, are the sum of the individual outcomes as a group, but these outcomes may also be linked with use of other services, such as additional home care, respite, GP appointments, hospital bed use or care home placements.

Many questions need consideration and decisions must be made before gathering outcomes data. For instance:

  • What is the purpose of gathering these data?
  • Is there in-house expertise on how to measure outcomes?
  • Which outcomes should be measured and how should this be undertaken?
  • How difficult/easy and time-consuming will it be to gather these data?
  • Is staff training necessary?
  • When should data be gathered – how often, after what period and during which part of a day centre day?
  • Are there staffing implications?
  • Should the exercise be undertaken in a private area?
  • Is the person willing to provide data about their own outcomes?
  • How will data be interpreted and presented?
  • Will data be reported on an individual basis or for groups of people? How should data be stored?

When gathering outcomes data (whether qualitative or quantitative), it is also important to bear in mind that:

  • brevity is key: the method should not be burdensome for any party involved, but some people are happy to fill in longer forms. People can be asked if they are happy to do so.
  • frailty can also be a barrier to collecting data: people may have too much going on in their lives and may not wish to add another exercise.
  • some conditions, for example dementia, are progressive and can mean that collecting reliable outcomes data is challenging. Carers may need to be involved in a respectful way. The best way to gain consent to complete surveys will need to be considered.
  • sensory loss may mean considering additional things. Tips on administering questionnaires with people with sensory loss appear on pages 30-31 of Campaign to End Loneliness’ Measuring your impact on loneliness in later life.
  • the process should be stress and anxiety free, especially for people living with dementia, and undertaken in a way that minimises discomfort about what’s going on for them and realisation of their loss of skills etc.
  • timing is important:
    i)  getting to know a person, building a rapport and letting them settle before asking lots of questions can lead to fuller responses
    ii)  some people do not mind being asked questions at first contact, but others may not wish to share personal feelings before having developed a relationship with staff / volunteers
    iii)  some people function better at certain times of day
    iv) outcomes for people with cognitive limitations (e.g. memory loss) may need to be measured ‘in the moment’ rather than a long while after an event.
  • carers’ responsibilities may mean some are reluctant to respond to requests
  • permission may need to be gained to report individual 'stories' (case studies) on websites, social media or in funding bids
  • when surveying staff/volunteers or older people attending a day centre across an organisation that provides various services, surveys may like to include some separate questions for each service to enable reporting of findings by service
  • if data are collected across several day centres, it is useful to be able to separate data for each centre
  • if any standardised questionnaires are being used, guidance on their use and interpretation is available (see below).
  • outcomes data gathered should be put to use somehow:  it is counterproductive to collect data from individuals (using any method) and then file it away in its original form without processing it somehow.
  • becoming more digitally literate is a target for some (e.g. local authorities), so being stretched by the use of digital outcomes monitoring data can be a good thing.
  • permission may need to be gained to report individual 'stories' (case studies) on websites, social media or in funding bids.
  • although it may be time-consuming to organise, collate and summarise qualitative data, this is usually time well-spent.
  • Some advice is provided in Campaign to End Loneliness’ Measuring your impact on loneliness in later life (pages 22-29) which covers how to introduce a questionnaire, encouraging staff and volunteers to use a questionnaire, at what points and how regularly a questionnaire might be used, how to choose people to use a questionnaire with (‘the sample’), getting consent, asking open questions and making sure enough information is gathered about the people it is used with.

Some ‘tools’ (questionnaires) or approaches to gathering data that could be used by day centres

This section gives an overview and links to further information about tools, questionnaires or approaches that may be useful for day centres to consider using when monitoring or gathering evidence about outcomes.

Further information includes hyperlinks, supporting explanations, and information about their use.

Practical suggestions are made for recording individual outcomes in Social Care Wales’ 12 principles for effective outcomes-focused recording

Social Care Wales suggests 12 principles for meaningful and effective outcomes monitoring and provides guidance and practice examples of personal outcomes-focused recording for each of the principles (e.g. case records, reflections).

Friend not foe: supporting meaningful outcomes focused recording in social care in Wales (2022) (see page 10).

Using ‘stories’ to convey change outcomes

Thinking about giving examples of how involvement with a day centre (attending, volunteering, being a carer) has improved people’s lives and writing ‘stories’ to communicate outcomes that have resulted in change, can build on the Most Significant Change technique.

The ‘Most Significant Change’ technique involves a person (e.g. day centre attender, volunteer) reporting, in their own words, the most significant change that has happened to them as a result of something (e.g. attending or volunteering at the day centre). The ‘story’ covers the most significant change they have experienced, what happened/where/what or who was involved/when, why it is significant for you/what difference it has made to the person’s life, and why this difference it has made for the person is so important.

Detailed guidance and examples: The ‘Most Significant Change’ (MSC) Technique. A Guide to Its Use (2005) by Rick Davies and Jess Dart

Examples appear in Magic Moments in adult service provision (numbers 1,6, 7, 20, 33 and 40 are about day services) developed by Swansea University and Swansea Council in 2016.

More detailed examples of stories that have been written are available further down this page under ‘Some local Examples’ (see Evidencing impact using individual stories written using the Most Significant Change approach: Knit one, purl one, see my confidence grow My wife – big difference)

The Adult Social Care Outcomes Toolkit  is a set of questionnaires specifically designed to measure aspects of quality of life that specific social care services may impact upon. Instead of focusing only on physical or psychological health, they cover a broad range of aspects of quality of life that are relevant and important to people who use social care and their carers (e.g. social relationships, control over daily life, feeling significant, feeling safe, time to yourself, feeling supported). These are called ‘domains’. As well as helping service providers understand client experiences, these questionnaires support care planning and quality monitoring. They are useful for monitoring individual progress and for demonstrating the impact of a service across a group of people. Scores/results can be shown visually using a spider chart. ASCOT has been recommended for use in economic evaluations of services (i.e. cost effectiveness) if unit cost data are available. 

There are several versions which include questionnaires to be completed by older people attending a day centre or carers themselves and questionnaires to be read out to older people/carers (i.e. ‘administered in an interview’):

  • ASCOT SCT4 (8 questions for service users) and ASCOT Carer SCT4 (7 questions) can be completed by older people attending a day centre or carers themselves. It can be used once or repeated to monitor change. These versions are currently used in local authority service user surveys. (‘ASCOT SCT4 four-level self-completion tool’, ‘Carer SCT4 four-level self-completion tool or family/friend (unpaid) carers’)
  • ASCOT INT4 (23 questions for service users), ASCOT Carer INT4 (21 questions) and ASCOT-ER (OP) are designed to be read out in an ‘interview’. ASCOT-ER (OP) is a revised version of ASCOT INT4 which is aimed specifically at older people with or without memory problems. These versions discover how much the service has impacted on someone. As well as overall benefit, it is helpful for identifying which are the areas (domains) in which people feel they have benefited the most and the least, so can also inform service improvement initiatives. The questionnaires include explanatory notes for the person reading it out and prompts they can use. These versions measure the current quality of life and ‘expected’ quality of life (i.e. how the person perceives their quality of life would be in the absence of the service). The difference between these scores is the “gain”, or the impact of the service on that person’s quality of life (expressed numerically). Because of this structure, service users/carers must be able to imagine hypothetical situations. (‘ASCOT INT4 four-level interview tool’, ‘Carer INT4 four-level interview tool or family/friend (unpaid) carers, ASCOT Easy Read (Older People)’)

ASCOT questionnaires are free of charge by not-for-profit organisations or if to be used for the benefit of the public (e.g. a LA using these for quality monitoring or service improvement). For-profit organisations are required to pay a fee. All will need to complete a registration form (explaining planned use) and agree to Terms and Conditions of use before a license is granted and the questionnaire and guidance for its used provided. Guidance documents include how to calculate scores manually. Data entry tools (Excel spreadsheets set up to do automatic calculations of scores – one for each questionnaire) cost £50 and can be downloaded from the website (cost at May 2024).

A 13 minute video explains how providers can use the ASCOT tools as part of a ‘circle of care’ interview.

Katharine Orellana shared her experiences of using ASCOT (INT4 and INT4 Carers) in day centres for older people in a talk at an ASCOT 10th Anniversary webinar series (October 2022). See https://www.youtube.com/watch?v=SUDgp1q16Tw (from minute 21).

Building on the ASCOT work, a care work-related quality-of-life questionnaire is under development (at the time of writing). Day centre staff wellbeing is likely to impact on the quality of service provided, and on outcomes related to patients and service users. Therefore, understanding the aspects of care workers’ quality of life most affected by their work is important. 

ICECAP-O (ICEpop CAPability measure for Older people) measures older people’s capability-wellbeing in 5 areas

The ICECAP-O measures older people’s wellbeing in 5 dimensions: Attachment (love and friendship), Security (thinking about the future without concern), Role (doing things that make you feel valued), Enjoyment (enjoyment and pleasure), Control (being independent). The questionnaire is short and easy to complete. Registration is straightforward. ICECAP-O has been recommended for use in economic evaluations.

Using ICECAP-O is free for non-commercial users, but users must sign a licensing agreement with the University of Bristol in which they agree not to change the wording of the measure. Scoring coding is provided on the website using STATA and SPSS statistical software. An Excel scoring spreadsheet is available upon request after registration.

The registration form, questionnaire and scoring system codes are downloadable from the ICECAP-O web page.

The Outcomes Star is a charged-for standardised tool in the shape of a star, each leg of which represents different levels of progress towards an outcome area. It is a keywork tool that increases awareness and informs action planning/interventions. Many versions are available including the Older Person’s Star (covering Staying as well as you can, Keeping in touch, Feeling positive, Being treated with dignity, Looking after yourself, Feeling safe, and Managing money and personal administration) and the Carers Star (covering Health, The caring role, Managing at home, Time for yourself, How you feel, Finances, and Work). Paper or online versions are available.

An annual licence and training for each staff member using an Outcomes Star is required. Supportive resources are available (action plans, guidance, flashcards etc). Costs start at £250 per annual licence for all published versions, the online version and supportive resources; additional licences cost £40 per person. Training starts at £960 (costs at May 2024).

Health Improvement Network’s (HIN) straightforward tools to measure outcomes for people with dementia in a range of support settings (including community groups)

The Health Improvement Network (HIN) developed a set of easy tools to measure outcomes for people with dementia in a range of support settings (including community groups) which may be useful in demonstrating the value of day services to people with dementia (see page 15).

Measuring staff time needed to support older people attending a day centre

This is another useful approach to evidencing individual changing needs and for organisational planning purposes. A day centre manager developed a 'time tool’ to do this.

The Time Tool is a spreadsheet used by a day centre to monitor the frequency of certain occurrences and staff time needed to intervene. This has enabled better understandings of changing needs and was also valuable for self-audit as staff could learn from each other about how best or most efficiently to support clients. It is straightforward and calculations are made automatically behind the scenes. Links to download the tool and detailed information about the areas of support it monitors, how to use it and its benefits appears further down this page under 'Some local examples'.

Two tools to measure how person-centred a day centre is: PERCCI and Making it Real

Social care is expected to be person-centred. Different understandings of what 'person-centredness' means make monitoring and reporting it difficult. A review of definitions concluded that understanding the person, engagement in decision-making and promoting the care relationship are central to service person-centredness. To ‘measure’ how person-centred a day centre service is from the perspective of the older people who attend it, two tools may be useful:

PERCCI – the Person-centred Community Care Inventory

PERCCI is a questionnaire with 12 questions about how people feel they are treated, to what extent they are involved in decisions about their care and how well the service addresses individual needs and preferences.

PERCCI (pronounced ‘percy’) a short (12 questions), straightforward standardised questionnaire that measures quality of care experiences (e.g. if people feel they are treated with kindness, respect and compassion, are involved in decisions about their care, if care is personalised and responsive to their needs and preferences). It was co-designed with people with lived experience of care services and is based on evidence.

It can also help to demonstrate whether the service/provider is well-led (how it promotes a person-centred culture, if it has a process for continuous quality improvement, if it gathers and acts upon people’s experiences of care or if it uses evidence-based processes within the service).

The questionnaire can be downloaded from the website; people wishing to download the questionnaire are asked for their details, how they heard about it and if they are willing to be contacted about their experience of using PERCCI.

Scoring is explained on the PERCCI website. An Excel spreadsheet for entering and storing scores is available on request. Each response is scored 0, 1, 2 or 3 (0 for 'rarely or never' and 3 for 'always') and scores for all 12 questions are added together. Individual scores range from 0-36. Scores can be compared between people (e.g. to identify those with particularly low scores) or for the same people over time (e.g. to see if care experiences are improving).

An infographic summarises PERCCI.

Read more about how it was developed in a presentation given to the Day Centre Research Forum in June 2020.

Making it Real

Making it Real is a framework built around 6 themes that reflect important parts of personalised care and support. It is for use in and beyond social care services (i.e. in housing, residential care, hospitals, public services) therefore, not all themes may be relevant to day centres. It was co-produced by Think Local Act Personal with input from a range of organisations and individuals.

Each theme has a set of “I” statements about how support is experienced by an individual (e.g. I have considerate support delivered by competent people; I am supported to make decisions by people who see things from my point of view, with concern for what matters to me, my wellbeing and health) and related “We” statements that cover the provider’s perspective of what needs to be in place for the “I” statements to happen (e.g. We don’t make assumptions about what people can or cannot do and don’t limit or restrict people’s options).

Three questionnaires to assess whether attending a day centre has reduced feelings of loneliness

Loneliness is one aspect of wellbeing that day services may help to reduce. Measuring loneliness is important because feeling lonely can be accompanied by worsening physical and mental health. Recognising that persistent loneliness can harm health, the Government published a strategy for tackling loneliness in 2018 and has continued its work in this area. Loneliness can be measured (at different points in time) to discover whether attending a day centre has reduced feelings of loneliness. Several questionnaires do this. They are short and useful for different purposes:

Campaign To End Loneliness Measurement Tool

The Campaign To End Loneliness Measurement Tool aims to measure the change that happens as a result of an intervention (e.g. day centre attendance), with a focus on how people’s scores change over time. This questionnaire is based on the idea that loneliness is felt when there is a mismatch between the social contact a person has and the social contact they want. Because it was co-designed with older people and service providers, it uses language that is non-intrusive and unlikely to cause any embarrassment or distress and, therefore, may be the most appropriate questionnaire for use in day centres. 

It is a three-question standardised questionnaire that is useful for service providers wanting a short questionnaire that uses positive and sensitively worded language (i.e. written in language which is non-intrusive and unlikely to cause any embarrassment or distress) and is easy to use. It was co-designed with older people and service providers. It is based on the idea that loneliness is felt when there is a mismatch between the social contact a person has and the social contact they want. Guidance covers the questionnaire itself and scoring system (pages 11-13), how to introduce it, encouraging staff and volunteers to use it, at what points and how regularly a questionnaire to measure loneliness might be used, how to choose people to use it with (sample), getting consent, asking the questions themselves and making sure enough information is gathered about the people it is used with (see pages 22-29).

National Loneliness Measure (UCLA Loneliness Scale)

This loneliness measure, recommended in the Government's Loneliness Strategy, is a 4 question scale (including 3 questions from UCLA scale and an additional question). It is helpful for service providers wanting a short tool that uses a simple scoring system. Although it does not mention 'loneliness', questions are worded negatively. The Government strategy recommends adding a fourth question that uses the word ‘loneliness’. This approach is widely used and, therefore, enables the comparison of data between different types of services or service user groups.

Details and scoring information can be found in Campaign to End Loneliness’ Measuring Your Impact on Loneliness in Later Life (see page 17-19), in What Works for Wellbeing’s Brief Guide to Measuring Loneliness (see page 10-11), on the ONS web page, and on What Works for Wellbeing’s pages on National Loneliness Measures on its ‘Measure Bank’

De Jong Giervald Loneliness Scale

A six-question standardised questionnaire developed for researchers. It distinguishes social loneliness (when the number of friendships or relationships someone has is smaller than desired) and emotional loneliness (when someone is missing intimacy from their relationships, friendships or acquaintances) and gives an overall measure of loneliness. Although it does not mention ‘loneliness’, questions are worded negatively. Details and scoring information can be found in Campaign to End Loneliness’ Measuring Your Impact on Loneliness in Later Life (see page 14-16), and on What Works for Wellbeing’s pages on National Loneliness Measures on its ‘Measure Bank’

Further information about these three tools can be found in Campaign to End Loneliness’ guide Measuring your impact on loneliness in later life (2015) and What Works Centre for Wellbeing’s A brief guide to measuring loneliness.

Timed Up and Go Test (TUG) for testing testing mobility, balance and falls risk

The Timed Up and Go Test (TUG) is a good indicator of balance and, therefore, falls risk, and may be helpful in determining levels of staff support needed in day centres. Data collected using it may be of interest to people working in health (e.g. occupational therapists, physiotherapists, GPs or commissioners).

The Timed Up and Go Test is a standardised test that assesses mobility, balance, walking ability and falls risk. If repeated at intervals, this test can monitor change; it may be useful, for example, before and after specific programmes of exercise, for example. It may also indicate levels of staff support that a person may need at their day centre (e.g. going to the toilet or moving between activities). The test is simple, quick and requires no special equipment or training; a person is asked to rise from a standard chair, walk to a marker 3 metres (10 feet) away, turn, walk back, and sit down again. They do this while wearing their usual footwear and using their usual mobility equipment, if any (e.g. walking stick, walking frame).

Different approaches are taken to scoring (i.e. being at risk of falling starts at 12 seconds for some, 13.5 seconds for some, or 20 seconds for others). The UK Chartered Institute for Physiotherapy defines a person being at risk of falls if the test takes 15 seconds or more.

The Chartered Institute for Physiotherapy has produced a video that demonstrates the test.

Short Warwick Edinburgh Mental Wellbeing Scale (SWEMWBS) and Warwick Edinburgh Mental Wellbeing Scale (WEMWBS)

Changes in individual wellbeing could indicate service impact and may be helpful for monitoring purposes. Data may also be of interest to people working in health (e.g. GPs or commissioners). The Warwick-Edinburgh Mental Wellbeing Scale (WEMWBS) is a standardised self-completion questionnaire developed to measure mental wellbeing in the general population and evaluate projects, programmes and policies which aim to improve mental wellbeing. Both ask positively worded questions about feelings and thoughts over the previous 2 weeks. Each has a choice of 5 response categories. The WEMBWS has 14 questions and the SWEMWBS has 7 questions.

Both scales are recommended by the What Works Centre for Wellbeing, with further details in its ‘Measures Bank’: WEMWBS and SWEMWBS Scoring and interpretation guidance is provided. The full (and rather complex) user guide covers both questionnaires. Frequently Asked Questions provide additional useful information about its use. The only drawback to this questionnaire is that it has not been validated for use with people aged 75 and older.

Both scales are free to use (non-commercial use) but users must register for a license.

SF-12 or SF-36 health outcomes questionnaires measure quality of life and functional health and wellbeing from a person’s own perspective

The SF-12 and SF-36 are standardised questionnaires that measure quality of life and functional health and wellbeing from a person’s own perspective. They cover physical functioning, bodily pain, limitations due to physical health problems and personal/emotional problems, emotional wellbeing, social functioning, energy/fatigue, general health perceptions, and a perceived change in health. They have 12 and 36 questions respectively.

These questionnaires may help day centres to present information about the health and wellbeing characteristics of older people using their service (as a group of service users) rather than to evidence outcomes and impact (i.e. background information to outcomes/impact data gathered in another way). Data collected using it may be of interest to people working in health (e.g. occupational therapists, physiotherapists, GPs or commissioners).

Use of the original (outdated) version of SF-36 is subject to a few Terms & Conditions, including crediting its development at RAND as part of the Medical Outcomes Study. The questionnaire (blank printable form; can also be completed online and printed) and scoring instructions (see Table 1-steps 1&2) are downloadable from RAND’s website.

Use of the most up-to-date versions (SF-12v2 or SF36v2) require a license (3-page form more suited to academic research than local monitoring of service users’ characteristics). See further information about these and the licences required

Assessment of Quality of Life (AQoL)

The Assessment of Quality of Life (AQoL) measures health-related quality of life in four domains: Independent Living, Relationships, Mental Health and Senses. It may be helpful for profiling older people attending a day centre, as a group, and for evidencing impact (even in a single domain, such as Relationships).

Several versions of the AQoL standardised questionnaire measure health-related quality of life. The shortest is AQoL-4D which has 12 questions covering four domains: Independent Living (self-care, household tasks, mobility), Relationships (friendships, isolation, family role), Mental Health (sleeping, worrying, pain) and Senses (seeing, hearing, communication).

The AQoL may be useful for presenting health information about older people attending a day centre (as a group of service users as background information to outcomes/impact data gathered in another way) as well as a means of evidencing outcomes and impact. A single score is calculated, or the score for each domain may be reported separately (e.g. Relationships) and the measure repeated at intervals to evidence any change.

Scoring assigned to each question differs according to whether the questionnaire is being used as a ‘psychometric’ measure (i.e. to get scores on health-related quality of life that can be used as a single score or separately by domain) or as a ‘utility’ measure (i.e. to be used in economic evaluations). Psychometric scores are more straightforward (e.g. 1-5 for each of 5 answer options). Utility scores are ‘preference weighted’ (i.e. reflect preferences more accurately than unweighted scores).

The analysis spreadsheet to calculate Psychometric (unweighted) scores is downloadable (see Step 1). A Zip Folder of SPSS and STATA documents to calculate Utility scores is downloadable from the web. Use is free, but users must register. Registration means that technical support can be provided.

Making records of healthcare service use

Making records of healthcare service use, suggested by US day service experts, may also form part of an approach to evidencing day centre attendance outcomes.

In their call for the development of a set of uniform set of outcomes measures to be used in US day services, Anderson and colleagues reported that simple records of healthcare use have been recommended as useful for evidencing impact. For example, records may include numbers of hospital admissions, numbers of Accident and Emergency visits, numbers of prescribed medications, or numbers of GP consultations or NHS professional home visits.

Monitoring staff job satisfaction

Matters concerning staffing may also form part of a day centre’s impact. Staff recruitment and retention are known to be challenging in social care. Day centres may wish to monitor their staff’s job satisfaction

What Works Centre for Wellbeing has developed a Workplace wellbeing question bank. One measure is of job satisfaction in terms of worker wellbeing. Seven response options range from ‘Completely dissatisfied’ to ‘Completely satisfied’. The question options can be found on the Measures Bank.

As a single question, it is easy to interpret. A longer, 10-question version, the Worker wellbeing snapshot survey is also available

Social Return on Investment (SROI)

SROI aims to measure the wider (social and environmental as well as economic) value of a service (or organisation). It uses financial calculations to indicate the ratio of input cost to wider value created (e.g. for every pound spent, the service achieved a return of £x in social value). The scope of this resource for day centres and their stakeholders does not include SROI, but readers may wish to read an example of an SROI study undertaken by the Health Improvement Network on Peer Support for People with Dementia.

Some local examples

Study participants have shared some examples developed locally and given permission for these to be downloaded and edited to suit local circumstances.

Download a PDF of this section which includes further information on understanding outcomes and measuring impact

Understanding outcomes and measuring impact

This document explains what is meant by ‘outcomes’, or ‘impact’, and discusses the collection of these data in English day centres.
Download Understanding outcomes and measuring impact
Download 4-Understanding%20outcomes%20and%20impact%20May2024.pdf