Understanding outcomes and measuring the impact of day centres

Outcomes data can track the progress of individuals who attend the day centre. People may wish to see how they’ve progressed since starting to attend their day centre (especially if feeling tired or having ‘a wobble’ about going). Carers may appreciate knowing how their relative/friend is doing, too.

Outcomes data can show (evidence) the usefulness (or impact) of the day centre to its users for funders, commissioners and others who make service-related decisions. Outcomes data can be helpful for funding applications/tender bids and service monitoring. They can inform service reviews. They can also contribute to economic evaluations or social value/social return on investment exercises. Business analysts who can help construct strong justification for commissioning decisions are less common in LAs now, and, therefore, easy-to-use data/evidence is appreciated.

Outcomes data inform new staff and staff moving roles within LA and NHS services where there is high turnover, reorganisations and resulting changing roles are common. Background information will be necessary for new or re-assigned staff and is particularly important within an environment that encourages joint commissioning.

Outcomes data can also help map service impact across to current policy goals, thus furthering the understanding of day centres’ relevance to policy. Policy goals include ensuring good quality of life for people with care and support needs, delaying or reducing the need for care and support, ensuring people have a positive experience of care and support, safeguarding adults whose circumstances make them vulnerable and protecting them from avoidable harm. Some contracts may specify policy-related outcomes for service providers to aim for.

Outcomes data can evidence the usefulness (impact) of the day centre to social care and NHS professionals and others who may consider making referrals, or signposting, to day centres (e.g. social workers, occupational therapists, social prescribing link workers, information and advice services). Any referrals made should be appropriate, and outcomes data may help with this.

Demonstrating how day centres make a difference to those who attend them supports marketing of the service to potential service users and to the general public. This is especially important because day centres will increasingly need to make their services attractive (marketable) to people who will pay for services themselves or their families. These ‘self-funders’ include people who do not meet a local authority’s criteria for services but who have other needs which could be met by a day centre (e.g. functional mental health needs such as depression). These people and their carers will need to understand exactly what they would get for their money. This will be even more important for services not subsidised by contracts and which, therefore, may make higher charges to users. The Resources Hub includes a section on marketing communications.

Outcomes data can support day centre providers to self-audit and undertake service improvement.

Monitoring staff and volunteers’ feelings/role satisfaction/views may improve staff/volunteer retention as it helps identify where changes need to be made. Publicising satisfaction data may also help market a day centre as a good place to work or volunteer.

Recruitment and retention in social care are often problematic. Volunteering at a day centre is not only an opportunity to help others and can also be beneficial to volunteers’ health and wellbeing, especially if they are older themselves.

Localities may wish to address particular problems that have been identified, such as high levels of loneliness. Outcomes data can inform themed briefings for staff or local strategies – such as how to reduce loneliness locally.

Outcomes data may also help demonstrate value for money (economic evaluation).

A participant in this research also explained how central registration of outcomes data for individuals on statutory databases used by other professionals, for example, can support reviews of an individual’s overall care and support package. At the same time, data support other staff within a directorate to better understand the service and raise the profile of the day centre.

• What is the purpose of gathering these data?
• Is there in-house expertise on how to measure outcomes?
• Which outcomes should be measured and how should this be undertaken?
• How difficult/easy and time-consuming will it be to gather these data?
• Is staff training necessary?
• When should data be gathered – how often, after what period and during which part of a day centre day?
• Are there staffing implications?
• Should the exercise be undertaken in a private area?
• Is the person willing to provide data about their own outcomes?
• How will data be interpreted and presented?
• Will data be reported on an individual basis or for groups of people? How should data be stored?

• brevity is key: the method should not be burdensome for any party involved, but some people are happy to fill in longer forms. People can be asked if they are happy to do so.
• frailty can also be a barrier to collecting data: people may have too much going on in their lives and may not wish to add another exercise.
• some conditions, for example dementia, are progressive and can mean that collecting reliable outcomes data is challenging. Carers may need to be involved in a respectful way. The best way to gain consent to complete surveys will need to be considered.
• sensory loss may mean considering additional things. Tips on administering questionnaires with people with sensory loss appear on pages 30-31 of Campaign to End Loneliness’ Measuring your impact on loneliness in later life.
• the process should be stress and anxiety free, especially for people living with dementia, and undertaken in a way that minimises discomfort about what’s going on for them and realisation of their loss of skills etc.
• timing is important:
  i)  getting to know a person, building a rapport and letting them settle before asking lots of questions can lead to fuller responses
  ii)  some people do not mind being asked questions at first contact, but others may not wish to share personal feelings before having developed a relationship with staff / volunteers
  iii)  some people function better at certain times of day
  iv) outcomes for people with cognitive limitations (e.g. memory loss) may need to be measured ‘in the moment’ rather than a long while after an event.
• carers’ responsibilities may mean some are reluctant to respond to requests
• permission may need to be gained to report individual 'stories' (case studies) on websites, social media or in funding bids
• when surveying staff/volunteers or older people attending a day centre across an organisation that provides various services, surveys may like to include some separate questions for each service to enable reporting of findings by service
• if data are collected across several day centres, it is useful to be able to separate data for each centre
• if any standardised questionnaires are being used, guidance on their use and interpretation is available (see below).
• outcomes data gathered should be put to use somehow:  it is counterproductive to collect data from individuals (using any method) and then file it away in its original form without processing it somehow.
• becoming more digitally literate is a target for some (e.g. local authorities), so being stretched by the use of digital outcomes monitoring data can be a good thing.
• permission may need to be gained to report individual 'stories' (case studies) on websites, social media or in funding bids.
• although it may be time-consuming to organise, collate and summarise qualitative data, this is usually time well-spent.
• Some advice is provided in Campaign to End Loneliness’ Measuring your impact on loneliness in later life (pages 22-29) which covers how to introduce a questionnaire, encouraging staff and volunteers to use a questionnaire, at what points and how regularly a questionnaire might be used, how to choose people to use a questionnaire with (‘the sample’), getting consent, asking open questions and making sure enough information is gathered about the people it is used with.

Social Care Wales suggests 12 principles for meaningful and effective outcomes monitoring and provides guidance and practice examples of personal outcomes-focused recording for each of the principles (e.g. case records, reflections).

Friend not foe: supporting meaningful outcomes focused recording in social care in Wales (2022) (see page 10).

Thinking about giving examples of how involvement with a day centre (attending, volunteering, being a carer) has improved people’s lives and writing ‘stories’ to communicate outcomes that have resulted in change, can build on the Most Significant Change technique.

The ‘Most Significant Change’ technique involves a person (e.g. day centre attender, volunteer) reporting, in their own words, the most significant change that has happened to them as a result of something (e.g. attending or volunteering at the day centre). The ‘story’ covers the most significant change they have experienced, what happened/where/what or who was involved/when, why it is significant for you/what difference it has made to the person’s life, and why this difference it has made for the person is so important.

Detailed guidance and examples: The ‘Most Significant Change’ (MSC) Technique. A Guide to Its Use (2005) by Rick Davies and Jess Dart

Examples appear in Magic Moments in adult service provision (numbers 1,6, 7, 20, 33 and 40 are about day services) developed by Swansea University and Swansea Council in 2016.

More detailed examples of stories that have been written are available further down this page under ‘Some local Examples’ (see Evidencing impact using individual stories written using the Most Significant Change approach: Knit one, purl one, see my confidence grow My wife – big difference)

The Adult Social Care Outcomes Toolkit  is a set of questionnaires specifically designed to measure aspects of quality of life that specific social care services may impact upon. Instead of focusing only on physical or psychological health, they cover a broad range of aspects of quality of life that are relevant and important to people who use social care and their carers (e.g. social relationships, control over daily life, feeling significant, feeling safe, time to yourself, feeling supported). These are called ‘domains’. As well as helping service providers understand client experiences, these questionnaires support care planning and quality monitoring. They are useful for monitoring individual progress and for demonstrating the impact of a service across a group of people. Scores/results can be shown visually using a spider chart. ASCOT has been recommended for use in economic evaluations of services (i.e. cost effectiveness) if unit cost data are available. 

There are several versions which include questionnaires to be completed by older people attending a day centre or carers themselves and questionnaires to be read out to older people/carers (i.e. ‘administered in an interview’):

• ASCOT SCT4 (8 questions for service users) and ASCOT Carer SCT4 (7 questions) can be completed by older people attending a day centre or carers themselves. It can be used once or repeated to monitor change. These versions are currently used in local authority service user surveys. (‘ASCOT SCT4 four-level self-completion tool’, ‘Carer SCT4 four-level self-completion tool or family/friend (unpaid) carers’)
• ASCOT INT4 (23 questions for service users), ASCOT Carer INT4 (21 questions) and ASCOT-ER (OP) are designed to be read out in an ‘interview’. ASCOT-ER (OP) is a revised version of ASCOT INT4 which is aimed specifically at older people with or without memory problems. These versions discover how much the service has impacted on someone. As well as overall benefit, it is helpful for identifying which are the areas (domains) in which people feel they have benefited the most and the least, so can also inform service improvement initiatives. The questionnaires include explanatory notes for the person reading it out and prompts they can use. These versions measure the current quality of life and ‘expected’ quality of life (i.e. how the person perceives their quality of life would be in the absence of the service). The difference between these scores is the “gain”, or the impact of the service on that person’s quality of life (expressed numerically). Because of this structure, service users/carers must be able to imagine hypothetical situations. (‘ASCOT INT4 four-level interview tool’, ‘Carer INT4 four-level interview tool or family/friend (unpaid) carers, ASCOT Easy Read (Older People)’)

ASCOT questionnaires are free of charge by not-for-profit organisations or if to be used for the benefit of the public (e.g. a LA using these for quality monitoring or service improvement). For-profit organisations are required to pay a fee. All will need to complete a registration form (explaining planned use) and agree to Terms and Conditions of use before a license is granted and the questionnaire and guidance for its used provided. Guidance documents include how to calculate scores manually. Data entry tools (Excel spreadsheets set up to do automatic calculations of scores – one for each questionnaire) cost £50 and can be downloaded from the website (cost at May 2024).

A 13 minute video explains how providers can use the ASCOT tools as part of a ‘circle of care’ interview.

Katharine Orellana shared her experiences of using ASCOT (INT4 and INT4 Carers) in day centres for older people in a talk at an ASCOT 10th Anniversary webinar series (October 2022). See https://www.youtube.com/watch?v=SUDgp1q16Tw (from minute 21).

Building on the ASCOT work, a care work-related quality-of-life questionnaire is under development (at the time of writing). Day centre staff wellbeing is likely to impact on the quality of service provided, and on outcomes related to patients and service users. Therefore, understanding the aspects of care workers’ quality of life most affected by their work is important. 

The ICECAP-O measures older people’s wellbeing in 5 dimensions: Attachment (love and friendship), Security (thinking about the future without concern), Role (doing things that make you feel valued), Enjoyment (enjoyment and pleasure), Control (being independent). The questionnaire is short and easy to complete. Registration is straightforward. ICECAP-O has been recommended for use in economic evaluations.

Using ICECAP-O is free for non-commercial users, but users must sign a licensing agreement with the University of Bristol in which they agree not to change the wording of the measure. Scoring coding is provided on the website using STATA and SPSS statistical software. An Excel scoring spreadsheet is available upon request after registration.

The registration form, questionnaire and scoring system codes are downloadable from the ICECAP-O web page.

The Outcomes Star is a charged-for standardised tool in the shape of a star, each leg of which represents different levels of progress towards an outcome area. It is a keywork tool that increases awareness and informs action planning/interventions. Many versions are available including the Older Person’s Star (covering Staying as well as you can, Keeping in touch, Feeling positive, Being treated with dignity, Looking after yourself, Feeling safe, and Managing money and personal administration) and the Carers Star (covering Health, The caring role, Managing at home, Time for yourself, How you feel, Finances, and Work). Paper or online versions are available.

An annual licence and training for each staff member using an Outcomes Star is required. Supportive resources are available (action plans, guidance, flashcards etc). Costs start at £250 per annual licence for all published versions, the online version and supportive resources; additional licences cost £40 per person. Training starts at £960 (costs at May 2024).

The Health Improvement Network (HIN) developed a set of easy tools to measure outcomes for people with dementia in a range of support settings (including community groups) which may be useful in demonstrating the value of day services to people with dementia (see page 15).

This is another useful approach to evidencing individual changing needs and for organisational planning purposes. A day centre manager developed a 'time tool’ to do this.

The Time Tool is a spreadsheet used by a day centre to monitor the frequency of certain occurrences and staff time needed to intervene. This has enabled better understandings of changing needs and was also valuable for self-audit as staff could learn from each other about how best or most efficiently to support clients. It is straightforward and calculations are made automatically behind the scenes. Links to download the tool and detailed information about the areas of support it monitors, how to use it and its benefits appears further down this page under 'Some local examples'.

Social care is expected to be person-centred. Different understandings of what 'person-centredness' means make monitoring and reporting it difficult. A review of definitions concluded that understanding the person, engagement in decision-making and promoting the care relationship are central to service person-centredness. To ‘measure’ how person-centred a day centre service is from the perspective of the older people who attend it, two tools may be useful:

PERCCI – the Person-centred Community Care Inventory

PERCCI is a questionnaire with 12 questions about how people feel they are treated, to what extent they are involved in decisions about their care and how well the service addresses individual needs and preferences.

PERCCI (pronounced ‘percy’) a short (12 questions), straightforward standardised questionnaire that measures quality of care experiences (e.g. if people feel they are treated with kindness, respect and compassion, are involved in decisions about their care, if care is personalised and responsive to their needs and preferences). It was co-designed with people with lived experience of care services and is based on evidence.

It can also help to demonstrate whether the service/provider is well-led (how it promotes a person-centred culture, if it has a process for continuous quality improvement, if it gathers and acts upon people’s experiences of care or if it uses evidence-based processes within the service).

The questionnaire can be downloaded from the website; people wishing to download the questionnaire are asked for their details, how they heard about it and if they are willing to be contacted about their experience of using PERCCI.

Scoring is explained on the PERCCI website. An Excel spreadsheet for entering and storing scores is available on request. Each response is scored 0, 1, 2 or 3 (0 for 'rarely or never' and 3 for 'always') and scores for all 12 questions are added together. Individual scores range from 0-36. Scores can be compared between people (e.g. to identify those with particularly low scores) or for the same people over time (e.g. to see if care experiences are improving).

An infographic summarises PERCCI.

Read more about how it was developed in a presentation given to the Day Centre Research Forum in June 2020.

Making it Real

Making it Real is a framework built around 6 themes that reflect important parts of personalised care and support. It is for use in and beyond social care services (i.e. in housing, residential care, hospitals, public services) therefore, not all themes may be relevant to day centres. It was co-produced by Think Local Act Personal with input from a range of organisations and individuals.

Each theme has a set of “I” statements about how support is experienced by an individual (e.g. I have considerate support delivered by competent people; I am supported to make decisions by people who see things from my point of view, with concern for what matters to me, my wellbeing and health) and related “We” statements that cover the provider’s perspective of what needs to be in place for the “I” statements to happen (e.g. We don’t make assumptions about what people can or cannot do and don’t limit or restrict people’s options).

Loneliness is one aspect of wellbeing that day services may help to reduce. Measuring loneliness is important because feeling lonely can be accompanied by worsening physical and mental health. Recognising that persistent loneliness can harm health, the Government published a strategy for tackling loneliness in 2018 and has continued its work in this area. Loneliness can be measured (at different points in time) to discover whether attending a day centre has reduced feelings of loneliness. Several questionnaires do this. They are short and useful for different purposes:

Campaign To End Loneliness Measurement Tool

The Campaign To End Loneliness Measurement Tool aims to measure the change that happens as a result of an intervention (e.g. day centre attendance), with a focus on how people’s scores change over time. This questionnaire is based on the idea that loneliness is felt when there is a mismatch between the social contact a person has and the social contact they want. Because it was co-designed with older people and service providers, it uses language that is non-intrusive and unlikely to cause any embarrassment or distress and, therefore, may be the most appropriate questionnaire for use in day centres. 

It is a three-question standardised questionnaire that is useful for service providers wanting a short questionnaire that uses positive and sensitively worded language (i.e. written in language which is non-intrusive and unlikely to cause any embarrassment or distress) and is easy to use. It was co-designed with older people and service providers. It is based on the idea that loneliness is felt when there is a mismatch between the social contact a person has and the social contact they want. Guidance covers the questionnaire itself and scoring system (pages 11-13), how to introduce it, encouraging staff and volunteers to use it, at what points and how regularly a questionnaire to measure loneliness might be used, how to choose people to use it with (sample), getting consent, asking the questions themselves and making sure enough information is gathered about the people it is used with (see pages 22-29).

National Loneliness Measure (UCLA Loneliness Scale)

This loneliness measure, recommended in the Government's Loneliness Strategy, is a 4 question scale (including 3 questions from UCLA scale and an additional question). It is helpful for service providers wanting a short tool that uses a simple scoring system. Although it does not mention 'loneliness', questions are worded negatively. The Government strategy recommends adding a fourth question that uses the word ‘loneliness’. This approach is widely used and, therefore, enables the comparison of data between different types of services or service user groups.

Details and scoring information can be found in Campaign to End Loneliness’ Measuring Your Impact on Loneliness in Later Life (see page 17-19), in What Works for Wellbeing’s Brief Guide to Measuring Loneliness (see page 10-11), on the ONS web page, and on What Works for Wellbeing’s pages on National Loneliness Measures on its ‘Measure Bank’

De Jong Giervald Loneliness Scale

A six-question standardised questionnaire developed for researchers. It distinguishes social loneliness (when the number of friendships or relationships someone has is smaller than desired) and emotional loneliness (when someone is missing intimacy from their relationships, friendships or acquaintances) and gives an overall measure of loneliness. Although it does not mention ‘loneliness’, questions are worded negatively. Details and scoring information can be found in Campaign to End Loneliness’ Measuring Your Impact on Loneliness in Later Life (see page 14-16), and on What Works for Wellbeing’s pages on National Loneliness Measures on its ‘Measure Bank’

Further information about these three tools can be found in Campaign to End Loneliness’ guide Measuring your impact on loneliness in later life (2015) and What Works Centre for Wellbeing’s A brief guide to measuring loneliness.

The Timed Up and Go Test (TUG) is a good indicator of balance and, therefore, falls risk, and may be helpful in determining levels of staff support needed in day centres. Data collected using it may be of interest to people working in health (e.g. occupational therapists, physiotherapists, GPs or commissioners).

The Timed Up and Go Test is a standardised test that assesses mobility, balance, walking ability and falls risk. If repeated at intervals, this test can monitor change; it may be useful, for example, before and after specific programmes of exercise, for example. It may also indicate levels of staff support that a person may need at their day centre (e.g. going to the toilet or moving between activities). The test is simple, quick and requires no special equipment or training; a person is asked to rise from a standard chair, walk to a marker 3 metres (10 feet) away, turn, walk back, and sit down again. They do this while wearing their usual footwear and using their usual mobility equipment, if any (e.g. walking stick, walking frame).

Different approaches are taken to scoring (i.e. being at risk of falling starts at 12 seconds for some, 13.5 seconds for some, or 20 seconds for others). The UK Chartered Institute for Physiotherapy defines a person being at risk of falls if the test takes 15 seconds or more.

The Chartered Institute for Physiotherapy has produced a video that demonstrates the test.

Changes in individual wellbeing could indicate service impact and may be helpful for monitoring purposes. Data may also be of interest to people working in health (e.g. GPs or commissioners). The Warwick-Edinburgh Mental Wellbeing Scale (WEMWBS) is a standardised self-completion questionnaire developed to measure mental wellbeing in the general population and evaluate projects, programmes and policies which aim to improve mental wellbeing. Both ask positively worded questions about feelings and thoughts over the previous 2 weeks. Each has a choice of 5 response categories. The WEMBWS has 14 questions and the SWEMWBS has 7 questions.

Both scales are recommended by the What Works Centre for Wellbeing, with further details in its ‘Measures Bank’: WEMWBS and SWEMWBS Scoring and interpretation guidance is provided. The full (and rather complex) user guide covers both questionnaires. Frequently Asked Questions provide additional useful information about its use. The only drawback to this questionnaire is that it has not been validated for use with people aged 75 and older.

Both scales are free to use (non-commercial use) but users must register for a license.

The SF-12 and SF-36 are standardised questionnaires that measure quality of life and functional health and wellbeing from a person’s own perspective. They cover physical functioning, bodily pain, limitations due to physical health problems and personal/emotional problems, emotional wellbeing, social functioning, energy/fatigue, general health perceptions, and a perceived change in health. They have 12 and 36 questions respectively.

These questionnaires may help day centres to present information about the health and wellbeing characteristics of older people using their service (as a group of service users) rather than to evidence outcomes and impact (i.e. background information to outcomes/impact data gathered in another way). Data collected using it may be of interest to people working in health (e.g. occupational therapists, physiotherapists, GPs or commissioners).

Use of the original (outdated) version of SF-36 is subject to a few Terms & Conditions, including crediting its development at RAND as part of the Medical Outcomes Study. The questionnaire (blank printable form; can also be completed online and printed) and scoring instructions (see Table 1-steps 1&2) are downloadable from RAND’s website.

Use of the most up-to-date versions (SF-12v2 or SF36v2) require a license (3-page form more suited to academic research than local monitoring of service users’ characteristics). See further information about these and the licences required

The Assessment of Quality of Life (AQoL) measures health-related quality of life in four domains: Independent Living, Relationships, Mental Health and Senses. It may be helpful for profiling older people attending a day centre, as a group, and for evidencing impact (even in a single domain, such as Relationships).

Several versions of the AQoL standardised questionnaire measure health-related quality of life. The shortest is AQoL-4D which has 12 questions covering four domains: Independent Living (self-care, household tasks, mobility), Relationships (friendships, isolation, family role), Mental Health (sleeping, worrying, pain) and Senses (seeing, hearing, communication).

The AQoL may be useful for presenting health information about older people attending a day centre (as a group of service users as background information to outcomes/impact data gathered in another way) as well as a means of evidencing outcomes and impact. A single score is calculated, or the score for each domain may be reported separately (e.g. Relationships) and the measure repeated at intervals to evidence any change.

Scoring assigned to each question differs according to whether the questionnaire is being used as a ‘psychometric’ measure (i.e. to get scores on health-related quality of life that can be used as a single score or separately by domain) or as a ‘utility’ measure (i.e. to be used in economic evaluations). Psychometric scores are more straightforward (e.g. 1-5 for each of 5 answer options). Utility scores are ‘preference weighted’ (i.e. reflect preferences more accurately than unweighted scores).

The analysis spreadsheet to calculate Psychometric (unweighted) scores is downloadable (see Step 1). A Zip Folder of SPSS and STATA documents to calculate Utility scores is downloadable from the web. Use is free, but users must register. Registration means that technical support can be provided.

Making records of healthcare service use, suggested by US day service experts, may also form part of an approach to evidencing day centre attendance outcomes.

In their call for the development of a set of uniform set of outcomes measures to be used in US day services, Anderson and colleagues reported that simple records of healthcare use have been recommended as useful for evidencing impact. For example, records may include numbers of hospital admissions, numbers of Accident and Emergency visits, numbers of prescribed medications, or numbers of GP consultations or NHS professional home visits.

Matters concerning staffing may also form part of a day centre’s impact. Staff recruitment and retention are known to be challenging in social care. Day centres may wish to monitor their staff’s job satisfaction. 

What Works Centre for Wellbeing has developed a Workplace wellbeing question bank. One measure is of job satisfaction in terms of worker wellbeing. Seven response options range from ‘Completely dissatisfied’ to ‘Completely satisfied’. The question options can be found on the Measures Bank.

As a single question, it is easy to interpret. A longer, 10-question version, the Worker wellbeing snapshot survey is also available

SROI aims to measure the wider (social and environmental as well as economic) value of a service (or organisation). It uses financial calculations to indicate the ratio of input cost to wider value created (e.g. for every pound spent, the service achieved a return of £x in social value). The scope of this resource for day centres and their stakeholders does not include SROI, but readers may wish to read an example of an SROI study undertaken by the Health Improvement Network on Peer Support for People with Dementia.