Category Archives: Business and Law

Determining the Real-World Value of Interventions in Field Research

Leave a reply

Written by Dr Nancy Doyle.

Co-director of the Centre for Neurodiversity at Work, Dr Nancy Doyle is a Research Fellow at Birkbeck, Chartered Psychologist in organisational and occupational psychology, and the founder and owner of Genius Within CIC, a social enterprise dedicated to facilitating neurodiversity inclusion.

Real world data is essential

Applied field research is really difficult – data can be messy and full of contradictions. I realised in my doctoral research that data from a large field study didn’t make sense. I wanted to flip open the ‘black box’ of coaching (Nielsen & Randall, 2013) to understand how being coached could improve the work performance of dyslexic adults in the workplace. My pilot studies had shown a large increase in self-rated and manager-rated performance (Doyle & McDowall, 2015). Support for dyslexic adults is much needed as they are at significant increased risk of career limitations, unemployment and incarceration than the general population (Jensen et al., 2000; Snowling et al., 2000). I wanted to find out how coaching changes their self-beliefs, their stress levels and their behaviour

Real world data is hard to collect

So, I had before-, immediately after- and 3-months after coaching data from 67 dyslexic adults, split into three cohorts of wait-list control, one-to-one coaching and group coaching. I had a working memory score, a generalised self-efficacy score, a stress indicator and a workplace behaviour score for each. Bonferroni corrections for multiple comparisons (Perrett & Mundfrom, 2010) were somewhat disabling. All my intervention group means headed in the same direction – up! But the three x time, three x condition, four x dependent variable with the 67 people (down from 85 at the start) was just not powerful enough to get a conclusive result. My control group had practice effects (grrr) which waned by the third interval but ruined the time 2 analysis. My one-to-one coaching participants had a sustained uplift from time two to time three and my group coaching condition went up at time two and then up again at time three. I was none the wiser as to how coaching might improve the difficulties associated with dyslexia at work.

Real-world data is messy

We considered if the measures were faulty. The strongest result had come from using backward digit span in the Weschler Adult Intelligence Scale (Weschler, 2008). The group coaching condition had increased from an average of seven to eleven (the standardised score ranges from 1-19; the average is 8-12; practice effects are reported by the publisher to be 0.6). Yet this score was still not significant following Bonferroni corrections. The self-efficacy scores initially went backwards for the coaching conditions. We wondered if this was some sort of methodological artefact; or perhaps it reflected an increased self-awareness of struggles. However, they recovered by time three. Perhaps a more workplace-focused self-efficacy scale would be more effective? With the behavioural measures, these were designed by me and, though their reliability analyses were decent, we wondered if we should use an established scale of strategies. So, I decided to re-run the study. All studies were triple-blinded – testers didn’t know to which condition testees were assigned, coaches didn’t know test scores and I didn’t know which condition was which until after I had done the analysis.

You can imagine my delight, six months later, when I had almost identical results from my second cohort of 52 dyslexic adults (this time split into group coaching and control only). Control group practice effects at time two, persistent increases from the intervention group but not powerful enough to placate Bonferroni. So I undertook some ‘abductive reasoning’ (Van Maanen et al., 2007) to try and understand the results. This is when I noted a conundrum – a pattern in the data that shouldn’t be there if it was a straightforward null result.

Real-world people don’t respond in a homogenous way

Looking solely at the time three minus the time one scores (total distance travelled, or the “magnitude of the effect”) the means for each measure went in the same direction. Up for the intervention groups, slightly up for controls. But they were not correlated. How could this be? Why would there be consistency at the group level (as measured by the group means) analysis but no consistency at the individual level (correlation works by assessing the consistency of paired trajectories for each participant)? There is only one answer – the group means were masking significant disparities for individuals within each group. Now, this is where is gets technical. I tried a person-centred cluster analysis (Morin et al., 2018). In the working memory variable, I found distinct cohorts, a bi-modal distribution for the intervention group:

Some of them scored similarly to the control – a zero to small uplift, probably a practice effect. Others increased dramatically. In the other measures, I found a platykurtic distribution of improvement, some similar to the control, a bit ‘meh’, a bit more, increasing to quite reasonable and then quite large levels of improvement:

Group effect measures versus individual effects variance

But these were not the same people, which is why the correlations were not significant. In other words, some coachees had improved on working memory, some on levels of stress, some on self-efficacy and some on implementation of behavioural strategies. The coachees had taken what they wanted from the coaching, and not invested their personal development resources in the other mechanisms of change. The group level of analysis had wiped out variability in response-to-treatment and masked the impact of the coaching. This has implications for research, which is broadly dependent on the framework of null hypothesis significance tests. T-tests, ANOVAs, MANOVAs – all these depend on some sort of consistency within the group. Psychological research depends on isolating a potential variable, measuring it for each individual in a group, and crossing our fingers that the group will all behave in a similar enough way to achieve the hallowed ground of a significant p-value. But humans don’t behave in similar ways, even if they are broadly similar in age, diagnosis, employer, job role. I started wondering how many psychological approaches were ignoring the individual variability in treatment responses in favour of what works best for the dominant average, and ignoring the needs of those who don’t respond or respond negatively: mindfulness, I am looking at YOU (Farias & Wikholm, 2016).

Personalised pathways, group effect: meta-impact

We decided that there should be a way to understand whether or not an intervention has a good chance of working in some way for most rather than the one mechanism that will often work in the same way for many. To do this, I constructed a method for demarcating a significant improvement at the individual level which could be then re-aggregated at the group level across all the dependent variables. I deemed my participants to have improved if they improved to equal / more than one standard deviation above the average level of improvement for the cohort. This reduced the number of people who could be improved, marked a line in the sand for my platykurtic distributions and isolated the improvers in the bimodal distribution. When I had a binary yes/no score for improvers I could then add up how many improvers there were in the intervention groups and how many there were in the control groups. And bingo! The intervention groups produced significantly more improvers than the controls. This could be analysed using odds-ratio, ANOVA, t-tests or non-parametric equivalents (Doyle et al., 2022).

Going into my PhD viva with a novel statistical method of analysis was a risk. However, after a decent grilling, my examiners concurred that the method was empirically sound. Almuth and Dr Ray Randall, my external examiner, helped corral the study into a single paper. Getting it past journal reviewers was another matter! Those with statistical pedigree seemed affronted at the “arbitrary dichotomization” but offered several avenues for statistical exploration which I undertook, leading me to a place where I am way more familiar with mathematical reasoning than is comfortable for most social scientists! I enhanced the Maths and roped in a mathematician, Dr Kate Knight, to lay out the process in algebraic formulae. Job done? Nope. Those with field study experience loved the idea, but struggled with the Maths. Grr. Eventually, a multi-disciplinary journal, PLOSONE, found an editor and some anonymous reviewers who could see the pragmatic, realist need for expanding the methods available to field researchers and after a year of wrangling it was published on the 17th March 2022.

Real world data needs real world analytic method

What does this mean? My editor, Dr Ashley Weinberg, suggested that the meta-impact analysis of interventions has the potential to increase our understanding of psychological interventions in situ, giving boost to field researchers. There are still limitations. For example, we need to understand more about the cut-off point- the method needs to be replicated in tandem with qualitative study to explore whether it chimes with self-reports of experience and real world value. I know many research students and field researchers will empathise with my plight. There is a general sentiment in organisational psychology that we are hampered in research by participant attrition and low power, which leads us to design studies that have the most chance of a successful result, even though this limits us to basic designs or using large cohorts in ways that don’t match reality. My hope is that we can use meta-impact analysis to bring more ecological validity to our work as psychologists and embed nuance for individuals into study designs.

References

Dixon, R. A., & Hultsch, D. F. (1984). The Metamemory in Adulthood (MIA) instrument. Psychological Documents, 14(3).

Doyle, N., & McDowall, A. (2021). Diamond in the rough? An ‘empty review’ of research into ‘neurodiversity’ and a road map for developing the inclusion agenda. Equality, Diversity and Inclusion: An International Journal, published. https://doi.org/10.1108/EDI-06-2020-0172

Doyle, N.E., & McDowall, A. (2019). Context matters: A review to formulate a conceptual framework for coaching as a disability accommodation. PLoS ONE, 14(8). https://doi.org/10.1371/journal.pone.0199408

Doyle, N.E., Mcdowall, A., Randall, R., & Knight, K. (2022). Does it work ? Using a Meta-Impact score to examine global effects in quasi-experimental intervention studies. PLoS ONE, 17(3), 1–21. https://doi.org/10.1371/journal.pone.0265312

Doyle, N., & McDowall, A. (2015). Is coaching an effective adjustment for dyslexic adults? Coaching: An International Journal of Theory and PracticeCoaching: An, 8(2), 154–168. https://doi.org/10.1080/17521882.2015.1065894

Farias, M., & Wikholm, C. (2016). Has the science of mindfulness lost its mind ? British Journal of Psychology Bulletin, 40, 329–332. https://doi.org/10.1192/pb.bp.116.053686

Jensen, J., Lindgren, M., Andersson, K., Ingvar, D. H., & Levander, S. (2000). Cognitive intervention in unemployed individuals with reading and writing disabilities. Applied Neuropsychology, 7(4), 223–236. https://doi.org/10.1207/S15324826AN0704_4

King, E. B., Hebl, M. R., Morgan, W. B., & Ahmad, A. S. (2012). Field Experiments on Sensitive Organizational Topics. Organizational Research Methods, 16(4), 501–521. https://doi.org/10.1177/1094428112462608

McLoughlin, D., & Leather, C. (2013). The Dyslexic Adult. Chichester: John Wiley and Sons.

Morin, A., Bujacz, A., & Gagne, M. (2018). Person-Centered Methodologies in the Organizational Sciences : Introduction to the Feature Topic. 21(4), 803–813. https://doi.org/10.1177/1094428118773856

Nielsen, K., & Randall, R. (2013). Opening the black box: Presenting a model for evaluating organizational-level interventions. European Journal of Work and Organizational Psychology, 22(5), 601–617. https://doi.org/10.1080/1359432X.2012.690556

Perrett, J. J., & Mundfrom, D. J. (2010). Bonferroni Procedure. In N. J. Salkind (Ed.), Encyclopedia of Research Design (pp. 98–101). Sage Publications Ltd.

Santuzzi, A. M., Waltz, P. R., Finkelstein, L. M., & Rupp, D. E. (2014). Invisible disabilities: Unique challenges for employees and organizations. Industrial and Organizational Psychology, 7(2), 204–219. https://doi.org/10.1111/iops.12134

Snowling, M. J., Adams, J. W., Bowyer-Crane, C., & Tobin, V. A. (2000). Levels of literacy among juvenile offenders: the incidence of specific reading difficulties. Criminal Behaviour and Mental Health, 10(4), 229–241. https://doi.org/10.1002/cbm.362

Van Maanen, J., Sørensen, J. B., & Mitchell, T. R. (2007). The interplay between theory and method. Academy of Management Review, 32(4), 1145–1154. https://doi.org/10.5465/AMR.2007.26586080

Weschler, D. (2008). Weschler Adult Intelligence Scale version IV. Pearson.

More information: 

 

Share

Inclusivity at an Organisational level

Leave a reply

This blog is a lay summary of Doyle, N. (2022). Chapter 11: Adapting Other Internal Organizational Resources to a Neurodiverse Workforce in Bruyère, S. & Colella, A. (Eds) Neurodiversity in the Workplace Interests, Issues, and Opportunities. New York, Routledge.

Researchers and practitioners who work in the field of neurodiversity consider neurological differences to be more than just a disability. This view lets us see neurodiversity as a power- a power that comes with challenges but ones that can be overcome with systemic inclusion.

So, who are we referring to when we use the terms neurodiversity, neurodivergent or neurominorities?

The range is broad, but it includes neurodivergent identities such as: ADHD, autism, dyscalculia, dyslexia, dyspraxia, and tic disorders etc. Those that do not identify with these are known as neurotypical people. Some neurological differences might be caused by brain injury, chronic neurological conditions such as multiple sclerosis and mild to moderate mental health needs such as anxiety and depression.

This blog outlines changes employers could implement in their workplace so neurominorities feel more inherently included. This can lead to improved productivity and success.

Disability legislation is shifting towards a social model, where disability is seen as an effect of our environment, such as one’s workspace. Traditionally, organisations have been managing this by helping employees on an individual basis. Very little is done to change organisational policies and procedures on a broader level. For example, action is taken when someone discloses their disability or if there is a drop in an employees’ performance. This usually takes place in the form of reasonable adjustments or conducting workplace needs assessments. Nevertheless, this carries the risk of someone with a disability not receiving the support they need in the workplace if they choose not to disclose their disability. Also, there is a risk of discrimination and stigma for those who choose to disclose a disability. Injecting inclusive thinking at an organisational level means everyone is potentially covered to an extent rather than just the few. This aims to reduce discrimination and stigma and enable everyone to thrive in their roles.

What can employers do to be more inclusive?

Universal Design

Universal design is an approach that enables organisations to build an infrastructure so some of the accommodations that are broadly helpful can be put in place before a problem arises. These are likely to benefit both neurodivergent and neurotypical workers. In order to get the best results, universal design tells us to consider the following factors:

  • Equitable use: An employer could consider offering accommodations to every employee rather than an individual and make this best practice. For example, if flexible working hours for some led to increased productivity, could flexible working hours be offered to anyone?
  • Flexibility in use: Understanding that individuals may approach tasks in a different way and offering alternatives.
  • Simple and Intuitive Use: Use of clear and concrete language to avoid misinterpretations or confusion.
  • Perceptible Information: Presenting information in different ways, such as dividing long texts into paragraphs, using visual aids, use of audios etc.
  • Tolerance of Error: Not every accommodation can be in place, therefore there has to be a system for individuals to review their work, go back and change.
  • Low Physical Effort: To minimise physical effort on employees, for example, offering flex time for an employee to get to work to avoid rush hour traffic. This is useful for employees who might experience noise sensitivity or time management issues.
  • Size and Space for Approach and Use: Consider how to adjust a work environment since neurominorities could find some environments more overwhelming than others. Examples include noise, temperature, lack of personal space and privacy, visual stimuli, movements and smells.

Employee Welfare

  • Counselling, Mindfulness and Cognitive Behavioural Therapy via Employee Assistance Programme: If an employer offers traditional psychological therapies to support neurominorities, they should consider making sure that a referral is made to a specialist in neurodiversity. These traditional therapies focus on reducing the level of stress. However, the cause of the stress could be the result of the environmental demands in the workplace.
  • Assessment from Occupational Health: These assessments are carried out by occupational therapists (OT) and focus on physical health problems. Employers passing on referrals for an OT assessment should ensure their chosen provider has an understanding and specialism in neurodiversity.
  • Health benefits: When we look at medical interventions such as anti-depressants for anxiety and depression or medication for ADHD, employers should consider if the employment context itself is providing the unhealthy stimulus for the difficulty. If so, can the employer offer accommodations in the workplace first, before going down the treatment route?

Employee Resource and Business Groups

Many large companies have put together employee resource groups. Employees report that being part of such groups helps ensure they are heard. It is important for organisations to get feedback from individuals within their organisations. Hearing from a wider range of your employees is an essential and an important step towards introducing inclusivity in the organisations. Furthermore, business leaders have a crucial role to play in shaping and role modelling these policies and practices within their businesses.

Assessing the Effectiveness of Neurodiversity Programmes

This can be achieved through:

  • Assessing the longitudinal outcome after the implementations are made within the business
  • Reductions in individual-level compliance-based adjustments
  • Friendly and inclusive language around inclusion.

In conclusion, universal design should become the norm for organisations to create and promote a welcoming climate for neurominorities. The active steps outlined in this blog can greatly benefit organisations, leaders, and employees to create a meaningful and truly inclusive organisation.

Further Information

Share

Understanding Internet Addiction

Leave a reply

This blog by Marianne Cole, Centre for Neurodiversity at Work, is a lay summary of Pontes, H.M., Satel, J., McDowall, A. (2022). Internet Addiction. In: Pontes, H.M. (eds) Behavioral Addictions. Studies in Neuroscience, Psychology and Behavioral Economics. Springer, Cham.

This chapter summarises existing research into Internet Addiction (IA) and the wide range of definitions in the literature. It considers a number of different models that researchers have used to understand more about IA and its related disorders. The authors summarise how IA is diagnosed, treated and make recommendations for future, more reliable, research.

What is Internet Addiction?

There has been much debate around the term ‘internet addiction’ and what it means.  This is important when comparing studies because each study may be working to a different definition and looking at the condition through a different interpretive lens.  Some researchers think this term is too general and that we should focus more on specific online behaviours instead as people tend to become addicted to specific online activities.  IA is currently not recognised as a mental health disorder and there is no agreement on terminology. However, all definitions link behavioural addiction with serious health-related changes common across addictive disorders.

The positives and negatives of internet use

There are positive and negative implications of internet use, including but not limited to:

Positives

  • Improved quality of life
  • Reduction in social isolation
  • Potential platform for positive lifestyle choices
  • Increased access to information
  • Improved educational, social and psychological outcomes for students
  • Mood-enhancing

Negatives

  • Increase in aggression and hostility (gaming)
  • Acts as a medium for addition, for example gambling, gaming, pornography
  • Increases depression and anxiety
  • Causes relationship difficulties
  • Disrupts sleep hygiene behaviours

Models to aid understanding of Internet Addiction

This chapter compares three interpretive models (others are mentioned) which help our understanding of IA: Cognitive Behavioural, Interaction of Person- Effect-Cognition-Execution (I-PACE) and a biological model.

  1. Seen through a Cognitive Behavioural lens, some people may be addicted to a particular online activity (e.g., games, social media, etc.), while others (over)use the internet without specific purpose. General internet addicts may feel that the internet is the only place where they feel good about themselves, and they may seek it out because of underlying conditions such as depression and/or social anxiety. Researchers have found a strong link between procrastination and general internet addiction.
  2. The I-PACE model looks more widely at behaviours and ways of thinking that might influence a specific internet addiction and short-term gratification. Some people may be more likely to react too strongly, for example, personality, mood regulation, impulse control and other mental processes may all play a part.
  3. IA has been linked to biological changes in the brain picked up during scanning, such as reduced grey matter and dopamine. We need to be cautious about interpreting these biological links to IA because there are too many variables to firmly state the cause.

Internet Addiction and related disorders

IA is linked to a range of disorders, including Attention-Deficit/Hyperactivity Disorder (ADHD), mood/sleep disorders, and autism.  Research suggests that people diagnosed with these conditions are more likely to use the internet as a coping mechanism.  Again, conclusions need to be cautious because study methods differ, and it can be difficult to untangle the two-way relationship between IA and underlying conditions.

Some research has compared IA across different countries, which is useful in understanding how widely it affects different populations. But more research needs to be conducted with larger participant groups and improved study design.  It is difficult to make reliable comparisons when different definitions of IA are used alongside different ways of assessing it.

Diagnosis

The chapter mentions a number of methods for assessing IA:

  • Questionnaires
  • Internet Addiction Test (the most popular)
  • Internet Disorder Scale-Short Form
  • Psychometric tests

These have been widely adopted across many countries but have been criticised for being unreliable as an assessment tool while there is no agreed standard for diagnosing IA.

Treatment

There are medical (e.g. anti-depressants) and psychological (e.g. cognitive behaviour therapy) treatments for IA with the aim of regulating rather than stopping use. More and better-designed research is needed into the effectiveness of both these treatments.

Conclusions

Researchers seem to favour clear descriptions of specific forms of IA to make clear that it has different levels of severity – as opposed to a broader and unspecific category.  As long as there is ongoing disagreement over definitions, IA cannot officially be recognised as an addictive disorder. The authors are concerned that if researchers abandon this field, those with IA will suffer harmful effects both psychologically and socially, feeling that their distress is being played down. Researchers need to work with clinicians, psychologists and therapists to find evidence-based treatment for what is a vulnerable group of people.

Further Information

Share

Physical workplace adjustments to support neurodivergent workers

Leave a reply

This blog is a layperson summary of the paper Weber, C., Häne, E., Yarker, J., Krieger, B., & McDowall, A. (2022). Physical Workplace Adjustments to Support Neurodivergent Workers: A Systematic Review. Applied Psychology – An International Review.

Who are we?

We are a group of five researchers:

  • Clara is an environmental psychologist at the Zurich University of Applied Sciences, where she researches how physical workplaces make people feel, think, and behave and reasons why.
  • Eunji is part of Clara’s team and is a workplace management researcher, who figures out how offices can be better managed.
  • Almuth and Jo are occupational psychologists at Birkbeck, University of London who research workplace health and diversity.
  • Beate is an occupational therapy researcher at Zurich University of Applied Science looking at making the work environment better for young people with autism.

What is neurodiversity and neurodivergence?

  • Neurodiversity means that humans are all different from each other with particular strengths and weaknesses. The different ways that humans feel, think and behave are associated with certain conditions;
  • Neurodivergence describes when someone’s brain learns or behaves differently from what is considered ‘typical’. Not everyone likes this word, but we use it because we looked at research carried out on people with certain conditions. We use the two words interchangeably here for this reason.

What did we do?

  • We looked at ‘physical workplace adjustments’ for neurodivergent workers. These adjustments are mostly ‘physical helpers’ that aim to make it more comfortable to work in an office.
    • For example, if bright light in the office is hard to tolerate, you might prefer a different artificial light that you can also control or choose to use sunglasses.
  • We wanted to know what research has been done, what physical helpers are used, and if they make a difference to people’s wellbeing;
  • We confirmed if studies were trustworthy and of good quality;
  • Finally, we confirmed where there are gaps in research.

We need this information to make recommendations and to guide future research.

Why did we do it?

  • Research shows that neurodivergent people are excluded from work. Many experience difficulties finding or remaining in work because workplaces do not easily accommodate different needs;
  • People who are neurodiverse often have unmet sensory needs in the workplace. This means that sounds, lights, the touch/feeling of things, or other people’s closeness can be too much. This can affect their health and work ability, with people commonly reporting headaches, feeling dizzy or sick;
  • We need to know how to make workplaces healthy and productive environments for neurodiverse workers and what types of physical helpers are best;
  • There is guidance from charities and advisory groups listing different physical helpers;
  • Workplace design companies also offer various creative physical helpers, but they don’t say if these actually work. Little is known about how these helpers are tried and tested.

What did we do specifically?

We did a systematic review of the research evidence. A systematic review follows very specific rules and steps in order to find studies and make sense of their findings. By using this method, we developed a picture of all the available research in a specific area;

We looked for any studies that considered at least one physical helper used in an office. We included studies if this helper had anything to do with:

  • how well people felt at work (health/well-being)
  • how well people were able to work (performance)
  • the extent to which people found it easier to stay in work (occupational longevity).

We searched academic literature and guidance documents from charities or advisory groups and:

  • found 319 studies connected with our research topic/question. Of these, only 20 studies mentioned our particular focus;
  • confirmed how trustworthy the results of the studies are. We rated the quality of their research design and reporting of information;
  • grouped all the physical helpers and their positive effects to see at a glance what types have what kind of effect.

What have we found out?

  • Few studies say anything concrete about links between these physical helpers and improved well-being, work ability or staying in a job;
  • Many studies are based on interviews asking people about experience, rather than testing over time to see if physical helpers make a difference;
  • No studies focused specifically on physical helpers. This means that studies only mentioned physical helpers if participants did so;
  • Some studies report general helpful office adjustments such as altering light or desk placement (if available);
  • No studies used strong, reliable research methods, meaning other researchers cannot test their findings by recreating them. Without robust studies, we cannot say anything about cause and effect;
  • So far, we can mostly say that study participants believed that some helpers contributed to a good experience but we have no evidence that they actually work.

Why is this important?

  • We urgently need more and better research;
  • Neurodivergent workers are likely to be better able to access helpers if there is evidence that shows they make a difference;
  • Organisations might be spending money on helpers which don’t help;
  • Charity and other guidelines should acknowledge the evidence base for the accommodations they recommend so that people are aware of the basis for this advice;
  • Developing a specification for return of investment would help researchers and organisations gather more data to inform our understanding of what works, for whom and when.

Further Information

Share