Law: It’s all a game

Happy New year! I am tempted out of my accidental blogging purdah by a genuinely fascinating story in Legal Week on Taylor Wessing’s use of Cosmic Cadet. Cosmic Cadet is not a replacement term for trainee solicitors indicating the uber-global commercial awareness of the modern day law student. No. It is a possibly cringe-worthy test designed to measure (per the maker‘s website):

  • Cognition. How an individual processes and uses information to perform mental operations.
  • Thinking Style. How an individual tends to approach and appraise problems and make decisions.
  • Interpersonal Style. An individual’s preferred approach to interacting with other people.
  • Delivering ResultsAn individual’s drive to cope with challenges and finish a task through to completion.
With such an awful title, there must be something in it, no?  Arctic Shores claim strong levels of scientific support for their approach, including that all of their ‘research’ (not all of their testing or application or interpretation, I note in passing [NB, I am reassured since posting this that, “our testing, interpretation and general validity has been independently reviewed” – see below in comments section] is reviewed (with what results we know not) by “independent subject matter experts“.
If I am sounding sceptical, in fact, I am more interested than sceptical. The attributes that Legal Week highlighted as measured by the test are particularly worthy of scrutiny:

Thinking style
Risk appetite
Managing uncertainty
Potential to innovate
Learning agility

Interpersonal style
Social confidence

Processing capacity
Executive function
Processing speed
Attention control

Delivering results
Performance under pressure

No mention of ethics was my first reaction – and remains my strongest one. Risk appetite is likely to be related to ethical inclination and some of the other measures may be too. It would be especially interesting to know what kinds of risk appetite users of the test want. The rather weakly evidenced assumption in the industry is that lawyers are risk averse, in the same way as lawyers are seen as both show-offs and introverts. An interesting part of the test will be the capacity of Arctic and the firms to learn more about the truth of such claims.

Fascinating too would be an explanation of how would-be trainees are supposed to manage uncertainty. There is an uncomfortable impression given by this list of the trainee as a machine, a resilient robot, a chip in the supercomputer that is big law. That’s an unfair impression, I am sure, but it is one which I hope the firms who are thinking along these lines think carefully about. Taylor Wessing, to be clear, seem to be thinking carefully about how the tests integrate with their wider processes of assessment.

Resilient, high performing people are one thing; systems that break them or lead them astray are another. I would not say law firms are broken, but there is plenty of evidence that they can and do lead some people astray. And it is absolutely vital that if firms are thinking along these lines they pay more than lip service to the moral capacities of their candidates and the ethical resilience of their systems and culture. I don’t see that in these tests. Perhaps it is to be found elsewhere.


Postscript: there’s another excellent story on this here

3 thoughts on “Law: It’s all a game

  1. Hi Richard, Note your comments about our game-based assessment. While everyone will have their own opinion about the name (and that will be subjective), the measurements we make have to be scientifically founded. In fact not just our research, but our testing, interpretation and general validity has been independently reviewed. We follow British Psychological Society standards and our reviewers include your colleague at UCL, Professor Adrian Furnham. Please do ask him about the exciting progress that an objective and data driven assessment provides, not just in terms of providing a level playing field for people from all backgrounds but a psychometrically advanced one too.
    Robert Newry, Managing Director & Co-Founder

    1. Hi and thanks. I’ve altered the text to reflect the substantial point around what the review covers. I know Adrian and will drop him a line.

      I agree there can be benefits of the kind you mention. Do you have data on the relationship between social confidence and background from your tests? Indeed, are you collecting data on background routinely?

      1. Thanks for that Richard. We collect anonymous demographic data on every applicant who uses our assessments (it’s an optional survey). We don’t tie the background to the test results rather we review the feedback by demographics – I don’t believe any other test provider does this but trying to make the selection process fairer and data-driven is why we started.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s