SQE -Underpowered after all these years

There’s an interesting story on Legal Cheek about the SQE. Existing evidence, informed readers will recall, suggests there may be an EDI problem with the assessment. They have been warned about this periodically, it is (I believe, and stand to be corrected) a known problem with some of the medical equivalents, and their assessment of SQE1 underlined the problems. Now some news about the evaluation of SQE2 has emerged. Here is the paragraph that caught my eye:

Kaplan, the organisation appointed to assess the SQE, found that “univariate” analysis (performance by individual candidate variables) shows a significant performance difference between white and BAME candidates compared with “multivariate” analysis (which seeks to identify the best true predictors of candidate scores). However, both analyses should be viewed with caution due to overlapping candidate variables and the small numbers in the different groups, Brannan stressed.

The first sentence is difficult to understand. It might be saying multivariate analysis suggests that factors other than ethnicity explain performance on SQE2. It’s the second sentence that is important. It is an admission that the evaluation of SQE2 is underpowered; that is it was run with insufficient numbers to test the question Julie is dealing with here. Unless one of the aims of the pilot was not to look at BAME differences, then this bit of the pilot is (I will allow you to pick the single best answer):

a) going to happen anyway because, well because
b) numbers-shmumbers
c) the Netherlands deserved to win
d) not fit for purpose.




One thought on “SQE -Underpowered after all these years

  1. These figures are valueless.

    First, the sample is too small and probably self-selected.

    Second, you would need to correct for other factors. Take for example the advantage of being the child of a lawyer or indeed two lawyers. A higher proportion of that group of mid-twenties is – at the moment – white than of the population at large; that will change as the profession changes. And they enjoy an advantage; they will have absorbed law by osmosis from the cradle up. Short of excluding them or marking them down you can’t change that.

    Third: the Law Society reported last year that the proportion of members who “prefer not to say” is 17% and rising – which suggests that it is younger members who are exercising that option so the proportion will rise. I have consistently since I was in articles in the mid-Seventies refused to provide my ethnic origin; a fortiori I have refused to answer the even more intrusive questions about religion and sexuality. For many of us “prefer not to say” is a euphemism for “mind your own damned business” and I hope nobody will suggest abolishing it. But it makes figures like these meaningless.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s