Garbage In, Garbage Out
January 10 2024
Foolish to believe that computers and their software are infallible. Even more foolish to change the presumption as the Law Commission recommended some 25 years ago. That change must now be reversed, as I argue here –
Law on computer evidence must change in wake of UK Post Office Horizon scandal
The full article is below –
One of the less appreciated aspects of the UK Post Office Horizon scandal is that it was made possible by a legal change in 1999 on the admissibility of computer evidence. That change was made following a flawed Law Commission analysis which showed two things:
- how poorly lawyers understand technical evidence, especially software and digital evidence generally;
- the arrogance of those who do not know what they do not know.
It was an ignorant arrogance not confined to the Law Commission. Its recommendation effectively reversed the burden of proof and made it practically impossible for a defendant to prove that computer-based evidence was not accurate. There was simply no understanding of how complex computer systems operate nor the importance of their reliability.
MPs who passed the relevant law treated this change with a frivolity which would be mildly amusing were it not for its baleful consequences. They made the mistake of thinking that because computer technology had become more complicated, thus making it more difficult to prove reliability, the answer was not to bother at all because this would be “impractical”.
The idea that it was precisely this complexity which made it imperative to find a way of ensuring and proving that it could be relied on did not – apparently – occur to them.
The relevant Minister, Paul Boateng (a solicitor) treated it as a trivial change, commenting about eight-year-old children being the only ones to understand computers. There was simply no understanding of how complex computer systems operate nor the importance of their reliability nor the vital necessity for a whole range of people and groups to be able to rely on and trust their reliability.
That was then. Now – more than two decades later with the knowledge of the Post Office Horizon scandal (and others) arising because of this change – is the government going to look at this again?
No. In 2022 the Justice Ministry said there were no plans to review this (despite in 2020 a paper having been prepared by lawyers and IT professionals setting out some detailed proposals as to how the law on computer evidence might be reformed). Having a law which is at odds with how society works today, which has the potential for prejudicial consequences and which undermines the trust essential to the working world is absurd.
Why? The relevant Minister has stated that the presumption that what a computer says is accurate “has wide application”. This is to repeat the error made in 1999. Then it was “impractical” to expect people to prove their evidence was reliable because this was too difficult. Now it is too much effort to review it because it is used so widely so it is again too difficult.
But it is precisely because it has wide application that it is imperative that the law catches up with the world as it is now, that it be based on a proper understanding of IT. We are in a digital world which is only going to become more so. Laws which do not reflect that and which are based on ignorance (or laziness) are unpardonable.
This not an issue which only matters when someone is prosecuted. The reliability of computers matters to customers: of banks, insurance companies, pension companies, investment firms. It matters to every one operating in the financial sector. It matters in relation to monitoring. It matters in relation to disclosure and discovery. It matters to customers of entities in a wide range of sectors. It matters to regulators of entities in a wide range of sectors. It matters to patients. It is hard to think of any commercial sector or activity where it does not matter. This is only going to become more important with the increasing use of AI.
If the law says that you don’t need to prove that your systems are working properly, in a court battle, this risks undermining the need to do so, rather than reinforcing it. If your customers and counterparties assume that your systems are reliable and they turn out not to be, how will they react on being told that, too bad, the law says they are and it is up to them to prove otherwise?
Having a law which is at odds with how society works today, which has the potential for prejudicial consequences and which undermines the trust essential to the working world is absurd. And will become absurder still as AI technology and its use expand.
Directors of companies have a duty to “promote the success of the company”. Directors of regulated companies have additional duties, particularly in relation to the technology and systems they use and the information which is captured, stored and derived from them. They need to be able to show their regulators – and other stakeholders – that their systems work effectively, are reliable and that the information coming from them can be trusted.
The government needs to look at this again. And this time involve and listen to IT professionals and others who really understand how computer systems work and how they can go wrong. It could do with listening to those who understand the consequences of such failing.
Fortunately, this time the Law Commission is led by someone who certainly does understand this. Mr Justice Fraser was the judge who decided the Bates vs The Post Office litigation which blew this scandal open. He was less than impressed with the Post Office’s factual statements about Horizon. He is now the new Chair of the Law Commission. Time for the government to stop finding excuses for inaction.
Photo by Markus Spiske on Unsplash
Abuse of Power
January 4 2024
In episode 1 of Mr Bates vs The Post Office, there is a scene in which Alan Bates’ wife tells him she has a job.
“Teaching?” he replies.
“No. Cleaning houses.”
They need the money to make ends meet. They have lost their savings and business.
He gives her a look – of love, gratitude & a hint of humiliation at what they’ve been reduced to and then – quietly but with determination – he says:
“I’ll get those bastards.”
It is a wonderful piece of writing and sublime, subtle acting, especially by Toby Jones. It captures both the humiliation inflicted on innocent people by the powerful and the former’s determination not to be ground down. It is about the subpostmasters. But like all good drama, including that based on real life, it shows something universal.
Those sentiments have been echoed before. They will, I am sorry to say, be repeated in the future. Because abuse of power is hard to eradicate. The powerful have no interest in doing so; the powerless find it hard to do so.
This abuse of power by the state or its organs has happened so many times before. This story about the Post Office is not an appalling one-off. It is only the latest of a series of scandals going back at least 60 years.
In so many ways, the misbehaviours exhibited by the Post Office are similar to those exhibited by the Coal Board in the Aberfan tragedy, by the police in Hillsborough, by the government in the blood contamination scandal and Windrush, by the NHS in numerous medical scandals, and in many others.
See https://www.cyclefree.co.uk/the-price-of-indifference/.
The substance may be different but the misbehaviours by the powerful are so very similar:
– the refusal to listen to concerns
– the lies and cover ups
– the stingy callous approach to apologies and compensation
– the refusal to accept responsibility
– the avoidance of accountability.
There are two behaviours above all which are repeated. The first is the arrogance of indispensability.
It is this which leads to the abuse of power which lies at the heart of the actions taken. The Post Office’s conduct over nearly two decades might best be described as a rampage of extortion with menaces, based on lies.
It is enabled by those who allow such organisations to behave as if they are unchallengeable. As if they are “Too Big To Fail” or “Too Important To Fail“.
It is abetted by such organisations being put by voters on a pedestal of some kind or trusted too blindly: the Post Office as a twinkly, trusted “Postman Pat-At-The-Heart-of-The-Community” who could not possibly do any wrong. Or the NHS. Or the police – who have often confused the vital importance of policing as a function with the importance themselves as an institution so making it much harder to challenge bad policing.
In this, these organisations have echoed the stance taken by much of the City in its pre-financial crash glory days, when it gave the impression that it was so lucrative and therefore indispensable that it could do whatever it wanted with little real regard for the rules. It was an attitude enabled by politicians so delighted at the large tax revenues that they ignored the dangers of the overmighty barons of that time.
And the second?
It is an indifference to ordinary people, to the human consequences of misbehaviour, to the impact on others.
This quote from the above article explains so much about the Post Office’s and government’s obduracy about putting this right.
“There is the indifference which can be one of the causes of a problem. But what is often worse is the indifference shown to victims after problems have arisen. It is hard to understand the callousness of some decisions. Perhaps it is made easier by forgetting or ignoring those who are affected.
It feels like indifference to those on the receiving end. But perhaps its impulse is less the effect on the victims but more a desire to save face by those responsible……
It harms an institution’s self-image and, often, of senior people within it. “We got it wrong.” is hard to say. If “we get it wrong” what sort of a “we” are we, really?”Avoiding the shame of having to admit that your actions or inactions have been responsible for the suffering of others is what drives this defensiveness and indifference.”
- You see this in the evidence given by Post Office staff in the Williams Inquiry.
- You see it in the evidence given by the Post Office’s internal and external lawyers.
- You see it in the response at Board level, which also manages to suggest that criticisms of its staff are somehow unacceptable and unfair and unkind, as if they were the true victims. The combination of arrogance and narcissism must be hard to bear for those who really have suffered.
- You see it in the response by the government. It gives the impression of being a random passer-by at the scene of accident caused by complete strangers ineffectually using a hankie to mop up blood and expecting huge thanks. In reality, it is the owner and funder of the Post Office and without its say-so and money the Post Office would cease to exist overnight.
What you also see in those other cases is how those responsible for harm done to others got away with it, were not made accountable, suffered no adverse consequences.
We are seeing that in this case too. Look at all the senior people in the period between 2000 – 2012 (when Paula Vennells was appointed CEO) when prosecutions were happening despite the knowledge that senior people in the Post Office, Fujitsu and government knew about Horizon’s difficulties and deficiencies. Look at how they have flourished in well paid jobs with their time in charge of an organisation at the heart of the worst miscarriage of justice in English history airbrushed away or ignored.
It feels as if this is more of the same: the powerful abusing the powerless. Because they can. Because they know they are untouchable. Because even if disciplinary or civil or criminal proceedings are brought, those Post Office prosecutors, investigators and lawyers, external lawyers and Fujitsu employees and others will benefit from the protections and rights and compliance with the rules and a fair trial which the Post Office denied the subpostmasters. We know why they should get those protections. But to those who have suffered as a result of their actions, it must feel like yet another unfairness to be added to those they’ve already endured, another example of how the powerful benefit at the expense of the powerless.
Will those who did wrong be held to account this time?
What about those others who set up the structures or took decisions or made laws which enabled this scandal to happen: the Ministers, the civil servants, the Law Commission, the MPs?
What sort of accountability should they face?
An affront to our conscience
January 1 2024
This evening there is the first episode of a four part drama – Mr Bates vs The Post Office. It is a must see, if only in the hope that it will bring home to a wider public the scale and human impact of what has rightly been called the worst miscarriage of justice in English legal history. It is to be hoped that such public interest might put pressure on politicians to put right – and without further delay – matters which are – or should be – an affront to the conscience of the British state.
It is shaming to see from the evidence given during the statutory public inquiry headed by Sir Wyn Williams how so many from my own profession behaved so unprofessionally, incompetently and potentially worse, both during the events which are the subject of the Inquiry and during the Inquiry itself.
If there is one thing to learn from it, it should be a reminder that practising law or carrying out investigations without any understanding of the ethical underpinning of one’s work and the necessity of ensuring that this informs everything you do is wrong. This is not what true professionalism requires. The question is never “Can I do this?“. But “Should I?“.
It is correct to say that this is the worst miscarriage of justice. But this description underplays the nature of the scandal. In reality, this is not just a scandal about the Post Office exploiting some flawed accounting software.
- It is a scandal about the development of flawed hardware and software systems, a flawed governmental and corporate procurement process and a flawed adoption and rolling out process.
- It is a scandal about how the Post Office, a state owned body with unlimited resources and its own prosecution service, operated with no effective corporate governance or Ministerial control or supervision and exploited flawed software, flawed contracts and the civil and criminal legal systems to extort money it was not owed from subpostmasters.
- It is a scandal about how the legal system failed – and continues to fail – to understand technical evidence.
- It is a scandal about how the legal system has failed for far too long those accused and convicted of crimes which did not happen. As the government’s own Compensation Advisory Board has said: “the justice system itself is called into question in the current circumstances.”
- It is a scandal about a failure of Parliamentary and Ministerial governance.
- It is a scandal about how the state fails to put right its mistakes and compensate those harmed by those mistakes.
Ultimately, it is a story about the abuse of power.
There are so many aspects to it that it can be hard to get your head round all of it. But these articles are an attempt to summarise some of the key issues. A work-in-progress, obviously. But I hope helpful.
1. An overview
2. The Business Secretary’s role
3. Compensation
4. Revelations from the Williams Inquiry
5. Ministerial and corporate governance of the Post Offiice
6. The reliability of computer evidence and how the Law Commission got this wrong
7. What Parliament did and did not do
