Tony Thorne was one of the Apollo Project officers, advising the team on the task of merging large volumes of data. Thorne, a former counter-terrorism officer with the Welsh Extremism and Counter-Terrorism Unit, said he was shocked by what he saw in Scotland. “We left Scotland with a process that was by no means complete or adequate,” he said.
Major issues highlighted during the test immediately reappeared, according to emails and internal memos from 2014 and 2015 reviewed by BuzzFeed News.
Agents described a system that “regularly crashed” and “stopped after 10 minutes”, with issues so severe that they dramatically increased “the time it takes to complete a simple task”.
Even basic searches caused problems. An agent had described how he entered a search term and got a result that was too broad. He started sifting through the documents manually to determine which ones he really needed, but in doing so, the system crashed. When he logged back in, he entered the same search terms and found that “the search result was not the same”.
Officers using the new systems also reported serious difficulties with the very problem the NCIA aimed to solve: communication with other forces and agencies. After a suspicious person flew into the UK, an officer reported receiving an important intelligence report from airport officials in an unreadable format. Another told an Apollo team member that the NCIA’s inability to share intelligence with the other regions still using the old system was a critical risk that “could lead to an intelligence failure.”
The quality of information entering the system was often poor. In some cases, the NCIA was inundated with irrelevant information; in others, vital information did not appear on the NCIA at all. One officer complained that the system “automatically ingests” documents that have nothing to do with terrorism. “This issue was something that was always talked about,” the officer wrote, “but now that we’re live, it looks like nothing more has been done about it.”
The NCIA was modeled after an already existing system called the Home Office Large Major Inquiry System, four sources told BuzzFeed News. The problem, one said, is that HOLMES is being used to investigate incidents that have already happened while the NCIA is meant to prevent attacks from happening. Another officer told BuzzFeed News that the NCIA’s build on top of the HOLMES system caused flaws that left large amounts of information hard to find.
Officers echoed these concerns in their official emails and reports. One of the main features borrowed from the HOLMES system was a search tool, much like Google, which was supposed to allow officers to quickly retrieve documents containing a certain word, regardless of where the particular word appeared in the case. If this worked, it would be much easier to find specific intelligence on potential terrorists from hundreds of thousands of files.
But the search tool was not working. Agents have found that if they put in the same search term multiple times, they often get a different result each time. The search tool was also not able to search for dates of birth, which made it much more difficult to find the right document.
This shortcoming joins another major problem. Early on, it became clear that many duplicate records would make their way to the NCIA – as it was compiling data from multiple forces that often had the same record on a given individual. An internal report seen by BuzzFeed News acknowledges that this would cause a ripple effect that would embarrass analysts. But the superiors eventually decided that “no deduplication would happen” until the whole UK was using the NCIA.
A Manchester-based officer who later started using the NCIA told BuzzFeed News that duplicates made finding what you were looking for like “trying to find a needle in a haystack” — such a struggle that “you could missing vital intelligence leads”.
Thorne, the counterterrorism detective who worked for the NCIA, grew increasingly worried. “Unfortunately,” he wrote to his colleagues in a February 2014 email, “because we are all fully aware that the NCIA has struggled to deliver on its promises and has not been fit for purpose. “.
The deployment of the NCIA continued.