“If Twitter was to ‘go in the morning’, let’s say, all of this—all of the firsthand evidence of atrocities or potential war crimes, and all of this potential evidence—would simply disappear,” says Ciaran O’Connor, senior analyst at the Institute for Strategic Dialogue (ISD), a global think tank. Information gathered using open-source intelligence, known as OSINT, has been used to support prosecutions for war crimes and acts as a record of events long after the human memory fades.
Part of what makes Twitter’s potential collapse uniquely challenging is that the “digital public square” has been built on the servers of a private company, says O’Connor’s colleague Elise Thomas, senior OSINT analyst with the ISD. It’s a problem we’ll have to deal with many times over the coming decades, she says: “This is perhaps the first really big test of that.”
Twitter’s ubiquity, its adoption by nearly a quarter of a billion users in the last 16 years, and its status as a de facto public archive, has made it a gold mine of information, says Thomas.
“In one sense, this actually represents an enormous opportunity for future historians—we’ve never had the capacity to capture this much data about any previous era in history,” she explains. But that enormous scale presents a huge storage problem for organizations.
For eight years, the US Library of Congress took it upon itself to maintain a public record of all tweets, but it stopped in 2018, instead selecting only a small number of accounts’ posts to capture. “It never, ever worked,” says William Kilbride, executive director of the Digital Preservation Coalition. The data the library was expected to store was too vast, the volume coming out of the firehose too great. “Let me put that in context: it’s the Library of Congress. They had some of the best expertise on this topic. If the Library of Congress can’t do it, that tells you something quite important,” he says.
That’s problematic, because Twitter is teeming with significant content from the past 16 years that could help tomorrow’s historians understand the world of today.
