The Cyber Safety Review Board Should Investigate Major Historical Incidents

“Some things that should not have been forgotten were lost. History became legend. Legend became myth.” ― J.R.R. Tolkien 

“There is a lack of reliable data about incidents that can be used to build a coherent and consistent narrative about what has actually happened in cybersecurity, much less around which to build policy and incident response plans. Often, when reliable data is produced, it can disappear as links rot on the internet. There are few sources of truth, and little incentive to build careful history.” – Robert Knake, Adam Shostack, and Tarah Wheeler, Learning From Cyber Incidents. 

One of the functions of the NTSB is to establish authoritative history that can be drawn upon for future research. Crew Resource Management (CRM) is a key element in modern aviation safety, and limits aircraft crew shift durations to maintain safe conditions in the air, among other things. CRM was created after the crash of United Airlines Flight 173, a major aviation incident in Portland, Oregon in 1978. The McDonnell-Douglas aircraft involved in the crash ran out of fuel due to the exhaustion and overwork of the crew who were too tired to safely monitor the state of the airplane. Ten people died when the plane crashed in downtown Portland.  

But now, imagine that the only history available on the United 173 crash is a few blog posts and the occasional “in memoriam” tweet on the website of McDonnell-Douglas and United Airlines, explaining in glowing terms all the safety improvements both companies had made since the bad old days. That’s very much the state of the history of major cyber incidents now. 

On corporate websites, white papers and blog posts are considered marketing tools to be discarded when no longer of service to revenue generation. More generally, internet information is ephemeral. In 2013, a Harvard study by Jonathan Zittrain, Kendra Albert, & Lawrence Lessig found that half of the internet citations in US Supreme Court opinions were already gone. The same phenomenon applies to cyber incident reporting; we’re losing the history of our disasters and as a result, failing to learn from them. The CSRB can and should create an authoritative history of major events by assembling and recording knowledge while it’s still reasonably fresh. Doing so will be powerful, institution-building work that can be interwoven with more recent cases. 

Is the CSRB hamstrung? 

The CSRB faces many challenges, some specific to Solarwinds, others more general. There’s a series of misconceptions, fears, and institutional issues that continue to arise in this and many other major investigations.  

First, there was a perception in Washington, D.C. in 2021-2022 that intelligence agencies had already established “everything we needed to know about Solarwinds.” Many policymakers believed this message, and also accepted the statements of intelligence agencies regarding how they had reached surety on knowing enough. Intelligence agencies described their surety with a request that policymakers simply believe them, and claimed that attributing information to the actual parties would harm their sources and methods. For policymakers who may not have the technical skills to know the difference between a complex incident and an incident they are told is complex, this lack of public reporting is extremely problematic. A public source of understandable truth around these cyber incidents assists policymakers in good decision-making and, not so incidentally, budget allocation. 

Second, there is hesitation to cast blame on a single vendor or company. Any framing that these reports are about blame is concerning, and the board will need to work to establish the norm that CSRB investigations are about learning lessons in the complex global cyber environment, not casting blame. The truth is that if one company sells a product that has captured a major portion of a market, and there’s an exploited flaw in that product that creates a major and devastating incident, that incident must be investigated to prevent it from happening again.  

More generally, the board, as established, may be hamstrung by confidentiality concerns, institutional factors such a lack of full time staff (the CSRB now has approximately 5 full time staff compared to the NTSB’s 400 employees), budget concerns ($2.8 million compared to the NTSB’s $153 million), or conflicts of interest arising from members’ full time external jobs as corporate executives or government officers. We hope those will be addressed by upcoming legislation. To be extremely clear, the authors have deep respect for each individual on the CSRB board. Each one of them is outstanding, accomplished, highly respected in cybersecurity. The institutional constraints on the board as a whole is where concern arises.  

The first confidentiality issue is around corporate secrets. Companies are especially protective of their assets and any potential loss of face during cyber incidents. It is common for firms to immediately begin shielding themselves by use of attorney-client privilege once they realize an incident has taken place. Attorney-client privilege has many costs, one of which is that disclosure to the CSRB may reduce the ability of firms to preserve that privilege during litigation. This, combined with the board’s lack of subpoena power, limits the ability of the board to investigate recent incidents. Associated with that, confidentiality agreements such as non-disclosure agreements (NDA) limit the ability of the board to quote, attribute, and otherwise provide specific facts, all of which would add to the credibility of its reports. However, unless the conflict of interest issue is resolved, the CSRB could be seen as a tool for board members to force open the doors of their competitors. 

The second confidentiality issue is around government secrets. In their “Review of Inaugural Proceedings,” the board suggests that it establish policies for handling classified information. This would be something between a bad use of time and actively counter-productive. The board needs the ability to talk publicly about the facts it considers. Having classified information in the mix adds substantial complexity and inhibits the demonstration of clear, logical thinking that befits a review board. If classified information is essential to understanding an incident, then either the declassification should be done before the board considers it, or the incident should be de-prioritized, because we’ll never learn lessons from it. Relatedly, it would be a mistake to require all board members to hold clearances, as that would severely limit the expert pool upon which to draw. 

The NTSB is a full time job. Congress should make CSRB membership a similarly full-time, compensated role. Doing so would not only increase the board’s capacity, but also increase its independence and status as an emergent institution. 

Unfortunately, many of the members of the CSRB work for firms that are impacted by these incidents, can easily see themselves targeted by an investigation, and are also sensibly trying to avoid the conflict of interest of investigating their competitors. It is essential for the credibility of the board that it demonstrates its independence and recusal processes for those incidents, and avoid any appearance that having a seat on the board influences what’s investigated. Resolving this will make giving the board subpoena power as discussed above much easier. 

Until these issues are resolved—some of which are in the power of the board to solve and some of which require Congressional action—the CSRB will remain at least partially hamstrung by institutional constraints and incentives. However, there is a workaround that would let the CSRB create massive value and trust for the information security industry and the entire world affected by these cyber incidents. 

What incidents should be investigated? 

The CSRB can create a common and shared narrative of U.S. cybersecurity history by investigating incidents that are a few years old. The present moment is a sweet spot for investigation of these major incidents, because most of the NDAs have expired, sensitivities around share prices or blame are gone, associated lawsuits have been resolved, current conflicts of interest are minuscule or absent, the technical and social impacts from these attacks continue in the long tail, and many people who still retain solid memory of these incidents might now prefer correcting the record to not reopening old wounds. 

“The purpose of [a] technical investigation is not to identify a singular cause for the cyber incident or to assign blame to a particular actor, but rather to ascertain and document the conditions under which the cyber incident occurred, decisions made before, during, and after the incident, and various software, systems, personnel, and environmental factors that contributed to the incident.”  Based on this, the CSRB should choose which incidents to investigate using these three criteria:  

• The breadth of the incident’s impact across not just the tech sector but all American citizens; 

• The incident’s implications for the reputation of the U.S. government or its relationships with allies; 

• And the ongoing effects of the incident. 

Using these parameters to choose which incidents to evaluate will push the CSRB toward investigating cyber incidents that are meaningful, current, and globally-impactful. The CSRB would function as an impartial archive and arbiter; its conclusions would be isolated from the incentives and desires of the companies being investigated and provide a definitive record of what went wrong where.  

Three historic incidents should be at the top of the list for the CSRB to investigate: the 2015 Office of Personnel Management (OPM) breach, the 2016 Uber data breach, and the 2017 Equifax data theft. These incidents represent three of the most important failures in U.S. cybersecurity, and the CSRB needs to provide a definitive, impartial recounting of the course of each incident, what can be improved, and the ongoing ramifications of the attacks. 

Office of Personnel Management 

In 2015, China attacked the U.S. Office of Personnel Management and stole (at least) the SF-86 forms for 21.5 million people, including most past and present employees of the U.S. government. The SF-86 contains highly personal information about employees and applicants,  including medical, personal, financial, and mental issues, and is used for most security clearance applications. The response to the hack was also exceptionally confusing. Different sources made competing claims, such as that an outside company discovered the breach, or that it was an internal cybersecurity system that found it, or even that no one knew who had found the breach. This lack of clarity still persists eight years after the breach was discovered. The continued lack of an official report from the CSRB or any nonpartisan body since the incident arguably does far more harm, by demonstrating that a lack of accountability for the loss of uncountably valuable data regarding U.S. clearance holders is simply accepted. A lengthy report was released in 2016 by the majority party on the House Oversight & Accountability Committee, but as a partisan political tool lacked the needed perceived objectivity. The Government Accounting Office released a follow-up compliance audit which is laudable for its technical depth but did not draw larger lessons or make recommendations for the field. 

The combination of the chaotic response and the severity of the theft of the SF-86 forms makes the OPM hack important for the CSRB to respond to. There are complexities here. It’s very likely that some of the employees at OPM during the breach are still in government and do not wish to see the CSRB shine a spotlight on any missteps. A good deal of the government response at the time may have been classified, but now that the class action suit has been resolved, digging into the technical incident and response should have far fewer sensitivities. 

It’s been eight years. If that timespan isn’t long enough to remove sensitivities around blame for this incident (and those sensitivities are used as a reason to prevent or halt an investigation), we may as well not have a CSRB to begin with. 

Uber 

The CSRB could usefully investigate Uber’s cover-up of a 2016 data breach at Uber. The company’s then chief information security officer (CISO), Joe Sullivan has been convicted of and now sentenced for obstructing an investigation into the breach, as well as misprision of a felony. If a criminal trial has been conducted, what could a CSRB investigation add? 

The work of criminal trials and investigative reports are related. Both have a goal of bringing out the facts. As the authors wrote in the 2021 “Learning from Cyber Incidents” report, “Indictments are written to convict criminals, not to help the defensive community learn.” That distinction polarizes participants, and creates a controversy, one which rises to the surface in the Government’s sentencing memorandum: 

“One of the themes that becomes evident in reviewing the letters submitted on Defendant Sullivan’s behalf is that many in the cybersecurity industry are not aware of the egregious conduct Defendant Sullivan has been proved guilty of—the witness tampering, the fraudulent corporate paperwork, the many lies. Letter after letter submitted to this Court suggests that this prosecution reflects simple second-guessing of a difficult decision, that Defendant Sullivan is nothing more than a scapegoat…. As the Court is aware after presiding over the trial in this matter, none of this is true. Additionally, as the Court may be aware, this false narrative has the real potential to drive a wedge between the cybersecurity community and law enforcement at precisely a time when the United States is facing an unprecedented array of cyber threats that require those two communities to work hand-in-glove.” 

The U.S. Attorneys involved are specifically saying that the whole truth has not come out, multiple false narratives are at play, and many people involved have specific incentives to muddy the waters. The CSRB could provide a less charged review of the facts than those based on a published trial record, and in doing so help avoid the wedge to which the government alludes. It’s quite clear that the details of the technical incident, the lack of internal controls, and the information security issues inside Uber led to the egregious and abusive loss of user location data—but how? What are the lessons we can learn from this incident? Pointing a single finger at the CISO of the company (presiding Judge “Orrick said he felt former Uber chief executive Travis Kalanick was equally responsible for what he considered a serious offense, and he wondered aloud why Kalanick had not been charged.” ) and anyone using the trial resolution to claim that the problem is solved is exactly why the CSRB should exist in general, and investigate this incident in particular.  

Equifax 

The third hack which the CSRB should investigate is the breach of the credit bureau Equifax in 2017, which allowed the Chinese hackers to steal the names, addresses, and social security numbers of over 147 million Americans. The Equifax hack was possible because the company failed to patch a vulnerability in Apache, an open-source web application it was using in its dispute resolution portal. The CSRB needs to investigate the Equifax hack partly because of the sheer amount of personal data stolen–and partly because Apache is the same web application that contained the Log4Shell vulnerability from the single CSRB investigation so far. The 147 million records stolen represent more than 45 percent of the entire American population and could allow the Chinese government to easily create a profile of most U.S. adults. The U.S. government later charged four members of the Chinese military with breaking into the network and stealing the data.  

The Equifax breach has fewer sensitivities than the other two examples; the U.S. government had nothing to do with enabling the breach, this was a case of a company failing to patch vulnerabilities due to a complex series of missed internal controls, and everyone else paying the price. In this case, it may be simpler to conduct a robust investigation of what went wrong, although determining conclusively how the attackers gained and maintained access to the network may be difficult, given other reports’ lack of conclusiveness on the subject and the amount of time that has passed since the breach. The Federal Trade Commission filed a lawsuit against Equifax and later reached a $575 million settlement, and as in the OPM breach a House majority committee issued a report. However, every Congressional majority committee report must struggle against a perception that it aims to serve a partisan goal. Unfortunately, this is the case regardless of which party produces the report, and so the non-partisan nature of the CSRB will help develop and distribute lessons. 

Conclusion 

Last month, Kim Zetter reported that “The DOJ Detected the SolarWinds Hack 6 Months Earlier Than First Disclosed.” What happened at the Department of Justice, Mandiant, and Microsoft to make that decision seems like a fascinating case study. The CSRB was initially tasked with investigating Solarwinds, and apparently decided, for reasons that are unclear, that it already knew enough. These incident and decision making processes would benefit from independent oversight, and the board may be well positioned to provide that oversight, if it establishes sufficient independence and a good track record of providing valuable historic incident investigation. 

Right now, the United States does not have a shared narrative of cybersecurity history and major cyber events. The CSRB should rectify this problem. A shared narrative allows the U.S. government to lead in cybersecurity, rather than ceding that initiative to private cybersecurity companies, which are incentivized to make themselves and their products look good, and journalists, who may lack the technical knowledge to be the definitive source on an event. It also sets an example for state and local government and federal agencies to follow regarding conducting thorough, impartial reviews after major cybersecurity incidents. Getting the CSRB to investigate these incidents will not be easy. Building the reputation of an agency like the NTSB has been hard work over decades, with the wholehearted support of the aviation sector; building the CSRB as an institution will be even harder. There is a tremendous amount of pressure from different organizations that have prevented investigations up until this point, but it will be beneficial to have an impartial, public record of failures, and how to avoid repeating them. 

Tarah Wheeler is a Senior Fellow for Global Cyber Policy at the Council on Foreign Relations.

Adam Shostack is a leading expert on threat modeling, founder of Shostack and Associates, and an Affiliate Professor at the Paul G. Allen School of Computer Science and Engineering at the University of Washington.