Co-published with Tech Policy Press.
What a difference two weeks can make.
Back on Thursday, August 11, Twitter announced– in a series of tweets – that it would begin implementing the “Civic Integrity Policy” in the context of the upcoming US midterm elections. or blog post flagging the notice describes the newly redesigned tags on posts that contain misinformation. The tags, called “prebunks,” are meant to “get ahead of misleading narratives on Twitter,” and they work together with information “nodes” to share state-specific information about the election. “Twitter plays a critical role in empowering democratic conversations, facilitating meaningful political debate, and providing information on civic participation — not just in the U.S., but around the world,” the company said. “People deserve to trust the election chatter and content they encounter on Twitter.”
Then came the buzzer.
On Wednesday, August 23, documents disclosed by a whistleblower — former Twitter security chief Peiter “Mudge” Zatko — suggests that contrary to its stated intent, Twitter chronically underplays efforts to protect election discourse on its platform and that the problem is much worse abroad than that. is in the United States. The documents were originally reported by Washington Post AND CNN. Senate Judiciary Committee issued a summons for Zatko to testify next month.
One of the documents Zatko uncovered is a redacted version of a report titled “Current condition assessment” — prepared for Twitter by Alethea Group, a consulting company. The report describes “the current state of Twitter’s misinformation and disinformation capabilities” and identifies a number of shortcomings in the company’s approach to these problems. Regarding the election, he outlines three key areas of concern:
- Human resource issues, including staff shortages, organizational challenges and morale problems;
- A lack of expertise, particularly in the languages and cultural nuances needed to mitigate threats outside the United States;
- Engineering challenges, such as substandard tools, complexity of legacy systems, and security concerns.
While Twitter has likely taken some steps to address these issues since Zatko left the company in January 2022, the challenges raised in the whistleblower’s documents could be helpful to congressional investigators and civil society groups seeking assurances. that Twitter is preparing for the 2022 United States midterm election cycle and for representatives of other governments abroad with similar, if not greater, concerns.
People’s problems
The Alethea Group report, which is based on internal Twitter information and stakeholder interviews, shows that civic and election integrity teams are understaffed and overburdened. The report explains that this was a particularly acute problem in the 2020 US election period, where the company relied on “100 full-time staff” across Twitter’s integrity and security teams, as well as ” volunteers from other parts of the company” united under what the company called the “Election Squad” framework. Employees who focused on the 2020 election described burnout, while senior managers reported an expectation that they would be “always on” during the cycle. election to respond to any issues that may escalate, particularly regarding influential accounts.
The 2020 US election was so resource-intensive that the company didn’t have the staff to properly support other product launches. For example, debut of Fleets — Twitter’s short-lived and ephemeral image feature — was delayed until after the election because the company was so consumed by addressing election misinformation. And, there were organizational issues and communication problems across departments. As another indication of poor organizational design, the report says that “the organizational structure within Twitter that responds to misinformation and disinformation is muted and not clearly defined,” due to its ad hoc evolution over time.
It’s bad in the US, but much worse abroad
While the 2020 US election may have pushed Twitter’s security and integrity teams to near breaking point, the report clearly states that the company “lacks the organizational capacity in terms of staffing, functions, language and cultural nuance to to be able to operate in a global context.” The company has a bias against the English language and English-speaking countries, so the problem is particularly acute in Africa, Latin America and Asia. For example, the Alethea Group found that Twitter was unable to provide “even a scaled-down version of the election support that was deployed for the 2020 US election” for the July 2022 Japanese election, which the company identified as a priority. An internal document revealed that “there were no Japanese speakers on the Site Integrity team, only one [Trust & Safety] staff member based in Tokyo, and limited Japanese language coverage among the elderly [Twitter Safety] Strategic Response Staff.”
The US-centric approach means that misinformation and disinformation policies “are often made in response to US-based events, such as the 2020 presidential election, QAnon’s content on the platform, the Speaker of the House’s manipulated media , Nancy Pelosi, and more,” and so “they often don’t take into account the various ongoing disinformation or disinformation campaigns in other parts of the world.” A group of teams with different criteria decide which global picks are worth paying attention to, but even those designated as “top tier” may not receive proportional resources. The report documents the frustration of Twitter’s team in Brazil, for example, which unsuccessfully sought more attention during the 2020 election cycle in that country with an information system that affects a population of over 210 million people.
“Hacked Together”
Just as the human resources and organizational structures dedicated to addressing election integrity issues were developed on an ad hoc basis, the legacy systems and tools the company relies on to address election misinformation and disinformation also have significant deficiencies. At the time of the report’s completion, the situation was partly the result of organizational constraints: site integrity teams had to “seek help from engineering teams in other parts of the company to do things like implement minor updates to existing tools or new construction. those that can further automate the process for both policy analysts and investigative analysts.” Engineering teams were not asked to work with site integrity teams, so such requests were placed on a waiting list. This meant that teams fighting election fraud relied on “manual and outdated tools, and the individual knowledge of its analysts, who often have to code their own solutions to complete their work”.
The disinformation tagging process was itself complex, according to internal documents, which detailed that “the process of applying tags is cumbersome,” “requires the use of backend interfaces,” and “the complex steps involved make the scalable application of labels difficult to expand beyond a very small group of highly trained agents.” While preparing the report, an Alethea Group representative “found that no fewer than five different tools were needed to tag a single tweet.”
Questions for Legislators
After the disclosure of the whistleblower’s documents, US lawmakers pounced issue cards raising security concerns and asking the Federal Trade Commission to investigate whether the company may have violated the 2011 consent decree. Senate Judiciary Committee seeks Zatko’s testimony on Twitter security failures and “interference by foreign actors” on the platform. But the documents also raise questions that congressional investigators and lawmakers in other countries should be asking about what the company is doing to specifically protect elections.
Another congressional body that may be interested in Zatko’s revelations on the subject is the House Select Committee investigating the January 6, 2021 attack on the US Capitol, which included anonymous testimony from another whistleblower on Twitter at the July 12 hearing earlier this summer. The documents published by Zatko show that he had a special concern that there could be “acts of internal protest [within the company] aligned with the protesters,” which led him to wonder how to limit employee access to Twitter’s production environment. According to whistleblower complaint filed with the Securities and Exchange Commission and sent to Congress, when he examined the systems for such insider security threats, “he learned that it was essentially nothing. There were no logs, no one knew where the data lived or if it was critical, and all engineers had some form of critical access to the production environment.” What gave Zatko reason to fear such actions by a rogue employee, and are sufficient steps taken to secure these systems?
Furthermore, on August 26, 2021, the Select Committee sent a letter then Twitter CEO Jack Dorsey asking for copies of any review — internal or external — about “[m]is information, disinformation and misinformation about the 2020 election” produced since April 1, 2020, as well as a variety of other information, including about the activities of violent extremists on the platform. The Alethea Group report makes multiple references to a document on Twitter called “2020 US Civic Integrity Policy/Options/Product Reflections.” Was this document given to the Select Committee? For that matter, was the Alethea report the “subsequent report” it promises or any of the “19 other internal documents, retrospectives and training guides” it refers to that have been provided to the committee?
More broadly, however, lawmakers in the US and other democracies need to understand in more detail how social media platforms like Twitter are organized, both in terms of human and financial resources, to mitigate misinformation and disinformation in the context of elections. What is the right level of spending or investment in power needed to ensure that citizens “trust the election conversations and content they encounter on Twitter,” as the company has promised? And how does a country know if its election cycle has been determined worthy of the attention of Twitter’s integrity and security teams, and if the company has the right language and cultural expertise to meet the challenge?
It is difficult to establish a baseline for what is a suitable investment. But for years in what is now a chronic problem, social media platforms like Twitter can no longer be allowed to respond to elections as ad hoc “events.” “Twitter is a critical resource for the entire world,” Zatko told CNN. “Your entire perception of the world is created by what you see, read and consume online. And if you don’t understand what’s real and what’s not – yeah, I think that’s pretty scary.”
