X owner Elon Musk is promising to “soon” address the lack of transparency around “shadowbanning” on the social network formerly known as Twitter — that is, not outright banning a user, but making changes that ensure their content is no longer circulated and kept out of public view. In replies to users on X, Musk apologized for why the process is taking so long and explained what’s holding up X from providing this data to users.
Shadowbanning has been a topic of concern even in Twitter’s earlier days, as users didn’t know they had been penalized in this way for their tweets. When enforced, users can post to the platform as usual, but would notice their tweets simply aren’t getting the traction and replies they normally would. The topic often became politicized, as Trump and other Republicans complained that Twitter was making their tweets invisible.
After taking over Twitter, Musk even attempted to prove that shadowbanning was a common practice at the social network, by releasing information to the public, though ultimately what was shared was just a behind-the-curtain look at the complicated task of social media moderation.
Still, Musk has insisted that users should have the right to know if they’ve been shadowbanned. At the Viva Tech conference, the Twitter owner said the company was working on a way to show users if their account was affected by the Twitter system in any way.
In a new tweet this morning, Musk explained why the problem was hard to tackle, complaining there are “so many layers of ‘trust & safety’ software that it often takes the company “hours to figure out who, how and why an account was suspended or shadowbanned.” He also said a ground-up rewrite was underway to simplify this codebase.
The response didn’t shed as much light on the difficulties facing X’s latest project as many would hope, but a thread from Twitter’s former Head of Trust and Safety Yoel Roth provided far more insight.
Posting on the decentralized Twitter rival Bluesky, Roth explained there was some legitimacy to Musk’s statements, despite the “word salad” he used.
Roth began by explaining that social media sites would generally make note somewhere when they’ve banned someone and why. For smaller companies, this might even be a spreadsheet or Google Doc, at first. Over time, as the system and network scale, the company would generally move that info into some kind of metadata directly attached to user accounts.
“Maybe it’s a free-text note on the account that says something like, ‘Yoel banned this user for posting death threats. Don’t unban them without, yknow, checking with me first, please,’” Roth said. Twitter, however, still stores “a lot of enforcement metadata in free-text notes attached to user accounts,” he noted. (This system is called Guano, because everything at Twitter had a cutesy bird name, he mentioned as an aside.)
Twitter didn’t prioritize moving away from free-text notes system because they were busy with other things, Roth said, but that led to problems because these sorts of notes are “very hard to parse programmatically — which is sort of what Musk is complaining about,” Roth said. “A human can understand and act on them, but if you wanted to provide automated user notice of account status, you need them in some structured format.”
Roth agreed that rewriting the code that handles enforcement attribution is a good idea — and one that was in the works just ahead of Twitter’s acquisition, in fact. It was due to be shipped this year. (Perhaps, then, the Musk takeover of Twitter and the subsequent widespread layoffs were what slowed the project down?)
“For things like spam, Twitter has hundreds (if not thousands) of different models and heuristics operating in parallel. That complexity is necessary to keep up with spam. But it makes it hard to explain exactly what caused a specific user to get “shadowbanned” (sigh…) at a specific moment,” Roth continued.
“Anyway, if Twitter are rebuilding a bunch of the infrastructure that handles this, that seems like a good thing and I wish them luck,” he added, joking he left documents describing how to do it in his Google Drive at the company.
