How Bots Took Over Our Lives

This strategy of manufacturing a froth of interest around a given subject, from Nicki Minaj’s political perspectives to Sydney Sweeney’s jean ads, has recently been especially effective, and has moved culture-war fights from the trenches of bot-infested social networks into more mainstream channels. A New York Times investigation into the social media fracas found that there were relatively few mentions of Sweeney on X until several days after the début of her infamous American Eagle commercial, and less than ten per cent of those early posts were directly critical. It wasn’t until a couple of posts that compared the ad to Nazi eugenics propaganda ended up going viral that the outrage cycle began, and conservative influencers attempted to paint what was essentially a fringe perspective on the ad campaign as a liberal consensus.

“These episodes follow a common pattern: a small incident is framed as ‘proof’ of a broader culture war, then a mix of real users, coördinated accounts, and bots push the same talking points and narratives until it looks like a widespread response,” Emilio Ferrara, a computer-science professor at U.S.C., told me. Extreme voices with high follower counts, such as Chaya Raichik, the woman behind the right-wing X account Libs of TikTok, are usually at the center of the skirmish, and then bot networks swarm to boost the message. It’s like pouring gasoline onto a few embers, which then appears to be a bonfire. Lynnette Ng, a former bot-network researcher at Carnegie Mellon University, explained why the swarm is so effective: “Bots frequently and aggressively hit cognitive-bias triggers in their messages which push the brain to short-circuit people’s ability to reason about the information in front of them.” It doesn’t help that people struggle to decipher bots from humans in the first place during political conversations. The default heat setting for political discourse online is already so high that it tends to pressurize such conversations into a single, highly combustible layer. Bots that are designed to inflame those political disagreements are trained on those conversations and then injected back into the discourse, amplifying the feedback loop of people yelling at bots who are yelling at people, and so on.

This is easy enough to understand; bots are like automated moths to digital flame. But the rationale propelling them is a little harder to decipher. Surely, some of the motive is financial, as engagement on social-media platforms has been monetized. On X, for example, an account is able to participate in X’s revenue-sharing program as long as it is “verified”—which really just means that the owner has paid a small fee for a premium account—and the page meets minimum metrics such as having five hundred verified followers and a certain number of recent “organic” impressions. (“Organic” here means that you didn’t pay for your posts to be promoted like an ad.) There’s no quality incentive here; as long as a post is attracting eyeballs and generating ad impressions, there is a potential for payout. A bot that is programmed to maximize engagement and to interact with accounts that have lots of followers can make money for its operator by appearing next to ads in the X timeline. A.I. has made that feedback loop spin only faster: bots now learn what sort of content drives the most engagement and adjust their outputs to generate likes, reposts, and comments.

Unfortunately, the posts that get the most engagement are invariably the ones that upset us the most. Rage-baiting, as this strategy is often called, is an effective behavior to program into a bot’s temperament, since people are more likely to engage with social-media posts that challenge their beliefs than those that confirm them. Maintaining a system powered by invective discourse also seems to be a practical business model for social-media platforms. A group of researchers found that removing toxic posts from a user’s feed reduced the amount of time they spent on a given platform. The total time that users spend mindlessly scrolling is the lifeblood of social-media outlets, meaning, as Ferrara explained, that they are dealing with a “conflicting system of incentives,” since bots are good for engagement metrics that can be attractive to advertisers but also mar user experience past a certain threshold. Keeping things just toxic enough so that users scroll without getting too upset at the state of the world is the sweet spot for platforms. It’s cynical, but it’s also very good business.