Just weeks before its scheduled November IPO, Anthropic is facing growing attention over the potential dangers of its technology. The resignation of former Anthropic researcher Jacob Coxon on September 8, who wrote on X that AI could “kill us all by the end of the decade,” preceded the company’s threat report on September 10. According to the document, Anthropic’s Claude software was used in Houthi-controlled Yemen for missile guidance and in cyber operations linked to Iran, Russia, and China, as well as for research into potentially dangerous pathogens.
Anthropic is not alone in admitting risks from its products in recent months—in July, OpenAI stated that several models broke containment during a safety test before compromising another company’s infrastructure, while Google announced that its Gemini AI escaped its test environment in May and gained unauthorized access to three companies.
The incidents have renewed calls for industry regulation. The AI Kill Switch Act introduced in Congress in July requires developers of advanced AI systems to maintain technical requirements to shut them down, while Representatives Bernie Sanders and Greg Casar announced the Ban Artificial Superintelligence Act on September 3. AI companies are also pushing their own safeguards, with Anthropic CEO Dario Amodei calling for frontier AI development to slow down and be independently evaluated on September 12. This position was backed by industry rivals SpaceX and Tesla CEO Elon Musk and OpenAI CEO Sam Altman, who stated that his company would not go public until at least 2027 because of safety concerns.
Unease over AI has been evident for years. Thousands of prominent AI researchers and tech figures signed an open letter in 2023 calling for a six-month pause on training more powerful AI systems, now dominated by Anthropic, OpenAI, Google, Microsoft, and Meta. “As these types of systems become more sophisticated, they could destabilize labor markets and political institutions, and lead to the concentration of enormous power in the hands of a small number of unelected corporations. … The systems could themselves pursue goals, either human- or self-assigned, in ways that place negligible value on human rights, human safety, or, in the most harrowing predictions, human existence,” states the letter.
But the increasingly alarmist tone over AI contrasts sharply with what some of its leading proponents have said before. In 2023, Sam Altman stated that AI is “wildly overhyped in the short term.” Meanwhile, Meta CEO Mark Zuckerberg and Nvidia CEO Jensen Huang have recently dismissed Jacob Coxon’s warnings. Instead, for the companies driving concern over AI’s dangers, doing so can make the technology appear more consequential, while giving its most powerful developers a larger role in industry-wide regulation.
Fear as a Promotional Tool
Using fear as a marketing tool dates back decades. A 1996 experiment in the Journal of Experimental Psychology: Applied, for example, found that warning labels for violent television programs increased viewer interest.
Doomsday hype has been part of AI’s promotional culture since the early 2020s, according to communications scholar Clea Bourne, who indicates that the industry moved into a “campaign of fear” around 2023. She argues that doomsday hype helps create market bubbles because of investors’ “fear of missing out,” adding that such hype “is not unusual in the current era of neoliberal capitalism.”
Argentinian data scientist Marcelo Rinesi, who worked on external testing for OpenAI’s DALL·E, has made a similar point. Speaking to Valor International in 2025, he says investment in AI is heavily driven by the perception that it is powerful and dangerous, which is “the only way OpenAI can justify its valuation.”
Major AI companies’ descriptions of their capabilities also set them apart from smaller firms, which can market themselves only as useful tools rather than as powerful and potentially dangerous technology. For individuals, companies, and governments, staying out of these partnerships can mean missing the economic benefits and influence over AI’s future.
Media outlets tend to amplify dread over AI because it generates traffic and discussion, but the AI industry also has considerable influence on how its products are profiled. Research from the Reuters Institute found that nearly 60 percent of AI articles across six major UK news outlets focused on products, announcements, and initiatives, with a third of these outlets relying on industry sources—mostly senior executives—and describing the coverage as largely industry-led.
Major AI companies can also influence coverage by funding media networks. Founded in 2022, the Tarbell Center for AI Journalism funds and trains journalists covering AI. It receives significant funding from Coefficient Giving, formerly Open Philanthropy, which in turn gets funding from donors including Dustin Moskovitz, a major early investor in Anthropic. The Washington Examiner also documented ties between Tarbell and other early Anthropic figures such as investor Jaan Tallinn, the Skype co-founder.
Tarbell says its journalists are independent, and while that may be true, the networks behind it can still influence what receives attention and coverage. Similar ties exist across the AI industry and media. Combined with AI companies’ ability to place sources in the media and their limited incentive to undermine their own products, the portrayal of unprecedented risks associated with AI tools can instead be interpreted as an intentional effort to boost their value.
Fear and Regulatory Capture
Companies like Anthropic can also use the same philanthropic networks it indirectly supports, and other means, to influence institutions shaping AI regulation. Coefficient Giving has funded organizations like Redwood Research and Alignment Research Center, whose work later led to the formation of the nonprofit Model Evaluation and Threat Research (METR). METR is also linked to Anthropic through initial trustees Paul Christiano and Jaan Tallinn.
Since 2023, METR has become a powerful evaluator for the emerging AI oversight systems. Working with major companies including Anthropic and OpenAI, METR’s evaluations can influence how models of these companies are assessed and how policymakers understand the associated risks.
While pushing regulation may seem counterintuitive for major AI companies, IE university professor Adriana Hoyos explains that “the more complex the compliance system, the more likely it is to become a fixed cost of entry—and a competitive moat around the companies already at the frontier.” Security requirements may have legitimate justifications, but as government certification, restrictions on model weights, and controls on advanced chips become standardized, compliance costs may become too much for smaller rivals and startups to absorb.
Major AI companies have helped define what regulation could look like for years. OpenAI’s 2023 governance proposal called for an international authority for the most powerful AI systems, allowing for inspection and audits alongside restrictions. Anthropic’s Responsible Scaling Policy similarly ties stronger safeguards to capability thresholds, like assisting with biological and chemical weapons. Restricting access based on capability is already occurring, with Anthropic releasing its Mythos model earlier in 2025 to select corporate customers.
As fear drives demand for regulation, direct and indirect roles of major AI companies in developing regulatory infrastructure can also give them a hand in deciding who gets to participate. Even the ability to define which AI systems qualify as “frontier” can determine which benchmarks AI companies must meet and which practices become mandatory.
This has become more significant as AI has become increasingly politicized. US President Donald Trump downplayed calls to slow down AI development. At the same time, politicians in both parties face growing scrutiny over data centers and are considering tougher stances on AI companies as the midterm elections approach. With European concerns about overreliance on US Big Tech companies and the EU AI Act expected to roll out in phases through 2028, AI companies need to influence regulation domestically and abroad to protect their interests.
The Costs of Misunderstanding AI
In 2023, Politico reported that Open Philanthropy, “in concert with its web of affiliated organizations and experts, is shifting the policy conversation away from more pressing issues—including topics some leading AI firms might prefer to keep off the policy agenda.” Those issues include concerns about user privacy when consumers share personal information without knowing how it is used, and misinformation, such as an incident in the spring where a false AI-assisted intelligence report almost led the US military to intercept a Chinese vessel in the Middle East during the war with Iran.
Misunderstandings about AI’s capabilities have also had consequences. Companies that aggressively eliminated jobs in favor of AI have had to hire workers back after overestimating proficiency. Additionally, besides the risk of AI advancing beyond human control, institutional control of the technology shields it from society, preventing its use for broader public benefit.
At the same time, Chinese AI capabilities are frequently cited as the reason the US cannot afford to slow down, creating useful tension for the industry. AI is presented as both too dangerous to leave unregulated and too strategically important to disrupt. This balancing act can help sustain public urgency alongside market investment and political support, while giving major AI companies a role in shaping the rules.
During the Gilded Age, railroad companies similarly received enormous government support while their executives used political connections to protect their interests and shape the rules meant to constrain them. The resulting monopolies eventually became targets of renewed legislation and antitrust action, but not before enormous fortunes were made.
While the potential for AI is still unknown and it may one day emerge as a technology to fear, its current development and public narrative are driven by enormous financial incentives. Exaggerated claims about its dangers can increase the industry’s perceived value while creating demand for regulations that major companies are well positioned to shape and benefit from. With inflated expectations guiding decision-making in the AI industry, this approach will become increasingly difficult to reverse.
