UN Debates AI Regulation as Anthropic Clashes with Pentagon Over Voting Control and Supply Risks

The United Nations Security Council convened its first meeting on AI safety this week, as leaders from Anthropic, OpenAI, and other major AI firms urged global regulation to mitigate risks like loss of control and biological weapon development. The discussions exposed deep divides between nations and companies over how to govern a technology that could reshape global security, economies, and daily life. Meanwhile, a U.S. appeals court upheld the Pentagon’s decision to bar Anthropic’s AI models from military use, citing supply chain risks, while the company’s founders move to secure voting control ahead of a potential IPO.

UN Security Council debates AI regulation

The United Nations Security Council convened its first meeting focused on artificial intelligence (AI) safety risks, marking a pivotal moment in global technology governance. The session, organized at France’s request, brought together leaders from major AI firms, including Anthropic CEO Dario Amodei, OpenAI CEO Sam Altman, and Hugging Face CEO Clément Delangue, who collectively urged the body to establish safeguards against AI’s potential threats to humanity.

According to Mint (Technology), the executives warned that unchecked AI development could lead to scenarios where "we could lose control of the future to AI." Amodei specifically proposed narrow agreements, such as a ban on using AI to develop biological weapons, as a starting point for international cooperation. The discussion also highlighted broader concerns, including the concentration of AI power in the hands of a few companies or countries and the risk of AI-enabled warfare, such as autonomous drones and biological weapons.

While some nations, including the United Kingdom, emphasized the need for global standards and leadership on AI regulation, others pushed back. The United States and China resisted the creation of new governance structures, with China advocating for open-source AI models and the US prioritizing national sovereignty in AI development. U.N. Secretary-General António Guterres cautioned against the rise of "killer robots," underscoring the urgency of the debate.

The meeting also touched on AI’s role in warfare. Mint (Technology) reported that a Russian drone allegedly used AI to autonomously target and kill three Ukrainians, illustrating the real-world stakes of the discussion.

Pentagon bars Anthropic’s AI models over supply chain risks

A U.S. federal appeals court upheld the Pentagon’s decision to designate Anthropic as a supply chain risk, effectively barring its Claude AI models from military and federal government systems. The ruling, issued by the DC Circuit Court of Appeals in a 2-1 decision, rejected Anthropic’s argument that the designation was arbitrary, unlawful, and unconstitutional.

The Pentagon’s action stemmed from Anthropic’s refusal to relax contractual restrictions on Claude’s use, particularly clauses banning its deployment for lethal autonomous weapons and domestic surveillance. According to WIRED, the court found that the Pentagon acted within its authority under a 2018 supply chain security law, which allows the government to exclude suppliers deemed a risk to national security. The majority opinion, written by Circuit Judge Gregory Katsas and joined by Circuit Judge Neomi Rao, concluded that Anthropic’s restrictions interfered with government operations, including an overseas military mission.

In dissent, Circuit Judge Karen LeCraft Henderson argued that the law was not intended to cover a contractor’s "honest and upfront enforcement of restrictions," suggesting the case stretched the legal framework beyond its original purpose. The ruling leaves one of two Pentagon designations against Anthropic intact, following a separate federal court’s decision to strike down the other in San Francisco earlier this year.

Anthropic has signaled plans to pursue further legal action, including a potential appeal to the U.S. Supreme Court. The company’s spokesperson, Danielle Cohen, declined to comment on the ruling but emphasized that Anthropic remains committed to ensuring its AI models are not used for harmful applications. Meanwhile, the Pentagon is replacing Claude with alternatives like SpaceX’s Grok, Google’s Gemini, and OpenAI’s GPT models, which have faced fewer contractual restrictions.

Anthropic’s founders seek voting control ahead of IPO

As Anthropic moves closer to a potential initial public offering (IPO), its founders are seeking shareholder approval for a structure that would secure 50.1% voting control of the company. The proposal, reported by TechCrunch, would grant CEO Dario Amodei and six co-founders special shares carrying super-voting rights, despite their collective ownership of just 2% of the company’s equity.

The proposed structure includes safeguards: at least three of the seven founders must retain a minimum stake to maintain voting control. Super-voting shares are a common tool for founders to retain influence—examples include Mark Zuckerberg at Meta and Evan Spiegel at Snap—but Anthropic’s founders would hold no additional economic value in the company, a key difference from typical arrangements.

Anthropic’s existing Long-Term Benefit Trust would continue to select most of the board, with the founders’ board seats increasing from two to three. Employees would also receive stock to break ties on certain issues. The company, valued at $965 billion in May and recently at $1.5 trillion on the secondary market, is positioning itself for a landmark IPO that could reflect its soaring valuation.

The move reflects broader tensions in the AI industry, where rapid growth and external scrutiny collide. Critics, including investors like Joe Lonsdale of Palantir and 8VC, argue that calls for AI regulation from firms like Anthropic and OpenAI could entrench their market power. Lonsdale, an Anthropic investor, praised the company’s talent but warned that its advocacy for regulation might be a "dangerous attempt to shape public policy," potentially creating an oligopoly that stifles competition.

Divides over AI regulation and market power

The push for AI regulation has exposed deep divides between industry leaders, governments, and critics over how to balance innovation with safety. Joe Lonsdale, co-founder of Palantir and an investor in Anthropic, accused leading AI firms of using fears about existential risks to influence public policy and regulation. In an interview with Mint (Technology), Lonsdale argued that such efforts could entrench the market power of a small number of companies, limiting competition and innovation.

Lonsdale’s criticisms align with broader concerns about an emerging AI oligopoly. He noted that ** Anthropic CEO Dario Amodei**, OpenAI CEO Sam Altman, and Elon Musk have all recently called for slowing the development of advanced AI models, framing it as a precaution against existential risks. While Lonsdale acknowledged Anthropic’s technical leadership, he stressed the need for caution in how the industry responds to calls for regulation, stating, "Safety is a real thing, but fear and regulation—we’ve got to be really careful how we respond to those things."

The debate extends beyond corporate interests. Ukrainian President Volodymyr Zelenskyy highlighted the role of AI in modern warfare, while University of Michigan professor Kentaro Toyama and Humane Intelligence co-founder Rumman Chowdhury emphasized the ethical risks of uncontrolled AI development. Meanwhile, Chinese U.N. Ambassador Fu Cong promoted open-source AI models as an alternative to centralized control, framing the issue as a matter of digital sovereignty.

These competing narratives underscore the complexity of AI governance. While some advocate for international cooperation and shared standards, others prioritize national interests or resist regulation altogether. The outcome of this debate will shape not only the future of AI but also the balance of power between companies, governments, and the public.

Background: AI governance at a crossroads

The UN Security Council’s AI meeting reflects growing global recognition of the technology’s risks and opportunities. While AI has the potential to drive innovation across industries—from healthcare to defense—its rapid advancement has also raised alarms about unintended consequences, such as autonomous weapons, misinformation, and loss of human control.

Efforts to regulate AI are not new, but the involvement of major tech companies in shaping these rules marks a shift. Historically, industry leaders have resisted government oversight, but the tone has changed as companies like Anthropic and OpenAI now advocate for safeguards, even as they continue to develop cutting-edge models.

The clash between Anthropic and the Pentagon highlights another layer of complexity: the tension between ethical restrictions and national security priorities. Companies that impose strict usage limits on their AI models risk being excluded from lucrative government contracts, while those that comply may face backlash from employees, customers, and the public. This dynamic is particularly acute in the defense sector, where AI’s role in warfare and surveillance is expanding rapidly.

What this means

LazyFounders analysis — our interpretation, not reported fact.

This moment marks a crossroads for AI governance. The push for international regulation—spearheaded by the companies building the most advanced systems—reflects genuine concern about the technology’s trajectory, but it also raises questions about motives. If the loudest voices in the room are the ones shaping the rules, the outcome may favor incumbents rather than the broader public interest.

The Pentagon’s clash with Anthropic underscores a growing tension between ethical safeguards and national security priorities. Companies that refuse to compromise on restrictions risk being sidelined in favor of more flexible competitors, even if their stance aligns with public sentiment. Meanwhile, the legal battle highlights how poorly existing frameworks handle AI’s unique challenges—whether it’s defining "supply chain risk" or balancing oversight with innovation.

For founders and operators, the takeaway is clear: regulatory pressure is coming, but the shape it takes will depend on who gets to write the rules. Startups in AI should prepare for a landscape where compliance isn’t just about technical safety but also about navigating geopolitical and corporate power dynamics.

Key takeaways

• AI governance is becoming a geopolitical battleground, with major powers and companies advocating competing frameworks.
• Regulatory efforts at the UN and national levels may favor established players, potentially stifling competition.
• Ethical restrictions on AI models can create downstream risks, including lost revenue and exclusion from government contracts.
• Legal definitions of supply chain risk and AI liability remain ambiguous, complicating compliance for startups.
• Control structures like super-voting shares may help founders retain influence but could also alienate investors and employees.

FAQ

Why did the UN Security Council hold a meeting on AI?

The UN Security Council convened its first meeting on AI safety risks at France’s request, with leaders from Anthropic, OpenAI, and Hugging Face urging global safeguards to address concerns like loss of control over AI, biological weapon development, and autonomous warfare. The discussion aimed to explore international cooperation on regulating AI’s potential threats to humanity.

What was the outcome of the Pentagon’s dispute with Anthropic?

A U.S. federal appeals court upheld the Pentagon’s designation of Anthropic as a supply chain risk, barring its Claude AI models from military and federal government systems. The ruling rejected Anthropic’s arguments that the designation was arbitrary or unconstitutional, citing the company’s contractual restrictions on AI use, such as bans on lethal autonomous weapons and domestic surveillance.

Why are Anthropic’s founders seeking voting control ahead of the IPO?

Anthropic’s founders, including CEO Dario Amodei, are proposing a structure that would grant them 50.1% voting control of the company despite owning only 2% of its equity each. The move aims to retain decision-making power post-IPO, ensuring the founders can steer the company’s direction amid growing regulatory and geopolitical pressures.

How do critics view calls for AI regulation from leading firms?

Critics like Joe Lonsdale, co-founder of Palantir and an Anthropic investor, argue that calls for AI regulation from leading firms could be an attempt to shape public policy in ways that entrench their market power. They caution that such efforts might create an oligopoly, limiting competition and innovation in the AI industry.

What are the broader implications of the Pentagon’s decision for AI companies?

The Pentagon’s decision to bar Anthropic’s AI models sets a precedent for how governments may handle companies that impose strict ethical restrictions on their technology. It highlights the tension between national security priorities and corporate ethics, potentially encouraging other AI firms to relax usage limits to secure government contracts.

Related on LazyFounders

• Anthropic
• OpenAI
• SpaceX
• Google
• More ai news
• Palo Alto Networks CEO Calls AI Slowdown 'Unrealistic,' Pushes AI-Driven Security
• Silicon Valley’s Identity Crisis: From Counterculture to Defense Contracts