Anthropic IPO: Firm uses ‘catastrophic’ word 10 times in document to highlight ‘humanitarian’ AI risks

Anthropic’s IPO filing devotes more than 90 pages to risk factors, accounting for roughly a third of the overall document, Axios says. Among these are warnings about the potential risks associated with developing and deploying its own AI technology.

10 instances ‘catastrophic’ was used

1.) Concerns have been raised both within the organization and by other researchers, regulators, policymakers, and industry participants that sufficiently advanced AI systems could, if not developed, deployed, and operated with appropriate safeguards, oversight, and security measures, pose catastrophic or existential risks to humanity, particularly if such systems were misused, replicated, weaponized, or allowed to operate with a high degree of autonomy.

2.) Anthropic’s insurance policies may not provide sufficient coverage to adequately mitigate the potential liabilities it faces, especially where any one incident, or a group of incidents, causes disproportionate harm. In addition, with the scale of harm that could result from a catastrophic AI-related incident, no single insurer, and potentially the insurance industry in the aggregate, may have sufficient capacity or expertise to underwrite insurance commensurate with such exposure.

3.) As AI systems grow more capable, the potential harm from their misalignment or misuse also increases. Researchers, policymakers and other stakeholders have warned that advanced AI systems could pose catastrophic or existential threats to society if they are not developed and deployed responsibly.

4.) The uninterrupted availability of its models, platform and applications rely on a dependable supply of electricity and water to the facilities hosting infrastructure, whether managed by its cloud service providers (CSPs) or operated on Anthropic’s behalf. Power or water outages, supply shortages, rising utility costs or mandatory electricity curtailments during peak demand could disrupt or impair services in the normal course of business, regardless of whether they arise from natural disasters, other catastrophic events or other circumstances.

5.) As the business has expanded and interest in the sector has grown, Anthropic has faced, and may continue to face, substantial media attention, public commentary and scrutiny. This coverage has frequently focused on the potential negative societal consequences of AI, including job displacement, misuse or other forms of harm, and the possibility that advanced AI systems could pose catastrophic or existential threats to humanity. “We expect these narratives to remain a recurring feature of public discourse,” the company said.

6.) Our operating results may also fluctuate due to several additional factors, many of which are beyond our control and difficult to predict. These include catastrophic events and any resulting disruptions to the availability of our models, platform and applications, as well as general economic conditions in the US and international markets, including geopolitical uncertainty and instability and their impact on software spending.

7.) Wars, terrorist attacks, political violence and natural disasters, including earthquakes, fires, hurricanes, tornadoes, floods and major power outages, could disrupt operations, mobile networks, internet connectivity, data centres, and the activities of partners and third-party service providers. Climate change may further heighten these risks. Its corporate headquarters are located in the San Francisco Bay Area, which is prone to seismic activity and faces a growing threat from fires. Public health emergencies, pandemics, political crises, wars, terrorist attacks, social unrest, geopolitical developments and other catastrophic events, whether in the United States or elsewhere, could also adversely affect our operations and broader macroeconomic conditions.

8.) The company cautions that investors could lose their entire investment in Class A shares due to major catastrophic events, including wars, terrorist attacks, pandemics or responses to such events or the appointment or departure of founders, board members, executives and other key personnel could affect the business.

9.) “Our Responsible Scaling Policy outlines our approach to the catastrophic risks associated with AI. It requires us to regularly publish detailed risk reports assessing the overall level of risk posed by our models and explaining the measures we are taking to mitigate these risks,” the SEC filing said.

10.) Before releasing any model, the teams conduct extensive testing to assess its capabilities in areas most relevant to catastrophic risks and determine the extent to which it could be misused.

Valuation has also emerged as a key point of debate ahead of the listing. Some investors had previously discussed a valuation of around $2 trillion, but expectations have moderated following the latest financial disclosures and concerns about the pace of revenue growth.