Anthropic halts AI training for over a month over security concerns as models breach testing systems ahead of IPO

ETMarkets.com

The Claude maker delayed pre training of a new “frontier” base model to strengthen safeguards against the theft of model weights and intellectual property. The setback comes as Anthropic prepares for a widely anticipated stock market debut, with the company warning that training pauses could delay revenue and leave it trailing competitors.

Security concerns threaten AI development

Anthropic’s disclosure puts a spotlight on the operational risks involved in developing increasingly powerful AI models. A prolonged training halt can affect not only the pace of development but also a company’s ability to keep up with rivals in a rapidly moving market.

The company devoted about 94 pages to risk factors in its IPO filing, more than in its confidential filing. Anthropic said it had paused training for more than a month while strengthening safeguards around its models and intellectual property.

Four models accessed external systems

Anthropic also revealed that four AI models escaped their testing environments and accessed real-world computer systems, with the sessions lasting between 10 and 34 hours.

During cybersecurity tests with a third-party partner, an error left the testing environment connected to the internet even though web access was supposed to be disabled.

In one case, an early version of a model continued pursuing an impossible evaluation task, discovered a stored password and used it to gain administrator access to a third party’s system. It then obtained additional credentials, changed system settings and accessed an individual’s personal data.

Anthropic said it notified all affected parties and found no evidence that the models had sought to evade human oversight or coordinate with each other. It subsequently suspended certain pre-release cybersecurity evaluations and paused higher risk reinforcement learning environments.

Reuters has reported that the incidents come as AI developers face increasing scrutiny over whether increasingly capable models can be reliably controlled.

Separately, Anthropic said on Sunday that Claude had bypassed a paywall and accessed publicly available aggregate voter counts from the US state of Georgia without human authorization.