At our online event, AMA: Software 2026, we’ll discuss all elements of the digital ecosystem around additive manufacturing. Register for free here.
A warning that a product could end civilisation is an unusual sales pitch. It advertises the product’s importance while raising an awkward question about the judgment of those selling it. For the companies developing artificial intelligence, it may also open a useful conversation about who else should be allowed into the business.
A recent claim by Evan Hubinger, a safety researcher at Anthropic that, “AI could kill all humans! I personally think it is >10% within the next decade” has been widely reported, and forms part of a longer history of extinction warnings. In 2023, Sam Altman, Dario Amodei and Demis Hassabis were among the signatories to the Center for AI Safety’s statement calling for the risk of extinction from AI to receive global attention alongside pandemics and nuclear war. The statement establishes their concern. It does not quantify the probability or prescribe a regulatory regime.
More recently, Cory Doctorow attacks the commercial usefulness of such alarm in “LLMs are real, AI is fake”. He argues that frightening accounts of AI capabilities help its developers raise investment. His immediate subject is the portrayal of the OpenAI/Hugging Face hacking incident as machines acquiring intentions. He directs readers towards failures of containment and supervision, and towards the companies responsible.
Doctorow does not develop an argument in that essay about extinction warnings being used to secure regulation that excludes smaller competitors. That is a separate question, and it deserves scrutiny without requiring agreement with his account of how language models work. A damaging automated attack needs no conscious attacker. Its occurrence does not, by itself, establish the likelihood of human extinction.
From extinction risk to market power
The political opportunity lies in the scale of the claim. If a technology threatens civilisation, ordinary product oversight may seem inadequate. Governments may seek exceptional powers over its development and turn to the leading laboratories for advice on exercising them. Those laboratories then have an opportunity to influence the conditions under which competitors can operate.
Their expertise is necessary. Their commercial interests deserve equal attention.
A rule requiring extensive testing before a powerful model is released might be entirely justified. It can nevertheless confer an advantage on a company that already employs the relevant specialists, owns the testing infrastructure and has sufficient revenue to absorb delays. A challenger must assemble that capacity while financing its attempt to catch up. The same formal obligation can impose very different competitive burdens.
There is evidence of interest in exceptional oversight. In May 2023, Altman, Greg Brockman and Ilya Sutskever proposed an international authority for superintelligence, comparable to the International Atomic Energy Agency, with powers to inspect systems, require audits and restrict deployment above a capability or computing threshold.
There is also an important qualification. The proposal explicitly opposed burdensome licences and audits for models below that threshold, including open-source projects. Such an approach could concentrate costs on the largest developers. It cannot fairly be described as a demand to license every start-up.
Much would depend on the details: how the threshold was set, whether independent assessors could challenge the largest laboratories’ judgments, and whether a new entrant had a workable route to approval. Rules that permit small firms to exist can still make it difficult for them to become serious rivals.
None of this proves that executives have invented an extinction threat to protect their businesses. A sincerely held fear can be commercially convenient. The public need not establish bad faith before examining who benefits from a proposed law. In 2024, Britain’s Competition and Markets Authority warned that powerful technology firms could shape foundation-model markets in their own interests. Safety rules will operate within those markets, with consequences for competition as well as risk.
What are the implications of AI for Additive Manufacturing?
For additive manufacturing, the connection begins with dependence on suppliers. An AM software business that builds a product around an external AI model inherits some of its provider’s commercial decisions. A market with fewer viable model suppliers could narrow its options on price, deployment and access. Restrictions on distributing model weights could also matter to manufacturers seeking to run software within their own facilities.
These are possible consequences of particular policies, not grounds for assuming that every AI rule burdens AM. A specialised defect-detection model and a frontier language model are different products. Regulation aimed at the latter would not automatically govern the former.
The closer parallel lies on the factory floor, where the cost of demonstrating safety can already influence the choice of supplier.
An additive manufacturing process is more than a machine producing the right shape. Its performance depends on material, build parameters and subsequent treatment, among other factors. Changing an element may change the evidence needed to establish that the finished component is suitable for its intended use.
The US Food and Drug Administration’s 2017 guidance on additively manufactured medical devices illustrates the point. It recommends assessing the risks introduced by changes and identifies software updates, material changes and altered build arrangements as possible triggers for revalidation. It also recognises workflows involving software from different suppliers. The engineering problem predates generative AI.
For a manufacturer with an established process, an alternative supplier’s offer therefore carries two prices: the purchase price and the cost of adopting it. A cheaper material or better build-preparation tool may be unattractive if the improvement cannot justify additional testing and production disruption.
That calculation can be perfectly rational. It does not establish that a supplier has captured a regulator. Customer requirements, industry standards and legal obligations are distinct, even when they produce similar commercial effects. An engineer who insists on evidence before changing an implant’s production process is doing the job.
When AI enters the build
AI can complicate that judgment. Consider a developer proposing software that identifies defects from images collected during a metal build. Flagging a suspect region for further inspection requires one level of assurance. Using the model’s output to release a component requires a much stronger case. A successful demonstration on one dataset will not answer every question about performance under different production conditions.
The developer also needs a policy for updates. A new model may improve average accuracy while becoming less reliable on a rare defect. Customers should be able to assess that change and retain an approved version where necessary. Neither automatic acceptance nor compulsory repetition of every previous test is a sensible default.
An established equipment supplier may have the production data, machine access and customer relationships needed to assemble the evidence. An independent software company may need permission to obtain them. Where the supplier also sells competing software, safety assurance and commercial exclusion can become difficult to disentangle.
Integration has benefits. One company responsible for the machine and its software can offer clearer accountability when something goes wrong. But the customer should examine whether that convenience depends on surrendering practical access to production records or accepting restrictions on competing tools. If replacing a supplier means losing the ability to interpret past builds, dependence has acquired an engineering justification.
Can standards keep the market open?
The alternative requires more than an appeal to openness. Common tests can give a small company a credible way to demonstrate competence without relying on its reputation. NIST’s AM-Bench programme, which provides publicly available measurements against which researchers can test manufacturing simulations, illustrates the value of shared evidence. A benchmark is not a production qualification, but it can reduce the amount of foundational work each developer must repeat.
Well-designed assurance can therefore help competition. Buyers need requirements tied to the consequences of failure, independent testing and records that remain usable when suppliers change. Manufacturers should be able to establish why a particular restriction is necessary and what evidence would allow an alternative. A small company should face demanding safety tests where the application warrants them; it should also have a fair chance of passing.
The danger for AM is that confidence in a production process becomes inseparable from loyalty to its supplier. That outcome requires neither a fabricated extinction forecast nor a conspiracy among vendors. It can emerge from individually reasonable decisions about risk, reinforced by rules whose competitive effects receive too little attention.
The laboratories warning about humanity’s future should expect scrutiny of the market they want to govern. Manufacturers considering AI have a more immediate question for their suppliers: once the system has been approved, what would it take to replace you?
Join AMA: Software 2026 on October 22nd to continue the conversation.
For regular AM news, insights, and original analysis, please subscribe to the 3D Printing Industry newsletter.
Featured image shows a 3D printed lampshade at Materialise HQ in Belgium. Photo by Michael Petch.
