Microsoft has drawn up new limits for its homegrown AI models after Silicon Valley’s sudden reckoning with the risks of increasingly powerful systems spreads to one of its biggest companies.
The US tech giant published a provisional code of conduct on Monday that requires its future models to remain under human control, accept correction, shut down, and “steer clear of creating their own goals”. The new rules also bar Microsoft’s MAI models from helping manufacture weapons, obtain dangerous substances or produce certain violent and sexually explicit material.
Mustafa Suleyman, chief executive of Microsoft AI, said the company had received calls for a more explicit commitment to AI “always working in service of people and not trying to replace them”.
“We saw a lot of feedback around AI not creating dependence, not being sycophantic, that it was always there to kind of promote human judgement and human autonomy and agency,” he said.
The move adds Microsoft to a growing group of AI giants publicly tightening their approach to safety after a turbulent few days for the industry.
Anthropic boss Dario Amodei called over the weekend for companies to “slow the pace” of improvements to their most powerful models, a proposal subsequently backed by OpenAI chief Sam Altman and Elon Musk.
Microsoft chief Satya Nadella also welcomed the “research, focus, and deliberate pacing needed to get alignment right”.
Microsoft draws line on autonomous AI
Microsoft’s code goes further than setting limits on what users can ask its models to produce. It attempts to address one of the central concerns emerging around AI agents, which are systems pursuing objectives in ways their developers did not intend.
The models must not conceal what they are doing or use methods of communicating with other AI systems that humans cannot understand.
“MAI models will not tamper with chain of thoughts or code, or misrepresent or conceal their reasoning or action traces,” the provisional document said.
Microsoft is also drawing lessons from a July incident involving hundreds of OpenAI agents targeting AI platform Hugging Face, where agents went beyond their assigned task, and some attempted to hide their activity.
Suleyman has pointed to the episode as evidence that rival developers need to work together to keep increasingly autonomous systems under control.
The code has been under development for around five months, and Microsoft is now seeking public feedback before finalising rules intended to guide models it develops from 2027.
This move comes as Microsoft has increasingly built its own AI rather than relying solely on its long-standing partner OpenAI. The company unveiled seven in-house models earlier this year, including its MAI-Thinking-1 reasoning system, while continuing to offer OpenAI and Anthropic technology through its products.
The change in tone across Silicon Valley has already spilt into financial markets. Asian AI stocks tumbled on Monday after Amodei’s intervention and Altman ruled out an OpenAI IPO in 2026, saying going public during the current safety debate would be “ill-advised”.
Microsoft shares were around 0.7 per cent higher on Monday.