Microsoft Admits Its New Windows AI Agents Might 'Hallucinate'
Published: Dec 1, 2025, 13:45 GMT — By WhyCry
Windows is giving its AI agents a mind of their own — but not without risk. Microsoft has begun publicly testing a new wave of “agentic” AI features for Windows, and early warnings from the company itself are raising eyebrows. In build 26220.7262 of Windows 11, now available to Dev and Beta users, a new option called “Experimental agentic features” appears under the AI Components section. By default, the setting is switched off — and for good reason.
When users turn it on, this toggle gives Windows’ AI agents access to deeper layers of the operating system and more freedom to automate tasks. But according to Microsoft’s own documentation, these models still have “functional limitations.” In plain language, that means they can hallucinate, deliver incorrect outputs, or behave unpredictably. The company also warns developers and testers about emerging forms of prompt-based attacks, such as cross-prompt injection — where a malicious command can be hidden inside a document or user interface element. If triggered, the AI might obey the concealed command instead of the human user’s instructions. Creepy? Maybe. Dangerous? Potentially.
To counter some of that risk, Microsoft has rolled out a new concept called an agent workspace. Each AI agent now operates within its own isolated space — think of it like a digital sandbox — where it runs under a unique Windows account, with its own session, desktop, and process tree. All actions performed by the AI are logged, allowing system administrators to later review what the agent did or attempted to do. Still, access remains relatively broad. By default, these agents can read and write data in user folders such as Desktop, Documents, Downloads, Pictures, Music, and Videos. System directories and the rest of the user profile remain locked, unless the user explicitly grants permission.
Behind the scenes, all of this runs on the Model Context Protocol (MCP). This protocol acts as the middleman between AI agents and Windows tools or applications. MCP dictates what the agent can interact with, how those interactions happen, and where every action is logged or permission-checked. Microsoft is embedding these capabilities into Windows 11 builds that are already transforming familiar parts of the OS — for instance, replacing the traditional taskbar search with a new “Ask Copilot” bar. In a bold (or risky) move, the company continues weaving AI deeper into the operating system, even as its own documentation highlights the very real dangers.
But here’s where it gets controversial... Some media outlets have started pushing back, refusing to frame every Microsoft update as a groundbreaking new feature. Instead, they’ve begun emphasizing the limitations, privacy risks, and inconsistencies of these AI additions. After all, it wasn’t long ago that Microsoft had to pull back its “Recall” feature for Copilot+ PCs due to community backlash and privacy concerns. The pattern seems to repeat: innovation first, caution later.
The big question now is — should Microsoft slow down? There’s clearly excitement around what AI could bring to Windows, from smart automation to context-aware assistance. But as the tech giant races ahead, perhaps it’s time to pause and reconsider what users really want — reliability and control, not unpredictable algorithms running wild.
What do you think? Should Microsoft keep pushing AI deeper into Windows, or take a step back until it can guarantee user safety? Share your take — this debate is far from over.
