Updated April 4, 2025: We have updated the rollout timeline below. Thank you for your patience.
Custom engine agents and Teams bots can now stream responses like Microsoft Copilot. They can provide real-time updates on their actions while delivering parts of the response as they generate it, making interactions feel faster and more engaging.
This message is associated with Microsoft 365 Roadmap ID 422261
When this will happen:
Targeted Release: We will begin rolling out mid-November 2024 and expect to complete by late November 2024.
General Availability (Worldwide): We will begin rolling out mid-December 2024 and expect to complete by late April 2025 (previously late March).
How this will affect your organization:
When users interact with a custom engine agent or Teams bot that streams its response, they will experience the new UX.
Users will first see informative updates from the agent on what it is doing while it generates a response to the user’s message:
Once the agent has performed prerequisite actions and is ready to respond, users will start to see the agent’s response flow in as incremental chunks until the full response is available. This experience is very similar to how the response iteratively appears on leading AI experiences like ChatGPT or Copilot:
Users will have the option to stop generating the response if they feel the agent is headed down an ineffective path. Users can simply click on the stop button underneath the response, and this will stop the agent’s response. Users can then prompt the agent again:
What you need to do to prepare:
You may consider updating your training and documentation as appropriate.