Allow models that only support json_object to also support structured output

What happened?

Describe the bug 描述 bug
Currently, JSON Output of deepseek already supports json_object, but does not support json_schema. However, it is still possible to generate JSON results as required by deepseek and generate structured output based on it.
目前，deepseek 的 JSON 输出已经支持 json_object，但不支持 json_schema。但是，仍然可以根据 deepseek 的要求生成 JSON 结果，并基于它生成结构化输出。
Because OpenAIChatCompletionClient strictly checks the structured_output support of the model, it is impossible to generate structured output when using deepseek.
由于 OpenAIChatCompletionClient 严格检查模型的 structured_output 支持，因此在使用 deepseek 时无法生成结构化输出。

In addition, when setting AssistantAgent's output_content_type to True and fixing the output_content_type detection code, it is possible to generate JSON results instead of json code blocks.
此外，当将 AssistantAgent 的 output_content_type 设置为 True 并修复 output_content_type 检测代码时，可以生成 JSON 结果而不是 json 代码块。

Not sure if this is the right place to submit this issue. If not, the issue will be deleted.
不确定这是否是提交此问题的正确位置。否则，该问题将被删除。

To Reproduce 重现
Use the following code to generate the structured output:
使用以下代码生成结构化输出：

import asyncio
from typing import Literal
from pydantic import BaseModel
from autogen_agentchat.agents import AssistantAgent
from autogen_agentchat.ui import Console
from autogen_ext.models.openai import OpenAIChatCompletionClient

class AgentResponse(BaseModel):
    thoughts: str
    response: Literal["happy", "sad", "neutral"]


async def main() -> None:
    model_client = OpenAIChatCompletionClient(
        model="deepseek-chat",
        base_url="https://api.deepseek.com/v1",
      # api_key="your_api_key",
        model_capabilities={
            "vision": False,
            "function_calling": True,
            "json_output": True,
            "structured_output": False,
        },
        # response_format=AgentResponse, # type: ignore
        # response_format={"type": "json_object"},
        # response_format={"type": "text"},
    )
    agent = AssistantAgent(
        "assistant", 
        model_client=model_client, 
      # model_client_stream=True, 
        system_message="""
        Categorize the input as following the JSON format.
        {
          "thoughts": "thoughts",
          "response": "happy, sad, or neutral"
        }
        """,
        output_content_type=AgentResponse,
        # output_content_type=True,
        # output_content_type=False,
    )
    await Console(agent.run_stream(task="I am happy.")) # nice, bad, hunger

asyncio.run(main())

The following error occurs when running:
运行时出现以下错误：

$ uv run python test_structured_output.py 
---------- TextMessage (user) ----------
I am happy.
Traceback (most recent call last):
  File "/data/gym/zkdh-006-code-00/projects/ai_agents/test_structured_output.py", line 45, in <module>
    asyncio.run(main())
  File "/home/dihuge/.local/share/uv/python/cpython-3.10.16-linux-x86_64-gnu/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/home/dihuge/.local/share/uv/python/cpython-3.10.16-linux-x86_64-gnu/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
    return future.result()
  File "/data/gym/zkdh-006-code-00/projects/ai_agents/test_structured_output.py", line 43, in main
    await Console(agent.run_stream(task="I am happy.")) # nice, bad, hunger
  File "/data/gym/zkdh-006-code-00/projects/ai_agents/.venv/lib/python3.10/site-packages/autogen_agentchat/ui/_console.py", line 117, in Console
    async for message in stream:
  File "/data/gym/zkdh-006-code-00/projects/ai_agents/.venv/lib/python3.10/site-packages/autogen_agentchat/agents/_base_chat_agent.py", line 175, in run_stream
    async for message in self.on_messages_stream(input_messages, cancellation_token):
  File "/data/gym/zkdh-006-code-00/projects/ai_agents/.venv/lib/python3.10/site-packages/autogen_agentchat/agents/_assistant_agent.py", line 852, in on_messages_stream
    async for inference_output in self._call_llm(
  File "/data/gym/zkdh-006-code-00/projects/ai_agents/.venv/lib/python3.10/site-packages/autogen_agentchat/agents/_assistant_agent.py", line 981, in _call_llm
    model_result = await model_client.create(
  File "/data/gym/zkdh-006-code-00/projects/ai_agents/.venv/lib/python3.10/site-packages/autogen_ext/models/openai/_openai_client.py", line 594, in create
    create_params = self._process_create_args(
  File "/data/gym/zkdh-006-code-00/projects/ai_agents/.venv/lib/python3.10/site-packages/autogen_ext/models/openai/_openai_client.py", line 496, in _process_create_args
    raise ValueError("Model does not support structured output.")
ValueError: Model does not support structured output.

Relax the detection of OpenAIChatCompletionClient on structured_output and modify it as follows:
放宽 OpenAIChatCompletionClient 在 structured_output 上的检测，并按如下方式进行修改：

_openai_client.py#L471

              # if self.model_info["structured_output"] is False:
                if self.model_info["structured_output"] is False and self.model_info["json_output"] is False:

_openai_client.py#L495

              # if self.model_info["structured_output"] is False:
                if self.model_info["structured_output"] is False and self.model_info["json_output"] is False:

_openai_client.py#L607

                  # response_format=create_params.response_format,
                    response_format=create_params.response_format if self.model_info["structured_output"] else ResponseFormatJSONObject(type="json_object"),

_openai_client.py#L1037

          # response_format=(response_format if response_format is not None else NOT_GIVEN),
            response_format=((response_format if self.model_info["structured_output"] else ResponseFormatJSONObject(type="json_object")) if response_format is not None else NOT_GIVEN),

Running the code again generates the structured output correctly:
再次运行代码会正确生成结构化输出：

$ uv run python test_structured_output.py 
---------- TextMessage (user) ----------
I am happy.
---------- StructuredMessage[AgentResponse] (assistant) ----------
{"thoughts":"The user is expressing a positive emotion.","response":"happy"}

Use the following code to generate the JSON output (instead of json code blocks):
使用以下代码生成 JSON 输出（而不是 json 代码块）：

import asyncio
from typing import Literal
from pydantic import BaseModel
from autogen_agentchat.agents import AssistantAgent
from autogen_agentchat.ui import Console
from autogen_ext.models.openai import OpenAIChatCompletionClient

class AgentResponse(BaseModel):
    thoughts: str
    response: Literal["happy", "sad", "neutral"]


async def main() -> None:
    model_client = OpenAIChatCompletionClient(
        model="deepseek-chat",
        base_url="https://api.deepseek.com/v1",
      # api_key="your_api_key",
        model_capabilities={
            "vision": False,
            "function_calling": True,
            "json_output": True,
            "structured_output": False,
        },
        # response_format=AgentResponse, # type: ignore
        # response_format={"type": "json_object"},
        # response_format={"type": "text"},
    )
    agent = AssistantAgent(
        "assistant", 
        model_client=model_client, 
      # model_client_stream=True, 
        system_message="""
        Categorize the input as following the JSON format.
        {
          "thoughts": "thoughts",
          "response": "happy, sad, or neutral"
        }
        """,
        # output_content_type=AgentResponse,
        output_content_type=True,
        # output_content_type=False,
    )
    await Console(agent.run_stream(task="I am happy.")) # nice, bad, hunger

asyncio.run(main())

The following error occurs when running:
运行时出现以下错误：

$ uv run python test_structured_output.py 
Traceback (most recent call last):
  File "/data/gym/zkdh-006-code-00/projects/ai_agents/test_structured_output.py", line 45, in <module>
    asyncio.run(main())
  File "/home/dihuge/.local/share/uv/python/cpython-3.10.16-linux-x86_64-gnu/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/home/dihuge/.local/share/uv/python/cpython-3.10.16-linux-x86_64-gnu/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
    return future.result()
  File "/data/gym/zkdh-006-code-00/projects/ai_agents/test_structured_output.py", line 28, in main
    agent = AssistantAgent(
  File "/data/gym/zkdh-006-code-00/projects/ai_agents/.venv/lib/python3.10/site-packages/autogen_agentchat/agents/_assistant_agent.py", line 688, in __init__
    self._structured_message_factory = StructuredMessageFactory(
  File "/data/gym/zkdh-006-code-00/projects/ai_agents/.venv/lib/python3.10/site-packages/autogen_agentchat/messages.py", line 340, in __init__
    self.StructuredMessage = StructuredMessage[self.ContentModel]  # type: ignore[name-defined]
  File "/data/gym/zkdh-006-code-00/projects/ai_agents/.venv/lib/python3.10/site-packages/pydantic/main.py", line 911, in __class_getitem__
    submodel = _generics.create_generic_submodel(model_name, origin, args, params)
  File "/data/gym/zkdh-006-code-00/projects/ai_agents/.venv/lib/python3.10/site-packages/pydantic/_internal/_generics.py", line 128, in create_generic_submodel
    created_model = meta(
  File "/data/gym/zkdh-006-code-00/projects/ai_agents/.venv/lib/python3.10/site-packages/pydantic/_internal/_model_construction.py", line 237, in __new__
    complete_model_class(
  File "/data/gym/zkdh-006-code-00/projects/ai_agents/.venv/lib/python3.10/site-packages/pydantic/_internal/_model_construction.py", line 597, in complete_model_class
    schema = gen_schema.generate_schema(cls)
  File "/data/gym/zkdh-006-code-00/projects/ai_agents/.venv/lib/python3.10/site-packages/pydantic/_internal/_generate_schema.py", line 711, in generate_schema
    schema = self._generate_schema_inner(obj)
  File "/data/gym/zkdh-006-code-00/projects/ai_agents/.venv/lib/python3.10/site-packages/pydantic/_internal/_generate_schema.py", line 1004, in _generate_schema_inner
    return self._model_schema(obj)
  File "/data/gym/zkdh-006-code-00/projects/ai_agents/.venv/lib/python3.10/site-packages/pydantic/_internal/_generate_schema.py", line 837, in _model_schema
    {k: self._generate_md_field_schema(k, v, decorators) for k, v in fields.items()},
  File "/data/gym/zkdh-006-code-00/projects/ai_agents/.venv/lib/python3.10/site-packages/pydantic/_internal/_generate_schema.py", line 837, in <dictcomp>
    {k: self._generate_md_field_schema(k, v, decorators) for k, v in fields.items()},
  File "/data/gym/zkdh-006-code-00/projects/ai_agents/.venv/lib/python3.10/site-packages/pydantic/_internal/_generate_schema.py", line 1206, in _generate_md_field_schema
    common_field = self._common_field_schema(name, field_info, decorators)
  File "/data/gym/zkdh-006-code-00/projects/ai_agents/.venv/lib/python3.10/site-packages/pydantic/_internal/_generate_schema.py", line 1372, in _common_field_schema
    schema = self._apply_annotations(
  File "/data/gym/zkdh-006-code-00/projects/ai_agents/.venv/lib/python3.10/site-packages/pydantic/_internal/_generate_schema.py", line 2297, in _apply_annotations
    schema = get_inner_schema(source_type)
  File "/data/gym/zkdh-006-code-00/projects/ai_agents/.venv/lib/python3.10/site-packages/pydantic/_internal/_schema_generation_shared.py", line 83, in __call__
    schema = self._handler(source_type)
  File "/data/gym/zkdh-006-code-00/projects/ai_agents/.venv/lib/python3.10/site-packages/pydantic/_internal/_generate_schema.py", line 2279, in inner_handler
    schema = self._generate_schema_inner(obj)
  File "/data/gym/zkdh-006-code-00/projects/ai_agents/.venv/lib/python3.10/site-packages/pydantic/_internal/_generate_schema.py", line 1009, in _generate_schema_inner
    return self.match_type(obj)
  File "/data/gym/zkdh-006-code-00/projects/ai_agents/.venv/lib/python3.10/site-packages/pydantic/_internal/_generate_schema.py", line 1127, in match_type
    return self._unknown_type_schema(obj)
  File "/data/gym/zkdh-006-code-00/projects/ai_agents/.venv/lib/python3.10/site-packages/pydantic/_internal/_generate_schema.py", line 639, in _unknown_type_schema
    raise PydanticSchemaGenerationError(
pydantic.errors.PydanticSchemaGenerationError: Unable to generate pydantic-core schema for True. Set `arbitrary_types_allowed=True` in the model_config to ignore this error or implement `__get_pydantic_core_schema__` on your type to fully support it.

If you got this error by calling handler(<some type>) within `__get_pydantic_core_schema__` then you likely need to call `handler.generate_schema(<some type>)` since we do not call `__get_pydantic_core_schema__` on `<some type>` otherwise to avoid infinite recursion.

For further information visit https://errors.pydantic.dev/2.11/u/schema-for-unknown-type

Fixing the output_content_type detection code:
修复 output_content_type 检测代码：

_assistant_agent.py#L687

      # if output_content_type is not None:
        if output_content_type is not None and isinstance(output_content_type, type) and issubclass(output_content_type, BaseModel):

_assistant_agent.py#L1016

          # if output_content_type:
            if output_content_type and isinstance(output_content_type, type) and issubclass(output_content_type, BaseModel):

_assistant_agent.py#L1237

      # if output_content_type:
        if output_content_type and isinstance(output_content_type, type) and issubclass(output_content_type, BaseModel):

Running the code again generates the correct JSON output:
再次运行代码会生成正确的 JSON 输出：

$ uv run python test_structured_output.py 
---------- TextMessage (user) ----------
I am happy.
---------- TextMessage (assistant) ----------
{
  "thoughts": "The user is expressing a positive emotion.",
  "response": "happy"
}

Expected behavior 预期行为
As mentioned above. 如上所述。

Additional context 其他上下文
Although the above modification can generate structured output and JSON output, the following problems still exist:
虽然上述修改可以生成结构化输出和 JSON 输出，但仍然存在以下问题：

When response_format=AgentResponse is set on model_client, the returned value is still TextMessage instead of StructuredMessage[AgentResponse].
在 model_client 上设置 response_format=AgentResponse 时，返回的值仍为 TextMessage 而不是 StructuredMessage[AgentResponse]。
When output_content_type=True is set on agent, the returned value is TextMessage containing JSON. (Is there a mechanism to automatically convert content to json?)
在 agent 上设置 output_content_type=True 时，返回值为 TextMessage containing JSON。（有没有自动将内容转换为 json 的机制？

Which packages was the bug in?

Python Extensions (autogen-ext)
Python 扩展（autogen-ext）

AutoGen library version.

Python 0.6.1 Python 版本 0.6.1

Other library version.

No response 无响应

Model used

deepseek-chat 深度搜索聊天

Model provider

None 没有

Other model provider

https://api.deepseek.com/v1

Python version

None 没有

.NET version

None 没有

Operating system

None 没有

Thanks for the issue. My opinion is we don't want to overload the same OpenAIChatCompletionClient with unstable parsing logics.
谢谢你的问题。我的观点是，我们不想用不稳定的解析逻辑使同一个 OpenAIChatCompletionClient 过载。

In this case, I highly recommend creating your custom agent instead of using AssistantAgent, so you can express your own parsing logic directly. It is generally a bad practice to rely on the built-in AssistantAgent when creating your application. AssistantAgent is a convient kitchen-sink for demostration and education purpose.
在这种情况下，我强烈建议您创建自定义代理，而不是使用 AssistantAgent，这样您就可以直接表达自己的解析逻辑。在创建应用程序时，依赖内置的 AssistantAgent 通常是一种不好的做法。AssistantAgent 是一个方便的厨房水槽，用于演示和教育目的。

My opinion is we don't want to overload the same OpenAIChatCompletionClient with unstable parsing logics.
我的观点是，我们不想用不稳定的解析逻辑使同一个 OpenAIChatCompletionClient 过载。

Thanks for the explanation, that's reasonable design.
谢谢你的解释，这是合理的设计。

Users can get JSON output by setting response_format of model_client to {"type": "json_object"} and manually construct structured output based on the result.
用户可以通过将 model_client 的 response_format 设置为 {“type”： “json_object”} 来获取 JSON 输出，并根据结果手动构建结构化输出。

In this case, I highly recommend creating your custom agent instead of using AssistantAgent, so you can express your own parsing logic directly. It is generally a bad practice to rely on the built-in AssistantAgent when creating your application. AssistantAgent is a convient kitchen-sink for demostration and education purpose.
在这种情况下，我强烈建议您创建自定义代理，而不是使用 AssistantAgent，这样您就可以直接表达自己的解析逻辑。在创建应用程序时，依赖内置的 AssistantAgent 通常是一种不好的做法。AssistantAgent 是一个方便的厨房水槽，用于演示和教育目的。

Thanks for your suggestion.
谢谢你的建议。
However, for an Autogen novice like me, it is very valuable to be able to generate JSON output by learning the code logic of AssistantAgent.Moreover, the amount of code modification is also very small.
但是，对于像我这样的 Autogen 新手来说，能够通过学习 AssistantAgent 的代码逻辑来生成 JSON 输出是非常有价值的。而且，代码修改量也非常小。

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Allow models that only support `json_object` to also support structured output #6727
允许仅支持 `json_object` 的模型也支持结构化输出 #6727

What happened?

Which packages was the bug in?

AutoGen library version.

Other library version.

Model used

Model provider

Other model provider

Python version

.NET version

Operating system

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Participants

Allow models that only support json_object to also support structured output #6727允许仅支持 json_object 的模型也支持结构化输出 #6727

Description

What happened?

Which packages was the bug in?

AutoGen library version.

Other library version.

Model used

Model provider

Other model provider

Python version

.NET version

Operating system

Activity

ekzhu commented on Jun 28, 2025

smalltalkman commented on Jun 29, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Participants

Issue actions

Allow models that only support `json_object` to also support structured output #6727
允许仅支持 `json_object` 的模型也支持结构化输出 #6727