Python: ChatCompletionAgent to produce structured output with different schemas

[vslepakov]

Hey everyone,

I'm using a ChatCompletionAgent with multiple plugins, like [WeatherPlugin(), MenuPlugin()], where each plugin contains multiple kernel functions.

What I'm aiming for is to have agent.get_response() return a structured output that includes both the natural language response and the structured data returned by the kernel function. Ideally, the result would look like this:

{
  "natural_text_response": "lorem ipsum...",
  "structured_response": <JSON object or string representation of the function's return value>
}

The challenge I’m facing is that the only mechanism I’ve found to define an output format is through AzureChatPromptExecutionSettings, which requires specifying a concrete output schema. This doesn’t seem to support dynamic return schemas across different functions with varying return types.

Is there a way to achieve this goal?