I would have suspected it too, but I’ve been struggling with OpenAI returning syntactically invalid JSON when provided with a simple pydantic class (a list of strings), which shouldn’t be possible unless they have a glaring error in their grammar.
You might be using JSON mode, which doesn’t guarantee a schema will be followed, or structured outputs not in strict mode. It is possible to get the property that the response is either a valid instance of the schema or an error (eg for refusal)
Comments
How sure are you that OpenAI is using that?
I would have suspected it too, but I’ve been struggling with OpenAI returning syntactically invalid JSON when provided with a simple pydantic class (a list of strings), which shouldn’t be possible unless they have a glaring error in their grammar.
You might be using JSON mode, which doesn’t guarantee a schema will be followed, or structured outputs not in strict mode. It is possible to get the property that the response is either a valid instance of the schema or an error (eg for refusal)
How do you activate strict mode when using pydantic schemas? It doesn't look like that is a valid parameter to me.
No, I don't get refusals, I see literally invalid json, like: `{"field": ["value...}`
https://github.com/guidance-ai/llguidance
OpenAI is using [0] LLGuidance [1]. You need to set strict:true in your request for schema validation to kick in though.
[0] https://platform.openai.com/docs/guides/function-calling#lar... [1] https://github.com/guidance-ai/llguidance
I don't think that parameter is an option when using pydantic schemas.
class FooBar(BaseModel): foo: list[str] bar: list[int]
prompt = """#Task Your job is to reply with Foo Bar, a json object with foo, a list of strings, and bar, a list of ints """
response = openai_client.chat.completions.parse( model="gpt-5-nano-2025-08-07", messages=[{"role": "system", "content": FooBar}], max_completion_tokens=4096, seed=123, response_format=CommentAnalysis, strict=True )
TypeError: Completions.parse() got an unexpected keyword argument 'strict'
You have to explicitly opt into it by passing strict=True https://platform.openai.com/docs/guides/structured-outputs/s...
Are you able to use `strict=True` when using pydantic models? It doesn't seem to be valid for me. I think that only works for json schemas.
class FooBar(BaseModel): foo: list[str] bar: list[int]
prompt = """#Task Your job is to reply with Foo Bar, a json object with foo, a list of strings, and bar, a list of ints """
response = openai_client.chat.completions.parse( model="gpt-5-nano-2025-08-07", messages=[{"role": "system", "content": FooBar}], max_completion_tokens=4096, seed=123, response_format=CommentAnalysis, strict=True )