Skip to content

Comment on Sampling and structured outputs in LLMs

Comments

Google's Gemini API is a bit odd with structured outputs. If you specify an Application/JSON response mimetype, it will reliably respond with a consistent JSON output without any prompt engineering shenanigans. For my workflows, this setting plus providing a JSON Schema in the system prompt works even with complex schema.

The Gemini API has a canonical implementation of structured outputs where you can instead pass the JSON schema as a separate parameter to control the grammar more closely. However, this setting will reorder the JSON schema fields to be alphabetical beforehand, which is especially not desired behavior as the order of JSON fields in a schema is often very deliberate to control generation.

I was burned by this for a while because I assumed structured output ordering would be preserved.

You can specify ordering in the Gemini API with propertyOrdering:

"propertyOrdering": ["recipeName", "ingredients"]

JSON is still not available when you enable Grounding with Search.

gemini api has propertyOrdering field for that

that only works for the outer level, not for any nested fields

nested field have their own propertyOrdering

can you provide an example please? The docs suggest that propertyOrdering can only be a list[str].

yes but you can put it on nested objects. Each type: object can have its own propertyOrdering, so you control whole tree

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.