NitpickLawyer on Sampling and structured outputs in LLMs

Comment on Sampling and structured outputs in LLMs parent

Perhaps I worded that poorly. What I mean by semantic correctness is that the model could output nonsensical values for some things. Say in a game, "normal" health is ~100hp and the model creates a wizard with 50hp but then a mouse with 10000hp. So you're guaranteed to get a parsable json object (syntactically correct) but what the values are in that json is not guaranteed to make sense in the given context.

viralpraxis 11mo

You can specify `minimum` and `maximum` property for these fields. So this schema

  {
    "$id": "https://example.com/test.schema.json",
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "title": "Person",
    "type": "object",
    "properties": {
      "hp": {
        "type": "integer",
        "description": "HP",
        "minimum": 1,
        "maximum": 15
      }
    }
  }

is converted to this BNF-like representation:

  hp ::= ([1-9] | "1" [0-5]) space
  hp-kv ::= "\"hp\"" space ":" space hp
  root ::= "{" space  (hp-kv )? "}" space
  space ::= | " " | "\n"{1,2} [ \t]{0,20}

michaelgiba 11mo

For anyone curious here is an interactive write up about this http://michaelgiba.com/grammar-based/index.html

lubujackson 11mo

I find it does pretty well given a reasonable prompt and (especially) well-named keys/JSON structure. So if you had boss.mouse.hp you would get higher HP than random_enemies.mouse.hp, or better: enemies.level_1.mouse.hp.

Comments