Skip to content

Comment on Our LLM-controlled office robot can't pass butterparent

Comments

It can't "lose" what it never had. :P A fictional character has a mind to the same extent that it has a gallbladder.

if the writings of humans who have lost their minds (and dialogue of characters who have lost their minds) were entirely missing from the LLM’s training set, would the LLM still output text like this?

I think should distinguish between concepts like "repetitive outputs" or "lots of low-confidence predictions the lead to more low-confidence predictions" versus "text similar to what humans have written that correlates to those situations."

To answer the question: No. If an LLM was trained on only weather-forecasts or stock-market numbers, it obviously wouldn't contain text of despair.

However, it might still generate "crazed" numeric outputs. Not because a hidden mind is suffering from Kierkegaardian existential anguish, but because the predictive model is cycling through some kind of strange attactor [0] which is neither the intended behavior nor totally random.

So the text we see probably represents the kind of things humans write which fall into a similar band, relative to other human writings.

[0] https://en.wikipedia.org/wiki/Attractor

Very good underappreciated comment.

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.