I agree that oversight is needed - but I'm not certain people are the right approach to this oversight. Everything described above represents an undesirable software output for the given business. Why can't we make the LLM smart enough to know it's current operating context and build guardrails to catch the cases which slip through?
The proper guardrails worthy of being entrusted, they best come from an ecology of human minds with diverse perspectives and ideas, to grok the whole possibility space with minimal blind spots.
The human machine is already tuned to do this navigating of the information landscape. The sense of curiosity itself lures us toward things that are just a little weird -- the goldilocksian zone between familiar and unfamiliar[1]. Our minds are designed to bridge gaps and seek paths into otherness, but only when the individual steps are small. We are built to seek, grow understanding and incorporate diverse thought :)
Comments
I agree that oversight is needed - but I'm not certain people are the right approach to this oversight. Everything described above represents an undesirable software output for the given business. Why can't we make the LLM smart enough to know it's current operating context and build guardrails to catch the cases which slip through?
The proper guardrails worthy of being entrusted, they best come from an ecology of human minds with diverse perspectives and ideas, to grok the whole possibility space with minimal blind spots.
The human machine is already tuned to do this navigating of the information landscape. The sense of curiosity itself lures us toward things that are just a little weird -- the goldilocksian zone between familiar and unfamiliar[1]. Our minds are designed to bridge gaps and seek paths into otherness, but only when the individual steps are small. We are built to seek, grow understanding and incorporate diverse thought :)
[1]: https://nautil.us/curiosity-depends-on-what-you-already-know...
Thing is, you need people to take care in selecting training data. GIGO - garbage in, garbage out - especially applies in AI.