Skip to content

Comment on Releasing 3B and 7B RedPajama

Comments

idea: linked parameters / models tree

build a model that can change the number of parameters in the vicinity of some meaning, effectively increasing the local resolution around that meaning

so parameter space becomes linked-parameter space, between models

links could be pruned based on activation frequency

another way of seeing the concept is a tree of models/llms

and one additional model/llm that all it does is manage the tree (ie. build it as it goes, use it to infer, prune it, etc)

Or is it too dumb what I’m saying?

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.