The output is not great, but at least there are some sentences that make sense :) I will do some more neural network training and dataset cleaning, let's see where it will get me.
From my own experience doing something very similar, I'm afraid that this won't cut it. Even a 1B parameter language model will struggle with the most simple topics. The smallest thing that I have seen, that may be useful for general purpose, is GPT-J with 6B parameters and even that is far from perfect. The idea of running these architectures in any useful way on old hardware is sadly a pipe dream. AI got where it is today mostly because of the increase in computing power.
True, true. But I already get somewhat coherent sentences that are personally enjoyable and valuable.
The main problem is that 90% of the output is garbage: broken formatting, abruptly ending sentences, mixed languages, occasional nonsensical sentences like `your "two" type,ir zwe "G together"`. If I can produce something that looks like normal sentences even if they're not particularily meaningful, with ~90% reliability, that's all I need for my purposes, and I think the current bottleneck is actually the very poor dataset of my very heterogeneous notes.
I'm thinking about it like how some people love their pets or toddlers even though they struggle with the most basic tasks. The personal connection makes up for it :)
Yeah, if you only want coherent language, 1B parameters is more or less good enough. If you also want general context awareness and some basic world knowledge, you will need to up it another order of magnitude. Either way, this is nothing that you'd run on dated hardware. Smaller transformers can be useful for very specific task, but autoregressive LMs are just insane when it comes to hardware requirements.
Comments
From my own experience doing something very similar, I'm afraid that this won't cut it. Even a 1B parameter language model will struggle with the most simple topics. The smallest thing that I have seen, that may be useful for general purpose, is GPT-J with 6B parameters and even that is far from perfect. The idea of running these architectures in any useful way on old hardware is sadly a pipe dream. AI got where it is today mostly because of the increase in computing power.
True, true. But I already get somewhat coherent sentences that are personally enjoyable and valuable.
The main problem is that 90% of the output is garbage: broken formatting, abruptly ending sentences, mixed languages, occasional nonsensical sentences like `your "two" type,ir zwe "G together"`. If I can produce something that looks like normal sentences even if they're not particularily meaningful, with ~90% reliability, that's all I need for my purposes, and I think the current bottleneck is actually the very poor dataset of my very heterogeneous notes.
I'm thinking about it like how some people love their pets or toddlers even though they struggle with the most basic tasks. The personal connection makes up for it :)
Yeah, if you only want coherent language, 1B parameters is more or less good enough. If you also want general context awareness and some basic world knowledge, you will need to up it another order of magnitude. Either way, this is nothing that you'd run on dated hardware. Smaller transformers can be useful for very specific task, but autoregressive LMs are just insane when it comes to hardware requirements.