If you wanted to generate a longer essay by GPT3 but make it really hard to tell, you could just get it started with a 1-2 sentences. Then let it generate N-1 tokens (where N is the window size, I guess 2048), and write a few tokens yourself (and repeat). Then I don't think there's any way to reverse engineer the internal state of the model at any point in the text. It would be like trying to find a string that hashes to a given value.
Comments
If you wanted to generate a longer essay by GPT3 but make it really hard to tell, you could just get it started with a 1-2 sentences. Then let it generate N-1 tokens (where N is the window size, I guess 2048), and write a few tokens yourself (and repeat). Then I don't think there's any way to reverse engineer the internal state of the model at any point in the text. It would be like trying to find a string that hashes to a given value.
Maybe like hashing though there can be weaknesses in the algorithm still.