IME GPT-4 is bad at doing things and great at looking things up for you. Rather than trying to get it to do things, I ask it how I should do it (and argue with it about any hallucinations that come out—a process that often teaches me a lot!) and then I go do the task myself, copying whatever I need from the conversation but not relying on it slavishly.
It’s basically a fuzzy inverted index for public docs and code. “What’s the normal way of doing X”-type queries work best, with quality falling off quickly as complexity increases. Stuff like “what is a ‘git log’ command that only shows commits containing a particular snippet in the diff, limited to merge commits on master”, for example.
For more complex tasks, a trick I’ve seen work well is “give me an outline for how to do large task X” followed by “let’s go through each step in the above outline. For each one, I’d like a description of how it should be solved, including example code. Let’s start with the first step.” But that trick is not totally reliable and has its own complexity limit.
Comments
IME GPT-4 is bad at doing things and great at looking things up for you. Rather than trying to get it to do things, I ask it how I should do it (and argue with it about any hallucinations that come out—a process that often teaches me a lot!) and then I go do the task myself, copying whatever I need from the conversation but not relying on it slavishly.
It’s basically a fuzzy inverted index for public docs and code. “What’s the normal way of doing X”-type queries work best, with quality falling off quickly as complexity increases. Stuff like “what is a ‘git log’ command that only shows commits containing a particular snippet in the diff, limited to merge commits on master”, for example.
For more complex tasks, a trick I’ve seen work well is “give me an outline for how to do large task X” followed by “let’s go through each step in the above outline. For each one, I’d like a description of how it should be solved, including example code. Let’s start with the first step.” But that trick is not totally reliable and has its own complexity limit.