Someone pointed out that some of the answers are using OpenAI's famous "As an AI...". Soooo you can roughly say that RLHF might still have had an impact here through the training data that came from an RLHF model.
But what we are seeing is a revolution in picking quality demonstration data that might make RLHF an optional last step for fine-tuned models.
I'm bullish on a new technique where the quality of instruction-tuned code model output are measured and evaluated automatically by a machine. I'm calling it RL<machine>F
Code Model outputs a test harness, an implementation, and how to run the implementation for a users inputs. Have RLMF evaluate implementations against the synthetic test harness and against hidden user inputs.
Comments
Yes and no.
Someone pointed out that some of the answers are using OpenAI's famous "As an AI...". Soooo you can roughly say that RLHF might still have had an impact here through the training data that came from an RLHF model.
But what we are seeing is a revolution in picking quality demonstration data that might make RLHF an optional last step for fine-tuned models.
I'm bullish on a new technique where the quality of instruction-tuned code model output are measured and evaluated automatically by a machine. I'm calling it RL<machine>F
Code Model outputs a test harness, an implementation, and how to run the implementation for a users inputs. Have RLMF evaluate implementations against the synthetic test harness and against hidden user inputs.