That is easy when you can control the teacher model yourself and you want to transfer its capabilities to a smaller model.
If the teacher model is run by an external entity, e.g. Anthropic or OpenAI, then the number of queries to the blackbox model that is required is so great that it should be easy for the owner of the teacher LLM to detect and stop any such attempts.
Comments
That is easy when you can control the teacher model yourself and you want to transfer its capabilities to a smaller model.
If the teacher model is run by an external entity, e.g. Anthropic or OpenAI, then the number of queries to the blackbox model that is required is so great that it should be easy for the owner of the teacher LLM to detect and stop any such attempts.