AI Assistant failing with Azure gpt-3.5-turbo-instruct
I've deployed a gpt-3.5-turbo-instruct in azure and tried configuring LINQPad7's AI Assistant to use it. It looks like LINQPad is not formatting the request the way Azure likes it.
This was the request and response:
{"model":"gpt-3.5-turbo","messages":[{"role":"system","content":"You are a C# code completer. Output only code. Do not output English. Output precisely what should appear in place of ~~~. Do not repeat existing text."},{"role":"user","content":"Given the following code:\n\n\.................\n\nPrecisely what code should appear in place of ~~~? Remember not to re-state existing code.\n"}],"temperature":0,"stream":true}
{"error":{"code":"OperationNotSupported","message":"The chatCompletion operation does not work with the specified model, gpt-35-turbo-instruct. Please choose different model and try again. You can learn more about which models can be used with each operation here: https://go.microsoft.com/fwlink/?linkid=2197993."}}
Despite what the error message says in the response, this model does work fine with completions if you format the request differently. It works if you omit the model property and in place of the messages array use an all-in-one prompt string named prompt. eg:
{"prompt":"You are a C# code completer. Output only code. Do not output English. Output precisely what should appear in place of ~~~. Do not repeat existing text. Given the following code:\n\n.................\n\nPr
ecisely what code should appear in place of ~~~? Remember not to re-state existing code.\n","temperature":0,"stream":true}
{"id":"cmpl-9BGn6L2ObiK7KBG7betFEk83BNNni","object":"text_completion","created":1712473180,"model":"gpt-35-turbo-instruct","choices":[{"text":"~~","index":0,"finish_reason":null,"logprobs":null}]}
I'm using v7.8.10 (X64)
Comments
According to the OpenAI documentation, gpt-3.5-turbo-instruct has similar capabilities as GPT-3 era models and is compatible with legacy Completions endpoint and not Chat Completions.
LINQPad uses the newer chat-based API. Support for the legacy API was dropped some time ago, as it lacks some important features.