Azure OpenAI's API unable to use at all for a while.

doggy · April 15

I'm sure my Azure OpenAI Service is able to use, but I just can't use in LINQPad.

Host runtime version: 8.0.15
Default query runtime version: 9.0.4
Default query reference assembly version: 9.0.4
Roslyn Version: 4.13.0-3.25120.11
FSharp.Compiler.Service version: 43.8.101.0
NuGet client version: 6.7.1.1
Windows DevDrive Status: Unavailable
Results rendering engine: Edge Chromium Engine 135.0.3179.73 (WebView2)

doggy · April 15

I attempted to set up the "Custom" provider, but it still doesn't work.

I'm confident that the following curl command should run successfully.

doggy · April 15

I think it's a bug in LINQPad. I traced the request payload from LINQPad and the response content from Azure OpenAI Service. I realized that the first response in Stream mode will produce an "EMPTY" choices and LINQPad think that's an error.

Here is the very first response chunk of the response from AOAI.

{
    "choices": [],
    "created": 0,
    "id": "",
    "model": "",
    "object": "",
    "prompt_filter_results": [
        {
            "prompt_index": 0,
            "content_filter_results": {
                "hate": {
                    "filtered": false,
                    "severity": "safe"
                },
                "jailbreak": {
                    "filtered": false,
                    "detected": false
                },
                "self_harm": {
                    "filtered": false,
                    "severity": "safe"
                },
                "sexual": {
                    "filtered": false,
                    "severity": "safe"
                },
                "violence": {
                    "filtered": false,
                    "severity": "safe"
                }
            }
        }
    ]
}

There is a related issue here: https://github.com/Azure/azure-sdk-for-go/issues/21086

Also here: https://github.com/streamlit/streamlit/issues/8084

I think the best way to solve this issue is by ignoring empty choices chunk.

JoeAlbahari · April 22

I'll get this fixed in the next build. In the meantime, you can make this work by deleting the API version override.

rs38 · April 24

I had exactly the same error but cannot workaround as without API version override it:

OpenAI returned HTTP error code BadRequest
Model {modelName} is enabled only for api versions 2024-12-01-preview and later

rs38 · April 24

with 40-mini I am able to get it running again.
Btw. why is there an optional model dropdown? isnt that determined by the deployment name?

JoeAlbahari · April 25

The deployment name can be anything - it's whatever you type in when creating the deployment in the Azure Portal. It doesn't have to be related to the model name.

doggy · April 28

@JoeAlbahari I confirmed that the latest LINQPad v8.9.2 has already fixed this problem. Thank you.

Azure OpenAI's API unable to use at all for a while.

Comments