Options
Support for custom AI services?
Hello,
I have my own license for linqpad (ie. my organization doesn't pay for it). I'd like to use the generative AI features of it, however my organization blocks all traffic to any services with generative AI (so, OpenAI's and Azure's AI services are blocked). However, we have a service with an internal GPT-4 model that we have access to for free within the company. This service doesn't follow any existing API specification. Would it be possible to write our own implementation of an AI service provider for LINQPad, so I can write my own code to connect to the internal AI service?
Comments
Have you tried choosing the "Custom" option in AI Settings?
I don't have an option in LINQPad 7 for 'Custom' in the AI settings dialog:
It's in the beta - click that download link.
Could you please check your e-mail?
Ah okay, wasn't aware it was a beta feature. I'll grab it and take a look.
After trying it a bit I don't think it will work - I think the service is a wrapper around Azure's AI offerings, but with a custom API in-between. Is it possible to allow something like having a query in charge of sending data to the AI service's API? I saw a similar thing was used for the automation.linq query (in the LINQPad plugins thread), maybe a strategy could be used, where you pass the information for the AI service request to an async method in the query, and it returns the response.
@JoeAlbahari I believe my questions under the V8 thread is related to this thread:
What is included in the request you send to OpenAI? Do you add any additional instructions we don't see inside LINQPad? Can we have access to the request parameters to add our own instructions/settings?
I will spend more time testing the custom option. I want to use OpenAI, I just would like to control the request data from LINQPad for the assistant features.
The Assistant UI uses the standard Chat API with default parameters:
https://platform.openai.com/docs/api-reference/chat/create
No additional instructions are sent. The system prompt is exactly as it appears on the screen.
What specifically would you like to control in the request?
I have LINQPad programs that use the API. I make heavy use of the prompt/json structure in my other programs and tools used outside of LINQPad. After saving the prompt file as json and adding additional information to the request/chat, I read information from the json file, it may include instructions, standard messages, messages and other data I added to the file but is not sent with the request.
I would also like to add information to the editor AI.
I think the "My Extensions" query is a great example. A query named "My AI Settings" could contain all the information for openai, azure or custom the AI assistant feature will use.
Sorry, I never got notifications for this thread 😅
The API I have from my company is a custom, internal one that isn't based on the OpenAI chat API at all, hence me asking if it would be possible to override the built-in functionality for the AI assistant via something like an extension query as @stephensmitchell mentioned.