LINQPad AI Assistant Preview - feedback needed

JoeAlbahari

The latest beta now has an AI coding assistant.

To activate the assistant, press Ctrl+P. This will display a list of prompts, each of which starts a ChatGPT-style conversation.

All messages in the conversations can be edited on each submission (including system prompts), and by clicking Save, you can create custom prompts that will appear in the main list. There are also options to compare assistant-generated code with your own, and replace or insert AI code into the editor.

Conversations can optionally incorporate selected code from your query, as well as the current runtime error or compilation error message.

See Help | What's New or the online release notes for more info.

Let me know your thoughts!

msGuy

After pressing ctrl p for AI assistant.

msGuy

The above error popped up after pressing "ctrl p", the AI assistant. The openAI api key has been entered at the edit AI Setting

msGuy

After I clicked on the "Cancel" button in the dialog, I got this:

msGuy

If I just go to the Edit --> AI --> Personal Assistant, I also see the null exception error.

JoeAlbahari

Thanks for the early heads-up. I've got your error report and it should be fixed now.

JBehrendsen

I use it in company environment and have to use a different proxy, like for nuget otherwise requests fail.

msGuy

Thanks. The problem is now fixed with new update.

Simutilator

I think it would be great if the contents of #load referenced queries could be included.

Swampy

Some of the formatting is lost receiving data from the Assistant. For example, the following would benefit from highlighting the method names and keywords in the backticks.

Sorax

I'd like to clean up AI Chat panels. For example, if Chat 1, Chat 2, and Chat 3 panels exist, close Chat 3. Currently I resort to closing and reopening the Query to clean up the AI panel.

JoeAlbahari

@Swampy This is a tricky one. Because the conversation is fully editable (including Assistant responses), it would require a WYSIWYG editor with a formatting toolbar.

@Sorax - will add a feature for that.

stephensmitchell

This is great!

@JoeAlbahari Can you add the query snippets directory where prompts are stored to search ( look in, search in)?

I have thousands of queries and will likely have as many prompts so search is important.

Sorax

@JoeAlbahari thank you for the Close option on the AI tabs.

Today I'm working on a project in which I collect information from source apis. I instinctually selected text in the results and pressed Ctrl+P to ask questions about it. Of course, this didn't work but it led me here to suggest adding the selection of text in the Results tab to be an input for the AI.

THenrich

@JoeAlbahari

How to switch between Curie, Davinci, Babbage, Ada models when using the Ai Assistant?

JoeAlbahari

@Sorax - Nice idea. Working with result data is certainly possible, although there are some challenges with token limits and formatting tabular data. What are some examples where you think it would be useful?

@THenrich - These are not chat-based models, and so would not couple well to LINQPad's AI Assistant interface. Is there a specific reason that you'd like to use these models?

THenrich

@JoeAlbahari Ok Thanks. I thought they are used in the APIs. They have different pricing structures and the better ones give better results according to their site.

stephensmitchell

It would be great if this UI could persist across queries as a standalone tool, optionally available for global use rather than being attached to specific queries.

sdflysha

Can LINQPad support ChatGPT in Azure? by specify openai endpoint, deployment id and apikey.

stephensmitchell

LINQPad AI features are GUI based. To use OpenAI APIs in Azure, you would need to either create your own integration or use a third-party integration with OpenAI. Azure offers OpenAI services that can be used for this purpose.

https://learn.microsoft.com/en-us/azure/cognitive-services/openai/overview
https://azure.microsoft.com/en-us/products/cognitive-services/openai-service
https://learn.microsoft.com/en-us/azure/cognitive-services/openai/
https://azure.microsoft.com/en-us/pricing/details/cognitive-services/openai-service/

JoeAlbahari

@stephensmitchell - Is this to allow you to close a query without its AI conversation disappearing, or is there another benefit that you have in mind? And would a "Detach from query" button work?

@sdflysha - Will probably add this option when Azure-hosted OpenAI becomes publicly available.

stephensmitchell

@JoeAlbahari can you add this for local installs also. That can be useful for using Open Source AI and ChatGPT alternatives.

I use LINQPad for building complete apps, tools or overall solutions. I have multiple LINQPad 7 and 5 apps running for different tasks.

I want to use the AI features standalone so I open another LINQPad and use it only for that purpose. I then clone the conversion for other tasks.

This works but has obvious drawbacks.

The AI panel being linked to a query doesn't work for me because I open and close queries constantly. I also want to use the AI panel at runtime or in production, not only when coding or debugging. In the first image above I need to update a 3D algorithm for more complex meshes. I need to see the results and work with different tools then make changes to the code in LINQPad, review results, etc.

I think it would be better to add a non-modal AI window with conversion and prompt saving so we don't lose work.
Similar to the prompt window.

A pinnable tab panel or something similar may work too.

sdflysha

@JoeAlbahari said:
@stephensmitchell - Is this to allow you to close a query without its AI conversation disappearing, or is there another benefit that you have in mind? And would a "Detach from query" button work?

@sdflysha - Will probably add this option when Azure-hosted OpenAI becomes publicly available.

Hey @JoeAlbahari, Azure hosted OpenAI already becomes publicly available, only with a form request, pleease try introduce OpenAI in Azure in LINQPad!!! Because it's only option available in China.

JoeAlbahari

There is Azure support now in 7.7.11:
https://www.linqpad.net/linqpad7.aspx#beta

Let me know how you get on.

stephensmitchell

Enhancement:

Add a check all button in the save prompt dialog to avoid clicking each one by one:

And or add a button to the Personal Assistant tab to check all and save at once (this is preferred to save time):

Thanks.

stephensmitchell

Issue:

When loading a prompt code blocks are duplicating.

Original Conversation: Everything works fine

Conversation loaded from prompt window: Weird duplicating issues

The "Language" property has the same value as "Text" property. "Language" should be "C#".

   {
      "Role": "Assistant",
      "IsCode": true,
      "Language": "string str = \u0022C#\u0022;\nforeach (char c in str)\n{\n    Console.WriteLine(c);\n}",
      "Text": "string str = \u0022C#\u0022;\nforeach (char c in str)\n{\n    Console.WriteLine(c);\n}"
    },

Here is the prompt:

http://share.linqpad.net/t2ev2p.linq

JoeAlbahari

Thanks - I'll get a fix out today. Will look into adding a "check all" option soon.

hepsubah

Awesome stuff Joe

JFalcon

@JoeAlbahari, would it be possible to open up more of the settings (maybe as "Advanced AI Configuration") so that I could configure the URL for the ChatGPT API and the model? I am running GPT4All on Windows, which has a setting that allows it to accept REST requests using an API just like OpenAI's.

There are a few benefits to this:
1. It doesn't require a subscription fee.
2. the information remains private and runs on the user's system.
3. different models can be used, and newer models are coming out often.
4. There is a recent plug-in to allow GPT4All to chat using your files and data as reference material.

The drawbacks:
1. The user should have a decent system (good GPU preferred) for inference.
2. The models are still not as good as GPT-4 but are getting closer to GPT-3/GTP-3.

There is a recent Github project called "Rift", which is a server that uses GPT4All with one of the chat models (used in a side chat window) and another model, the recently released replit model for code completion. They've created, in conjunction, a plug-in for a few different editors (one of them being VS Code). It's basically like a free, locally sourced Codepilot.

I really like the idea of having an AI coding system in LINQPad, but I think that it would be really cool to allow users to extend it with other options.

Thank you!

stephensmitchell

I would love a custom provider option some other ideas...

Maybe include the provider data inside the prompt file useful when working in the AI Personal Assistant tab.

{
  "Messages": [
    {
      "Role": "System",
      "IsCode": false,
      "Text": "You are a helpful assistant."
    },
    {
      "Role": "User",
      "IsCode": false,
      "Text": "c# class example"
    },
    {
      "Role": "Assistant",
      "IsCode": false,
      "Text": "Sure, here\u0027s an example of a C# class:"
    }
  ],
  "OpenAIRequest": {
    "url": "https://api.openai.com/v1/completions",
    "headers": {
      "Content-Type": "application/json",
      "Authorization": "Bearer $API_KEY"
    },
    "data": {
      "model": "text-davinci-003",
      "max_tokens": 7,
      "temperature": 0
    }
  }
}

Give complete control of the API parameters:

{
  "max_tokens": 60,
  "temperature": 0.5,
  "top_p": 1,
  "frequency_penalty": 0,
  "presence_penalty": 0,
  "model": "text-davinci-002",
  "best_of": 5,
  "n": 1,
  "stream": false,
  "stop": ["\n"],
  "echo": false,
  "log_level": "info",
  "logprobs": 10
}

When invoking with shift + space in the editor, it would use the provider from the AI settings dialog by default.

But maybe inside a query we could use a compiler flag like syntax to pick which provider to use per query.

#LINQPad openai  
#LINQPad azure    
#LINQPad custom

This would be great for debugging and evaluating the differences between AI models and settings on-the-fly per query instead of switching in the settings every time. I can imagine some use cases would require a custom provider while during general usage the OpenAI provider would be preferred.

openai.provider, azure.provider and custom.provider file:

{
  "url": "",
  "method": "",
  "headers": {
    "Content-Type": "application/json",
    "Authorization": "Bearer $API_KEY"
  },
  "data": {
    "model": "text-davinci-003",
    "max_tokens": 60,
    "temperature": 0.5,
    "top_p": 1,
    "frequency_penalty": 0,
    "presence_penalty": 0,
    "model": "text-davinci-002",
    "best_of": 5,
    "n": 1,
    "stream": false,
    "stop": ["\n"],
    "echo": false,
    "log_level": "info",
    "logprobs": 10
  }
}

• A file loaded once per query file or session and not at startup (loading settings every time shift + space is invoked may be too slow)?
• Some mechanism that would allow us to make changes to the AI parameters without needing to restart LINQPad every time we tweak a parameter.

Just some ideas and thoughts.

Oh and thanks @JoeAlbahari for the select all and select none buttons!

stephensmitchell

If we can define the request ourselves this could allow us to make use of the latest OpenAI feature function calling.
https://openai.com/blog/function-calling-and-other-api-updates

I need to read the docs and experiment but I can think of at least one use case for it inside LINQPad:

Using the Personal Assistant a completion function can run LINQPad queries based on you chat and return the result.

User: Create a feet to inch converter LINQPad query, run it and give me the result.
User: 24
Assistant with function calling:
Generate the code from the prompt
Write it to file
Run with LPRun or LINQPad
Assistant: 288

This example is dumb, the point is adding ChatGPT Plugin like feature to LINQPad using function calling.
https://openai.com/blog/chatgpt-plugins