Including one .linq file inside another - please comment

JoeAlbahari

I've just written a proposal for an upcoming new feature - referencing .linq files from other queries:
https://www.linqpad.net/LinqReference.aspx

Please let me know what you think!

nescafe

Can you make the link to this thread absolute? It's now referring to https://www.linqpad.net/forum.linqpad.net/..

Any thoughts about sharing queryies using Upload to instant share function? Currently queries referencing My Extensions are not runnable stand alone. Perhaps a little warning dialog "It looks like you're referencing another query... which will not be included in the uploaded file" with "Do not show this again" would be enough, both for referenced .linq files as well as My Extensions.

user

Will the include work for extension methods?

JoeAlbahari

nescafe: thanks, the link is now fixed. A warning would be a good idea. It might even be possible to merge the queries, although there are edge cases that could cause this to fail.

JoeAlbahari

user: there are no special restrictions on the kind of code that referenced queries can include, so classes that define extension methods are fine.

If you were to reference several queries that all defined extension methods in a class with the same name (e.g., "Extensions"), you'd need to include the 'partial' keyword to avoid a collision.

NetMage

Given this 'The F12 shortcut ("go to definition") will also work with symbols defined in a reference query.' can My Extensions references also be made to work with F12 pulling up the source in My Extensions instead of decompiling?

Lack of nested references will hurt but I will take the 80/20 rule any day on this.

NetMage

My initial use case for this was a project with multiple queries against a complex set of CSV files. The (copied and pasted) base query defines sets of variables as LINQ queries that convert and standardize the data, which are then used in the other queries.

How would this method of referencing a query work with trying to initialize lots of (query) global variables?

JoeAlbahari

Supporting F12 with My Extension functions could end up being tricky but I'll look into it. I don't think there's any Roslyn support for this because My Extensions is compiled to a DLL.

Yes, it would work well in your CSV scenario. You'd declare your CSV variables as fields in the query, and populate them either with field initializers or in the OnStart() hook method.

NetMage

I don't think that would work then - they use (lots of) anonymous types, so they are declared var?

NetMage

While they are barely documented, don't CSX files expect recursive #load to work?

JoeAlbahari

Is your query of type statements or program?

NetMage

Originally they were of type statements, but since #region doesn't work in statements, the "base" query is of type program just to use #region, but I copy only the body of main and put it at the top of main in the child queries.

JoeAlbahari

Check out the LINQPad 6 preview:

forum.linqpad.net/discussion/1877/linqpad-6-for-net-core-3-preview-now-available

It now supports the #load directive, and should work well in your scenario.

NetMage

Unfortunately, my queries heavily use the CSV driver, and it looks like no external drivers are available for LINQPad 6 yet. It seems like LINQPad 5 will never add what I want, and LINQPad 6 won't be usable for me until next year, so I will wait.

ordinaryorange

Am I correct that this is only supported for C# ? Got it working on C# and it is a nice feature, but does not appear to work on F#. The FSharp compiler complains because the #load directive is specific to the FSharp Interactive tool.
Is there an F# way ?

JoeAlbahari

Yes, unfortunately it's C#-only for now. It would be a big job to get it working well in F#. For one, it relies on partial classes to minimize the performance hit to autocompletion and tooling.

JeffMercado

Would this coming to LINQPad 5 as well or is this going to be 6 only?

JoeAlbahari

LINQPad 6 only. It required some major refactoring.

thealexdresko

I believe I found a bug:

With the following two files, running 2.linq produces the following error:

1.linq at line 11 (Ctrl+F11 to open): CS0246 The type or namespace name 'Something' could not be found

1.linq

void Main()
{

}

public class Something {

}

public static class Boogers {
	public static void Test(this Something something) {
		something.Dump();
	}
}

2.linq

#load ".\1.linq"

void Main()
{
}

If I change the extension method parameter to `this string something` instead of `this Something something`, the error goes away.

So it seems there's a problem using reference types in extension methods?

JoeAlbahari

Thanks for reporting. I overlooked adding the automatic import directive to #load-ed queries (static global::UserQuery). This is easy to fix, so it will make the next build.

JeffMercado

Are scripts loaded recursively?

Meaning, if I load a script that itself has a load as well, will this work? I'm trying to create a series of scripts similar in structure to javascript modules and it doesn't seem to load beyond the first script.

A.linq:

#load ".\B"
void Main()
{
    MethodB();
}

B.linq:

#load ".\C"
void MethodB()
{
    MethodC();
}

C.linq:

void MethodC()
{
}

Attempting to run A.linq, I would get an error along the lines of:

Error in ...\B.linq at line 4: CS0103 The name 'MethodC' does not exist in the current context

This applies to all methods and types (with varying error messages).

A "workaround" I've found for now is to load all scripts depended on in the main script which I would rather not have to do (there's a lot of scripts).

JoeAlbahari

Yes, that's a deliberate limitation, to simplify tooling and ensure good performance. Recursive #load-ing would be simple for me to enable, but making it work well would be a fairly big project.

raboof

I am looking for an expanded explanation of the following passage from “Referencing other .linq files”:

To avoid compilation errors, any namespaces that the #load-ed query imported via the Query Properties dialog, are also implicitly imported.

While I could test out all the questions I have but I thought it would be much more helpful to get the official version straight from the horse's mouth (sorry, Joe, don't mean to call you a horse but so goes the expression ¯\_(ツ)_/¯) in case I accidentally stumble across a scenario that works incidentally but never designed so.

Suppose the very simple case of a program query A.linq (A) hash-loading also a program query B.linq where B.linq (B) is some mini-library. What would here?

1. Will A's and B's imports get merged?
   
2. Will A imports follow B imports or vice versa?
   
3. Do imports include any static and aliasing imports?
   
4. If B has an import then does A need it too?
   
5. What happens to NuGet references between B and A? Do these also get merged as one set?
   
6. Can B reference a NuGet package that's then used by A without A needing the same reference?
   
7. If B and A reference different versions of the same package, which one will win?
   

Thanks!

raboof

I've noticed that a query hash-loading (#load) another inherits its imports except when the hash-loaded query is a Program query. Is this a bug or by-design behaviour? If it's the latter, what's the reason for the exception?

JoeAlbahari

Yes, it's by design. With expression and statements-based queries, LINQPad uses lexical injection to resolve #load, as if you pasted the #load-ed query into your query. (If it did otherwise, you wouldn't have access to the variables that you declare in your loaded query.) This means that the namespaces of loaded query must be imported, otherwise it wouldn't compile.

With program-based queries, LINQPad creates a separate syntax tree, rather like including another .cs file in a project. This allows the loaded query to have its own imports, which avoids potential interference with the main query.

raboof

Thanks for the clarifications.

The hash-loading feature is a fantastic addition but I have hundreds of queries in production and I am trying to understand how to use hash-loading sensibly and for an optimal setup of mini-libraries. My fear is that someone changes a loaded query in a way that breaks all loading queries. The modification could be as simple as changing the query kind but it has far-reaching impacts for the loading query. You see, right now an empty statements-query can be used or abused to import namespaces as well as NuGet packages via hash-loading. Change it to a program query just to add some hooks and now you've broken the loading queries since they won't see the imports and references. It would be nice if LINQPad could help navigate the space of loading and loaded queries to avoid being surprised at run-time. I'm thinking:

• A menu option and shortcut to open all queries referenced in #load directives
  
• A menu option and shortcut to open all queries referencing a query open in the editor
  

Also, since a query with references is no longer a single unit of exchange and deployment, it would be nice to see LINQPad offer native support for exporting a query along with referenced ones as a .zip or .tar.gz file (ideally both) as well as loading them back without going through the extra step of extracting those archives.

JoeAlbahari

Regarding the shortcuts, right now you can Ctrl+Click or press F12 on any #load directive to open the query. You can also press F12 on any symbol in a loaded query and it will open the symbol in the loaded file.

Regarding NuGet packages and references, these are always imported into the host query regardless of whether the query is an expression, statement or program.