Avoiding bloated and contested MyExtensions. Thoughts?

Tormod

I posted the following suggestion:
http://linqpad.uservoice.com/forums/18302-linqpad-feature-suggestions/suggestions/7124601-myextensions-folder-multiple-queries-alternativ

Basically, it is considering the tendency of MyExtensions to bloat into the dreaded "Common.dll" we all know from countless software projects. This tendency is greater in LINQPad because there few other options for communal code. Add to this the fact that it your regular queries may span many different topics/domains/efforts and MyExtensions is serving all.

I outline two alternative ways of addressing this.
* MyExtensions should have multiple files. It could be a folder.
* Queries should be able to reference another query (in the normal "add reference").

Searching this forum and the uservoice forums, I was surprised how few people seems to run into this problem.
How do YOU deal with, or avoid, bloated dependency-heavy MyExtensions? (Or is there a feature that I'm unaware of?)

JoeAlbahari

My own "My Extensions" is 800 lines, and I use regions to divide it up. So far, it's not caused me any trouble; I put the things that I'm working on at the top. If it were to get a lot bigger, I'd move some of the code into VS project(s) and compile the .DLLs into the LINQPad extensions folder (LINQPad automatically references all assemblies in this folder.)

If LINQPad were to allow multiple extension files, or if queries could reference other queries, you'd end up with potentially complex dependency graphs, not only between the queries, but between the assemblies that they reference. This could lead to nasty wtf-style scenarios, essentially the same complexities that are possible in VS, but without the overarching solution/project structure to make it clear what's going on.

Tormod

As a seasoned software developer, I agree. However, LINQPad, in conjunction with git, to many of my users is the entire toolset and constitute their entire developer career.
They need to share code. They need to reuse, share, merge and they need to deal with duplication issues. And, when the common code has some heavy .dll dependencies, it would be nice/neat to contain it to where it is needed.
In short, they are experiencing normal developer pains.

Chowder

@Tormod This might help, if you make MyExtensions partial, it makes it very easy to remove, and add sections of functionality to it.
#region MyExtension public static partial class MyExtensions { public static string Stuff1; } #endregion MyExtension #region OtherExtension public static partial class MyExtensions { public static string Stuff2; } #endregion OtherExtension

@JoeAlbahari
Here is a solution that wouldn't cause the issues you listed:

1. Turn MyExtensions into a folder under MyQueries.
2. Change the old MyExtensions.linq to MyExtension.linq, and make it a partial class.
3. For each linq file in the plugins folder, show it in the new folder.
4. When compiling, merge all linq files in the plugin folder into a big linq file, and compile it.

Tormod

Thank you, I appreciate it.

Yes, regions, proper naming and sectioning does help. And there is something to learn for new programmers about dealing with a large code base.
The problem, or lost opportunity, is how multiple files would effectively emulate a proper developer environment and nurture software architectural practices. Instead, MyExtensions is becoming a dependency magnet (both afferent and efferent), clogging down the overall LINQPad experience. In addition, it is the place for personlized stuff (hence the name). Bad things happen when MyExtensions is team/activity specific.
Don't get me wrong. MyExtensions is an excellent addition to LINQPad, but gets some unfortunate use as it is the only available unit of reuse, from users who do not have Visual studio and may not even realize that they are in fact becoming hybrid domain experts/developers.

I suspect that if the solution was as simple as concatenating into a MyExtensions, then we would already have the feature. I have considered on-the-fly stuff myself (having a separate query that builds myextensions). I suspect what makes this a tall order is to support intellisense.

JoeAlbahari

A textual concatenation simplifies matters in some ways, but creates complications of its own. As you pointed out, Intellisense (and debugging) is one issue, but the problem is broader than that.

To give an example, suppose some change that you make to the current extensions query causes the compiler to report a compilation error in another extensions query. (Or perhaps, the other extensions query had recently changed due to a newer copy being fetched from source control or a network location). LINQPad now has to be aware of that, open the other query, and show the error there. Without this behavior, it would be unworkable. To give a more extreme example, let's say there are 20 extension queries, and one of them is missing a closing brace somewhere. Unless LINQPad points you to file where the compiler reports the *first* compilation error, it will be almost impossible to diagnose the problem.

Another issue is that each extensions query might have its own set of references, which will need to be merged. These might be incompatible: Two queries might each point to a different non-identical copy of an assembly. LINQPad would have to identity this and display appropriate warnings. Essentially, LINQPad would need to take on the concept of a "project".

Of course, these problems can (and might one day) be solved, but it's non-trivial to do it in a way that won't frustrate users. I don't want LINQPad to become a second-rate Visual Studio.

Tormod

I appreciate the desire to dissuade LINQPad scope creeping into a second rate visual studio.

As far as I can tell, however, most of the other points are already introduced on account of having a MyExtensions in the first place and even more pronounced by a singleton MyExtensions than would be the case of referencing individual queries. Individual queries can be testrun individually. They can emulate packages. Automated LPRun -compileOnly query can serve as CI pre-checkin. Teams are already sharing MyExtensions through source control, which is wrong in its own right.

LINQPad is certainly a scratchpad, but it also has a pedagogical/tutorial aspect. If users developed disciplines around elements of reuse and package stability, then the transition into visual studio would be more natural. Both developer wise and from management perspective.

But I'm fully aware that I do not know the complexity of the issue and you have absolute saying in the direction you want to take LINQPad.

kingkeith

My solution for preventing a bloated MyExtensions is to create separate NuGet packages for related methods. Then, using MyGet to host a private NuGet feed, I can then choose which packages to include in each query

Tormod

So instead of MyExtensions, you use NuGet? Which means that the extension code is in Visual Studio and not in LINQPad?

kingkeith

Yeah, it's not ideal - I copy the code into LINQPad when I need to debug it or make changes, then put it back into a NuGet package when I'm done... But I don't bother with Visual Studio, I just manually create/edit a package using NuGet Package Explorer (https://npe.codeplex.com/)