Home

BenchmarkDotNet in LinqPad (AnyCPU) throws an Exception

Recently I posted an issue with BenchmarkDotNet (https://forum.linqpad.net/discussion/1872/benchmarkdotnet-in-linqpad#latest). This turned out to be issue of completely different kind.

But during investigation I tried several LinqPad versions. Beta versions, x86 or AnyCPU I was able to run BenchmarkDotNet in LinqPad, but only in x86 build.
AnyCPU build will throw an FileLoadException.

Assembly is still being loaded. (Exception from HRESULT: 0x80131016)
 at BenchmarkDotNet.Portability.RuntimeInformation.get_IsFullFramework()
   at BenchmarkDotNet.Portability.RuntimeInformation.GetCurrentRuntime()
   at BenchmarkDotNet.Characteristics.Resolver.<>c__DisplayClass1_0`1.<Register>b__0(CharacteristicObject obj)
   at BenchmarkDotNet.Characteristics.Resolver.Resolve[T](CharacteristicObject obj, Characteristic`1 characteristic)
   at BenchmarkDotNet.Toolchains.ToolchainExtensions.GetToolchain(Job job)
   at BenchmarkDotNet.Running.BenchmarkRunnerClean.<>c__DisplayClass12_0.<GetSupportedBenchmarks>b__2(BenchmarkCase benchmark)
   at System.Linq.Enumerable.WhereArrayIterator`1.MoveNext()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at BenchmarkDotNet.Running.BenchmarkRunnerClean.<>c__DisplayClass12_0.<GetSupportedBenchmarks>b__0(BenchmarkRunInfo info)
   at System.Linq.Enumerable.WhereSelectArrayIterator`2.MoveNext()
   at System.Linq.Enumerable.WhereEnumerableIterator`1.MoveNext()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at BenchmarkDotNet.Running.BenchmarkRunnerClean.Run(BenchmarkRunInfo[] benchmarkRunInfos)
   at BenchmarkDotNet.Running.BenchmarkRunner.RunWithDirtyAssemblyResolveHelper(Type type, IConfig config)
   at BenchmarkDotNet.Running.BenchmarkRunner.RunWithExceptionHandling(Func`1 run)
   at BenchmarkDotNet.Running.BenchmarkRunner.Run[T](IConfig config)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()
I don't know if this can be called a defect and to which project it actually belongs ...

Comments

  • I can't run it at all. I've looked into why, and there's layer upon layer of nested problems. Some I can fix, but for some it doesn't look good. It's related to complex chains of .NET Standard dependencies, duplicated types, and type-forwards.

    It might be a while before I can work further on this because the priority now is on LINQPad 6. The dependency problems are not an issue with .NET Core (and also, because LINQPad 6 has a re-written dependency resolver which looks at reference assemblies as well as runtime assemblies). There are still other unrelated barriers with Benchmark.NET in LINQPad 6 - these will either be remedied or I'll ship a built-in benchmarking tool which will be more ergonomic. I'm thinking of an interface whereby you can highlight any code and press Shift+Ctrl+B to benchmark it.
  • Hi Joe,
    by
    I can't run it at all.
    you mean LINQPad 6 ?

    I find your idea of running benchmark with the shortcut great !

    I like benchmark dot net's output, which compares times and is also able to compare garbage collections and allocations.
    |    Method |            Mean |       Error |      StdDev |    Gen 0 |    Gen 1 |    Gen 2 | Allocated |
    |---------- |----------------:|------------:|------------:|---------:|---------:|---------:|----------:|
    |  SortLinq |        53.03 ms |   1.0579 ms |   2.1609 ms | 222.2222 | 222.2222 | 222.2222 | 1000064 B |
    | SortArray |        22.01 ms |   0.4109 ms |   0.3844 ms |        - |        - |        - |         - |
    |    Bubble | 1,016,419.24 ms | 606.5232 ms | 567.3422 ms |        - |        - |        - |         - |
  • Thanks - I'll look into what features I can provide. It's actually possible to write your own benchmarking right now in LINQPad 6 by #load-ing a script that has hook methods:
    https://www.linqpad.net/LinqReference.aspx

    Here's a more elaborate script that allows marking methods with the Benchmark attribute:
    http://share.linqpad.net/lorrft.linq

    To test it:
    #load "Benchmark"
    
    void Main()
    {
    }
    
    [Benchmark] void Test1() => Thread.SpinWait (1000);
    [Benchmark] void Test2() => Thread.SpinWait (2000);
    [Benchmark] void Test3() => Thread.SpinWait (3000);
Sign In or Register to comment.