Home

Performance regression in Linqpad 8 for GetType + typeof?

DaiPlusPlusDaiPlusPlus
edited March 1

I revisited Jon Skeet's 2008 benchmark for GetType() vs typeof() and thought I'd try it out in .NET 7 and .NET 8 and I get unexpected results - specifically, Linqpad 8 seems to exhibit worse performance for some simple tests in both .NET 7 and .NET 8 compared to Linqpad 7 + .NET 7.

I note that the .NET team said they specifically improved GetType() performance in .NET 8 compared to previous versions, which makes this odd.

Attached are the 3 .linq scripts I ran. Screenshot proof (with a bonus guest appearance of Linqpad 5):

(I made one modification to get the .linq script to run in LinqPad 5 for .NET Framework 4.8.1: I changed Elapsed.TotalMicroseconds to Elapsed.TotalMilliseconds * 1000d)

Running in Linqpad 7 x64 and Linqpad 8 x64, with /o+ and both Break-on-exceptions buttons switched off, I get these results (I also pressed F5 repeatedly and ensured the result values were stable; the numbers never changed more than +/-1,000us between runs):

.NET typeof(Test) field = typeof(Test) test.GetType()
Linqpad 7.8.10 x64 7.0.16 132,983 133,200 159,128
Linqpad 8.1.12 x64 7.0.16 133,879 158,588 159,495
Linqpad 8.1.12 x64 8.0.2 135,368 158,534 270,105
  • So Linqpad 8 runs the simplest test (which simply dereferences a static field) in 20% more time than Linqpad 7 (158ms vs 133ms).
  • Curiously, Linqpad 8 with .NET 8 experiences a 70% time increase for type.GetType() vs .NET 7. This is the weirdest part.

Just to make sure it wasn't something else, I converted the code into a BenchmarkDotNet test (attached) and I got these results:

BenchmarkDotNet v0.13.12, Windows 10 (10.0.19045.4046/22H2/2022Update)
Intel Core i7-10700K CPU 3.80GHz, 1 CPU, 16 logical and 8 physical cores
.NET SDK 9.0.100-preview.1.24101.2
  [Host]     : .NET 6.0.27 (6.0.2724.6912), X64 RyuJIT AVX2
  DefaultJob : .NET 6.0.27 (6.0.2724.6912), X64 RyuJIT AVX2


| Method                       | Mean      | Error     | StdDev    |
|----------------------------- |----------:|----------:|----------:|
| Operator_typeof_Test         | 2.1312 ns | 0.0461 ns | 0.0409 ns |
| Operator_typeof_TestSubclass | 1.8909 ns | 0.0428 ns | 0.0379 ns |
| Static_field_operator_typeof | 0.4901 ns | 0.0031 ns | 0.0027 ns |
| Method_Test_GetType          | 1.3882 ns | 0.0071 ns | 0.0060 ns |
| Method_TestSubclass_GetType  | 1.4349 ns | 0.0197 ns | 0.0185 ns |
| Method_Test_virtual_GetType  | 1.4311 ns | 0.0035 ns | 0.0030 ns |


-----

BenchmarkDotNet v0.13.12, Windows 10 (10.0.19045.4046/22H2/2022Update)
Intel Core i7-10700K CPU 3.80GHz, 1 CPU, 16 logical and 8 physical cores
.NET SDK 9.0.100-preview.1.24101.2
  [Host]     : .NET 8.0.2 (8.0.224.6711), X64 RyuJIT AVX2
  DefaultJob : .NET 8.0.2 (8.0.224.6711), X64 RyuJIT AVX2


| Method                       | Mean      | Error     | StdDev    |
|----------------------------- |----------:|----------:|----------:|
| Operator_typeof_Test         | 0.5283 ns | 0.0007 ns | 0.0006 ns |
| Operator_typeof_TestSubclass | 0.5359 ns | 0.0153 ns | 0.0143 ns |
| Static_field_operator_typeof | 0.5285 ns | 0.0008 ns | 0.0007 ns |
| Method_Test_GetType          | 0.5318 ns | 0.0056 ns | 0.0053 ns |
| Method_TestSubclass_GetType  | 0.5194 ns | 0.0058 ns | 0.0051 ns |
| Method_Test_virtual_GetType  | 0.5234 ns | 0.0006 ns | 0.0005 ns |

...which shows that .NET 8 actually gets the expected (documented) performance gains... weird.

So far, it seems something might be wrong with Linqpad 8, but lemme try one more thing...

...I also compiled the exact same code in a VS2022 .csproj for <TargetFrameworks>net481;netcoreapp3.1;net6.0;net7.0;net8.0 and got these results:

.NET typeof(Test) field = typeof(Test) test.GetType()
net481 132,185 158,945 185,785
netcoreapp3.1 193,089 131,998 131,826
net6.0 158,877 158,824 132,355
net7.0 134,114 132,208 132,189
net8.0 159,324 237,973 264,532

...which is not what I got with BenchmarkDotNet. This is getting weird...

Any ideas?

· Share on FacebookShare on Twitter

Comments

  • sgmooresgmoore
    edited February 29

    Don't have an answer, but have you tried using LinqPad's built in benchmark support?

    They say you should never roll your own encryption and when I started looking at the inards of BenchmarkDotNet I came to the conclusion that perhaps the same should apply to benchmarking. It goes to extraordinary lengths to try to separate methods so nothing that has happened before in one test could affect another and also to exclude outliers etc.

    That's not to say you haven't discovered something weird that I can't explain.

    · Share on FacebookShare on Twitter
  • sgmooresgmoore
    edited February 29

    Curiously, Linqpad 8 with .NET 8 experiences a 70% time increase for type.GetType() vs .NET 7. This is the weirdest part.

    Definitely weird.

    If you change the order of the tests, so that the type.GetType() test runs first then this big increase disappears.

    It's not a linqpad issue as a VS2022 project with the attached file which runs your type.GetType() test twice shows the second run takes much longer under net8.0

    dotnet run --configuration Release --framework net7.0
test.GetType(): 116,618us
test.GetType(): 117,394us

dotnet run --configuration Release --framework net8.0
test.GetType(): 116,120us
test.GetType(): 265,777us

    Can't replicate this under BenchmarkDotNet.

    · Share on FacebookShare on Twitter
Sign In or Register to comment.