Antivirus Performance Test
Hi,
I just bought LinqPad 8 and immediately ran into something I wasn't even aware of until I started it and it showed me a warning about slow loading of assemblies.
On my new Lenovo T16 laptop that has Windows 11 23H2 I have huge antivirus test script result times (ranging from 450ms to few seconds - longest was 10 seconds). As is mentioned the cause is antivirus (built-in Windows Defender). So I added exlusions for process name, nothing changed at all. I added some folder and file exclusions nothing, as if exlusion settings are ignored completely. I checked group policy and local group policy, nothing is forced so should be fine.
Then I disabled WiFi and test times droped to 100ms almost consistently. This triggered some ideas, so I enabled back WiFi and disabled "Cloud-delivery protection", "Automatic sample submission" and this got me to 70ms. Further disabling "Real-time protection" got me to around 50ms for assembly load time. Exclusions are ignored as far as I can tell becaus enumbers don't change whether LinqPad8.exe is excluded or not.
Which is still many times longer than I get on my 8 year old laptop with Windows 10. On it and my other Windows 10 PC I get 0.1ms with exclusion to LinqPad8.exe.
In the meantime I ran into some articles that 23H2 causes CPU performance problems and Windows Defender is to blame, and there are some powershell commands to restart Windows Defender, but it didn't work for this issue.
Does anyone here have 23H2 and have this working correcly?
To me it looks like exclusions are completely ignored, and it sends every generated temp.dll file as a sample to online servers for scanning.
I have newly installed laptop and this is now getting on my nerves, especially since laptop work flawlessly and it's fast otherwise. The only solution that I can think of is to disable "Cloud-delivery protection" and "Automatic sample submission".
Comments
With realtime protection disabled, 50ms is still 100 times slower than your old laptop. Have you checked for I/O contention on your hard drive from other processes? On new laptops, there's often a lot of updates and scans going on in the background. One way to check is with Task Manager, Show All Processes, Details, select columns and choose I/O bytes read/written (there may be a better way).
Also, the fact that process-based exclusions for Defender don't appear to work could be a bug in 23H2 - maybe you should report this to Microsoft, especially if someone else can repro.
Nothing seems to be using disk (I've checked what you suggested), and everything else works much faster than old laptop. It's not that new anymore, I started setting it up 10 days ago so it had time to analyse, scan and index everything. And disk in tests shows nice speeds as modern NVMe should.
Write of that DLL file from your script is fast, but when reading starts (assembly load) then MsMpEng.exe takes over and that is very visible in ProcessMonitor. It's time is matching the delays shown in script results.
What is even stranger is that on old windows 10 laptop I have Malwarebytes installed, and with it the times are 10ms. When I exit Malwarebytes then it is briefly 0.1ms while transitioning to Windows Defender and when it takes over it is 20ms, so two times slower (but with every option turned on in Windows Defender). Also exclusions are respected and when LinqPad8.exe is ecluded it drops to 0,1ms.
On new laptop I also tried with Malwarebytes, even with ESET, but whether they work, or Windows Defender works the times are the same. Like Windows Defender works in parallel with them even when it shouldn't.
I'll do some more tests :-)
If someone reading this has 23H2 I'll be grateful if they check how it works on their machines.
Thanks!
I'll drop this info for others using Windows Defender:
I added the suggested exclusions for the LinqPad exe files. The times went up from ~400ms to ~8000ms ... super weird.
After playing around a bit I unchecked "Cloud-based protection" (don't know the exact english term for this, since I am using a german windows version). Turning this off, gives me ~20ms. Turning everything off, results in ~10ms
Using 23H2 as well
Edit:
Example, turning off the cloud-based protection mid test:
Edit 2:
Since I had other problems with my ethernet connections, I decided to install windows 10. Now I am at 1-2ms ...
It seems that perhaps Defender in 23H2 does not honor process exclusions. I've added to the thread on https://aka.ms/AAo3539 - can I encourage anyone else having this issue to do the same.
I'm on 22H2 but I'm seeing similar results.
With Windows Defender on, even with the LINQPad process added as an exclusion, I see times ranging from 400-700ms.
If I turn off "Cloud-delivered protection", the times drop down to the expected 20-40ms
I've reported it and I see you also commented on mine (the link above). Thanks.
I did a bit more tests (also added to the feedback ticket).
I've also checked on another colleague's new laptop who also installed Win 11 23H2 and he is experiencing the same issue. So it's not only mine laptop. He also gets 500ms loading times in this test, and Windows Defender exclusions don't work, Turning off "Cloud-delivery protection" and "Automatic sample submission" helps a bit (80ms) but not as when exclusion would be applied. So basically the same issue as mine.
We also tested on another colleague's laptop who has Win 11 22H2 with all updates for that version, and he doesn't experience this issue, it works the same as on my old laptop with Windows 10 22H2. When exclusions are applied loading time is less then 1ms and without exclusion (when Windows Defender does it's job in full) loading times are not that high (20ms compared to mine 500ms).
Found this page while trying to solve this exact problem on my new high-end HP ZBook laptop. Times in the 400-800ms range, exclusions have no effect. Replaced Defender with MalwareBytes, if anything, times seemed even slower.
Windows 11 23H2 Build 22631.2861
Addendum: Went back to Defender, disabled "Cloud-Delivered Protection" and can verify that times are in the 80-100ms range. Disabling "Device Security"->"Core Isolation"->"Memory integrity" reduces times to around 30ms. Still 10x slower than expected for a high-end G10 laptop.
Hi, I worked with this test as well. Such a great tool! We use Symantec. For Symantec folders (with sub-folders) can be excluded (that is a bad idea cause a virus could find out what folders are excluded, but simply guessing some candidates like). One thing that is interesting for LINQPad 8 64 bit.
If you add a line
Directory.GetCurrentDirectory().Dump();
the output surprisingly is:
C:\Users\\AppData\Local\Temp\LINQPad8_hqpnhwjq\shadow-1
instead of the folder where the query file is stored.
So all DLL files in the test will be created in the this folder - which in my case is not excluded.
So even if i stored a query in my exception folder this will not work.
I tried the same with a Visual Studio C++ project.
include
include // for file I/O
int main()
{
std::ofstream outfile("test.txt");
outfile << "my text here!" << std::endl;
outfile.close();
}
This will, as expected, create the file test.txt in the same folder as the executable.
So my question is: why is the current directory for LINQPad8 C:\Users\\AppData\Local\Temp\LINQPad8... ? Is that done by LINQPad itself or this caused by some Windows 11 virtualization feature?
It doesn't harm me - but I'm curious.
More precisely: it will create the file in the process's current directory. This is for the Windows shell the folder of the executable. From cmd is is the current directory of cmd. So it is a bad practice to use relative file path instead of fully qualifying them - but for the purpose of the test program it actually helped revealing the issue.
Today I tried this again, with all updates applied in last few weeks. And it seems that it is resolved now, at least on my PC.
And yes, like someone above mentioned, malwarebytes adds even bigger delays. This was the case even on my old PC, the test was faster with Windows Defender than with Malwarebytes.
The next LINQPad beta will support Windows Dev Drive.
Stay posted!
The new beta is out:
https://www.linqpad.net/linqpad8.aspx#beta
This will automatically detect Windows Dev Drive and use it for compilation output.
this topic is thrilling me!
I get results from 300ms (managed PC online), down to 30ms (managed offline) and below 1ms (AV excluded).
But what does that acutally mean and what is actually happening in the perf test?
I compiled and ran the code outside Linqpad and it's not slow at all, even with worst AV settings. Is it actually testing a special technique used only in Linqpad?
LINQPad just runs .NET code. You should get the same performance, unless you're running off a different hard drive with different antivirus settings. Have you tried setting up a dev drive?
managed PC is Windows 10, no dev drive. code shows 300ms when executing the embedded LinqPad8 AV test.
running the same code in a .net8 console app doesn't seem to trigger the AV and is below 1ms.
So this is why I am asking. probably that does not trigger AV but hosted by Linqpad it does?
Windows Defender uses undocumented heuristics to determine what and when to scan, and what kind of scanning to use. I also don't know how your machine has been set up. Maybe you've got exclusions that cover the output folder, or maybe Defender has just decided not to scan your console app files for some reason.
When I run the same tests in Visual Studio, I get identical results.
talking about Symantec and plus a more shitload of AV to make you feel secure.
Always looking for simple benchmarks that prove the slow down of these tools.
using on DEV drive w/ windows defender real-time protection off and cloud-delivered protection off and also the "Core Isolation"->"Memory integrity" off, results are now between 30-90
but I guess might be something related to xeon and dual cpu - but reason that I bought this machine used was to improve performance, now I'm not sure if I got a bad machine, CPU test from Intel and all DELL tests pass w/out problem.
The following Microsoft diagnostic tools will help you find out whether antivirus software is actually still running, and what else might be slowing your system:
https://learn.microsoft.com/en-us/sysinternals/downloads/procmon
https://github.com/microsoft/perfview
Let us know what you discover.