DumpContainer.Refresh() memory leak in connection with arrays of reference types (LINQPad8+)

DarthGizka
edited 6:13PM in General

I've encountered a massive memory leak in LINQPad 8 and 9 in connection with dumping arrays of reference types into a DumpContainer and then repeatedly calling Refresh() on it. This makes LINQPad exhaust even 32 GB or 64 GB of RAM in mere minutes but my script needs to run for a couple of hours.

Note: the problem does not occur in LINQPad 5, only in versions 8 and 9 (both Windows and MacOS). In LINQPad 7 the query process is not affected but the associated msedgewebview2.exe process eats gigabytes like there's no tomorrow (which I didn't notice at first because it was listed under the process group for a LINQPad 7 beta, not under the process group for the LINQPad 7 instance itself).

A minimal example for reproducing the effect can be derived from the code given in memory leak when dumping exceptions by CodesInChaos with a small change to the DumpContainer constructor argument:

- var dump = new DumpContainer(state).Dump();
+ var dump = new DumpContainer(new [] { state }).Dump();

This gives:

void Main ()
{
    var state = new State { Ex = new Exception() };
    var dump = new DumpContainer(new [] { state }).Dump();

    for (var i = 0; ; i++)
    {
        state.Counter = i;
        dump.Refresh();
    }
}

class State
{
    public int Counter;
    public Exception Ex;
}

(Note: I removed the loop condition because it does not serve any purpose here)

Here is an alternative version that is closer to my use case and which eats memory much more rapidly:

void Main ()
{
    var content = Enumerable.Range(0, 1000)
        .Select(i => new StatsLine(i))
        .ToArray();
    var dump_container = new DumpContainer(content).Dump();

    for (var i = 0; ; ++i)
    {
        content[0].A = i;
        dump_container.Refresh();
    }
}

class StatsLine 
{
    public int A, B, C, D, E, F;

    public StatsLine (int i)
    {
        A = B = C = D = E = F = i;
    }
}

Is there a way the shed the excess memory via some trick or other? I've tried calling DumpContainer.ClearContent() or Util.ClearResults() but to no avail.

I've attached the script I'm working on as illustration for where this is coming from; I was trying to make a simple UI for a script that sends millions of API requests to a herd of slave computers, with different degrees of parallelism for each slave. Depending on network weather and other factors I need to tweak the DOP for individuals slaves on the fly in order to maximise overall throughput, hence the detailed statistics.

This script also shows the effect of the foreground process boost under Windows. The framerate goes up by about 25% when LINQPad is in the foreground and it goes down again when another process gets the focus.

Answers

  • DarthGizka
    edited 4:56PM

    Update: LINQPad 6 and 7 can be tamed by switching to the older, IE-based rendering engine. No such luck with LINQPad 8 and 9, though, since there the query process itself leaks memory as well, not only the msedgewebview2.exe process.

    I also learned another useful thing during this investigation: cancelling the query process does not free any memory hogged by its msedgewebview2.exe companion. That goes away only when the correspoding tab (or the whole LINQPad instance) is closed.