Memory usage in 5.31

ulfius · November 2018

I'm extracting a lot (1000s) of varbinary records to the file system and memory usage starts out at a couple hundred MB, but slowly grows until the system crashes (if I'm not around to stop and start the script).

Here's the code using a Linq to SQL connection.

var folderBase = "\\\\myserver\\share$\\files\\";

var records = Forms.Where(r => r.PDF != null && r.ImageGuid == null).Select(r => r.ID).ToList();
foreach (var rec in records)
{
	var image = Forms.Single(i => i.ID == rec);
	var file = image.PDF.ToArray();
	var docGuid = Guid.NewGuid();
	if (!Directory.Exists(folderBase + docGuid)) Directory.CreateDirectory(folderBase + docGuid);
	File.WriteAllBytes(folderBase + docGuid + "\\" + image.ID + ".pdf", file);
	
	image.ImageGuid = docGuid;
	
	SubmitChanges();
	image = null;  // try this to release memory
	file = null; // try this to release memory
}

nescafe · November 2018

I'm not really familiar with the object change tracking of LINQ-to-SQL, but you could bypass this by creating (and releasing) a data context for each record (using new TypedDataContext, var rec = Forms.FirstOrDefault(), update, dispose by end using block).

Or simply bypass EF for updating:

ExecuteCommand("UPDATE Forms SET ImageGuid = @p0 WHERE ID = @p1", docGuid, rec);

Memory usage in 5.31

Comments