System.OutOfMemoryException when checking Contains on a sufficiently large list

jaydavid

I suppose I'm not certain this is strictly a LINQPad problem and not a LINQ problem in general. If that is the case, let me know and I'll take this issue to the MS LINQ forums.

Take the following example:

// I haven't included the entire string of IDs due to the character limit, assume all even numbers are included
string idString = "500,502,...,4998,5000";

List idList = new List();

foreach (string id in idString.Split(','))
{
idList.Add(Convert.ToInt32(id));
}

(from r in records
where idList.Contains(r.External_ID)
select r.Internal_ID).Dump();

When I run this query, I encounter an out of memory exception. The details of the exception make it seem as though some sort of tree is being built based on the items in the idList. The SQL generated in the end by a smaller query that can actually run ends up essentially just using the IN operator with a comma delimited list of IDs.

Any ideas on how to avoid this, or better practices to follow when operating with a sufficiently large list?

Thanks.

JoeAlbahari

No, I don't know what's causing this. The following works without error for me:

var idList = Enumerable.Range (250, 2000).Select (e => e * 2);

(from r in Customers
where idList.Contains(r.ID)
select r.Name).Dump();

In terms of best practices, this kind of query is not ideal, because you're only 100 parameters away from the maximum number that SQL Server can handle.