Optimize PowerShell Performance and Memory Consumption

PowerShell can be pretty resource intensive especially if you use it to retrieve data from your directory. I thought it will be useful to share this issue and workaround which were recently discussed in the PowerShell newsgroup.

SYMPTOMS

When trying to retrieve all user objects for subsequent processing with the following code:

$users = Get-QADUser -IncludeAllProperties -SizeLimit 0

the computer throws the following exception:

Get-QADUser : Exception retrieving members: "Exception of type
'System.OutOfMemoryException' was thrown."

(Looks like a correct line, right? SizeLimit set to zero makes PwerShell retrieve all user objects, IncludeAllProperties makes it retrieve whole user objects (not just the default set of attributes), then the collection is assigned to a variable for subsequent use. So why the exception? Read on!)

CAUSE

In reality on a fairly big domain this single line quoted above consumed all the RAM the machine had by basically retrieving the whole AD database wrapped into PowerShell objects.

You should avoid retrieving and keeping in memory the data you don’t need in your scripts. See information on how this can be done below.

RESOLUTION

Here are a few important tips to keep in mind to optimize your PowerShell code when working with Active Directory:

Use Pipeline

Don’t save the whole collection of objects to a variable. This way you make PowerShell retrieve all the objects and keep them the whole session. Use pipeline instead – which makes PowerShell pass the retrieved objects one by one to the next cmdlet.

So instead of:

$users = Get-QADUser
ForEach ($user in $users) { here goes the code }

Use:

Get-QADUser | ForEach { here goes the code }

Retrieve Only What You Need

-IncludeAllAttributes is a dangerous parameter because it, well, includes all attributes. Even the binary blobs you won’t have any idea on how to use. If you are ok with the attributes the cmdlets retrieves by default (to get the list just run Get-QADUser | Get-Member) simply use:

Get-QADUser -SizeLimit 0

If you need a couple additional parameter, use -IncludedProperties switch to add them and just them (not all the attributes!)

Get-QADUser -SizeLimit 0 -IncludedProperties proxyAddresses

If you need to optimize even further, limit the retieval only to the attributes you need by using the DontUseDefaultIncludedProperties switch:

Get-QADUser -DontUseDefaultIncludedProperties -IncludedProperties
SamAccountName,proxyAddresses

Filter Using cmdlet Parameters

Finally, if you need only a subset of objects use cmdlet parameters to do the filtering – and not the where clause.

So instead of:

Get-QADUser | where { $_.City -eq Amsterdam} | { here goes the code }

Use:

Get-QADUser -City Amsterdam | { here goes the code }

The difference is huge. In the former case, PowerShell retrieves all user objects and then does the filtering on the client. In the latter, the filtering becomes a part of the LDAP query and thus is made automatically on the domain controller during data retrieval.

SUMMARY

The ideal PowerShell script:

1. Does not keep the whole set of objects but processes them one by one upon retrieval.
2. Retrieves only the attributes it needs.
3. Filters objects during retrieval.

Get-QADUser -City Amsterdam -DontUseDefaultIncludedProperties -IncludedProperties
SamAccountName,proxyAddresses | ForEach { here goes the code }


Tags: PowerShell, AD cmdlets, cmdlets, KB, Known Issues, Knowledge Base, Active Directory, AD