Exchange cmdlets or PowerShell issue?

It looks like there are more and more ForEach-related topics coming out of blogosphere. This time I came across this post by Matthew Byrd on Exchange PowerShell behavior. In the post he compares two versions of Exchange 2007 one-liners: one with ForEach-Object and the other with direct pipelining. He then notices that the one with ForEach-Object is much faster:

The reason for this is simple and is tied into how PowerShell executes commands that are pipelined together. PowerShell executes the commands in a linear manner. So it will execute completely the first command in then pass the whole output to the next command in the pipeline. That means that if you have 30,000 mailboxes in your org and you run the shorter command you must enumerate all 30,000 mailbox objects before you pass it down the pipeline.

I am not sure this is PowerShell behavior really. Please correct me if I am wrong but I was under impression that if the left-hand cmdlet is using WriteObject(object, true) this cmdlet sends objects into the pipeline one at a time, and if the right-hand side processes the objects in ProcessRecord method, then all processing should be happening asynchronously and not after the first cmdlet is fully done.

So apparently something is not working “as designed” and either Exchange cmdlets are not using the interfaces right, or PowerShell v1 is not working “as advertised” (and using some undocumented “secret hooks” internally to make out-default, out-list, out-table, foreach-object, and other built-in cmdlets work with pipeline correctly – one object at a time).

Which actually does not change the outcome I guess: with Exchange 2007 ForEach-Object can be much more efficient than direct pipelining. Check your script performance – PowerShell gives you tons of ways to do the same thing and, as Matthew reports, some ways can be better than others!

But still, if you know whose issue this is please leave a comment. Maybe we need to post another CR to Microsoft Connect here…

Tags: , , , , , ,

5 Responses to “Exchange cmdlets or PowerShell issue?”


  1. 1 kirkamunro October 2, 2007 at 12:59 pm

    Nice. I just read that article and it certainly has a lot of mistakes in it! Hopefully my reply to that post will get read and hopefully they’ll respond to it. Posts like that are just creating more confusion when working with ForEach-Object and foreach and pipelining.

  2. 2 Vivek October 2, 2007 at 7:09 pm

    Actually, your analysis is not correct either🙂 Don’t jump before you claim something is buggy.

    Take it easy guys, the great thing about powerShell is its flexibiilty, and Matthew’s article is a great example of how people of varying abilities can utilize it. The idea is to attract users, not scare them away.

    Thanks, ~vivek

  3. 3 dmitrysotnikov October 2, 2007 at 8:04 pm

    Vivek,

    I don’t try to point fingers or anything but it would be great if someone could provide details of what is actually happening. To me it sounds that something is not working as designed: either the cmdlets, or PowerShell.

    My suspicion is that it is PowerShell which is to blame and that ForEach-Object and other “native” cmdlets are indeed using some kind of backdoors to provide “real” per-object pipelining.

    It would be great if someone could confirm and provide details rather than just saying that the analysis is not correct.😉

    I agree (and I mentioned in my post) that PowerShell is flexible to provide workarounds, but it would be great to know what is it that we are working around.🙂

    Dmitry

  4. 4 Bruce Payette October 5, 2007 at 9:35 pm

    Hi Folks,

    Matt has added a comment to his original post explaining what’s going on.

    Just to be clear, the default PowerShell cmdlets don’t have any special backdoors for pipelining data. By design, there are no “special built-in” commands in PowerShell. The shell works as I have described in my book. It’s up to the cmdlet to decide how and when it writes objects. Unfortunately, if the underlying store that the cmdlet is accessing doesn’t allow for incremental reads, then the cmdlet has no choice but to read everything at once which is essentially what’s happening here. See Matt’s comment for more details.

    Thanks,
    -bruce

    ————————-
    Bruce Payette
    Principal Developer, Windows PowerShell
    Microsoft Corporation.

  5. 5 dmitrysotnikov October 8, 2007 at 4:07 pm

    Thanks Bruce! Your comment here as well as Matt’s and Evan’s responses in the original thread helped a lot!


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s




My Recent Tweets

Legal

The posts on this blog are provided “as is” with no warranties and confer no rights. The opinions expressed on this site are mine and mine alone, and do not necessarily represent those of my employer - WSO2 or anyone else for that matter. All trademarks acknowledged.

© 2007-2014 Dmitry Sotnikov

October 2007
M T W T F S S
« Sep   Nov »
1234567
891011121314
15161718192021
22232425262728
293031  

%d bloggers like this: