Autotesting with PowerShell

We have started working on automating PowerGUI testing.

While this will not bring immediate benefit such as new features, etc. this should make our releases starting with 1.0.6 and beyond much more stable and bug-free. With our current schedule of new releases every 2-3 weeks manual testing simply cannot provide for high quality and allow many changes between releases.

We had a few options on the software to use to automate tests and decided to go with… PowerShell scripts!

Here’s what we’ve done:

1. Created a dll proving a wrapper which gives us things like global variables, STA threading model (we need this for the web browser control on the welcome page), initialization in a separate thread (so we have both PowerGUI and scripts running in separate threads), etc.

2. Created a PowerShell autotest script which basically runs all tests one by one and catches and reports error messages.

3. For each individual tests we create separate ps1 files doing just one of the tests. For example, this script “clicks” all nodes in PowerGUI left-hand tree without waiting them to complete.

P.S. This last test by the way already caught us a bug in 1.0.5 caused by our changes to asynchronous behavior. This fast click-through sometimes results in an error message being displayed. This is not fatal and the workaround is to simply click Refresh but nevertheless this is an issue which we now have covered in our tests and which will for sure go away forever in our next releases.

You can read more about Lightweight Testing with Windows PowerShell in May edition of MSDN Magazine.

Tags: QC, PowerGUI, autotest, PowerShell, software testing, .NET, QA, software