Using the RadEditor with Fitnesse

Since moving over to the WebDriver code for Fitnesse as a fixture, including the SpiderFixture extension, I have had better luck in getting some additional tasks to work. One item I had been trying to do in Selenium, and never could get it to work, was in making entries into the RadEditor we use on the SharePoint server the product runs on. With WebDriver there is an option to Execute JavaScript, I don't think it does well with multiple lines but I hadn't tried very hard to get things working in that way; what I ended up with was a single line of JavaScript that finds the iframe, adds the text then moves back to the base frame. The frame moving took me a few minutes to realize, once you move to a particular frame on the page going back to the base frame is a good follow-up.

For those who can use it, the JavaScript to get the RadEditor to work is:

execute javaScript

$find(${RadEditoriFrame}).set_html(${test_text});

My RadEditoriFrame was set up as: ctl00_PlaceHolderMain_zoneMain_ctl00_txtPost

Best way to verify that this is the one to use, especially on SharePoint, is to use an Object Viewer on the page and verify the editor box is highlighted.

Hope that helps you out, it took me awhile to get this working.

Setting up Pylot

I was in need of a Load Test Tool, now that I have Fitnesse up and running in my environment, I wanted to be able to check and see how our environment was going to behave under a significant amount of Users. Significant at this point is more than 5, so in looking around I came back to Pylot, not that I needed another language installed on my machine but I liked the simplicity and the ease by which within a day I was up and running with tests that were already confirming to me that the pages I knew were slow, are indeed slow.

Check the Getting Started Guide on how to get it up and running. But there is one thing missing, if you want to run the Script Agent you will need the Python Win 32 Extensions in addition to the normal set up.

When running the Script Agent and running back the XML script I did get a couple of errors about:
ERROR: can not parse testcase file: not well-formed (invalid token): line #, column #
It took a bit to work through this but basically what happened, in my case, was that the recording took in a NULL value somewhere in the CDATA line on the login and logout of my site, removing those allowed my script to run without a problem.

Once you get used to the XML format its easy to parameterize the testcases, and adding in new test steps for text validation and new pages, while the URL's are not the best mechanism in my site to use they do allow me to do a check against bookmarks on the site rather than clicking on the page elements.

I'm liking this tool a lot.

Fitnesse and Fitnium Fun

Some things I have learned when using Fitnium and Fitnesse.

SuiteSetUp and SuiteTearDown are not for actions, those need to go into regular tables, the SetUp and TearDown are not for Action Fixtures.

fit.exception.FitParseException: Can't find tag: table - typically means malformed HTML in a page but MOSTLY means you don't have a SetUp or TearDown in your Suite, and within those pages you need something, ANYTHING, even if its simple text.
Another possibility is if the wiki page containing the test has both the Suite and Test properties showing in the left property bar, both properties can also generate the error as its trying to run a Test but has nothing to do.

The jar file usually needs to be on any page its called on, don't trust the inheritance with the Parent pages or the Root page, you can try it but if you start getting function errors on the page - well now you know what needs to be added.


When adding URL's make sure they are not in camel case, especially URL's that have been parameterized otherwise the URL may not be sent to Selenium the way you think...and oh yes, PARAMETIZE everything.

Fitnesse Parameterization

In order to adjust for the multiple environments that the Fitnesse tests will be running in, within my environment later on, I have spent the time to parameterize the framework now and just added in some Symbolic Links. The Symbolic Links is a wonderful feature for Fitnesse, as it allows a wiki page to be called from another, so I can set up one page with a test, or a set of tests, and have it called from another. The true power in this comes from being able to set up multiple high level pages with specific environment information, so one set of tests can be run on Server A if its called from the Server A Test Page which has a Symbolic Link to Server Tests, I can also call the same tests from the Server B Test Page, which has different test settings, and reuse Server Tests within it. Simple, yet powerful.

It did take a little bit of playing around to get the set up working, so here is what I found out.

• Create the page that will contain the Symbolic Link
• Go to Properties and set the Suite property on the page and save it, as this page is calling another Suite is needed so the Page will go down the Hierarchy
• Find the path that will be in the Symbolic Link
• Go to Properties and set the Name for the Link then the path to the Page, click on Create/Replace
• Save the settings and edit the page with whatever is needed, the Symbolic Link if used once is automatically called

I don't know what will happen with the need for multiple calls, I have seen examples where instead of looping a page was set up as follows:
[Username1]
[Password1]
[Symbolic Link]
[Username2]
[Password2]
[Symbolic Link]
This should allow testing each Username and Password on that Linked Test Page, I have not tried this yet, have not had a need to, but it should be workable as I believe Fitnesse will take the settings it comes to last.

There is probably more to get out of Fitnesse that I haven't discovered yet, I can't wait to see what I find next.

Books On My Shelf

Over the years I have collected quite a few books, not always on testing as I used to be able to get enough online and through discussions with people to keep up. Occasionally I still bought the book that to me, was useful, or was a necessary part of the books I wanted to always take with me from Job to Job. So here is the list of what I have (your mileage may vary):

Testing Books (some of these come into the office with me, especially Lessons Learned In Software Testing, which I have pulled out from time to time to get a point across to Dev Managers), this is not in any special order:

• Quality Software Management - Volume 1: Systems Thinking, Weinberg
• The Complete Guide To Software Testing, Hetzel
• Effective Software Testing, Dustin
• How To Break Software, Whittaker
• Lessons Learned In Software Testing, Kaner & Bach & Petticord
• Managing The Testing Process, Black
• Customer Orientated Software Quality Assurance, Ginac
• Software Testing In The Real World, Kit
• Change-Based Test Management, Sistowicz & Arell
• How We Test Software At Microsoft, Page, Johnston, Rollison
• Writing Solid Code, Maguire
• Debugging The Development Process, Maguire
• How To Break Software Security, Whittaker, Thompson
• Implementing Automated Software Testing, Dustin, Garrett, Gauf

Technology Books (I use Perl ALOT as you can see), since I am mostly self taught these are all the things I have needed for my job in past...most of this is O'Reilly, a geeks best friend.

• Learning Perl On Win32 Systems, Programming Perl, Advanced Perl Programming, Perl Best Practices, Perl CD Bookshelf, Mastering Regular Expressions, Perl Cookbook
• Debugging Perl
• Learning Bash Shell
• Sed & Awk
• JavaScript
• CGI Programming
• SQL In A Nutshell
  
• Unix Shell Programming
• Unix In A Nutshell
• Python Visual Quickstart Guide
• Python Programming For Beginners
• Python Essential Reference
• Unit Test Frameworks
• Teach Yourself Java
• Java Development With Ant
• Open Source Programming with CVS
• Unit Test Framework, O'Reilly
• Pragmatic Unit Testing, Hunt, Thomas
  
• Practical Performance Analyst, Gunther
• Aptana RadRails, Ramirez
• Test Driven .NET Development with Fitnesse, Adzic
• Fit For Developing Software, Mugridge, Cunningham
• Head First C#
• Windows PowerShell In Action

I've been asked if I really had to choose my top 3 I'd say:

1. Hetzels book The Complete Guide (as it was my first)
2. Maguire's Debugging The Development Process
3. Lessons Learned (which I actually bring into my office a lot)

Others I received about the same time I originally made this post are:

Web testing handbook - Splaine and Jaskiel
Software testing - Ron Patton
Systematic software testing - Craig and Jaskiel
Software testing techniques - Boris Beizer
Black box testing - Boris Beizer
Test process improvement - Koomen & Pohl
The testing practitioner - Erik van Veenandaal
The Art of Software Testing – Glenford Myers (yes, I know it is old, but still good) 

PowerShell fun with Fitnesse

Fitnesse is the newest automation tool I have introduced here, we also use PowerShell to drive a lot of the builds, because there is some integration between the two (think Smoke Test) I worked on getting the TestRunner working with PowerShell. I couldn't get the .Net version to run properly, it kept dying when it would render the page, but the Java runner worked great once I was able to work through the java command line with the BuildMaster. If not for his help I wouldn't have been able to finish.

This script launches the Java TestRunner, points it to the Fitnesse server, and runs the three tests (one Test and two Suites) I currently have set up, then scans through the XML results to display some results in the command line window.

# Variables for the script
[string]$fitniumLocale = "C: itnium"
[string]$fitnesseJar = "fitnesse.jar"
[string]$fitnesseClass = "fitnesse.runner.TestRunner"
[string]$serverName = "my_server"
[string]$portName = "8888"
# Use a Test Array to be able to bundle all the tests
$testArray = ( "Test1" , "Test2", "Test3" )

function main()
{
Write-Host "Running Fitnesse Tester.`n"
cd $fitniumLocale
# Run through each test in the bundle
$Java = "java.exe"
foreach ($test in $testArray)
{
# Make a date string to get the right file date/format
$dateStr = [string] (get-date)
$Month,$Day,$Year,$Hour,$Minute,$Second = $dateStr.split(" :/")
$testResults = "$test" + "$Month" + "$Day" + "$Year" + "$Hour" + "$Minute" + ".xml"
[string]$opts = "-xml $testResults"
# Start the test
Write-Host "`nRunning $test`n"
# Get the output from Java so we can scrape it for the test passes and fails
& $Java "-cp" "$fitnesseJar" "$fitnesseClass" "-xml" "$testResults" "$serverName" "$portName" "$test"
Write-Host "Completed $test`n"
Write-Host "Test Results"
# Scan through the output file, we want that for archiving
[System.Xml.XmlDocument] $xd = new-object System.Xml.XmlDocument
$file = resolve-path($testResults)
$xd.load($file)
$nodelist = $xd.selectnodes("/testResults/finalCounts") # XPath is case sensitive
foreach ($testCaseNode in $nodelist)
{
$right = $testCaseNode.selectSingleNode("right").get_InnerXml()
$wrong = $testCaseNode.selectSingleNode("wrong").get_InnerXml()
$ignores = $testCaseNode.selectSingleNode("ignores").get_innerXml()
$exception = $testCaseNode.selectSingleNode("exceptions").get_InnerXml()
}
# Send a summary of the results to the console
Write-Host "Passed: $right" -ForegroundColor Green
Write-Host "Failed: $wrong" -ForegroundColor Red
Write-Host "Ignored: $ignores"
Write-Host "Exceptions: $exception" -ForegroundColor Yellow
}
}
main


It needs some cleaning up, but its a good first step that runs well, I especially like the color coding for Pass, Fail and Exceptions so I can visually be drawn to the values I want to see.

I'm in QA and I'm a Doubter

Yes, I doubt. I doubt a lot, especially on projects that tell me that things are working good and we're on target, I sit back and think - am I missing something? I think its partly endemic to people who work in Test, because let's face it we are in the position of being the person next in line from when something is done and there needs to be a step to make sure that the something is done right. If you've been in this position this may be familiar to you:

Developer: Hey, the code is ready I'm going to hand it over to you later, I'm done with it and it looks good!
Tester: Thanks for letting me know, I'll get back to you once I check it out.

Then you do, and your getting back to the Developer involves defect reports. Was it good? Probably, it met what the Developer was trying to do with it, there were Unit Tests which ran and passed, or what was the minimun to pass did, so it was handed to Test. Now, Test has its own tests to run, its plan and its criteria to be able to say that things are stable enough to put it into Production and let Customers use it. Testers usually have Tests that Developers never run, or want to get into, so they are looking at things from a higher level, sort of like the Ranger in the Watchtower, we have our scope up and we are looking all around. Meanwhile, the Developer was just doing something over in one place and getting that to look nice, while nearby another Developer was doing the same thing but somehow they will collide. Testers are there to raise the warning, hey there is some cross-purposes here or to see that there may be, if its been communicated that Developers are working together on something and maybe no one involved realizes there is some dependency.

So where does the doubting come in? Remember those Customers? Well, if a Customer sees something wrong and calls up with a problem, or is angry that there is a problem, many companies will look to find out why that problem got out there. Someone has to take responsibility for it, and because we have that person in Test between the Developer and Production its that person's fault that anything got past, not that the Developer put it there - by mistake or accident - but that the Customer saw it. To avoid that I became a doubter.

Now, being a doubter can be a strength, it makes you question everything and you look for issues that may need resolution, but to be KNOWN as a doubter is pretty bad. Then you get a reputation, and sometimes people won't want to work with you because they know you'll come back with issues, complaints or will be negative right away. What is needed is something I like to call Constructive Doubt.

Constructive Doubt is what I do when I get code and need to look at it, I won't immediately snipe back and say "heh, more bugs for me to find?" when a Developer comes and says something is ready. Diplomacy is part of Constructive Doubt, because you want to build with it, anyone can tear down something, but it takes real skill to build something. Use that doubt to figure out some new way to test, use a new tool, a framework or something that will allow you to doubt in an automated fashion. Sounds like I am talking about Testing, but in reality part of a Test is a doubt, you doubt something is going to follow the requirement, sure you could call it a Verification or a Validation but in the middle of it all is a seed of doubt; you want to KNOW that something is doing the right thing in the right way. Sometimes when I have tested something enough I know it inside and out I sit back and think, how else could this fail? How else could I figure out that something failed? Where else could I doubt the process is working? Then you look, you learn and sometimes you find something new.

I'm in QA, I doubt and I build up on that.

Updating Subversion

Yesterday I went through the process of upgrading our soon to be used Subversion server, we are moving from VSS to SVN and the initial install of Subversion was done by our BuildMaster. I got asked to do the upgrade and bring things up to current versions and document the steps, so for posterity here they are.

Subversion installation used 1.6:

http://subversion.tigris.org/svn_1.6_releasenotes.html

From Compatibility Concerns

• Older clients and servers interoperate transparently with 1.6 servers and clients. However, some of the new 1.6 features may not be available unless both client and server are the latest version. There are also cases where a new feature will work but will run less efficiently if the client is new and the server is old.
• There is no need to dump and reload your repositories. Subversion 1.6 can read repositories created by earlier versions. To upgrade an existing installation, just install the newest libraries and binaries on top of the older ones.
• Subversion 1.6 maintains API/ABI compatibility with earlier releases, by only adding new functions, never removing old ones. A program written to the 1.0, 1.1, 1.2, 1.3, 1.4 or 1.5 API can both compile and run using 1.6 libraries. However, a program written for 1.6 cannot necessarily compile or run against older libraries.

Subversion upgrade process

o Turn off the Apache Service

o Prepare Subversion for upgrade (back-ups)

1. Zip up directories to keep a rollback in place

1. Back up the configuration files under Apache (can be done within the Zip file)

1. svnaccessfile

2. conf/httpd.conf

3. other stuff!

2. Move ZIP files to a new location as storage

o Upgrade Subversion

1. Unzip the upgrade

2. Move the binaries into place

1. Copy /bin directory to install location

2. Copy D:Program FilesSubversion inmod_dav_svn.so and mod_authz_svn.so to the Apache modules directory

3. Copy the /iconv directory to install location (this helps to update the version numbers for Subversion within the Apache footer - at least I think it did, my cache was acting up in FireFox)

o Upgrade Tortoise

1. Install Updated software package

o Reboot

o Create a new Repository:

1. Svnadmin create d:SourceControlSVNMAIN

o Test/Verify Upgrade

1. Connect using the Tortoise Repo-Browser from a remote machine

1. Connection should allow viewing of repository

Backup Process

Using FSFS as the database backend.

Criteria for the backup:

The process should run [nightly?? (this can be adjusted depending on how long it takes)] using Windows Scheduled Tasks.

Use the following command from a Windows Scheduled Task:

svnadmin hotcopy (repository path) (backup path)

The backup process will copy server configuration and hook scripts.

Keep the backups on an area that is backed up (get location from IT)

Is it possible to "share files" in SVN?

If you use the svn:externals –r option you can, but it seems SVN can only do directories not files

http://blog.kowalczyk.info/article/Short-tutorial-on-svn-propset-for-svnexternals-p.html

Security/branches:

· Authentication via AD

· Groups and user access defined in svnaccessfile

· We want to have two groups with users defined to each

· By default all Company group users will have all access; Consultant group users have access to only a branch (and possibly build stuff)

· Out of the gate just the trunk, all users edit there. More branches down the road.

Sites with additional information:

http://www.sublimesvn.com/blog/2009/02/configuring-subversion-with-active-directory-authentication/

http://concise-software.blogspot.com/2009/02/instant-windows-svn-server-with-ssl-and.html

Fitnesse and Selenium as Services

Because I am working with Fitnesse and Selenium, I wanted a way to be able to remotely access tests on a central server, right now I am setting up the services on my own machine. These are the instructions I used to get things in place.

Set-up Fitnesse as a Service:

• Install the Windows Resource Kit if its not already in place
• Run the following command:

instsrv Fitnesse "C:\Program Files\Windows Resource Kits\Tools\srvany.exe"

• If the resource kit is not in that location update the path, but you MUST use the full path to srvany.exe or it will not work
• Modify the Registry key in HKLM\SYSTEM\CurrentControlSet\Services\Fitnesse
• Change the Image Path to point to Srvany.exe's full path (if not already done so)
• Add a new key - Parameters HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Fitnesse\Parameters
• and three string values,
  AppDirectory - AppDirectory is the root directory of our service where the Fitnesse bat file resides
  Application - is the java.exe executable needed to run our service
  AppParameters - is the parameters used by the java.exe to start our Fitnesse service.
• In the Services panel verify that the Account being used to start the service has the right permissions
• Start the service
• To access the Fitnesse Web UI use the URL - http://local_ip_address:port/

Items to note:

• If the Service starts, but does not open the port make sure Windows Firewall is set to allow access to the port.
• If the Port is open but the open port is not appearing in a netstat -a then use the command line arguments to verify that the command line options work.

Example settings for Fitnesse Service:
AppDirectory - c:\fitnium
Application - C:\Program Files\Java\jre6\bin\java.exe
AppParameters - -cp fitnesse.jar;selenium-java-client-driver fitnesseMain.FitNesseMain -p 8888

Selenium Server:

• In order to assure that Fitnesse can run its tests Selenium needs to be set up the same way
• Modify the Registry key in HKLM\SYSTEM\CurrentControlSet\Services\Selenium
• Change the Image Path to point to Srvany.exe's full path (if not already done so)
• Add a new key - Parameters (HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Service\Selenium\Parameters)
• and three string values,
  AppDirectory - AppDirectory is the root directory of our service where the Selenium bat file resides
  Application - is the java.exe executable needed to run our service
  AppParameters - is the parameters used by the java.exe to start our Selenium service.
• In the Services panel verify that the Account being used to start the service has the right permissions
• Start the service

Example settings for Selenium Service:
AppDirectory - c:\fitnium
Application - C:\Program Files\Java\jre6\bin\java.exe
AppParameters - -jar .\selenium-server-1.0\selenium-server.jar   

Fitnesse and Selenium

I had decided to go with Fitnesse and Selenium as part of our Acceptance Test Framework, it was a challenge to say the least, and with the options that were out there it was bit of work and testing to find the best combination to get things to work in our environment. Basically, I am testing an update to a HealthCare Process Improvement site, nothing like being a QA person on a project like that I must say! So my tests boil down to being able to login and do certain functions, as well as just roam the site and check for pages that are not working, in the midst of this I wanted some structure so that I could do things like have login and logout before the tests so I just set up a session and do everything. Sadly, it did not work out that way.

WebTest Fixture
I went through Gojko Adzic's Web Test Fixture for Selenium and Fitnesse, this I struggled with for a little while but I liked the ability to use .Net or Java and the language to call Selenium was pretty simple. Though you have to remember to use those !'s on certain commands, something I found through trial and error with Start Browser, and it worked out pretty well. Getting Suites to work is something that I started with here, and had a lot of problems with, I kept getting fixture errors in trying to set up my suite that really became a stumbling block, seems as if I am destined never to get the fixture library to carry through the suite and putting it on every page became a problem after awhile. Biggest problem though was the version that is complete on the site is an older version of Selenium that does not work with FireFox 3, so if you have that on your machine you can either do your own buildup, or try to update the Selenium libraries in the installation. You could probably compile your own code and get things working that way as well, but its not the method I wanted to go with so I eventually gave up on it.

Fitnium
Something I came across and this worked out pretty well, moreso once I resolved my whole Login and Logout issue, and the fixture on every page problem that seems to plague me is something that carried here as well. Still, I was able to work out my issues before getting responses to my posts on the Fitnesse Yahoo Group, not sure if that is good or not but I will say its a positive that I could do it. Again a big win for me was being able to work out the fixture and suite problem here, although some of the Fitnium commands could be a little easier to remember but that I can work through.

Fitnesse
I do like the Acceptance Tests you can make, and the Suites, they will definitely help with some of the work we need to do for our product, I think my biggest issue is like what most people say - you need to write the tests. Structuring them in a logical way, or what seemed logical tome, did not work as well and gave me trouble until I used SetUp and TearDown for my tests and put the Login and Logout steps in those pages rather than the SuiteSetUp and SuiteTearDown. I don't know why that happens, but it'd be interested to know. Writing the tests can get time consuming, and I still want an easy way to get Selenium scripts to output into Wiki pages, Cory Foy's script did not work for me and I have no idea why (I don't do JavaScript much). The documentation can be confusing and incomplete, I mean where is that !r command defined? I even bought Gojko Adzic's book to help me get going, and I was able to get some nice stuff out of it, but being a doer rather than a reader I didn't get as much out of it as I thought I would, but I'm keeping it handy, who knows, I may get more into this.

Overall, it worked out well, but I would say if you are going to set these up:

• Plan out your tests, and how you want them to work
• Consider a few alternatives, and your Users and see what works
• Read up on the tutorials and see if you can go through the code there, if not getting knee deep in this will only be frustrating in the future
• Be sure these are the tools for you, Fitnesse can be daunting to a techie, and it will be moreso to a business user

Other than that, I'm glad I went through this and now have Selenium and Fitnesse working in my environment.

Automation in its many forms

So in order to get things working smoother I have spent a couple weeks researching some automation tools so that once we really start getting into coding we have something in place to make testing easier, not only for me but anyone else who wants to check out their code.  I've gone through a number of demos on my machine, yes there has been much installing and uninstalling the past couple weeks, in order to assure that I have chosen the right tools.  Not just GUI test tools, since the project is an update to an existing web site, but test case tools and frameworks that will help with automation, also being able to do load and performance testing at some point would be nice.  This is what I have come up with:

Seleniun is the GUI tester of choice, why was because it was easy to generate tests and although the Ajax and Rad Editors on the site don't work as well with Selenium as I'd like I am ok with the manual checking of a couple parts of the site.  Better than all of it, plus being able to export the scripts into C# has allowed me to take the tests we have, import them into a test library and then run those through the PowerShell build scripts with Gallio to give us test reports.

I wanted a Test Case Management Tool so went through some research and the only ones I came up with as possibilities were:

• Test Link was my first choice, since I had heard good things about it from people in my last company, though I didn't like having to install Apache, PHP and MySQL on my machine I figured it's worth a shot.  Basically I wanted something simple, and I got around the MySQL option by utilizing my local MSSQL install, with some work, and then I got to work.  While it was interesting I thought the interface was a little unwieldy, not that intuitive, and even with the help files and forums I had a hard time getting my fields to show up on Test Cases at the times I wanted them.  My simple case was to show Bugs added to a Test Case, so they could be seen on early reviews, and modified when they were worked on, I couldn't get it to work right even after a struggle.  Some of the optioins were right but with some other comments about large numbers of test cases causing issues, I decided it was not worth it, considering half my Users were going to be non-Engineering folks it was not going to work for me.
• Fitnesse was my next choice, as a wiki with some human readable language would be a good way to generate some Acceptance Tests, and Test Cases, for the project.  I figured at least some of the Business Users could get used to a wiki, after all most are writers for the sites content, although I was later told that no it would not happen I knew the wiki idea would be more acceptable in some form.  Besides, with Selenium, if I could get the scripts exported into a wiki format then I could easily script tests then put them on the wiki, so we'd have them as smoke tests for the build and as automated tests on the wiki.  All I needed was something simpler for the Business folks.
• Web Test is an add-on developed by Gojko Adzic that works with Fitnesse, he does a lot with Fit and Fitnesse with training, seminars and books (one of which is being delivered to my home today [Sept 11, 2009] as I write this).  This add-on makes writing the tests in the wiki simpler, and works in Java and .Net forms, but I had a lot of trouble getting this to work properly with the tests when I was trying to write my Suites, and really get some power out of Fitnesse.  Not sure this is being actively developed since the last comments on the site were from 2008, so I had to let this go and see if there was another way.  Note: Gojko has noted in the Fitnesse Yahoo Group that he is no longer maintaining this, so I'd skip it as an option. 
• Fitnium looked to be that other way, right now I am implementing my tests in it, the install is easy and it did not take much for me to get the libraries in place, WebTest was tougher for me to use since the libraries I could get did not work with the 1.2 version of Selenium I wanted to use and I was not sure how to compile new ones, yet Fitnium worked right away.  It's language is even simpler, and I had all I needed right out of the box, so its the one I am going with because of that ease of use, and hopefully understandable code for Users who can then write their own tests in the future.  It's very nice, and I like it alot, once I get my Fitnesse and Fit books I can expand my use of it.

Now that I have Selenium and Fitnium going I am starting to look at some Load and Performance tools, so far I have the following to check out:

• JMeter, which I think I used in the past but it was almost 10 years ago.
• Pylot, a python based tool developed by Corey Goldberg, a fellow forum member
• DieselTest, never heard of this but its Windows based and matches out environment
• OpenSTA, which I used many years ago and had a hard time with, maybe this time will be different.

It's nice starting up a group from scratch, even when you are really the only official QA person on the project and in the group, but its also nice when you can share tools with people who also take a look and they all agree, "that one is cool!".

Defining Agile

I was running through some blog links this morning and came across this gem from Elizabeth Hendrickson, someone whose books I love but hadn't put her blog in my aggregator for some strange reason.

From this blog post Defining Agile Results, Characteristics and Practices
comes this gem:
"I define Agile in terms of results. Specifically, Agile teams:

• Deliver a continuous stream of potentially shippable product increments
• At a sustainable pace
• While adapting to the changing needs and priorities of their organization"

I like the way its spelled out simply, concise and is the goal of many of the Agile groups I have worked with, or have been trying to get that way.  I am definitely going to be using it, and thinking of it, when I work on my upcoming projects.

Starting in a New Place can be Fun

After starting at my new position I needed to come up with some new test strategies, it was a new product that had not yet had an experienced tester on it, and it was still under development with changes coming.  Spending a couple of days on the web application I got to learn all about how Sharepoint seems to work, and how Silverlight is delivering content, so having to jump back up to GUI level testing from back-end system testing I decided to take a close look at the future.  To me UI testing gets repetitive fast, while you are new you can find a lot, and that's originally what my bosses wanted, someone new to the project with a testers ability to see and plan things out.  So I decided to look around and see what could I eventually use to script this, and it had to be something to script things fast, I ended up trying and looking at one or two tools but ended up with Selenium.

It gave me what I wanted in quick scripting, allowed me to deal with IE and FireFox, which were the browsers we were using and while there is no official decree on support they are the ones the VP has on his box, and as has been said "we support what WE use."  WE being the Engineering team.  So I was covered there, and since the Build work is being done with C# and PowerShell the ability to export Selenium scripts as C# code that I could combine into a test library was a boon, I've built some simple scripts and made a small library that I run daily now.  This was exciting to our build master, who does some checking now but its all manual and right now he sees the ability to script it all with PowerShell, so I took that on and I run my test library with PowerShell.  How you ask?

Well I looked at .NET frameworks, because it would be nice to run tests with the builds and do some continual integration, but the build system is VSS and is going to be upgraded to Subversion soon, but no one knows when soon is but I was told not to worry about integration yet.  After all, they've gotten along fine without it so far.  So what was left?  PowerShell, but there is no reporting in PowerShell, so I went out and found something called Gallio which has a command line runner that can take my test library and run it.  So I can run my Selenium tests, through PowerShell and from Gallio I get a report of my test, including timings on how long the site takes to respond; useful information for a product we can all say acts slow but no one can quantify it or wants to look at that yet.  I am gathering timings as I run my tests and recording them for the future, at some point we will want to look at this and how the site has responded over time, but not yet.  So I'm planning for it.

In a few short weeks I've been able to capture a lot of material in a test plan, review major portions of the site and come up with some nice test cases, generate User Scenarios, talk to Support and some of the Content people to come up with additional cases based on what exists now.  After all, this will be a transition to a new site, and Users will need to migrate too, so why not make sure they are covered?  Plus I've been able to play around with PowerShell, C#, Selenium and Gallio all stuff that's new for me, and who says learning can't be fun?  I love being in spots like this, and its nice to see people excited about testing, frameworks and tools so that we can all do our jobs better.

I'll be adding more material here as I go along, so if you've been in a similar situation you might get some ideas.

Some useful links:

Selenium Home Page
Selenium Help Forums
Gallio Automation Platform

Microsoft's PowerShell Script Center