MIA - well I have been in action but not really missing...

So, 2011 was a busy year working with SharePoint. We did a major site upgrade where the old public site running on IIS was moved over to SharePoint, that was a major effort and took a lot of coordination with a lot of people. It came off really well, and has been successful since not many people noticed the underlying change, only the new look of the site. To me, that is a major success.

In addition I went through the task of learning SharePoint, as well as it's Search functionality, we had it going well for awhile then things just stopped. We still have no idea why, but it's been slightly educational in learning how things work. A nice environment where people just jump into things, help out where needed, and take the time to learn. That's what makes this job great.

Testing was a big focus, not only because we added a new person to help test last year, though she moved on it was good to sort of settle things and document much more. Teaching is always interesting since everyone comes in with a different focus and idea, that helps me readjust my thinking when I start to train someone. From that I have more wiki pages, more places to document what we have been doing and more documentation on just everything. I also got a start on automated testing, I now have a Framework with SpecFlow where I am now using BDD to create automated cases for Functional and Acceptance testing. First time the Acceptance Test was used after a deploy we found an error, that is success to me.

I needed to add goals for the year, which I am bad at, for myself I have overall goals but they get subsumed by whatever is going on at work at the time. Priorities come up and I need to learn something, say about writing a PowerShell script here or there, always something new. Though I think I am better now at focusing more on what I need to do and I will be going through that as a task for 2012. Should be an interesting year and I hope to do better about recording things here as well.

Troubleshooting SharePoint Issues

Since I am still in the midst of lots of SharePoint work, one of the recent items I started on was a quick list of things to check when troubleshooting. As SharePoint is a framework and a platform it's often hard to find out where issues are coming from, so I use this as a guide to track down errors:

• Check ULS logs with the ULS Viewer
• Verify web.configs for the affected Site
• Check the Event Log on the Central Administration machine or the Web Front End for errors
• Turn on/off Custom Error Reporting for Local/Developer/Test environments only
• Check IIS logs on the Web Front End for any errors not caught by ULS (this is a rare case)
• Clear page cache
• Assure the SharePoint User Permissions are correct

I also have a few locations I check to see if anyone else has my same issues

• MSDN Forums
• SharePoint Overflow (soon to be StackExchange Overflow)
• The SharePoint Homepage, to check for updates

I've found SharePoint to be difficult to check sometimes because its not just a web server, often pages are encapsulated within what are called WSP files, these are basically templates that pull data from a database and build the pages. It makes tracking issues down complex as often the problem is code related, not a missing bracket or end tage on a page.

SVN Release Notes with PowerShell

The environment I work in now is hevily Window-ified. This is due to the extensive use of PowerShell for the Build/Release processes and the fact the major project is a SharePoint installation. Now that we have moved to Subversion and have packages coming out, sometimes with only small fixes, I decided I needed a way to be able to get some information on a package. To be able to include this in the Build scripts I needed something in PowerShell that would query Subversion and give me a list of comments from all commits since the last release version. This worked out well, took a bit to figure out, but it came out nice in the end.

function GetLog([string]$target, [int]$base_version, [int]$target_version, $relNotesDir="C:\Temp")
{
if(($relNotesDir -eq $null) -or ($relNotesDir -eq "")) {
"The " + $relNotesDir + " is empty. `n"
$relNotesDir="C:\Temp"
}
if (!(Test-Path -Path $target)) {
$target + " does not exist.`n"
"Try the $target path again, make sure it is typed correctly and exists.`n"
exit
}
if (!(Test-Path -Path $relNotesDir)) {
"The Release Notes directory " + $relNotesDir + " does not exist.`n"
"Enter an existing directory and try again.`n"
exit
}

"Looking for changes from " + $base_version + " to " + $target_version + " in " + $target
"Writing Release Notes out to " + $relNotesDir
$releaseNotes = Join-Path -Path $relNotesDir -ChildPath( "releaseNotes_" + $target_version + ".txt")
$Command = "svn.exe"
$Range = "" + $base_version + ":" + $target_version
$Params = "log",$target,"-r",$Range,"--xml"
$svnlog = [xml](& $Command $Params)
$svnlog.log.logentry | foreach {
if ( $_.revision -gt $base_version ){
$a = ( $_.revision + " : " + $_.author + " `n " + $_.msg )
$a >> $releaseNotes
}
}
}

Hope this helps someone out!

Running Fitnesse with PowerShell - RESTful Tests

Previously I noted how I had written my original tests with Fitnesse to use the TestRunner and access Fitnesse in a way that worked, but as I recently discovered is no longer supported. Oh well, so I update my tests. Which was necessary anyway, since on the Build Machines where these tests would be run Fitnesse is not going to be running, and if it was I shouldn't need to usurp it for my tests. So I moved over to using the RESTful method, which I had a hard time with earlier, but now is so insanely simple I wonder why I had a hard time with it before.

Without futher ado, here is the RESTful method to run Fitnesse tests.

First I set up my variables, I've also added in an archive file now, so I can keep track of my results, the RESTful results are nice so I actually output each test into its own results file, and I make an aggregate to be an archive. Eventually I will get rid of the individual results and keep the archive around - this also let's me go onto any machine that ran a test and see how things have been going. The Arrays I have are suites that I set up in Fitnesse to contain my tests, I have one for QA that I would run, and a Smoke suite that is a selected set to verify the build worked.

# Get the current directory Push-Location .
# Variables for the script
# This location should not change, since its the standard checkout location [string]$fitnesseLocale = "C:\MAIN\trunk\AutomatedTests\Fitnesse\Fitnesse"
[string]$fitnesseJar = "fitnesse.jar"
# Need to get the local machine here, it will be how Fitnesse runs [string]$serverName = $env:COMPUTERNAME
[string]$portName = "8080"
# These are needed for the CheckFitnessProcess function,
# placing them here $fitRunning = $false
# Set Java executable to use within PowerShell $Java = "java.exe"
# Archive file $archiveFile = "fitnesseArchive.txt"
# Use a Test Array to be able to bundle all the tests
# Tests that are commented out are having issues being run
# Smoke Testing is Development Based
# although its now geared towards checking Login $smokeArray = ( "SpringsLoginLiteTests" )
# Tests for the QA environment
# although its now geared towards checking page links $qaArray = ( "SpringsLinkChecks" )
# Tests to check how this script runs, this is a hidden command
# This just validates the Registration Verify Fields $checkArray = ( "SpringsTest.UserLoginFfTests" )
# Location of the Test Results, this is hardcoded within the SVN structure
# so it should always be available, I am sure there are better ways of doing this $testResultsDir = "C:\IHI_MAIN\trunk\AutomatedTests\Fitnesse\Fitnesse\testResults"
if (!(Test-Path -path $testResultsDir))
{
New-Item $testResultsDir -type directory > $null
}

Then I still run my check on whether Fitnesse is running, it doesn't really need to be running anyway; so I decided that since REST only runs as needed then I was going to shut down Fitnesse if it was going.

# CheckFitnesseProcess determines if Fitnesse is running, if Fitnesse is not running
# then the script continues. The current test configuration does not require Fitnesse
# to run and does a better job when it's not.
# Adapted from: http://odetocode.com/Blogs/scott/archive/2006/07/17/5330.aspx function CheckFitnesseProcess {
param ([string]$AppPoolName="")
$ProcessId = $null
if ($AppPoolName.ToUpper() -ne $NOT_PASSED) {
[string]$filter = $null
if ((Get-Process java -ErrorAction SilentlyContinue) -ne $null) {
Get-Process java | ForEach {
# use WMI API to get process details
$filter = "Handle='" + $_.Id + "'"
$wmip = Get-WmiObject Win32_Process -filter $filter
# CommandLine property has AppPool name; grab name using contains
if($wmip.CommandLine.contains("fitnesse")) {
# Uncomment the next line for debugging
# Write-Host "Found this java process: $wmip.CommandLine `n"
$fitRunning = $true
# Saving the Process ID to stop the process later,
# using a Global as I could not get this work if I declared
# the variable at the beginning.
$Global:fitProcess = $_.Id
}
}
}
if ($fitRunning -eq $false) {
# All is good
}
if ($fitRunning -eq $true) {
Write-Host "Fitnesse is running, it's best if its not - shutting it down...`n" -ForegroundColor Red
# Stop the Fitnesse process
Stop-Process -Id $fitProcess
}
}
}

I'm going to skip the Help function, if you need that it should be in my previous post, the important one is the Main function.

function main()
{

# Need to be in the Fitnesse directory to access TestRunner, and

# the Fitnesse batch file in case Fitnesse needs to be started

cd $fitnesseLocale

#Check and see if Fitnesse is running

CheckFitnesseProcess

foreach ($test in $testArray)

{

# Start the test

$myDate = Get-Date

Write-Host "`nRunning $test`n"

# Build RESTful URL

$testLink = $test +"?" + $testType +"&format=text"

# Get the output from TestRunner so we can scrape it for test passes and fails

$testResults = & $Java "-jar" "$fitnesseJar" "-c" "$testLink"

Write-Host "Completed $test`n"

# Write-Host "Test Results..."

# Scan through the output, then display the results

# Dump $testResults out to a File

$testFile = $test + $myDate.Day + $myDate.Month + $myDate.Year + $myDate.Hour + $myDate.Minute + ".txt"

$testResults | Out-File "$testResultsDir\$testFile"

# Print out date to an archive file

$testDate = "Last run time: " + $myDate.Day + "/" + $myDate.Month + "/" + $myDate.Year + " at " + $myDate.Hour + ":" + $myDate.Minute

$testDate >> "$testResultsDir\$archiveFile"

foreach ($line in $testResults)

{

if ($line -match "\A\.\s"){

# This is a Pass

Write-Host $line -ForegroundColor Green

# Output to archive file

$line >> "$testResultsDir\$archiveFile"

}

if ($line -match "\AX\s") {

# This is an Exception, a warning but not a Failure

Write-Host $line -ForegroundColor Yellow

# Output to archive file

$line >>  

Fitnesse TestRunner Scripting in PowerShell

I'm running Fitnesse with SharePoint and in trying to automate my tests I ended up wrapping everything up in PowerShell, this is my first full script that handles all the tests I wanted to do.

First set up all the variables needed.
[string]$fitnesseJar = "fitnesse.jar"
[string]$fitnesseClass = "fitnesse.runner.TestRunner"
# Need to get the local machine here, it will be where Fitnesse runs
[string]$serverName = $env:COMPUTERNAME
[string]$portName = "8080"
# These are needed for the CheckFitnessProcess function, placing them here
$fitRunning = $false
# Since we need to manipulate this variable in a few different functions, its Global
[int]$Global:startTries = 0
# Set Java executbale to use within PowerShell
$Java = "java.exe"
# Use a Test Array to be able to bundle all the tests
$smokeArray = ( "FitNesseTestPageName1" )
$qaArray = ( "FitNesseTestPageName2" )

I use a function to check if Fitnesse is running, I wanted to only run it when I needed to and since this script can be automated I cannot be sure that Fitnesse will always be running
# CheckFitnesseProcess determines if Fitnesse is running; in which case
# it jumps down to running the tests, as long as Fitness is running the Check does
# not need to do anything else.
# If Fitnesse is not running then the script attempts to start it, does a
# check to make sure that Fitnesse started then runs the test. If this script
# starts Fitnesse then it will attempt to clean up after itself and stop
# Fitnesse by doing a check on $startTries, if this is > 0 then this script
# started Fitnesse and can shut it down, otherwise it leaves the existing
# process running
# Adapted from: http://odetocode.com/Blogs/scott/archive/2006/07/17/5330.aspx
function CheckFitnesseProcess {
param ([string]$AppPoolName="")
$ProcessId = $null
if ($AppPoolName.ToUpper() -ne $NOT_PASSED) {
[string]$filter = $null
if ((Get-Process java -ErrorAction SilentlyContinue) -ne $null) {
Get-Process java | ForEach {
# use WMI API to get process details
$filter = "Handle='" + $_.Id + "'"
$wmip = Get-WmiObject Win32_Process -filter $filter
# CommandLine property has AppPool name; grab name using contains
if($wmip.CommandLine.contains("fitnesse")) {
# Uncomment the next line for debugging
# Write-Host "Found this java process: $wmip.CommandLine `n"
$fitRunning = $true
# Saving the Process ID to stop the process later,
# using a Global as I could not get this work if I declared
# the variable at the beginning.
$Global:fitProcess = $_.Id
}
# The next statement was put in to check for the processes, there was a problem
# when updating Java that the process find command did not work, if this happens
# again just uncomment the Write-Host line to find out what is going on, otherwise
# this just gets bypassed but caught later on.
if ($fitRunning -eq $false)
{
# Uncomment the next line for debugging
# Write-Host "Did not find a Java/Fitnesse Process. Found $wmip.CommandLine `n"
}
}
}
if (($fitRunning -eq $false) -and ($Global:startTries -lt 3)) {
# Don't need to note that we have had 0 attempts
if ($Global:startTries -gt 0) {
# Need to give Fitnesse some attempts at starting, 3 is a good max
Write-Host "Fitnesse is not running, have tried to start $Global:startTries time(s),`n"
}
$Global:startTries++
startFitnesse
}
if (($fitRunning -eq $false) -and ($Global:startTries -eq 3)) {
# At some point we need to give up starting it
Write-Host "Fitnesse is not running, tried to start $Global:startTries time(s).`n"
Write-Host "Strike Out!" -ForegroundColor Red
Write-Host "Looks like Fitnesse cannot be started or found,`n"
Write-Host "check the server and try again.`n"
# Go back to where we started
Pop-Location
exit;
}
if ($fitRunning -eq $true) {
# Success!!
Write-Host "Fitnesse is good to go.`n" -ForegroundColor Green
}
}
}

Then I needed something to start Fitnesse, this uses the Fitnesse Batch script that comes with many of the installs
function startFitnesse()
{
Write-Host "Attempting to start up Fitnesse...`n"
# Starts the Fitnesse process using the batch configuration file
# Not sure we need the runAs Admin but that can be adjusted
Start-Process -FilePath fitnesse.bat
# Giving it time to wake up
Start-Sleep -Seconds 2
# Need to check that the process actually started, do the check again
CheckFitnesseProcess
}

Every script needs a help option
function myHelp()
{
Write-Host "The Fitnesse Tester understands the following command line arguments:`n"
Write-Host " qa - will run the QA Test Suites, a long set of tests`n"
Write-Host " smoke - which will run the Build Smoke Tests, a short set`n"
Write-Host "`n Running this script with no arguments will display this message`n"
}

Then there is the main and rest of the script that runs everything
function main()
{
# Need to be in the Fitnesse directory to access TestRunner, and
# the Fitnesse batch file in case Fitnesse needs to be started
cd $fitnesseLocale
#Check and see if Fitnesse is running
CheckFitnesseProcess
foreach ($test in $testArray)
{
# Start the test
Write-Host "`nRunning $test`n"
# Get the output from TestRunner so we can scrape it for test passes and fails
$testResults = & $Java "-cp" "$fitnesseJar" "$fitnesseClass" "-v" "$serverName" "$portName" "$test"
Write-Host "Completed $test`n"
Write-Host "Test Results..."
# Scan through the output, then display the results
# Dump $testResults out to a File
$testResults | Out-File "$test.txt"
foreach ($line in $testResults)
{
if ($line -match "Assertions:") {
# Write-Host "$line"
# Do a regular expression to get the Assertion info
# Example: Assertions: 8 right, 0 wrong, 0 ignored, 0 exceptions
$line -match ":\s(?\d+)\sr.*(?\d+)\sw.*(?\d+)\si.*(?\d+)\se.*"
# Send a summary of the results to the console
Write-Host "Passed: "$matches.right -ForegroundColor Green
Write-Host "Failed: "$matches.wrong -ForegroundColor Red
Write-Host "Ignored: "$matches.ignores
Write-Host "Exceptions: "$matches.exception -ForegroundColor Yellow
}
}
}
# Go back to where we started
Pop-Location
# Stop the Fitnesse process so we don't leave things running
# But only if this script started it, which means $startTries is > 0
if ($Global:startTries -ge 1){
Write-Host "`nSince this script started Fitnesse, it'll be shut down now.`n"
Stop-Process -Id $fitProcess
}
}
# Check command line values to see what test is running
if ($args -eq "smoke")
{
$testArray = $smokeArray
Write-Host "Running Smoke Test.`n"
}
elseif ($args -eq "qa")
{
$testArray = $qaArray
Write-Host "Running QA Test Suite.`n"
}
elseif ($args -eq "check")
{
$testArray = $checkArray
Write-Host "Running Check Suite.`n"
}
else
{
myHelp
exit;
}
# Run main
main

I've moved onto REST but in case anyone else is trying to work with the TestRunner and PowerShell you have something you can start with, and not have to work it all from the beginning like I did.

Testing with SharePoint

I've been learning a lot about SharePoint over the past 9 months, the project I am working on is to bring a non-profit health community onto SharePoint and increase the offerings the non-profit has and to expand the social networking. My work is revolving around testing the conversion of the site, making sure that what we end up with is going to be equivalent with what is currently in place. It's interesting work and I've had some interesting tasks over the past few months. Some of the more difficult work has been dealing with SharePoint and its eccentricities in order to assure that what we end up with is what was asked for.

Web Parts are like little applications that SharePoint uses to do all kinds of work, this could be anything from displaying a document library to a search result. While they are like little applications Web Parts are quite different and aren't very testable on their own, I haven't seen much on them regarding Unit Testing yet and with their reliance on SharePoint data its probably not that easy. Mostly the plan is to test the Web Parts in an exploratory fashion, when they are ready and added to pages they will be used and abused to put them through their paces. Not the way I'd like to do it, but I am also pulling some information from the Microsoft page on how to test Web Parts.

Automation with SharePoint is kind of tough, not that its difficult to work with, but the page layouts and Web Parts don't always make things run smooth. Web Parts can update like Ajax driven pages will, some of the ID's given by SharePoint for page elements can also be cryptic or dynamic. It's not impossible, I have done tests with Selenium, Fitnium and Fitnesse after some time to work out the issues in calling page elements. It just seems to take more time than it should.

The Social Networking aspects make things a little hard to automate everything, it would be nice to free myself up for other work but that's more a dream than a reality. Testing Discussion Groups, Comments and Ratings means eyes need to be on the page as things are done in order to verify that SharePoint responds properly.

When trying to find errors there are two important places to look:

1. ULS Logs - Get the Microsoft ULS Log Viewer, if there is something happening real time, this is the place to see it happen. Access to the SharePoint server under test is required to get this to work though.
2. Event Log - the Event Log is where alot of the captured errors go, if the Developers have not set this up in the code they should otherwise you lose and important resource.

I'll be putting in more links to some of the material I have captured over the past few months.

Cost of Bug Fixing

I got an IM from a developer I used to work with and he had that good old familiar question "do you have a copy of that graph that shows the cost of fixing a bug?"

Well I didn't at the time, mostly I dealt with this with the book I keep on Quality Assurance, it has the graph and a little blub about it. Since he needed one online, and pretty much everything is on the internet I did a check and lo and behold, I found a developer centric version.
Rate of Diminishing Returns Of Fixing Software Bugs
He needed an online version because he had a VP who wouldn't drink the water on catching defects early, I guess there are still plenty of them out there. Either way, I found the link and he went away happy, another QA-centric developer satisfied.

It feels good to help on a Monday.