Showing posts with label automation. Show all posts
Showing posts with label automation. Show all posts

Monday, September 5, 2016

Adding an external file to JIRA

As a part of my companies personal project time, a yearly occurrance they call Off The Grid, I was part of a project to add external files to a JIRA ticket.  JIRA is used to track EVERYTHING in the company.

This is the python script I hobbled together.

#!/usr/bin/env python

from stat import S_ISREG, ST_CTIME, ST_MODE
import os,sys,re,requests
from requests.auth import HTTPBasicAuth
import urllib3
urllib3.disable_warnings()
def openFile(location,key,filetype,fileKey):
    """
    Tests that the file location exists and searches through each file for a specific keyword
    :param location:
    :param key:
    :param filetype:
    :param fileKey
    :return: outputFile:
    """
    outputFile = filetype + "output.txt"
    try:
        os.remove(outputFile)
    except OSError:
        pass
    f = open(outputFile,'w')
    print >> f, "Looking in %s" % location
    for dirpath, dirnames, files in os.walk(location):
        if not files:
            print dirpath, 'is empty'
            exit()
    entriesf = (os.path.join(location, fn) for fn in os.listdir(location))
    entries = ((os.stat(path), path) for path in entriesf)
    entries = ((stat[ST_CTIME], path)
               for stat, path in entries if S_ISREG(stat[ST_MODE]))
    # for cdate, path in sorted(entries):
    #    print time.ctime(cdate), os.path.basename(path)
    check_file = re.compile(fileKey)
    for file in entriesf:
        if os.path.isfile(file): # and check_file.search(file):
            for i,line in enumerate(open(file)):
                #if re.finditer(key,line):
                match = re.search(key,line)
                if match:
                    print >> f, "Found in %s \n %s" % (file, line)
                else:
                    print >> f, "No entries that matched keyword %s" % fileKey
        else:
            print >> f, "No files that matched %s" % fileKey
    f.close()
    return outputFile

def addSqlTraceToJira(location,ticket):
    """
    Add a SQL Trace file to the Jira Ticket
    :param location:
    :param ticket:
    :return:
    """
    entriesf = (os.path.join(location, fn) for fn in os.listdir(location))
    check_file = re.compile('trc$')
    check_ticket = re.compile(ticket)
    for file in entriesf:
        if os.path.isfile(file) and check_file.search(file) and check_ticket.search(file):
            attachFileToJira(file,ticket)
        else:
            print "No files found with %s" % ticket
def attachFileToJira(logfile,ticket):
    """
    Attach the output file to the Jira ticket
    :param logfile:
    :param ticket:
    :return:
    """
    url = "https://jira-test.fakedomain.comnet/rest/api/2/issue/%s/attachments" % ticket
    headers = {'X-Atlassian-Token':'no-check'}
    files = {'file': open(logfile, 'r')}
    auth = HTTPBasicAuth('username', 'password')
    # print "Sending %s to ticket %s" % (logfile,url)
    r = requests.post(url,headers=headers,files=files,auth=auth,verify=False)
    print 'Jira file attach sent me a %s with \n %s' % (r.status_code,r.text)

def addCommentToJira(keyword,ticket):
    """
    This attaches a comment about the search parameter that was added to the ticket
    :param keyword:
    :param tickiet:
    :return:
    """
    url = "https://jira-test.fakedomain.comnet/rest/api/2/issue/%s/comment" % ticket
    headers = {'X-Atlassian-Token': 'no-check'}
    auth = HTTPBasicAuth('username', 'password')
    comment = "Just added an attached log file about the results for the %s search." % keyword
    json = {"body": comment}
    r = requests.post(url,headers=headers,json=json,auth=auth,verify=False)
    print 'Jira adding a comment sent me a %s with \n %s' % (r.status_code, r.text)

def main():
    if len(sys.argv) < 5:
        from os.path import basename
        print ('Usage: %s location key type ticket' % basename(__file__))
    else:
        # fileLocation
        LOCATION = sys.argv[1]
        # searchKey
        SEARCHKEY = sys.argv[2]
        # type
        FILETYPE = sys.argv[3]
        # Jira Ticket
        TICKET = sys.argv[4]
        # File Keyword
        FILEKEY = sys.argv[5]
        #timeStart
        #TIMESTART = sys.argv[3]
        #timeend
        #TIMEEND = sys.argv[4]
    # logfile = openFile(LOCATION,SEARCHKEY,FILETYPE,FILEKEY)
    # attachFileToJira(logfile,TICKET)
    # addCommentToJira(SEARCHKEY,TICKET)
    addSqlTraceToJira(LOCATION,TICKET)
if __name__ == '__main__':
    main()

I claim some responsibility for it, mostly by hacking together a lot of python and doing a lot of TDD as I built out the functions.


Tuesday, September 9, 2014

SSIS Task Variables

In some current Testing I am doing work with SSIS.  One task imports data into a database that we baseline for future Test Cases, but to get the data in I need to modify some of the dates.  With SSIS there is an option to adjust data using Derived Columns, with that it's possible to adjust data using Task Scoped Variables; I'll have to find a reference but using project scoped variables did not work initially.

Basically the variables were created this way:

  1. Gave a descriptive name (need to have those so you know what to look for later!)
  2. Scope, was selected to be the Task being worked on, which was easy since the project only had one task for data loading.
  3. Data Type, since it was for Dates I made this DateTime
  4. Value, just hit enter/return here and it took the current date and time
  5. Expression, this is the real work here
    1. For Now this was just left as the default - GetDate()
    2. For a Year I made an expression - DATEADD("Year",1,(GetDate()))
    3. Same adjustments for Adding Minute or Month, just adjust Year
When using these, in the Derived Columns select the Expression column and in the upper right panel there is a tree for Variables and Parameters.  Select the User: and this will update the value for that column when the Derived Columns task runs.

Learning about DateAdd came from the Microsoft Site

Friday, April 18, 2014

Sauce Connect Script

I am currently working with Sauce Labs and using their Cloud Based testing to work with some automation, what I wanted first was an easy way to handle starting up the tunnel to Sauce Labs.  This was a jar file so I tried a few other methods to set the environment up the way I want, but considering that most environments I am working with are Windows I figured may as well use the old stand by of PowerShell.

Here is my current version of the script, this just checks to see that the environment is set up so I can output the log files in the places I want, and checks if a job is running then just continues on and uses the existing one - for now.  I heavily comment code when I first write it, so it should be enough to answer most questions, if not comment here and let me know!

----

# Sauce Labs Connect script
# Description:
# This script checks and sees if there is an existing Job that has an open tunnel
# to Sauce Labs in the current session; because of the way PowerShell works you
# cannot easily detect what is running in another session.  Basically this script
# was intended to generate the necessary Tunnel and run the test scripts with
# fewer keystrokes
# Author: Michael Furmaniuk
# Last Updated: April 16, 2014

# Command line values, so there is no reliance on the Username and Access Key
# being set in the environment or in the configuration files (although if they already are...)
param([string]$u, [string]$k)
Set-StrictMode -v 2

#######################################
# Check and make sure some of the specific needs are met
# Sauce Connect
[string]$jobName = "sauceConn"
[string]$rootPath = ""
[string]$sauceLog = "sauceLog.log"
[string]$sauceReady = "sauceReady.log"
[string]$sauceJar = "C:\Sauce-Connect-jar\Sauce-Connect.jar"
# If necessary the following could be completed to make this easier to run, otherwise these are
# command line arguments given as $u $k
# [string]$u = ""
# [string] $k = ""

#######################################
# Start the Sauce Jar
function start-SauceConnectJar()
{
<#
.Synopsis
This function starts the Sauce Connect Jar utilizing some known and command line parameters
.Description
This function starts the Sauce Connect Jar utilizing some known and command line parameters
#>
[CmdLetBinding()]
Param(
[Parameter(Mandatory=$true)][string]$sauceLog,
[Parameter(Mandatory=$true)][string]$sauceReady,
[Parameter(Mandatory=$true)][string]$sauceJar,
[Parameter(Mandatory=$true)][string]$u,
[Parameter(Mandatory=$true)][string]$k
) #end param
Process
{
# Let's check and see what drives and temp files we have
if (Test-Path "C:\TEMP") {
$rootPath = "C:\TEMP"
} elseif (Test-Path "D:\TEMP") {
$rootPath = "D:\TEMP"
} else {
Write-Host "I can't seem to detect any disk drives here with a TEMP directory, that is a problem.`n"
exit
}
# Build up the file paths
$sauceLog =  $rootPath + "\" + $sauceLog
$sauceReady = $rootPath + "\" + $sauceReady
# If these already exist we want to remove them
if (Test-Path $sauceLog) {
Remove-Item $sauceLog
}
# Note: if this call fails then there is an existing job in another session
# need to somehow handle this and fail gracefully: this is a TODO
if (Test-Path $sauceReady) {
Remove-Item $sauceReady
}
# Start Sauce Connect and see what is returned for the process
# Build up the latter part of the script block
$commandLineOption = "-l $sauceLog -f $sauceReady -d"
# Now for the full command line
$scriptblock = [scriptblock]::Create("java -jar $sauceJar $u $k $commandLineOption")

# Now let's start this as a Job, next line is for debugging
# Write-Host "Starting Sauce Connect with: $($scriptblock).`n"
# Should now be suppressing the start text
$output = Invoke-Expression "start-job -name $jobName -ScriptBlock { $scriptblock } 2>&1"

# Check for the Ready File
while (!(Test-Path $sauceReady)) {
Write-Host "Sauce Connect is still starting...`n"
sleep(10)
}
# Next statement is for debugging
# Since you only get a Tunnel ID when the connection is complete using this as a defining
# point to know the tunne is up, next statement was for debugging
# Write-Host "Sauce Connect should be up...getting the Tunnel ID.`n"
# Now acually getting the Endpoint/Tunnel ID as a verification step
get-SauceEndPoint $sauceLog
}
}

#######################################
# Get the endpoint ID from the log
function get-SauceEndPoint()
{
<#
.Synopsis
This function scans the initial Sauce Connect log file for the EndPoint ID to pass back
.Description
This function scans the initial Sauce Connect log file for the EndPoint ID to pass back
to the User through the console, so it can be used by the User to access the active Tunnel.
At some point this may be used for further automation
#>
[CmdLetBinding()]
Param(
[Parameter(Mandatory=$true)][string]$sauceLog
) #end param
Process {
# Now that the Connection is up, and the Ready File is in place
# read the log file to get the Tunnel ID or endpoint in the case of Java
$runningLog = Get-Content $sauceLog
# Run a regex to get the endpoint line, only one in the file at a time
# when this script runs, then print this out so it can be used (or saved for something else later)
$myMatch = $runningLog -match "endpoint ID:\s.*$" | %{$_ -split "ID:\s"}
# Next statement was a way to message that the Tunnel is up, if not needed comment out
Write-Host "Endpoint ID this time is: $($myMatch[1])"
}
}

#######################################
# The Sauce branch depending on what is running,
# I can either start a new Job or shut it down as/if necessary
$jobs = Get-Job 2>&1
if ($jobs) {
if ($jobs.State -eq "Running") {
# There is an existing Job, either we want to stop it and restart or drop out
# Right now going with using an existing Tunnel
Write-Host "An existing job is running, quitting for now."
Write-Host "...but getting the EndPoint ID to be useful."
get-SauceEndPoint $sauceLog
exit
}
elseif ($jobs.Name -eq $jobName -and $jobs.State -eq "Completed") {
Write-Host "Found $($jobs.Name) that was $($jobs.State) getting rid of it...`n"
Remove-Job $jobName
# Now that the previous one is gone, let's move on
Write-Host "Removed the old job, so starting a new Tunnel.`n"
start-SauceConnectJar $sauceLog $sauceReady $sauceJar $u $k
} else {
Write-Host "Weird, found a job but its not running, it's $($jobs.State).`n"
}
} else {
# Shouldn't be any other states to worry about at this point
# so just start the Jar file and move along
Write-Host "Nothing running, so starting the Tunnel.`n"
start-SauceConnectJar $sauceLog $sauceReady $sauceJar $u $k
}