v1.0.3 release

* Fixes #55 Start-RemotelyJobPorcessing logic - Fixes use of alias within the code - Fixes indentation * Fixes #48 naming convention for tests file and the NUnit XMl result file - based on the nodename the above files are named. * Adds unit test for Start-RemotelyJobProcessing function logic - Fixes Write-VerboseLog to handle Pester mocks - Fixes Start-RemotelyJobProcessing to stop when ProcessRemotely* function fail in pipeline * Fixes PSRemotely.json of the empty element in ModulesRequired * Adds Enter-PSRemotely function and Integration tests for it * Adds integration tests for issue #48 - naminv convention for tests & results file * Refactors config data tests * Adds another fix for #48 - the tests file dumped should also use nodename * Add -PassThru switch to Enter-PSRemotely * Fixes typo in the Debug.Integration.Tests * Fixes another set of typos in Debug.Integration.Tests * Adds check for invalid filename chars #56 * Adds check for invalid filename chars to Invoke-PSRemotely & bootstrap.ps1 #56 * Fixes integration tests for #48 & #56 * Removes the empty element in the modulesrequired @ PSRemotely.json * Fixes the naming logic for the Nunit XML file * Fixes typo in the Debug.Integration.Tests - replaces the localhost with Node1 in nodename * Adds unit and integration tests for issue #27 - issue with parsing describe blocks * Refactor the integration tests for issue #27 * Adds code logic for issue #27 * Adds quick refactor to the integration test for issue #27 * Fixes typo * Swaps $Env:BHPSModulePath with $ENV:BHModulePath * Fixes Invoke-Pester call in the remote Invoke-PSRemotely function * Swaps BHPSModulePath with BHModulePath in psremotely.psdeploy file * Adds logo #58 * Initial draft of the PITCHME.md #57 * Removing PITCHME.MD * Adds back PITCHME.md * Adds pitchme badge to README * Fixes typo in doc & adds content to PITCHME * Fixes for presentation at PSBUG * Updates doc TroubleShoot.md - now references new Enter-PSRemotely function * Fixes typos in Integration tests * Fixes for the integration tests failing - Not enough storage for IPv6Address.Integration.Tests - Typos in the ConfigData.Integration.Tests * Adds enable-rdp to Appveyor * Block RDP in appveyor.yml * Block Appveyor VM on build finish * Fixes possible race condition issue with IPv6 integration tests
DexterPOSH · Jun 5, 2017 · 0d885af · 0d885af
1 parent 6d15547
commit 0d885af
Show file tree

Hide file tree

Showing 36 changed files with 920 additions and 150 deletions.
diff --git a/PITCHME.md b/PITCHME.md
@@ -0,0 +1,98 @@
+## Operations Validation
+
+- <span style="font-size:0.9em; color:gray">Validating your Infrastructure as Code.</span>
+- <span style="font-size:0.9em; color:gray">Tests if the infrastructure components are functional.</span>
+
++++
+
+<span style="font-size:1.0em; color:gray">Fits into the DevOps ecosystem.</span> |
+<span style="font-size:1.0em; color:gray">Is my infrastructure functioning as expected?</span>
+
+---
+## PowerShell Scripts ?
+
+- <span style="font-size:0.9em; color:gray">Can be used.</span>
+- <span style="font-size:0.9em; color:gray">Maintenance nightmare.</span>
+
++++
+
+<span style="font-size:1.0em; color:red">Tip - Avoid writing scripts for validating your infrastructure.</span>
+
+---
+
+## Pester and PoshSpec
+
+Pester is a Unit testing framework.
+Only the code logic is tested.
+
+```powershell
+Function Get-ServerInfo {
+    Get-CIMInstance -Class Win32_ComputerSystem | 
+        Select-Object -Property *
+}
+
+Describe 'Get-ServerInfo Unit tests' {
+    # Arrange
+    Mock -Command Get-CimInstance -FilterParameter {$Class -eq 'Win32_ComputerSystem' }
+    # Act
+    Get-ServerInfo 
+    # Assert
+    It "Should query the Win32_ComputerSystem class" {
+        Assert-MockCalled -Command Get-CimInstance -Times 1 -Exactly -Scope Describe
+    }
+}
+```
+
++++
+### Pester for Ops validation
+
+But Pester can be extended to validate/test Infrastructure as well.
+
+```powershell
+Describe "TestIPConfiguration" {
+    It "Should have a valid IP address on the Management NIC" {
+        (Get-NetIPAddress -AddressFamily IPv4 -InterfaceAlias 'vEthernet(Management)' | Select-Object -ExpandProperty IPAddress) |
+            Should be '10.10.10.1' 
+    }
+}
+```
+
++++
+
+### PoshSpec fits in
+
+PoshSpec adds yet another layer of abstraction on our infrastructure tests.
+The tests look concise and easy to maintain.
+
+```powershell
+Describe "TestIPConfiguration" {
+    Context "Validate the Management NIC " {
+        # Custom public type added to PoshSpec for our use.
+        IPv4Address 'vEthernet(Management)' {Should be '10.10.10.1'} 
+    }
+}
+```
+
+---
+
+## Challenges for Solution stack Ops validation
+- <span style="font-size:0.9em; color:gray">Targeting remote nodes for ops validation is still an overhead.</span>
+- <span style="font-size:0.9em; color:gray">Remote nodes need to bootstrapped before invoking the ops validation.</span>
+- <span style="font-size:0.9em; color:gray">Challenge in specifying node and solution configuration data.</span>
+
+
+---
+
+## Remote Ops validation
+Targeting tests written to remote node(s).
+PSRemotely was engineered with solution stack operations validation in mind. Few of its features:-
+- <span style="font-size:0.6em; color:gray">Target Pester/PoshSpec based operations validation tests on the remote nodes.</span>
+- <span style="font-size:0.6em; color:gray">Decouples node and solution configuration data using DSC config data syntax.</span>
+- <span style="font-size:0.6em; color:gray">Self-contained framework, bootstraps remote node(s) for running the ops validation.</span>
+- <span style="font-size:0.6em; color:gray">Allows re-running failed tests.</span>
+- <span style="font-size:0.6em; color:gray">Easier debugging.</span>
+---
+
+## Demo Validating a S2D cluster using PSRemotely
+
+![alt](PSRemotely.png)
diff --git a/PITCHME.yaml b/PITCHME.yaml
@@ -0,0 +1,2 @@
+theme : black
+highlight : monokai
diff --git a/PSRemotely.png b/PSRemotely.png
diff --git a/PSRemotely/private/ASTFunctions.ps1 b/PSRemotely/private/ASTFunctions.ps1
@@ -17,37 +17,82 @@ Function Get-TestNameAndTestBlock {
 	$ast = Get-ASTFromInput -Content $Content
 	$commandAST = $ast.FindAll({ $args[0] -is [System.Management.Automation.Language.CommandAst]}, $true)    
 	$output = @()
+	# fetch all the Describe blocks using AST
 	$describeASTs = $commandAST | Where-Object -FilterScript {$PSItem.GetCommandName() -eq 'Describe'}
 	if ($describeASTs) {
+		# iterate over each Describe block, this means that PSRemotely allows usage of multiple Describe
+		# block within a Node block.
 		foreach ($describeAST in $describeASTs) {
-			$TestNameElement = $describeAST.CommandElements | Select-Object -First 2 | Where-Object -FilterScript {$PSitem.Value -ne 'Describe'}
 
-			Switch -Exact ($TestNameElement.StringConstantType ) {
-
-				'DoubleQuoted' {
-					# if the test name is a double quoted string
-					$output += @{
-						#Add the test name as key and testBlock string as value 
-						$($ExecutionContext.InvokeCommand.ExpandString($TestNameElement.Value)) = $($describeAST.Extent.Text)
-					}
-					break
-				}
-				'SingleQuoted' {
-					# if the test name is a single quoted string
-					$output += @{
-						$($TestNameElement.Value) = $($describeAST.Extent.Text)
-					}
-					break
-				}
-				default {
-					throw 'TestName passed to Describe block should be a string'
-				}
+            $ScriptBlock = [scriptblock]::create("$($describeAST.Extent.Text)") 
+			#$TestNameElement = $describeAST.CommandElements | Select-Object -First 2 | Where-Object -FilterScript {$PSitem.Value -ne 'Describe'}
+			$ParametersPassedToDescribe = Get-ParametersPassedToDSLKeyword -FunctionInfo $(Get-Command -Module Pester -Name Describe) -ScriptBlock $ScriptBlock
+            # since there is a limitation while generating parameters from the proxy command
+            # we need to explicitly check that the name and fixture are not empty here
+            if (-not $ParametersPassedToDescribe['name'] -or (-not $ParametersPassedToDescribe['Fixture'])) {
+                throw 'Name or Fixture missing in the Describe block'
+            }
+			$output += @{
+				$ParametersPassedToDescribe['Name'] = $($describeAST.Extent.Text)
 			}
+
 		} # end foreach block
 		Write-Output -InputObject $output
 	}
 	else {
 		throw 'Describe block not found in the Test Body.'
 	}
-
+}
+
+Function Get-ParametersPassedToDSLKeyword {
+	<#
+		This function is a magic function. Specify it a module's DSL keyword function metadata along
+		with the actual usage of the DSL and it will return the PSBoundParameters being passed to the
+		original DSL keyword.
+
+		This was written in order to determine the test names of the Describe block wrapped inside 
+		PSRemotely DSL. Since these test names are later used while dropping individual .Tests.ps1 
+		files on the PSRemotely node(s).
+	#>
+	[cmdletbinding()]
+    param(
+        # Supply the script info object, output of Get-Command -Name <DeploymentScript>.ps1
+        [Parameter(Mandatory)]
+		[System.Management.Automation.FunctionInfo]$FunctionInfo,
+
+		[Parameter(Mandatory)]
+		[ScriptBlock]$ScriptBlock
+	)
+
+	TRY {
+		$Metadata = [System.Management.Automation.CommandMetadata]::New($FunctionInfo)
+		$CmdletBinding = [System.Management.Automation.ProxyCommand]::GetCmdletBindingAttribute($Metadata)
+		$Parameters = [System.Management.Automation.ProxyCommand]::GetParamBlock($Metadata)
+
+	# bad formatting due to usage of here-string
+$FunctionBody = @"
+	$CmdletBinding
+	param(
+		$Parameters
+	)
+	`$returnHashtable = `$PSBoundParameters    
+	`$returnHashtable
+"@
+		$DummyFunction = [scriptblock]::Create($FunctionBody)
+		$Null = New-Item -Path Function:\ -Name pSRemotelyDescribeOverride  -Value $DummyFunction -Force
+		# create a temporary override for the Pester's Describe keyword
+		$null = New-Alias -Name Describe -Value pSRemotelyDescribeOverride -Force
+
+		# Now invoke the scriptblock
+		$ParametersHash = & $ScriptBlock
+        Write-Output -InputObject $ParametersHash
+	}
+	CATCH {
+		Write-Warning -Message "$($PSItem.Exception)"
+		$PSCmdlet.ThrowTerminatingError($PSItem)
+	}
+	FINALLY {
+		# Clean up the alias
+		Remove-Item -Path Alias:\Describe -Force -ErrorAction SilentlyContinue
+	}
 }
diff --git a/PSRemotely/private/Helper.ps1 b/PSRemotely/private/Helper.ps1
@@ -6,45 +6,44 @@ Function ProcessRemotelyJob {
     )
     $NodeName = $InputObject.Key
     $Job = $InputObject.Value
-
-    foreach($childJob in $Job.ChildJobs)
-		{
-			if($childJob.Output.Count -eq 0){
-				[object] $outputStream = New-Object psobject
-			}
-			else {
-				[object] $outputStream = $childJob.Output | % { $_ }
-			}
-
-			$errorStream =    CopyStreams $childJob.Error
-			$verboseStream =  CopyStreams $childJob.Verbose
-			$debugStream =    CopyStreams $childJob.Debug
-			$warningStream =  CopyStreams $childJob.Warning
-			$progressStream = CopyStreams $childJob.Progress    
-
-			$allStreams = @{ 
-								Error = $errorStream
-								Verbose = $verboseStream
-								DebugOutput = $debugStream
-								Warning = $warningStream
-								ProgressOutput = $progressStream
-							}
-
-			$outputStream = Add-Member -InputObject $outputStream -PassThru -MemberType NoteProperty -Name __Streams -Value $allStreams
-			$outputStream = Add-Member -InputObject $outputStream -PassThru -MemberType ScriptMethod -Name GetError -Value { return $this.__Streams.Error }
-			$outputStream = Add-Member -InputObject $outputStream -PassThru -MemberType ScriptMethod -Name GetVerbose -Value { return $this.__Streams.Verbose }
-			$outputStream = Add-Member -InputObject $outputStream -PassThru -MemberType ScriptMethod -Name GetDebugOutput -Value { return $this.__Streams.DebugOutput }
-			$outputStream = Add-Member -InputObject $outputStream -PassThru -MemberType ScriptMethod -Name GetProgressOutput -Value { return $this.__Streams.ProgressOutput }
-			$outputStream = Add-Member -InputObject $outputStream -PassThru -MemberType ScriptMethod -Name GetWarning -Value { return $this.__Streams.Warning }
-			$outputStream = Add-Member -InputObject $outputStream -PassThru -MemberType NoteProperty -Name RemotelyTarget -Value $NodeName
-
-			if($childJob.State -eq 'Failed'){
-				$childJob | Receive-Job -ErrorAction SilentlyContinue -ErrorVariable jobError
-				$outputStream.__Streams.Error = $jobError
-			}
-
-			Write-Output -InputObject $outputStream
-		}
+    foreach($childJob in $Job.ChildJobs){
+        if($childJob.Output.Count -eq 0){
+            [object] $outputStream = New-Object psobject
+        }
+        else {
+
+            [object] $outputStream = $childJob.Output | Foreach-Object -Process { $_ }
+        }
+
+        $errorStream =    CopyStreams $childJob.Error
+        $verboseStream =  CopyStreams $childJob.Verbose
+        $debugStream =    CopyStreams $childJob.Debug
+        $warningStream =  CopyStreams $childJob.Warning
+        $progressStream = CopyStreams $childJob.Progress    
+
+        $allStreams = @{ 
+                            Error = $errorStream
+                            Verbose = $verboseStream
+                            DebugOutput = $debugStream
+                            Warning = $warningStream
+                            ProgressOutput = $progressStream
+                        }
+        #Write-Host -Object "$($outputStream.RemotelyTarget) NodeName -> $NodeName" -ForegroundColor red
+        $outputStream = Add-Member -InputObject $outputStream -PassThru -MemberType NoteProperty -Name __Streams -Value $allStreams
+        $outputStream = Add-Member -InputObject $outputStream -PassThru -MemberType ScriptMethod -Name GetError -Value { return $this.__Streams.Error }
+        $outputStream = Add-Member -InputObject $outputStream -PassThru -MemberType ScriptMethod -Name GetVerbose -Value { return $this.__Streams.Verbose }
+        $outputStream = Add-Member -InputObject $outputStream -PassThru -MemberType ScriptMethod -Name GetDebugOutput -Value { return $this.__Streams.DebugOutput }
+        $outputStream = Add-Member -InputObject $outputStream -PassThru -MemberType ScriptMethod -Name GetProgressOutput -Value { return $this.__Streams.ProgressOutput }
+        $outputStream = Add-Member -InputObject $outputStream -PassThru -MemberType ScriptMethod -Name GetWarning -Value { return $this.__Streams.Warning }
+        $outputStream = Add-Member -InputObject $outputStream -PassThru -MemberType NoteProperty -Name RemotelyTarget -Value $NodeName
+
+        if($childJob.State -eq 'Failed'){
+            $childJob | Receive-Job -ErrorAction SilentlyContinue -ErrorVariable jobError
+            $outputStream.__Streams.Error = $jobError
+        }
+
+        Write-Output -InputObject $outputStream
+    }
 }
 
 Function ProcessRemotelyOutputToJSON {
@@ -86,13 +85,13 @@ Function GetFormattedTestResult {
         $outputHashArray = @()
         $testsGroup = $testResult |Group-Object -Property Describe  
         foreach ($testGroup in $testsGroup) {
-            $result = ($TestGroup.Group | select -ExpandProperty Passed ) -Notcontains $false
+            $result = ($TestGroup.Group | Select-Object -ExpandProperty Passed ) -Notcontains $false
             $outputHashArray += @{
                 Name = $testGroup.Name
                 Result = $result
                 TestResult =  @($testGroup.Group | 
-                                Where -Property Result -eq 'Failed' |
-                                Select -Property Describe, Context, Name, Result, ErrorRecord)
+                                Where-Object -Property Result -eq 'Failed' |
+                                Select-Object -Property Describe, Context, Name, Result, ErrorRecord)
                 }
             }
             Write-Output -InputObject $outputHashArray
@@ -130,7 +129,10 @@ Function Write-VerboseLog
             $Message = $ErrorInfo.Exception.Message
             $Functionname = $ErrorInfo.InvocationInfo.InvocationName
             $LineNo = $ErrorInfo.InvocationInfo.ScriptLineNumber
-            $scriptname = $(Split-Path -Path $ErrorInfo.InvocationInfo.ScriptName -Leaf)
+            if ($ErrorInfo.InvocationInfo.ScriptName) {
+                # this is done to correctly recieve the original error back from Pester mocks
+                $scriptname = $(Split-Path -Path $ErrorInfo.InvocationInfo.ScriptName -Leaf)
+            }
             Write-Verbose -Message "$scriptname - $Functionname - LineNo : $LineNo - $Message"           
             #$PSCmdlet.ThrowTerminatingError($ErrorInfo)
             #Write-Error -ErrorRecord $ErrorInfo -ErrorAction Stop # throw back the Error record 
@@ -151,10 +153,10 @@ Function Start-RemotelyJobProcessing {
             $AllJobsCompletedHash.Add($PSItem, $False)
         }
 
-        $CloneJobHash = $AllJobsCompletedHash.Clone() # used to iterate over the Hashtable
-
         do {
+            $CloneJobHash = $AllJobsCompletedHash.Clone() # used to iterate over the Hashtable
             foreach ($nodeJobStatus in $CloneJobHash.GetEnumerator()) {
+                Write-VerboseLog -Message "Processing for Node -> $($nodeJobStatus.key) "
                 if ($nodeJobStatus.Value) {
                     # node job status is True, it has been processed
                     Write-VerboseLog -Message "PSRemotely job already processed for Node -> $($nodeJobStatus.key) "
@@ -167,13 +169,11 @@ Function Start-RemotelyJobProcessing {
 
                     if ($enum.Value | Where-Object -Property State -In @('Completed', 'Failed')) {
                         Write-VerboseLog -Message "PSRemotely job finished for Node -> $($nodeJobStatus.key). Processing it now."
-                        $enum | ProcessRemotelyJob | ProcessRemotelyOutputToJSON
+                        $enum | ProcessRemotelyJob -ErrorAction Stop | ProcessRemotelyOutputToJSON -ErrorAction Stop
                         $AllJobsCompletedHash[$enum.key] = $true
                     }
-
                 }
             }
-
         }  until (@($allJobsCompletedHash.Values) -notcontains $False)
     }
     CATCH {