Serge van den Oever [Macaw] - Archives - 2008

Powershell output capturing and text wrapping: strange quirks... solved!

Tuesday, December 9, 2008

14 Comments

Summary

To capture (transcript) all output of a Powershell script, and control the way textwrapping is done, use the following approach from for exmaple a batch file or your Powershell code:

PowerShell -Command "`$host.UI.RawUI.BufferSize = new-object System.Management.Automation.Host.Size(512,50); `"c:\temp\testoutputandcapture.ps1`" -argument `"A value`"" >c:\temp\out.txt 2>&1

Note the '`' (backtick characters).

The problem: output capture and text wrap

Powershell is a great language, but I have been fighting and fighting with Powershell on two topics:

to capture the output
to get it output text in the max width that I want without unexpected wrapping

After reading the manuals you would think that the PowerShell transcripting possibilities are your stairways to heaven. You just start capturing output as follows:

start-transcript "c:\temp\transcript.txt"
do your thing
stop-transcript

Try this with the following sample file c:\temp\testoutputandcapture.ps1:

Start-Transcript "c:\temp\transcript.txt"
function Output
{
    Write-Host "Write-Host"
    Write-Output "Write-Output"
    cmd.exe /c "echo Long long long long long long long long yes very long output from external command"
    PowerShell -Command "Write-Error 'error string'"
}
Output
Stop-Transcript

When you execute this script you get the following output:

PS C:\temp> C:\temp\testoutputandcapture.ps1
Transcript started, output file is c:\temp\transcript.txt
Write-Host
Write-Output
Long long long long long long long long yes very long output from external command
Write-Error 'error string' : error string
Transcript stopped, output file is C:\temp\transcript.txt

If we now look at the generated transcript file c:\temp\transcript.txt:

**********************
Windows PowerShell Transcript Start
Start time: 20081209001108
Username  : VS-D-SVDOMOSS-1\Administrator 
Machine	  : VS-D-SVDOMOSS-1 (Microsoft Windows NT 5.2.3790 Service Pack 2) 
**********************
Transcript started, output file is c:\temp\transcript.txt
Write-Host
Write-Output
**********************
Windows PowerShell Transcript End
End time: 20081209001108
**********************

The output from the external command (the texts Long long long long long long long long yes very long output from external command and Write-Error 'error string' : error string) is not captured!!!

Step one: piping output of external commands

This can be solved by appending | Write-Output to external commands:

Start-Transcript 'c:\temp\transcript.txt'
function Output
{
    Write-Host "Write-Host"
    Write-Output "Write-Output"
    cmd.exe /c "echo Long long long long long long long long yes very long output from external command" | Write-Output
    PowerShell -Command "Write-Error 'error string'" | Write-Output
}
Output
Stop-Transcript

This will result in the following output:

PS C:\temp> C:\temp\testoutputandcapture.ps1
Transcript started, output file is c:\temp\transcript.txt
Write-Host
Write-Output
Long long long long long long long long yes very long output from external command
Write-Error 'error string' : error string
Transcript stopped, output file is C:\temp\transcript.txt

Note that the error string Write-Error 'error string' : error string is not in red anymore.

The resulting transcript file c:\temp\transcript.txt now looks like:

**********************
Windows PowerShell Transcript Start
Start time: 20081209220137
Username  : VS-D-SVDOMOSS-1\Administrator 
Machine	  : VS-D-SVDOMOSS-1 (Microsoft Windows NT 5.2.3790 Service Pack 2) 
**********************
Transcript started, output file is c:\temp\transcript.txt
Write-Host
Write-Output
Long long long long long long long long yes very long output from external command
Write-Error 'error string' : error string
**********************
Windows PowerShell Transcript End
End time: 20081209220139
**********************

This is what we want in the transcript file, everything is captured, but we need to put | Write-Output after every external command. This is way to cumbersome if you have large scripts. For example in our situation there is a BuildAndPackageAll.ps1 script that includes a lot of files and cosists in totla of thousands of lines of Powershell code. There must be a better way...

Transcripting sucks, redirection?

Ok, so transcript just does not do the job of capturing all output. Lets look at another method: good old redirection.

We go back to our initial version of the script, and do c:\temp\testoutputandcapture.ps1 > c:\temp\out.txt. This results in a c:\temp\out.txt file with the following contents:

Transcript started, output file is c:\temp\transcript.txt
Write-Output
Long long long long long long long long yes very long output from external command
Write-Error 'error string' : error string
Transcript stopped, output file is C:\temp\transcript.txt

We are missing the Write-Host output! Actually, the Write-Host is the only line ending up in the transcript file;-)

This is not good enough, so another try, but now using the Powershell command-line host on the little modified script c:\temp\testoutputandcapture.ps1 that will showcase our next problem:

function Output
{
    Write-Host "Write-Host"
    Write-Output "Write-Output Long long long long long long long long yes very long output from Write-Output"
    cmd.exe /c "echo Long long long long long long long long yes very long output from external command"
    PowerShell -Command "Write-Error 'error string'"
}
Output

We now do: Powershell -Command "c:\temp\testoutputandcapture.ps1" > c:\temp\out.txt 2>&1 (2>&1 means: redirect stderr to stdout). We capture stderr as well, you never know where it is good for. This results in:

Write-Host
Write-Output Long long long long long long long l
ong yes very long output from Write-Output
Long long long long long long long long yes very long output from external command
Write-Error 'error string' : error string

I hate automatic text wrapping

As you may notice some strange things happen here:

A long output line generated by PowerShell code is wrapped (at 50 characters in the above case)
A long line generated by an external command, called from Powershell is not truncated

The reason the Powershell output is wrapped at 50 characters is because the dos box I started the command from is set to 50 characters wide. I did this on purpose to proof a case. Ok, so you can set your console window really wide, so no truncation is done? True. But you are not always in control of that. For example if you call your Powershell script from a batch file which is executed through Windows Explorer or through an external application like in my case CruiseControl.Net where I may provide an external command.

Powershell -PSConfigFile?... no way!

I hoped the solution would be in an extra parameter to the Powershell command-line host: -PSConfigFile. You can specify a Powershell console configuration XML file here that you can generate with the command Export-Console ConsoleConfiguration. This results in the ConsoleConfiguration.psc1 XML file:

<?xml version="1.0" encoding="utf-8"?>
<PSConsoleFile ConsoleSchemaVersion="1.0">
  <PSVersion>1.0</PSVersion>
  <PSSnapIns />
</PSConsoleFile>

I searched and searched for documentation on extra configuration like console width or something, but documentation on this is really sparse. So I fired up good old Reflector. After some drilling I ended up with the following code for the Export-Console command:

internal static void WriteToFile(MshConsoleInfo consoleInfo, string path)
{
    using (tracer.TraceMethod())
    {
        _mshsnapinTracer.WriteLine("Saving console info to file {0}.", new object[] { path });        
         XmlWriterSettings settings = new XmlWriterSettings();
        settings.Indent = true;
        settings.Encoding = Encoding.UTF8;
        using (XmlWriter writer = XmlWriter.Create(path, settings))
        {
            writer.WriteStartDocument();
            writer.WriteStartElement("PSConsoleFile");
            writer.WriteAttributeString("ConsoleSchemaVersion", "1.0");
            writer.WriteStartElement("PSVersion");
            writer.WriteString(consoleInfo.PSVersion.ToString());
            writer.WriteEndElement();
            writer.WriteStartElement("PSSnapIns");
            foreach (PSSnapInInfo info in consoleInfo.ExternalPSSnapIns)
            {
                writer.WriteStartElement("PSSnapIn");
                writer.WriteAttributeString("Name", info.Name);
                writer.WriteEndElement();
            }
            writer.WriteEndElement();
            writer.WriteEndElement();
            writer.WriteEndDocument();
            writer.Close();
        }
        _mshsnapinTracer.WriteLine("Saving console info succeeded.", new object[0]);    }
}

So the XML that we already saw is all there is... a missed chance there.

A custom command-line PowerShell host?

I have written a few Powershell hosts already, for example one hosted into Visual Studio, and there I encountered the same wrapping issue which I solved by the following implementation of the PSHostRawUserInterface:

using System;
using System.Management.Automation.Host;

namespace Macaw.SolutionsFactory.DotNet3.IntegratedUI.Business.Components.Powershell
{
    /// <summary>
    /// Implementation of PSHostRawUserInterface.
    /// </summary>
    public class PowershellHostRawUI : PSHostRawUserInterface
    {
        :

        public override Size BufferSize
        {
            get
            {   
                // The width is the width of the output, make it very wide!
                return new Size(512,1);
            }
            set
            {
                throw new Exception("BufferSize - set: The method or operation is not implemented.");
            }
        }

        :
    }
}

I solved transcripting in my own Powershell hosts by capturing all text output done by Powershell through custom implemented functions for the PSHostUserInterface you need to implement anyway, works like a breeze.

The solution, without custom host...

I was just getting started to solve the transcripting and wrapping issues by implementing again a Powershell host, but now a command-line Powershell host when I did one last digging step. What is you can set the BufferSize at runtime in your Powershell code. And the answer is.... yes you can! Through the $host Powershell variable: $host.UI.RawUI.BufferSize = new-object System.Management.Automation.Host.Size(512,50). This says that the output may be max 512 chars wide, the height may not be 1 (as in my code), bu this height value is not important in this case. It is used for paging in a interactive shell.

And this last statement gives us full control and solves all our transcripting and wrapping issues:

PowerShell -Command "`$host.UI.RawUI.BufferSize = new-object System.Management.Automation.Host.Size(512,50); `"c:\temp\testoutputandcapture.ps1`" -argument `"A value`"" >c:\temp\out.txt 2>&1

And this concludes a long journey into the transcripting and wrapping caves of Powershell. I hope this will save you the days of searching we had to put into it to tame the great but naughty Powershell beast.

Powershell: Generate simple XML from text, useful for CruiseControl.Net merge files

Monday, December 8, 2008

PowerShell

No Comments

I had the problem that I need to include ordinary text files into the CruiseControl.Net build output log file. This log file is a merge of multiple files in xml format. So I needed to get some text files into a simple xml format. I ended up with the following Powershell code to convert an input text file to a simple output xml file

Powershell code:

function Convert-XmlString
{
    param
    (
        [string]$text
    )
    # Escape Xml markup characters (http://www.w3.org/TR/2006/REC-xml-20060816/#syntax)
    $text.replace('&', '&').replace("'", ''').replace('"', '"').replace('<', '<').replace('>', '>')
}

function Convert-TextToXml
{
    param
    (
        $rootNode = 'root',
        $node = 'node',
        $path = $(Throw 'Missing argument: path'),
        $destination = $(Throw 'Missing argument: destination')
    )
    Get-Content -Path $path | ForEach-Object `
        -Begin {
            Write-Output "<$rootNode>"
        } `
        -Process {
            Write-Output " <$node>$(Convert-XmlString -text $_)</$node>"
        } `
        -End {
            Write-Output "</$rootNode>"
        } | Set-Content -Path $destination -Force
}

You can call this code as follows:

Convert-TextToXml -rootNode 'BuildAndPackageAll' -node 'message' -path 'c:\temp\in.txt' -destination 'c:\temp\out.xml'

where the file c:\temp\in.txt:

This is line 1
This is a line with the characters <, >, &, ' and "
This is line 3

Will be converted the the file c:\temp\out.xml:

<BuildAndPackageAll>
<message>This is line 1</message>
<message>This is a line with the characters <, >, &, ' and "</message>
<message>This is line 3</message>
</BuildAndPackageAll>