Efficient Base64 conversion in PowerShell

In one of my previous blogpost I was converting binary files to/from base64

I used naive straightforward approach

[Convert]::ToBase64String($bytes)
[Convert]::FromBase64String($base64String)

The problem with this approach is that if you need to convert big binary file, you have to read the whole file into memory. This consumes a lot of memory and a lot of time.

Better approach would be to read file with chunks and convert the chunks. Due to the base64 nature size of chunks should be multiplier of 3 when convert binary->base64 and a multiplier of 4 for reverse conversion

function ConvertTo-Base64
{
    param
    (
        [string] $SourceFilePath,
        [string] $TargetFilePath
    )

    $SourceFilePath = Resolve-PathSafe $SourceFilePath
    $TargetFilePath = Resolve-PathSafe $TargetFilePath
    
    $bufferSize = 9000 # should be a multiplier of 3
    $buffer = New-Object byte[] $bufferSize
    
    $reader = [System.IO.File]::OpenRead($SourceFilePath)
    $writer = [System.IO.File]::CreateText($TargetFilePath)
    
    $bytesRead = 0
    do
    {
        $bytesRead = $reader.Read($buffer, 0, $bufferSize);
        $writer.Write([Convert]::ToBase64String($buffer, 0, $bytesRead));
    } while ($bytesRead -eq $bufferSize);
    
    $reader.Dispose()
    $writer.Dispose()
}

function ConvertFrom-Base64
{
    param
    (
        [string] $SourceFilePath,
        [string] $TargetFilePath
    )

    $SourceFilePath = Resolve-PathSafe $SourceFilePath
    $TargetFilePath = Resolve-PathSafe $TargetFilePath

    $bufferSize = 9000 # should be a multiplier of 4
    $buffer = New-Object char[] $bufferSize
    
    $reader = [System.IO.File]::OpenText($SourceFilePath)
    $writer = [System.IO.File]::OpenWrite($TargetFilePath)
    
    $bytesRead = 0
    do
    {
        $bytesRead = $reader.Read($buffer, 0, $bufferSize);
        $bytes = [Convert]::FromBase64CharArray($buffer, 0, $bytesRead);
        $writer.Write($bytes, 0, $bytes.Length);
    } while ($bytesRead -eq $bufferSize);
    
    $reader.Dispose()
    $writer.Dispose()
}

function Resolve-PathSafe
{
    param
    (
        [string] $Path
    )
     
    $ExecutionContext.SessionState.Path.GetUnresolvedProviderPathFromPSPath($Path)
}

Note that we used [System.IO.Path]::GetFullPath instead of Resolve-Path, because Resolve-Path works with existing files only, and we need to deal with non-existent $TargetFilePath files

With that approach, when I converted 10mb file it completed within a second, whereas for the naive approach it took hours

UPD: Later after the blogpost I realized that [System.IO.Path]::GetFullPath won’t work properly and we need to use Resolve-PathSafe approach

Code above was updated to reflect that

Advertisements

About mnaoumov

Senior .NET Developer in Readify
This entry was posted in Uncategorized and tagged . Bookmark the permalink.

4 Responses to Efficient Base64 conversion in PowerShell

  1. Pingback: How to reach unreachable or copy files to RDP – Part 2 | mnaoumov.NET

  2. Anonymouse says:

    I just want to thank you for posting this article. It was quite helpful! I love it when a script comes together…

  3. david says:

    Is it possible to have the base64 string in a variable and decode it using a buffer?

  4. Pingback: How to get the bytes from a file and convert them to Base64String in PowerShell – Luis Carrazana's Blog

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s