Accessing the Kudu REST API from Powershell

Tuesday, March 2, 2021

Scott Hanselman said it and I admire him. So.

Spoiler: the final result, a Powershell module to access Kudu can be found on GitHub as Kudu-API.psm1.

For a particular website I host on Azure, content is uploaded to the web site every fifteen minutes by a scheduled task. This uses FTP and because FTP is old fashioned and generally sucks, it uses WinSCP for the actual upload. While WinSCP is a really great product, it feels like overkill to employ a third party product just to upload a file. But hey, whatever works.

But yesterday I decided to give Kudu another try. It's a companion web site to every Azure web app. It hosts extensions, takes care of deployment, interfaces with Git, etc. And one of the services it offers is a Virtual File Service (VFS) capable of enumerating the files on the site (and those around it), but also uploading, downloading and deleting files. These services are accessible through a REST API, which is succinctly documented.

The Kudu REST API

Basically, if your web app is called MyWebApp, it'll be hosted on mywebapp.azurewebsites.net and the Kudu services will be hosted on mywebapp.scm.azurewebsites.net. "Simply" use Invoke-RestMethod on the scm-containing URL, GETting, PUTting and POSTing to perform the various operations.

Authentication

Authentication is straightforward, using an Authorization header. The header is called "Authorization" and its contents are "Basic <token>". The token itself is a combination of a user name and a password, separated by a colon and Base64-encoded to make transmitting weird characters in the password easier. The English version is almost longer than the PowerShell version:

# Create a token from username and password
$token = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes("$($Username):$($Password)"))

Then:

Invoke-RestMethod -Uri ... ... -Headers @{ 'Authorization' = "Basic $Token" } ...

There are two sets of username/password pairs that can be used to access Kudu on an Azure Web App: the one used for publishing, or the one set for the Azure App Service Plan. This is explained on the Kudu Wiki.

Exploring Kudu in the browser

Kudu is also accessible in the browser, which uses the same authentication as the Azure Portal. Simply visit the scm-URL of your site (mywebapp.scm.azurewebsites.net) or select Advanced Tools from the your web App in the Azure Portal:

Click on the link that appears in the right pane to go to Kudu for your Web App. (This will open the scm-url). Once there, you can modify the address by adding /api/environment and you will see:

{"version":"91.30201.5050.0","siteLastModified":"2021-02-23T12:36:06.0600000Z"}

Simple JSON, which Invoke-RestMethod should parse automatically.

Testing the REST API

So here we go:

$token = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes("$($Username):$($Password)"))
Invoke-RestMethod https://mywebapp.scm.azurewebsites.net/api/environment `
    -Headers @{ 'Authorization' = "Basic $Token" }

Catch #1: use https or you'll get an HTML page containing an error message!

Sure enough, the result is:

version siteLastModified
------- ----------------
91.30201.5050.0 23-2-2021 12:36:06

Yay!

Using Kudu VFS

But that's not what we came here for. We want to use the Virtual File System, so we boldly type:

Invoke-RestMethod https://mywebapp.scm.azurewebsites.net/api/vfs/ `
    -Headers @{ 'Authorization' = "Basic $Token" }

And we get a list of objects with a name, a size, creation and modification dates, etc., in fact: the contents of the Kudu VFS root directory. Success.

Then we try to get the list of files in the "data" folder:

Invoke-RestMethod https://mywebapp.scm.azurewebsites.net/api/vfs/data `
    -Headers @{ 'Authorization' = "Basic $Token" }

Oops: an HTML page again, titled "Sign in to your account". This is an unfortunate, documented "feature" of VFS (and therefore catch #2): VFS path names have to end in a slash. When you try this from the browser, you will be redirected to the version of the URL with the slash, so it's not that obvious. The new version works:

Invoke-RestMethod https://mywebapp.scm.azurewebsites.net/api/vfs/data/ `
    -Headers @{ 'Authorization' = "Basic $Token" }

​We get the contents of the "data" directory.

Downloading a file

Eventually, we want to upload files, but to make sure we understand everything, we'll try a download first. When using the browser, you can see there is a file called eventlog.xml in the LogFiles directory, which we should be able to download using:

Invoke-RestMethod https://mywebapp.scm.azurewebsites.net/api/vfs/LogFiles/eventlog.xml `
    -Headers @{ 'Authorization' = "Basic $Token" }

The result is:

Events
------
Events

Interesting! The file contains XML and Invoke-WebRequest turns it into a structured XmlDocument automatically, so we can enumerate and query it. Noice.

Uploading a file

Uploading should be easy too. Just use the PUT method (not the default GET) and supply the contents of the file. This is easiest using the -InFile parameter of Invoke-RestMethod:

Invoke-RestMethod https://mywebapp.scm.azurewebsites.net/api/vfs/data/test.txt `
    -Headers @{ 'Authorization' = "Basic $Token" } `
    -Method 'PUT'
    -InFile localtest.txt

This copies the local file "localtest.txt" to the data-folder and names the resulting file "test.txt".

Catch #3: If you do this again, you wil get:

ETag does not represent the latest state of the resource.

This is, again, documented. For safety reasons you have to specify the current ETag of a file when overwriting or deleting it. The ETag is not returned in the directory listing, so I don't know how to obtain it. Fortunately, you can disable the "Etag-check" by supplying a header named "If-Match" with contents "*". So this will work:

Invoke-RestMethod https://mywebapp.scm.azurewebsites.net/api/vfs/data/test.txt `
    -Headers @{ 'Authorization' = "Basic $Token"; 'If-Match' = '*' } `
    -Method 'PUT' `
    -InFile localtest.txt

An alternative to using the -InFile parameter is specifying a -Body:

Invoke-RestMethod https://mywebapp.scm.azurewebsites.net/api/vfs/data/test.txt `
    -Headers @{ 'Authorization' = "Basic $Token"; 'If-Match' = '*' } `
    -Method 'PUT'
    -Body "line 1`r`nline 2"

This will set the contents of the file to

line 1
line 2

Executing commands

OK, task accomplished. But while we're here, /api/command looks interesting, too:

POST /api/command
Executes an arbitrary command line and return its output

So here we go, trying to execute "dir" in the default working directory:

Invoke-RestMethod https://mywebapp.scm.azurewebsites.net/api/command `
    -Headers @{ 'Authorization' = "Basic $Token"; 'If-Match' = '*' } `
    -Method 'POST' `
    -Body (ConvertTo-Json @{ command = 'dir'; dir = '' })

Oops:

Error 403 - This web app is stopped.

Catch #4: When using POST, you have to supply UserAgent and ContentType parameters. The error message is, unfortunately, rather unhelpful, but the fix is easy:

Invoke-RestMethod https://mywebapp.scm.azurewebsites.net/api/command `
    -Headers @{ 'Authorization' = "Basic $Token"; 'If-Match' = '*' } `
    -Method 'POST' `
    -Body @{ command = ''; dir = '' } `
    -UserAgent 'powershell/1.0' `
    -ContentType 'application/json'

Omitting the -ContentType parameter will get you:

The request entity's media type 'application/x-www-form-urlencoded' is not supported for this resource

With it, this returns an object with properties Output and (for non-builtin commands) Error and ExitCode.

All together now

Combining all we learned so far, our basic Invoke-KuduApi looks like this:

<#
    Call the Kudu REST API. SiteName, Token, Method and Path are mandatory.
    Optional Body, InFile and OutFile parameters are passed to Invoke-RestMethod.
#>
Function Invoke-KuduApi()
{
    [CmdletBinding()]
    Param(
        [Parameter(Mandatory=$true)]
        [string]$SiteName,
        [Parameter(Mandatory=$true)]
        [string]$Token,
        [Parameter(Mandatory=$true)]
        [Microsoft.PowerShell.Commands.WebRequestMethod]$Method,
        [Parameter(Mandatory=$true)]
        [string]$Path, # Relative after .../api/
        [Parameter(Mandatory=$false)]
        [string]$Body,
        [Parameter(Mandatory=$false)]
        [string]$InFile,
        [Parameter(Mandatory=$false)]
        [string]$OutFile
    )

    [string]$url = "$(Get-KuduUrl $Sitename)api/$Path";

    $arguments = @{}

    if ($Body) { $arguments.Body = $Body }
    if ($InFile) { $arguments.InFile = $InFile }
    if ($OutFile) { $arguments.OutFile = $OutFile }

    # Call the Kudu service. The If-Match header bypasses ETag checking
    return Invoke-RestMethod `
        -Method $Method `
        -Uri $url `
        -Headers @{ 'Authorization' = "Basic $Token"; 'If-Match' = '*' } `
        -UserAgent 'powershell/1.0' `
        -ContentType 'application/json' `
        @arguments
}

The mandatory parameters are -SiteName, -Token and -Path. SiteName is the name of your web app, e.g. 'mywebapp'. The Token is created using a function called New-KuduAuhorizationToken:

<#
    Create a new Kudu authorization token
#>
Function New-KuduAuthorizationToken()
{
    [CmdletBinding()]
    Param(
        [Parameter(Mandatory=$true)]
        [string]$Username,
        [Parameter(Mandatory=$true)]
        [string]$Password
    )

    # Note that the $username here should look like `SomeUserName`, and **not** `SomeSite\SomeUserName`
    return [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes("$($Username):$($Password)"))
}

The helper function Get-KuduUrl makes sure we use https:

<#
    Get the Kudu base-url for a site name. The URL ends in a slash
#>
Function Get-KuduUrl()
{
    [CmdletBinding()]
    Param(
        [Parameter(Mandatory=$true)]
        [string]$SiteName
    )

    return "https://$SiteName.scm.azurewebsites.net/"
}

Combining these, we can download a file from VFS-path $Path and saves it to $OutFile:

Invoke-KuduApi $SiteName $Token 'GET' "vfs$Path" -OutFile $OutFile

A Kudu module

I put everything in a Powershell module called Kudu-API.psm1:

[CmdletBinding()]
Param()

<#
    Powershell access to the Kudu REST API

    See https://github.com/projectkudu/kudu/wiki/REST-API

    All functions take SiteName and Token arguments. SiteName is the part before '.scm.azurewebsites.net'.
    The Token is created by calling New-KuduAuthorizationToken with the publishing user name and password
    of the Azure web App.
#>

<#
    Get the Kudu base-url for a site name. The URL ends in a slash
#>
Function Get-KuduUrl()
{
    [CmdletBinding()]
    Param(
        [Parameter(Mandatory=$true)]
        [string]$SiteName
    )

    return "https://$SiteName.scm.azurewebsites.net/"
}

<#
    Create a new Kudu authorization token
#>
Function New-KuduAuthorizationToken()
{
    [CmdletBinding()]
    Param(
        [Parameter(Mandatory=$true)]
        [string]$Username,
        [Parameter(Mandatory=$true)]
        [string]$Password
    )

    # Note that the $username here should look like `SomeUserName`, and **not** `SomeSite\SomeUserName`
    return [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes("$($Username):$($Password)"))
}

<#
    Call the Kudu REST API. SiteName, Token, Method and Path are mandatory.
    Optional Body, InFile and OutFile parameters are passed to Invoke-RestMethod.
#>
Function Invoke-KuduApi()
{
    [CmdletBinding()]
    Param(
        [Parameter(Mandatory=$true)]
        [string]$SiteName,
        [Parameter(Mandatory=$true)]
        [string]$Token,
        [Parameter(Mandatory=$true)]
        [Microsoft.PowerShell.Commands.WebRequestMethod]$Method,
        [Parameter(Mandatory=$true)]
        [string]$Path, # Relative after .../api/
        [Parameter(Mandatory=$false)]
        [string]$Body,
        [Parameter(Mandatory=$false)]
        [string]$InFile,
        [Parameter(Mandatory=$false)]
        [string]$OutFile
    )

    [string]$url = "$(Get-KuduUrl $Sitename)api/$Path";

    $arguments = @{}

    if ($Body) { $arguments.Body = $Body }
    if ($InFile) { $arguments.InFile = $InFile }
    if ($OutFile) { $arguments.OutFile = $OutFile }

    # Call the Kudu service. The If-Match header bypasses ETag checking
    return Invoke-RestMethod `
        -Method $Method `
        -Uri $url `
        -Headers @{ 'Authorization' = "Basic $Token"; 'If-Match' = '*' } `
        -UserAgent 'powershell/1.0' `
        -ContentType 'application/json' `
        @arguments
}

<#
    Execute a command in the Kudu console.
#>
Function Invoke-KuduCommand()
{
    [CmdletBinding()]
    Param(
        [Parameter(Mandatory=$true)]
        [string]$SiteName,
        [Parameter(Mandatory=$true)]
        [string]$Token,
        [Parameter(Mandatory=$true)]
        [string]$Command,
        [Parameter(Mandatory=$true)]
        [string]$WorkingDirectory
    )

    [string]$body = (ConvertTo-Json @{ command = $Command; dir = $WorkingDirectory })

    return Invoke-KuduApi $SiteName $Token 'POST' 'command' -Body $body
}

<#
    Download a file from Kudu vfs (virtual file system)
    If the path ends in a slash, it's interpreted as a 
    directory and its contents are returned
#>
Function Get-KuduItem()
{
    [CmdletBinding()]
    Param(
        [Parameter(Mandatory=$true)]
        [string]$SiteName,
        [Parameter(Mandatory=$true)]
        [string]$Token,
        [Parameter(Mandatory=$true)]
        [string]$Path, # Must start with a slash
        [Parameter(Mandatory=$false)]
        [string]$OutFile # Save the file under this name. If not specified, return the contents as a string
    )

    return Invoke-KuduApi $SiteName $Token 'GET' "vfs$Path" -OutFile $OutFile
}

<#
    Upload a file to Kudu vfs (virtual file system)
#>
Function Set-KuduItem()
{
    [CmdletBinding()]
    Param(
        [Parameter(Mandatory=$true)]
        [string]$SiteName,
        [Parameter(Mandatory=$true)]
        [string]$Token,
        [Parameter(Mandatory=$true)]
        [string]$Path, # Path must start with a slash
        [Parameter(Mandatory=$true)]
        [string]$InFile # The local file to upload
    )

    return Invoke-KuduApi $SiteName $Token 'PUT' "vfs$Path" -InFile $InFile
}

<#
    Get the Kudu environment
    Properties: version and siteLastModified
#>
Function Get-KuduEnvironment()
{
    [CmdletBinding()]
    Param(
        [Parameter(Mandatory=$true)]
        [string]$SiteName,
        [Parameter(Mandatory=$true)]
        [string]$Token
    )

    return Invoke-KuduApi $SiteName $Token 'GET' 'environment'
}

It's far from complete, but it contains remedies to all catches we found so far ;-)

Use it like this:

[CmdletBinding()]
Param()

Import-Module .\Kudu-Api -Verbose:$false

$sitename = 'mywebapp'
$username = '$mywebapp'
$password = 'your-publishing-or-deployment-password'

# Create a token for the site
$token = New-KuduAuthorizationToken $username $password

# Create a hash table with the site name and token
# By using @site we can supply the -SiteName and -Token arguments
$site = @{ SiteName = $sitename; Token = $token }

# Get the Kudu version
"Kudu version: $((Get-KuduEnvironment @site).version)"

# Get the contents of the wwwroot-folder. Needs trailing slash!
(Get-KuduItem @site '/site/wwwroot/').Path

Note: Splatting When you make multiple calls to Kudu on a web app, you will have to supply the -SiteName and -Token parameters every time. To make this a little easier, the code above uses a Powershell trick called "Splatting". Basically, it allows you to set up a hashtable with parameter names and values, and to supply those parameters using @site. Short and convenient.