Getting Started - Utilizing the Web: Part 2 (Invoke-RestMethod)
Welcome to my Getting Started with Windows PowerShell series!
In case you missed the earlier posts, you can check them out here:
- Customizing your environment
- Command discovery
- Using the ISE and basic function creation
- A deeper dive into functions
- Loops
- Modules
- Help
- Accepting pipeline input
- Strings
- Jobs
- Error handling
- Creating Custom Objects
- Working with data
- Utilizing the Web: Part 1 (Invoke-WebRequest)
We will be exploring:
Invoke-RestMethod
If Invoke-WebRequest had a brother, it would be Invoke-RestMethod. Well… brother, cousin, sister… you get the idea! They are related. Invoke-RestMethod allows us to send requests to REST web services (Representational State Transfer), and then returns to us a lovely object to use as we need to.
If we get an RSS feed as a response, we’ll get XML as the returned data type. When working with APIs, and using Invoke-RestMethod, you’ll get a custom object back. This is because more times than not we'll get the information back as JSON (JavaScript Object Notation). PowerShell automatically converts that into the custom object, which you can then dive into the properties of.
Here are the different methods we can send:
- Default
- Delete
- Get
- Head
- Merge
- Options
- Patch
- Post
- Put
- Trace
Whether we’re working with an RSS feed, or an API of sorts, Invoke-RestMethod allows us to do so with ease. After learning how to use it, it has quickly become one of my favorite commands in PowerShell.
Using Invoke-RestMethod
Let's go into some examples of how to use Invoke-RestMethod. To start out, we will use it to get some information via RSS, and then dive a little deeper into APIs.
RSS Feeds
Invoke-RestMethod can be used to gather information from RSS feeds. For this example, we'll get the feed for http://www.reddit.com/r/powershell. To do that, we simply append .rss to the URL.
Here's the command:
$redditRSS = Invoke-RestMethod 'http://www.reddit.com/r/PowerShell.rss'
Let's pipe our variable to Get-Member, and see what's up.
$redditRSS | Get-Member
It looks like we indeed have an XML object, and its associated methods/properties.
Let's see what the data looks like!
$redditRSS
The object we have here has a lot of properties that we can dive into.
For instance, let's say we want the author information for the first element in the array...
$redditRSS[0].author
Under the author property there are more properties that contain the information we're looking for. To access those, we need to drill down to them.
$redditRSS[0].author.name
$redditRSS[0].author.name.uri
This is part of the discovery you'll want to do when you receive an object back after using Invoke-RestMethod. Dig around, check documentation from the website if there is any, and discover all that you can. There is a lot of information returned, and the more you know, the more things you can do!
Let's dig into the content of the first array element.
$redditRSS[0].content
It looks like we want the '#text' property to see the actual returned content.
$redditRSS[0].content.'#text'
The content we received is in HTML. With some parsing we could make it more readable in the console.
Here's one way to clean it up, by stripping out the HTML tags:
$redditRSS[0].content.'#text' | ForEach-Object { $_ -replace '<[^>]+>','' }
With this object ($redditRSS[0]), we can also get the title and URL to the actual post.
$redditRSS[0].title
$redditRSS[0].link.href
The link value was buried in the href property under link.
What can we do with this information? Well... whatever you need/want to! That's the awesome part about PowerShell. If you need it to do something, there's a way to do it! We could setup a script that watches for a certain post with a specific title, and then have it email you or send you a text.
You can even make a make-shift post explorer, wherein you can check out the posts and comments of a specific subreddit. I have some example code for doing that as the conclusion to this section.
Code for subreddit exploration:
function Invoke-RedditBrowsing { #Begin function Invoke-RedditBrowsing [cmdletbinding()] Param( [Parameter(Mandatory)] [String] $subReddit ) #Get the RSS feed for the subreddit requested $redditRSS = Invoke-RestMethod "http://www.reddit.com/r/$subReddit.rss" #This is a hashtable for the text we'll replace in order to make the comments more readable $textToReplace = [hashtable]@{ '<[^>]+>' = '' '"' = '"' ''' = "'" ' ' = ' ' } #Make sure there is content before we proceed if ($redditRSS) { #Begin if for $redditRSS existence #Use the evil Write-Host to display information on what to do next Write-Host 'Select a [#] from the options below!' `n -ForegroundColor Black -BackgroundColor Green #Set $i to 0 so our loop iterates and displays $i correctly $i = 0 #ForEach $post in the subReddit's feed... ForEach ($post in $redditRSS) { #Use evil Write-Host to display the current value of $i and the title, then the author's name and post updated date Write-Host "[$i] -> $($post.Title)" Write-Host `t"By: [$($post.author.name)], Updated [$($post.updated)]"`n`n #Iterate $i by 1 $i++ } #Write-Host (new line) to make the next line look prettier Write-Host `n #Try / Catch to make sure we can convert the input to an integer. If not, we error out Try { #Ask for the post selection so we can proceed [int]$selection = Read-Host 'Which [#]? (any invalid option quits)' -ErrorAction SilentlyContinue } Catch { #If we can't make it an int... quit! Write-Host "Invalid option, quitting!" -ForegroundColor Red -BackgroundColor DarkBlue Break } #Switch statement for $selection. This makes sure it is in bounds for the array of posts. Switch ($selection) { #Begin Switch for the $selection value #if $i is less than or equal to the total number of values in the array $redditRSS... {$_ -le ($i - 1)} { #Begin option for valid selection #Get the comments RSS feed from the link provided $redditComments = Invoke-RestMethod -Uri "$($redditRSS[$_].link.href).rss" #Write-Host the title we'll be viewing comments for Write-Host `n Write-Host "Title: [$($redditRSS[$_].Title)]" -ForegroundColor Black -BackgroundColor Green #ForEach comment in the comments feed ForEach ($comment in $redditComments) { #Null out anything set in $commentText $commentText = $null #Set the comment text as the property which contains the actual comment $commentText = $comment.Content.'#text' #Go through each key in the hashtable we created earlier ForEach ($text in $textToReplace.Keys) { #Re-create the $commentText variable while replace the key ($text), with the value ($textToReplace.$text) #For example: it will match '<[^>]+>', then replace it with '' $commentText = $commentText -replace ($text,$textToReplace.$text) } #Use Write-Host to write out the author/comment text we cleaned up Write-Host `n`n Write-Host "By: [$($comment.author.name)]" -ForegroundColor Black -BackgroundColor Green Write-Host `t"Comment: [$commentText]" } } #End option for valid selection #If the number does not match a value that is valid, quit! Default { Write-Host "Invalid option, quitting!" -ForegroundColor Red -BackgroundColor DarkBlue Break } } #End Switch for the $selection value #If there is nothing in $redditRSS... quit! } else { Write-Host `n"Unable to get RSS feed for [$subReddit]." -ForegroundColor Red -BackgroundColor DarkBlue } #End if $redditRSS } #End function Invoke-RedditBrowsing Invoke-RedditBrowsing
Now to run it! It will prompt for the subreddit, which can be any subreddit. I will go with PowerShell.
The code will then run, and get the list of posts:
I will choose option [0], which is the first post.
There you have it, one example of what you can do with the data returned via Invoke-RestMethod. The primary motivation for doing something like this, at least for me, is that I like to visualize the data.
Using Write-Host is not suitable for automation, as it will not be seen by anyone. There are better options as well when you want to make specific things verbose, and others specifically for debug / errors. So why use it? Well, when visualizing the data you have available, it really can help certain bits of text stand out.
With a visual representation of what's available, it can lead to even more discovery and experimenting, which can in turn lead to more script ideas and ways to handle the data later.
Invoke-RestMethod and APIs
Invoke-RestMethod allows us to work with REST APIs with relative ease. When working with APIs, the most important thing is to familiarize yourself with their documentation. I'll link to the documentation for the different APIs I'll be using here.
- Weather Underground's Autocomplete API
- Weather Underground's API
- If you'd like to get your own API key to work with this API, they have a free option for developers
Example 1: No Authentication
For this example we will use Weather Underground's Autocomplete API. This API returns a list of locations that best match the input it receives. It can be used in tandem with their full API to get the weather information.
Let's try the following:
$city = 'Chicago' $lookupResults = Invoke-RestMethod -Uri "http://autocomplete.wunderground.com/aq?query=$city" $lookupResults
Looks like we have an object we can explore now!
$lookupResults | Get-Member
It is a custom object with one property (RESULTS). Let's check it out.
$lookupResults
Quite a few results! We can access the first result by using $lookupResults.RESULTS[0].
The 'l' property is something we will actually use later when getting a forecast for Chicago.
Let's wrap this up in a nice little function.
function Get-WeatherCity { #Begin function Get-WeatherCity [cmdletbinding()] param( [parameter(Mandatory=$true)] [string] $city ) Try { $lookupResults = Invoke-RestMethod -Uri "http://autocomplete.wunderground.com/aq?query=$city" } Catch { $errorMessage = $_.Exception.Message Write-Host "Error looking up [$city]: [$errorMessage]" Return } if ($lookupResults.results) { Return $lookupResults.Results[0] } else { Write-Host "Unable to find a result for: [$city]" -ForegroundColor Red -BackgroundColor DarkBlue } } #End function Get-WeatherCity
Now we can use (with that function in memory):
Get-WeatherCity -city Chicago
Example 2: API With a Key
function Get-WeatherCity { #Begin function Get-WeatherCity [cmdletbinding()] param( [parameter(Mandatory=$true)] [string] $city ) Try { $lookupResults = Invoke-RestMethod -Uri "http://autocomplete.wunderground.com/aq?query=$city" } Catch { $errorMessage = $_.Exception.Message Write-Host "Error looking up [$city]: [$errorMessage]" Return } if ($lookupResults.results) { Return $lookupResults.Results[0] } else { Write-Host "Unable to find a result for: [$city]" -ForegroundColor Red -BackgroundColor DarkBlue } } #End function Get-WeatherCity
With that function in memory, let's get a returned object for Chicago.
$cityInfo = Get-WeatherCity Chicago
Now for the setup to use the API to get weather information.
This code is one time code to keep my key secure so somebody reading the script can't see it in plain text.
To do this I will:
- Set the API key as a secure string in a variable
- Create a credential object to store the key
- Export that credential object via Export-Clixml
$apiKey = 'EnterYourKeyHere' | ConvertTo-SecureString -AsPlainText -Force $apiCredential = New-Object System.Management.Automation.PSCredential('apiKey',$apiKey) $apiCredential | Export-Clixml .\apiCred.xml
With that one time setup out of the way, you can then use:
$apiKey = (Import-Clixml .\apiCred.xml).GetNetworkCredential().Password
The key will be available to the script (as plain text), but it is stored in the $apiKey variable.
With all the setup done, here's the block of code we will be running:
$apiKey = (Import-Clixml .\apiCred.xml).GetNetworkCredential().Password $baseURL = 'http://api.wunderground.com/api/' $cityInfo = Get-WeatherCity Chicago $fullURL = $baseURL + $apiKey + '/features/conditions/hourly/forecast/webcams/astronomy/alerts' + "$($cityInfo.l).json" $weatherForecast = Invoke-RestMethod -Uri $fullURL
With the above code executed, $weatherForecast will now contain a plethora of data for us to work with.
$weatherForecast | Get-Member
This is one rich object! For alerts, you can use: $weatherForecast.alerts.
You can do a lot with the information available. I wrote a script that uses it to email a forecast (just the way we want it) to my significant other and myself. I took a parts of the script, and heavily modified them to fit this example. In the following example we will:
- Setup the API information needed
- Set a limit on use during the script's execution to 10
- Declare 3 cities we want the forecast for
- Create a PowerShell class (PowerShell version 5 only!)
- This class will have a method that formats the forecast in HTML
- Lookup the city information
- Pass the city information to a function that gathers the API data
- Create an object using the class
- Loop through the cities, get the information, update the object we created with the class, and export the HTML forecast after using the object's method to create it
Here's the code:
$apiKey = (Import-Clixml .\apiCred.xml).GetNetworkCredential().Password $baseURL = 'http://api.wunderground.com/api/' $cityInfo = Get-WeatherCity Chicago $fullURL = $baseURL + $apiKey + '/features/conditions/hourly/forecast/webcams/astronomy/alerts' + "$($cityInfo.l).json" $weatherForecast = Invoke-RestMethod -Uri $fullURL $citiesToGet = ('Chicago','San Francisco','Portland') $script:apiLimit = 10 Class Weather { #Begin Weather Class [pscustomobject]$apiData [string[]]formatForecastHTML() { #Begin method formatForecastHTML [System.Collections.ArrayList]$alertText = @() [System.Collections.ArrayList]$forecastText = @() $phases = $null $todayForecast = $this.apiData.forecast.simpleforecast.forecastday $city = $this.apiData.current_observation.display_location.full $selCam = Get-Random $this.apiData.webcams.count $camImg = $this.apiData.webcams[$selCam].CURRENTIMAGEURL $camName = $this.apiData.webcams[$selCam].linktext $camLink = $this.apiData.webcams[$selCam].link $curAlerts = $this.apiData.alerts if ($curAlerts) { foreach ($alert in $CurAlerts) { $typeName = $null $typeName = Get-WeatherInfo -Weather 'alert' -value $alert.type $alertText.Add(@" <p><b><font color=`"red`">Weather Alert! ($typeName)</font></b></p> <p>Date: $($alert.date) Expires: $($alert.Expires)</p> <p>$($alert.Message)</p> "@) | Out-Null } } $phases = @" <p><b>Sun Information</b></p><p></p> <p>Sunrise: $($this.apiData.sun_phase.sunrise.hour):$($this.apiData.sun_phase.sunrise.minute)</p> <p>Sunset: $($this.apiData.sun_phase.sunset.hour):$($this.apiData.sun_phase.sunset.minute)</p> <p></p> <p><b>Moon Information</b></p><p></p> <p>Moonrise: $($this.apiData.moon_phase.moonrise.hour):$($this.apiData.moon_phase.moonrise.minute)</p> <p>Moonset: $($this.apiData.moon_phase.moonset.hour):$($this.apiData.moon_phase.moonset.minute)</p> <p>Age: $($this.apiData.moon_phase.ageOfMoon) Days</p> <p>Phase: $($this.apiData.moon_phase.phaseofMoon)</p> <p>Illumination: $($this.apiData.moon_phase.percentIlluminated)%</p> "@ foreach ($day in $todayForecast) { $dayImg = $day.icon_url $dayMonth = $day.date.monthname $dayDay = $day.date.day $dayName = $day.date.weekday $dayHigh = $day.high.fahrenheit $dayLow = $day.low.fahrenheit $maxWind = $day.maxwind.mph $aveWind = $day.avewind.mph $aveHum = $day.avehumidity $conditions = $day.conditions [int]$dayPrecip = $day.pop $popText = Get-WeatherInfo -Weather 'preciptext' -value $dayPrecip $forecastText.Add(@" <p></p> <p></p> <p><b>$dayName, $dayMonth $dayDay</b></p> <p><img src=`"$dayImg`">$conditions</p> <p>Chance of precipitation: $dayPrecip% / $popText</p> <p>High: $dayHigh`F Low: $dayLow`F</p> <p>Ave Winds: $aveWind`mph Max Winds: $maxWind`mph</p> <p>Humidity: $aveHum%</p> "@) | Out-Null } $body = @" <p></p> <p>Here is your 4 day forecast!</p> <p>Random webcam shot from: <a href=`"$camLink`">$camName</a></p> <p><img src=`"$camImg`"></p> $($phases | Out-String) $($alertText | Out-String) $($forecastText | Out-String) "@ Return $body } #End method formatForecastHTML [string[]]formatHourlyForecastHTML() { #Begin Class format4DayForecast [System.Collections.ArrayList]$alertText = @() [System.Collections.ArrayList]$forecastText = @() $phases = $null $todayForecast = $this.apiData.forecast.simpleforecast.forecastday $city = $this.apiData.current_observation.display_location.full $selCam = Get-Random $this.apiData.webcams.count $camImg = $this.apiData.webcams[$selCam].CURRENTIMAGEURL $camName = $this.apiData.webcams[$selCam].linktext $camLink = $this.apiData.webcams[$selCam].link $curAlerts = $this.apiData.alerts $hourlyForecast = $this.apiData.hourly_forecast if ($curAlerts) { foreach ($alert in $CurAlerts) { $typeName = $null $typeName = Get-WeatherInfo -Weather 'alert' -value $alert.type $alertText.Add( @" <p><b><font color=`"red`">Weather Alert! ($typeName)</font></b></p> <p>Date: $($alert.date) Expires: $($alert.Expires)</p> <p>$($alert.Message)</p> "@) | Out-Null } } $phases = @" <p><b>Sun Information</b></p><p></p> <p>Sunrise: $($this.apiData.sun_phase.sunrise.hour):$($this.apiData.sun_phase.sunrise.minute)</p> <p>Sunset: $($this.apiData.sun_phase.sunset.hour):$($this.apiData.sun_phase.sunset.minute)</p> <p></p> <p><b>Moon Information</b></p><p></p> <p>Moonrise: $($this.apiData.moon_phase.moonrise.hour):$($this.apiData.moon_phase.moonrise.minute)</p> <p>Moonset: $($this.apiData.moon_phase.moonset.hour):$($this.apiData.moon_phase.moonset.minute)</p> <p>Age: $($this.apiData.moon_phase.ageOfMoon) Days</p> <p>Phase: $($this.apiData.moon_phase.phaseofMoon)</p> <p>Illumination: $($this.apiData.moon_phase.percentIlluminated)%</p> "@ foreach ($hour in $hourlyForecast) { $prettyTime = $hour.fcttime.pretty $hourTemp = $hour.temp.english $hourImg = $hour.icon_url $hourChill = $hour.windchill.english if ($hourChill -eq -9999) { $hourChill = 'N/A' } else { $hourChill = $hourChill + 'F' } $hourWind = $hour.wspd.english $windDir = $hour.wdir.dir $hourUV = $hour.uvi $dewPoint = $hour.dewpoint.english $hourFeels = $hour.feelslike.english $hourHum = $hour.humidity $conditions = $hour.condition [int]$hourPrecip = $hour.pop $popText = Get-WeatherInfo -Weather 'preciptext' -value $hourPrecip $forecastText.Add( @" <p></p> <p></p> <p><b>$prettyTime</b></p> <p><img src=`"$hourImg`">$conditions</p> <p>Chance of precipitation: $hourPrecip% / $popText</p> <p>Current Temp: $hourTemp`F Wind Chill: $hourChill Feels Like: $hourFeels`F</p> <p>Dew Point: $dewPoint</p> <p>Wind Speed: $hourWind`mph Direction: $windDir</p> <p>Humidity: $hourHum%</p> <p>UV Index: $hourUV "@) | Out-Null } $body = @" <p></p> <p>Here is your hourly forecast!</p> <p>Random webcam shot from: <a href=`"$camLink`">$camName</a></p> <p><img src=`"$camImg`"></p> <p>$city Radar:</p> <p><img src=`"$($this.radarURL)`"></p> $($phases | Out-String) $($alertText | Out-String) $($forecastText | Out-String) "@ return $body } #End method formatHourlyForecastHTML Weather([String]$apiURL, [String]$radarURL, [PSCustomObject]$apiData) { $this.radarURL = $radarURL $this.apiData = $apiData } Weather() {} } #End class Weather function Get-WeatherCity { #Begin function Get-WeatherCity [cmdletbinding()] param( [parameter(Mandatory=$true)] [string] $city ) Try { $lookupResults = Invoke-RestMethod -Uri "http://autocomplete.wunderground.com/aq?query=$city" } Catch { $errorMessage = $_.Exception.Message Write-Host "Error looking up [$city]: [$errorMessage]" Return } if ($lookupResults.results) { Return $lookupResults.Results[0] } else { Write-Host "Unable to find a result for: [$city]" -ForegroundColor Red -BackgroundColor DarkBlue } } #End function Get-WeatherCity function Get-WeatherAPIData { #Begin function Get-WeatherAPIData [cmdletbinding()] param( [parameter(Mandatory)] [PSCustomObject] $city ) [System.Collections.ArrayList]$weatherCity = @() $weatherProperty = $null $weatherO = $null $fullURL = $baseURL + $apiKey + '/features/conditions/hourly/forecast/webcams/astronomy/alerts' + "$($city.l).json" $radarURL = "http://api.wunderground.com/api/$apiKey/animatedradar/animatedsatellite" + "$($city.l).gif?num=6&delay=50&interval=30" Try { if ($script:apiCount -le $apiLimit) { Write-Host "Attempting to get API information for: [$($city.Name)]." $weatherForecast = Invoke-RestMethod -Uri $fullURL $script:apiCount++ $weatherForecast | Export-Clixml ("$outputDir\$($city.Name)APIData{0:MM-dd-yy_HHmm}.clixml" -f (Get-Date)) Write-Host "The API has been used [$script:apiCount] times, against the limit of [$apiLimit]" } else { Write-Host "The API limit of [$apiLimit] has been reached!" -ForegroundColor Red -BackgroundColor DarkBlue Break } } Catch { $errorMessager = $_.Exception.Message Write-Host "Unabled to get API data from Weather Underground: [$($_.Exception.Message)]." -ForegroundColor Red -BackgroundColor DarkBlue } if ($weatherForecast.forecast) { Write-Host "API information found for: [$($city.Name)]" $weatherO = [PSCustomObject]@{ APIData = $weatherForecast RadarURL = $radarURL } $weatherCity.Add($weatherO) | Out-Null Return $weatherCity } else { Write-Host "API information not found for: [$($city.Name)]" -ForegroundColor Red -BackgroundColor DarkBlue } } #End function Get-WeatherAPIData function Get-WeatherInfo { #Begin function Get-WeatherInfo [cmdletbinding()] param( [string] $weather, $value ) Switch ($weather) { {$_ -eq 'preciptext'} { Switch ($value) { {$_ -lt 20} { $popText = 'No mention' } {$_ -eq 20} { $popText = 'Slight Chance' } {($_ -lt 50 -and $_ -gt 20)} { $popText = 'Chance' } {$_ -eq 50} { $popText = 'Good chance' } {($_ -lt 70 -and $_ -gt 50)} { $popText = 'Likely' } {$_ -ge 70} { $popText = 'Extremely likely' } } Return $popText } {$_ -eq 'alert'} { Switch ($value) { 'HEA' {$typeName = 'Heat Advisory'} 'TOR' {$typeName = 'Tornado Warning'} 'TOW' {$typeName = 'Tornado Watch'} 'WRN' {$typeName = 'Severe Thunderstorm Warning'} 'SEW' {$typeName = 'Severe Thunderstorm Watch'} 'WIN' {$typeName = 'Winter Weather Advisory'} 'FLO' {$typeName = 'Flood Warning'} 'WAT' {$typeName = 'Flood Watch / Statement'} 'WND' {$typeName = 'High Wind Advisory'} 'SVR' {$typeName = 'Severe Weather Statement'} 'HEA' {$typeName = 'Heat Advisory'} 'FOG' {$typeName = 'Dense Fog Advisory'} 'SPE' {$typeName = 'Special Weather Statement'} 'FIR' {$typeName = 'Fire Weather Advisory'} 'VOL' {$typeName = 'Volcanic Activity Statement'} 'HWW' {$typeName = 'High Wind Warning'} 'REC' {$typeName = 'Record Set'} 'REP' {$typeName = 'Public Reports'} 'PUB' {$typeName = 'Public Information Statement'} } Return $typeName } } } #End function Get-WeatherInfo $weatherObject = [Weather]::New() Write-Host `n`n ForEach ($cityName in $citiesToGet) { $findCity = Get-WeatherCity -city $cityName $weatherCity = Get-WeatherAPIData -city $findCity -ErrorAction Stop if ($weatherCity) { $weatherObject.apiData = $null $weatherObject.apiData = $weatherCity.APIData $foreCastFile = $null $foreCastFile = ".\$cityName.html" Write-Host `n"Exporting HTML formatted forecast to [$foreCastFile]."`n $weatherObject.formatHourlyForecastHTML() | Out-File ".\$cityName.html" } else { Write-Host "No data in [`$weatherCity] variable, halting execution on [$($userEmail.FirstName) $($userEmail.LastName)]." -ForegroundColor Red -BackgroundColor DarkBlue Break } } Write-Host `n`n
There should be 3 files in the folder now.
There they are. Let's take a look at Chicago's forecast.
You can do anything you put your mind to with the data received.
Homework
- Explore the class created here to see how the information in the HTML generated forecast is created.
- A lot of this information is transferable to functions and other methods of creating the HTML formatted forecast if you do not have PowerShell version 5.
- See if a web service you use often has an API, read up on the documentation, and see if there's something you can automate doing with PowerShell!
I hope you've enjoyed the series so far! As always, leave a comment if you have any feedback or questions!
-Ginger Ninja