This post has already been read 7450 times!
When using Citrix NetScaler to present a web accessible service using GSLB the NetScaler and GSLB has the ability to automatically fail over between sites and return the public IP Address of the currently live service. This is great and a massive plus for any deployment, however, what if the internet connection goes down at a specific site but the NetScaler services are still up. This will give the user the IP Address of the live service but they will not be able to connect to it as there is no inbound connection.
This post will show you how to use Citrix Octoblu to monitor those site connections and if required force a NetScaler service to be put into a down state and therefore fail the service over to the currently live site. I know that you can do this with additional monitors on the GSLB service polling the sites upstream router, but lets face it, using Octoblu gives you more alerting capability and, and the most important, its damn cool !
So, lets get going. First I will describe what I have in play here to mimic the scenario.
- Site A – Primary DC
- Site B – Fail Over DC
- vServer Site A – 192.168.0.230 – Port 1234 (I don’t have multiple external IP Addresses so use ports to separate traffic)
- vServer Site B – 192.168.0.231 – Port 5678 (I don’t have multiple external IP Addresses so use ports to separate traffic)
- gslboctoblu.bretty.me.uk – GSLB Domain Name (Site A Primary, Site B Failover)
- Firewall Rule Port 1234 –> vServer Site A
- Firewall Rule Port 5678 –> vServer Site B
The Code
You will need to add the GSLB Failover module to your existing NetScaler Management Script. If you do not have a copy of this script then you can find the original script in this post
Add the code at the bottom of this post to your ps1 file and save it in a directory on your machine with your Octoblu Shell Connector. You can also find out more about configuring these in the post mentioned above.
The Module listed in this post will take in 5 variables:
- $sitea – A yes/no field generated from Octoblu to determine the state of the internet service hosted in Site A
- $siteb – A yes/no field generated from Octoblu to determine the state of the internet service hosted in Site B
- $gslbservicea – the GSLB Service Name Bound to Site A
- $gslbserviceb – the GSLB Service Name Bound to Site B
- $nsip – The NetScaler IP Hosting the GSLB Services
NetScaler Username and Password
I have hard coded the nsroot username and password into the module (bad I know) and I would NOT reccoment doing this in production but for the purpose of this example it works fine. You could alter the Parameters and pass in $NSUserName and $NSPassword to make the script more generic.
NITRO Limitations
As we are dealing with GSLB there are a few differences to the process we would use when dealing with standard Load Balancers.
If you look at the NITRO documentation you will notice that not all properties for a particular object on the NetScaler are Read/Write
If we look more at the documentation you can see there are certain parameters we can pass into the payload to update the service.
NOTE: Status is not one of them!
So we need a way to directly disable a GSLB Service using only NITRO. There are probably loads of different ways to do this but I am going to use a monitor that is deliberately set up to fail, and will therefore mark the service as down.
Force Down Monitor
Create a new monitor on your NetScaler called mon_force_down and set it to ping an IP Address that will fail. This way any service that you bind this monitor to will go down as the monitor is set up to fail the test
We will use this monitor to force a GSLB service to fail if the internet connection is down and therefore force a GSLB failover
For those interested we will be using the gslbservice_lbmonitor_binding NITRO command to bind this service to the GSLB service
Octoblu Flow
Next we head over to Octoblu and create a flow that looks like the below
I will explain in more detail during the video how to build this flow as writing all of that down in text would be a blog series in itself! What I am trying to achieve with this flow is bullet pointed below
- Set 2 variables SiteA and SiteB to down
- perform a HTTP Get for both Sites (If one fails that flow branch will stop there
- If the HTTP GET passes then set the Site Variable to UP
- Gather both Variables for Site A and Site B
- Compose the {{msg}} with Site Status
- Pause for 5 Seconds to let things Settle
- Run through a quick function to set a variable for SiteXActive
- Pass Site Status variables into Powershell as Environment Variables and execute the NITRO Powershell Code to action the GSLB Services
Octoblu Flow Example and Demo
[embedyt] https://www.youtube.com/watch?v=cqvDUfQNKCs[/embedyt]
Thats it, hopefully this has been a good example of what you can achieve with Octoblu, NetScaler and NITRO
Laters,
b@m
Code
function NS-Octoblu-GSLB-Failover ([string]$sitea, [string]$siteb, [string]$gslbservicea, [string]$gslbserviceb, [string]$NSIP) {
<#
.SYNOPSIS
Enables and Disables GSLB Services based on external URL Checks from Octoblu .
.DESCRIPTION
Takes input from Obtoblu about the external FQDN for a GSLB Service and enables / disables a service accordingly.
.PARAMETER sitea
String from Octoblu tests stating if site a is up
.PARAMETER siteb
String from Octoblu tests stating if site b is up
.PARAMETER gslbservicea
String from Octoblu stating what service is bound to site a
.PARAMETER gslbservicea
String from Octoblu stating what service is bound to site b
.PARAMETER NSIP
NetScaler IP Address that hosts the GSLB Service
.NOTES
Name: NS-Octoblu-GSLB-Failover
Author: David Brett - Citrix CTP
Date Created: 20/03/2017
.CHANGE LOG
David Brett - 20/03/2017 - Initial Script Creation
.LINK
https://bretty.me.uk
#>
#Check is Site A is active. If so then ensure that the GSLB Service Bound to it is active also
if ($sitea -eq "yes") {
#Login and enable the vserver passed into the function
ns-login $NSIP nsroot nsroot
#Check if the GSLB Service exists and is already enabled
$Url = "$NSIP/nitro/v1/config/gslbservice/$gslbservicea"
$Method = "GET"
$ContentType = "application/json"
#Try to connect to the GSLB Service and Catch the error output
try {
$GSLBStatus = Invoke-RestMethod -uri $Url -WebSession $nsSession.WebSession -ContentType $ContentType -Method $Method
#Get the service status and IP Address
$GSLBServiceStatus = $GSLBStatus.gslbservice.svrstate
} catch {
#Cannot find a GSLB Service with the same name as the valus passed in
$ErrException = $_.Exception.Response.StatusCode.value__
if ($ErrException -eq 404) {
cls
write-host -ForegroundColor Red "GSLB Service is not found, quitting"
break
}
}
if ($GSLBServiceStatus -ne "UP") {
$Url = "$NSIP/nitro/v1/config/gslbservice_lbmonitor_binding/" + $gslbservicea + "?args=monitor_name:mon_force_down"
#Set the HTTP Method
$Method = "DELETE"
#Remove the force down monitor
Invoke-RestMethod -uri $Url -WebSession $nsSession.WebSession -Method $Method
cls
write-host -ForegroundColor Red "GSLB Service Force Down Monitor Removed"
}
ns-logout $NSIP
} else {
If ($sitea -eq "no") {
#Login and enable the vserver passed into the function
ns-login $NSIP nsroot nsroot
#Check if the GSLB Service exists and is already disabled
$Url = "$NSIP/nitro/v1/config/gslbservice/$gslbservicea"
$Method = "GET"
$ContentType = "application/json"
#Try to connect to the GSLB Service and Catch the error output
try {
$GSLBStatus = Invoke-RestMethod -uri $Url -WebSession $nsSession.WebSession -ContentType $ContentType -Method $Method
#Get the service status and IP Address
$GSLBServiceStatus = $GSLBStatus.gslbservice.svrstate
} catch {
#Cannot find a GSLB Service with the same name as the valus passed in
$ErrException = $_.Exception.Response.StatusCode.value__
if ($ErrException -eq 404) {
cls
write-host -ForegroundColor Red "GSLB Service is not found, quitting"
break
}
}
if ($GSLBServiceStatus -eq "UP") {
#Set teh NITRO URL
$Url = "$NSIP/nitro/v1/config/gslbservice_lbmonitor_binding"
#Set the HTTP Method
$Method = "PUT"
#Set the Request Header for Content Type
$ContentType = "application/json"
#Set up the JSON Payload to send to the netscaler
$PayLoad = ConvertTo-JSON @{
"gslbservice_lbmonitor_binding"=@{
"servicename"=$gslbservicea;
"monitor_name"="mon_force_down";
"monstate"="enabled";
"weight"=1
}
}
Invoke-RestMethod -uri $Url -WebSession $nsSession.WebSession -ContentType $ContentType -Body $Payload -Method $Method
cls
write-host -ForegroundColor Red "GSLB Service Force Down Monitor Added"
}
ns-logout $NSIP
}
}
#Check is Site B is active. If so then ensure that the GSLB Service Bound to it is active also
if ($siteb -eq "yes") {
#Login and enable the vserver passed into the function
ns-login $NSIP nsroot nsroot
#Check if the GSLB Service exists and is already enabled
$Url = "$NSIP/nitro/v1/config/gslbservice/$gslbserviceb"
$Method = "GET"
$ContentType = "application/json"
#Try to connect to the GSLB Service and Catch the error output
try {
$GSLBStatus = Invoke-RestMethod -uri $Url -WebSession $nsSession.WebSession -ContentType $ContentType -Method $Method
#Get the service status and IP Address
$GSLBServiceStatus = $GSLBStatus.gslbservice.svrstate
} catch {
#Cannot find a GSLB Service with the same name as the valus passed in
$ErrException = $_.Exception.Response.StatusCode.value__
if ($ErrException -eq 404) {
cls
write-host -ForegroundColor Red "GSLB Service is not found, quitting"
break
}
}
if ($GSLBServiceStatus -ne "UP") {
$Url = "$NSIP/nitro/v1/config/gslbservice_lbmonitor_binding/" + $gslbserviceb + "?args=monitor_name:mon_force_down"
#Set the HTTP Method
$Method = "DELETE"
#Remove the force down monitor
Invoke-RestMethod -uri $Url -WebSession $nsSession.WebSession -Method $Method
cls
write-host -ForegroundColor Red "GSLB Service Force Down Monitor Removed"
}
ns-logout 192.168.0.201
} else {
If ($siteb -eq "no") {
#Login and enable the vserver passed into the function
ns-login $NSIP nsroot nsroot
#Check if the GSLB Service exists and is already disabled
$Url = "$NSIP/nitro/v1/config/gslbservice/$gslbserviceb"
$Method = "GET"
$ContentType = "application/json"
#Try to connect to the GSLB Service and Catch the error output
try {
$GSLBStatus = Invoke-RestMethod -uri $Url -WebSession $nsSession.WebSession -ContentType $ContentType -Method $Method
#Get the service status and IP Address
$GSLBServiceStatus = $GSLBStatus.gslbservice.svrstate
} catch {
#Cannot find a GSLB Service with the same name as the valus passed in
$ErrException = $_.Exception.Response.StatusCode.value__
if ($ErrException -eq 404) {
cls
write-host -ForegroundColor Red "GSLB Service is not found, quitting"
break
}
}
if ($GSLBServiceStatus -eq "UP") {
#Set teh NITRO URL
$Url = "$NSIP/nitro/v1/config/gslbservice_lbmonitor_binding"
#Set the HTTP Method
$Method = "PUT"
#Set the Request Header for Content Type
$ContentType = "application/json"
#Set up the JSON Payload to send to the netscaler
$PayLoad = ConvertTo-JSON @{
"gslbservice_lbmonitor_binding"=@{
"servicename"=$gslbserviceb;
"monitor_name"="mon_force_down";
"monstate"="enabled";
"weight"=1
}
}
Invoke-RestMethod -uri $Url -WebSession $nsSession.WebSession -ContentType $ContentType -Body $Payload -Method $Method
cls
write-host -ForegroundColor Red "GSLB Service Force Down Monitor Added"
}
ns-logout $NSIP
}
}
}
Pingback: Citrix NetScaler GSLB Management with Octoblu and NITRO