Monday, March 11, 2013

Netscaler Nitro API, surge queues and servicegroup members

we just upgraded to 9.3 61.5 , and nitro changed , citrix call it 'tidying up', but all I can say is that not making your API backwards compatible in a minor release is bad bad bad.

so, to get the service group members, their surge queue and other stats is now a multi step process

poll the config , get the lb names, and the service groups bound

  1. call /nitro/v1/config/lbvserver  to get a list of vserver names
  2. call /nitro/v1/config/lbvserver_servicegroupmember_binding/{lbservername} to get list of members
  3. call /nitro/v1/stat/servicegroupmember?args=servicegroupname:{servicegroupname},serverName:{ip},port:{port}"
suddenly a simple call is now N* bigger and more complex :-(

Mobile redirects and user agent device detection on a Netscaler

This is essential for integrated caching if you do redirects on apache/nginx based on mobile device etc.

bind policy patset User_Agent_Mobile Blackberry -index 260 -charset ASCII
bind policy patset User_Agent_Mobile iPod -index 200 -charset ASCII
bind policy patset User_Agent_Mobile iPhone -index 220 -charset ASCII
bind policy patset User_Agent_Mobile iPad -index 210 -charset ASCII
bind policy patset User_Agent_Mobile Android -index 250 -charset ASCII
bind policy patset User_Agent_Desktop Linux -index 100 -charset ASCII
bind policy patset User_Agent_Desktop Macintosh -index 120 -charset ASCII
bind policy patset User_Agent_Desktop Windows -index 110 -charset ASCII

add policy expression is_mobile_ua "HTTP.REQ.HEADER(\"User-Agent\").CONTAINS_INDEX(\"User_Agent_Mobile\").BETWEEN(200,299)"
add policy expression is_desktop_ua "HTTP.REQ.HEADER(\"User-Agent\").CONTAINS_INDEX(\"User_Agent\").BETWEEN(100,199)"

then you can do policies like

add cache selector mytest  HTTP.REQ.URL is_mobile_ua is_desktop_ua

which will store different versions of the content based on being a mobile device or desktop device.

GSLB on the netscaler

There are lots (and lots) of articles around GSLB, but none of them really worked for my brain.  I recently had to implement GSLB, to handle persistence of a java app between datacenters.

The Scenario:  we have an internet java app in a datacenter, it uses jsession id's to track sessions.  We wanted to do active-active between our two datacenters, and have a big fat pipe between them .

The solution:

  1. LB rules for jsession id persistence, like this
  2. GSLB for site persistence using connection proxy to make it travel our pipes between dc
The problems encountered.
  1. we use ultradns for managing dns load balancing and failover between our datacenters, this duplicates what gslb does, but we didn't want to open udp 53 to our dc.  this is slightly slower than using gslb and it has to wait for health checks and polling
  2. gslb is all about dns for failover, connection proxy is just for site persistence, we thought it would be smarter
  3. our testing showed if you hit DC2 , and the vip went down, GSLB kept sending you to the downed datacenter, as thats all it does.  it would withdraw dns for that site and make you fail that way. but as modern OS and web browser cache dns, this takes minutes
The solution to these problems
  1. we use backup vips in each dc, pointing to the opposite DC.  this is not a tidy config, but works well
  2. in dc1 for example, we have a main CS as a landing point for the app, then a second CS which is the landing point for dc2's backup LB config.
  3. to keep them in sync, we moved our policies to policy labels, and used the same label on each CS.
  4. in dc2 we created an LB which had a single destination, the CS in dc1.
  5. we had to be very careful here, jsessions don't play nice this way and you run a big risk of exposing session data.  as netscalers try to be smart, they try to re-use connections.  java isn't aware of this , and once the tcp stream is open it assumes all packets are the same session, so when customer 2 comes along, and hits the backup vip, this goes into the same tcp session as customer 1 , and same java session.  so you use max requests=1 and connection keep alive off.

How to sync your keys to your server farm / clusters

you need sshpass and ssh-copy-id , then do;

echo "StrictHostKeyChecking no" >> .ssh/config

echo "UserKnownHostsFile=/dev/null" >> .ssh/config

cat mylistofservers.txt | xargs -P10 -I {} sshpass -p 'mypassword' ssh-copy-id {}

Fire Alarms