Apr 23, 2012

How do you evaluate SaaS performance?

Like most companies, we have increasingly turned to the Cloud, both for storage and SaaS. Overall, this has been beneficial. Costs are generally down, and reliability is as good or better as it was prior to embracing the cloud. The one area that I have misgivings is SaaS. When everything works it is great, and to be fair, that is usually the case. But not always. In the past, I could use an application management system to keep track of performance and resources, and I could often head off problems before they became an issue. That's essentially out of my hands now. How can I monitor SaaS applications and evaluate performance now that everything is physically out of my hands?


The internet is a key issue when evaluating SaaS or cloud providers. RIght now ThousandEyes has the ability to monitor and disagnos the issues the this unknow can create. I know you posted this a few months ago, but I can send you a link for a free trial of the software.





Here's an article that provides five key things to evaluate a SaaS vendor. It might be useful if you apply these items in a performance review of a vendor you are currently using.

How to Evaluate SaaS Vendors: Five Key Considerations

1. Disaster Recovery
2. Security
3. Flexiblity and Quality of Service
4. SLAs
5. Global Reach

I use CA's APM Cloud Monitor, which is helpful.  But there is a monkey wrench thrown into evaluation of SaaS performance - the internet.  It is often challenging to determine whether my SaaS provider is the one responsible for an unresponsive application, or if it is "the internet".  Want to venture a guess at which one gets the blame every single time that I have contacted my provider?  If you guessed they said, "our bad, we'll take care of it," I have a bridge that I'd like to sell you.  They are probably right a lot of the time, but it isn't as if I can't ping/traceroute and get an idea of network performance.  It's a lot harder to measure performance inside of the provider's infrastructure, and let's be honest, if I was them I wouldn't let everyone past my firewall either.  Still, I think we have a little way to go to find the balance between provider security and user control/access. 

Answer this