Thursday, June 19, 2008

Environment Building on ten cents a day ....

I have been giving project team members a hard time about advance notice.  You know what I mean, giving the Deployment Team more than 15 minutes notice that they need a deployment done.  Well, I'm not going to bug developers in this note, I'm heading higher up the food chain for the project managers.

I've talked about how we are short on space and short on power.  Apparently many project managers (and I am talking about a good dozen or more at the moment) think that these facts don't apply to them because they are special.  You know what?  Everyone is special.  When there are so many special projects, the appellation is meaningless.  So, let's talk turkey.

I want a performance test environment, and I need it next Monday.  OK, to be honest I haven't heard this .. too often.  Most projects give us a lot more warning, although sometimes there is not a lot I can do for them.  (Did you miss my notes on not having space and being electrically challenged?)  Let's take a look at a typical application and see what a performance test environment entails.  A smallish application that access Stakeholder Registry.

So, what do we need for this performance test environment?  Well, we could go with clustered web servers, but let's give ourselves some leeway and just pick one.  We then need a SQL Server box for SHR and a SQL Server box for the application database.  Wait, in Production these are SAN attached, should we duplicate that?  Well, that would mean buying another SAN and the storage and configuring it, etc., so for the purposes of our example let's not go there.  Oh, wait, in order for this to be even remotely representative we need to emulate the existing servers, not buy new ones.  So now we need Dell 6850s, but they're not made anymore.  Have no fear, eBay is here!!  Oh, darn, that reminds me, we need to get an Dell 1850 for the web server.  I hope we haven't gotten rid of all of them yet. 

OK, we done yet?  No?  What else do we need?  Well, if we are using SHR we need to hook in ISM, which requires another license for SSA-NAME3 as well as a Dell 2650.  (Remember eBay?  It's your friend.)  Here is an interesting fact for you:  If you restore the SHR database, you need to rebuild the database that SSA uses for searching.  This probably isn't more than a six hour job these days, but it might be longer.  I guess this means that if you run a test at 9:00 AM in the morning we might be able to rebuild everything for a 3:30 PM test.

I'm pretty sure that you're not going to be using the stopwatch facility on your Timex watch to record everything, so that means we need some tools for testing and recording.  Let's assume that we have these, but the tools we might need are Intercept Studio licenses (aka Avicode) to pinpoint some performance issues for us.  These are not necessary, but they have proven themselves to be quite valuable in the past.

There are some other infrastructure things like backup tapes, disk space for intermediate backups

So what do we have?  Four computers that need to be purchased, refurbished and installed in a computer room where we are already short on space and power.  Additional licenses for software we don't have.  A lot of effort on part of a lot of different people just to get things in place.  Just think what would happen if we needed PRS (Oracle server and license) or perhaps CGES (two servers and SharePoint) or, heaven forbid, all of those, plus more!!!!

Or, *shudder*, you have multiple projects coming to you at the same time with the identical requirements.  Welcome to my world.

Setting up a performance testing environment is very expensive in terms of hardware, software and sheer manpower needed to set it up and keep it running.  The question is, what are your really trying to test?

Don't get me wrong, I like the fact that you want to performance test.  I guess what we really need to understand is what you are trying to test.

No comments: