When not to choose the default

I talked recently about how you should leave the defaults the way they are in many cases because, well, for most circumstances they are probably the best values to use.  Sometimes, however, the default doesn't work that well and you need to understand the reasons why changing the default is a good thing.

Suppose you went to a web site and it asked you to fill in 10 fields.  You then hit submit and it came back and told you that field number 1 is suppose to be numeric, not alphanumeric.  You make the change and then submit.  It then comes back and tells you that filed number 4 is suppose to be less than field number 3.  You keep doing this for a number of changes until you finally say "Forget it" and you leave that site forever.  It's happened to me, so I can honestly speak from experience.

My biggest problem with the process wasn't so much the one error message at a time, but rather the fact that there was a round trip to the server for every error.  I had over half a dozen interactions with the server to fill out a darn form!!!  By default .NET sets the controls you place on an ASP.NET page to process interactions at the server (runat="server").  If you provide complete error checking for each page, then this may be a suitable method of operating.  However, if you only respond to the user one error message at a time, this is sure fired way of getting  someone annoyed with you.  And quickly.

To be brutally honest, some error checking should be done at the client side.  If you have a popular application, or even if it is not that popular, there is still a certain amount of overhead involved in getting IIS to receive the request, process the header information, pass the information along to the application pool and then have the application do what it needs to do in order to tell you that the field is only supposed to contain numbers.  Then the whole thread has to go backward, towards the user, in order to give them the message.  It is faster, more efficient, and less costly from an infrastructure point of view if you let the client take care of this sort of data validation.  Your application will also check when it gets the data from the client (don't ever assume the client is sending you perfect data), but many checks can be performed at the client end, decreasing turn around time for error processing, distributing the work load, and, more importantly, providing a better user experience.

Remember, when you're designing your application think in terms of what provides the best user experience.  Think about your experiences, what you've liked or, more importantly, what you've disliked, and go from there.  The default, while usually good, does not have to remain if there is a good reason to change.

Validity vs. Reasonableness

While most of our applications do validity checks on data, not all of them do reasonableness checks.  Let me explain the difference.

Data Validation.  Let us suppose you have a number of fields on the screen:  name, address, birth date, phone number, and spouse's name, spouse's address, spouse's birth date, spouse's phone number and a marriage date.  Data validation would ensure that if there is a birth date, it is a valid date.  So in this case it would check to ensure that all of the date fields are valid.  It would also check to ensure that the phone number follows any one of a number of different standards, but predominantly the fact that it is numeric in nature.  You can also extend data validation to more complex tasks such as determining if the postal code is correct.   In general terms, data validation serves to ensure that a single piece of data is a valid for that data type.

Data Reasonableness.  OK, now that we've gotten the basics out of the way, there are still a number of checks that we can perform.  If there is a marriage date, then the date must be a certain time period after the birth date of both parties.  This is not just a simple "if marriageDate > spouseBirthDate then Happiness()".  We need some additional logic to ensure that even if the data is valid, it also must make sense.  Having data make sense is as important as ensuring that it is valid.

While there are many schools of thought on this, most post secondary training lumps both data validation and data reasonableness together under the "validation" banner.  This, unfortunately, has had the effect, in most cases, of putting data reasonableness checks in the background or has the checks embedded deep within the business logic of the application.  In most cases these checks can be done at the UI level, really quickly and prevent a lot of background processing that clogs the servers.  The other big problem is that some of these reasonableness checks are missed because "I thought the client was going to do that".

Just remember, there is a lot of data checking that needs to be done and reasonableness is just another item in the list.

Temporary (Expires yyyy/mm/dd)

My wife renewed her drivers license recently, as she turned 29, again, on September 24th.  She went into the Registry near our place, filled out the form, paid her money, got her picture taken and was given a "temporary" drivers license to last her until her real license was sent to her.  If the real license takes too long her temporary license is going to expire on her and that could cause no end of grief.

My bus takes a little bit longer to get to work in the morning as there is a "temporary" detour on it's normal route.  The city has dug a hole in the road, the entire width of the road, in order to do some emergency repair on the pipes.  They don't have a lot of time to get this done as it impacts a lot of traffic, a lot of buses (both ETS and school) and a lot of residents.  The only way to get to certain houses is to wind your way through back alleys.  Oh yeah, this is definitely going to be temporary.

My daughter has a temporary spacer in her mouth.  One of her baby teeth, incorrectly filled by an earlier dentist, developed some severe damage and needed to be pulled.  A temporary spacer was inserted so that her teeth would grow in their proper place until the adult tooth comes in.  The spacer is going to be removed either when the adult tooth comes in, or when the baby tooth to which it is attached falls out.  No choice, it is temporary.

We have servers, both physical and virtual, which were set up for temporary purposes.  We now have to face the task of upgrading them from NT 4 to something supported.  (OK, I lied, but you get the point, don't you?)  If things are temporary, give us a date and we will set up the system to self destruct on the day after.  If things aren't temporary, for goodness sake, tell us!!!!!  The "Oh, it was temporary, but the client said ..." story is getting old and, quite frankly, has been done better by other teams. 

Remember temporary means that it goes away.  Pick a date.  Any date.  Please....

Weapons of Mass Destruction

America went to war with Iraq because they wanted to find and destroy the Weapons of Mass Destruction.

In many respects that what I do by looking at the way applications are installed, operate and behave when encountering errors:  I'm looking for weapons of mass destruction.

My first IT job was with a construction company and my crowning achievement was an application that accurately allocated work site costs to various job codes in the accounting system on a daily basis.  It was rather tricky due to the fact that the company I worked for was a multinational company and each country, indeed province/state, had different holidays so it needed to take into account when costs should be allocated based on whether or not the previous day had been a holiday or not, in the location where the construction site was located.  Jobs on which 7x24 construction was occurring had other conditions that needed to be met.  All in all it was a masterpiece of software engineering.

Almost.

You see, I was under a tight timeframe for getting this done, as the VP in charge of construction had told the CEO that it would be in place by July 1st.  I only had 4 more weeks to finish the coding and then implement the application in the main production batch jobs.  Time was tight so I did what every rookie (and many seasoned professionals) do when faced with something that is difficult to compute:  I hard coded the answers.  I hard coded the holidays for all jobs sites for the next 18 months inside the application. 

It was simple to do and saved me a lot of trouble, because, you see, I left the company 2 months later, leaving behind a ticking time bomb in their production systems.  In sixteen months things were going to blow up, all because I took the easy way out instead of doing it properly.

Tick, tick, tick, tick ...

Groceries and Software

When you buy groceries from a grocery store, one of the things that they do for you is pack your groceries into plastic bags so you can take them home.  (OK, some grocery stores don't do that, but they will charge you a couple of pennies to give you a plastic bag so that you can do it yourself.)  When they pack the plastic bag they have certain, spoken and unspoken, rules.  For instance, you don't put a pound ... err, kilo... of hamburger in the same bag as fruits and veggies unless one of them is wrapped in an additional plastic bag.  You don't put a bag of potato chips at the bottom of the bag and potatoes on top of them and eggs are packed flat.

When creating a deployment for DeCo, think of ZIP files as being the similar to plastic bags from the grocery store.  Here are a few simple rules that you can follow to fill those bags:

• Create a ZIP file for each item type that you are migrating.  For instance, if you are migrating a web components and COM+ components, create two zip files, one for each.
• In each ZIP file put all of the pieces necessary to do the work for that type of deployment.  If it is a web zip file, include the MSI, any config files and even the documentation. 

The rules are really simple, but make life much easier for everyone.  For instance, by using ZIP files instead of a list of separate files, you reduce the number of times you need to attach a file to the request and you reduce the amount of space needed on the back end to store the files.  We current have over 9760 files that we are tracking from over 6800 deployments and these files take up over 19,680,000,000 bytes.  While some of those files are compressed, not all of them are. 

By using ZIP files you can help us keep a handle on the storage requirements for DeCo as well as making it easier for the Deployment Analyst or DBA to get all of the files they need for the deployment in one simple package.

Schopenhauer's Law of Entropy

So, just what is Schopenhauer's Law of Entropy?  Simply put, it is this:

If you put a spoonful of sewage in a barrel full of wine, you get sewage

So, what does sewage have to do with programming?  It's not sewage that I'm looking at, but rather the concept behind it.  In IT terms, what Schopenhauer is saying is that no matter how good the overall application, if one part doesn't work the whole application gets tarred with the bad brush. 

It is unfortunate that a single poorly designed, written or executed page can make someone believe that the entire application is poor.  Their perception of the application is what is important, not reality.  Kind of scary, isn't it, when perceptions are more important than reality.  But this is what happens in our business and it is something that we need to understand and do our best to influence.

So what influences this perception?  Well, consider this:  two web applications side by side on your desktop.  You push a button on the left one and you get the ASP.NET error page:  unfriendly, cryptic and somewhat unnerving.  You push a button on the right one and you get an error message in English, that explains there is a problem and that steps are being taken to resolve the issue.  Which one would you perceive to be better written and robust? 

How about another example?  You push a button on the left application and you get an error message that says "Unexpected Error.  Press the OK button".  You push a button on the right application and you get an error message that says "Our search engine is currently experiencing some difficulties and is offline.  Please try again later."  Which one do you perceive to be better?  Which one do you think your business clients will think is better?

It's not just one thing (error message or not) that gives you a feeling of confidence when dealing with an application, it is a multitude of little things.  Making things more personalized helps.  Translating from Geek ("Concurrency error") to English ("Someone else has updated the data before you") helps a lot.  Making it seem that you spent some effort to foolproof the system (i.e. don't make every error number in your application the same error number).

No matter how good the rest of your application, one bad move can create sewage.

Initialize All Variables at Their Points of Declaration

I was reading a book recently called Code Craft - The Practice of Writing Excellent Code and one of the comments struck a particular chord with me as it brought back memories of an upgrade that went horribly wrong.  At least for me.

There was a brief section called "Initialize All Variables at Their Points of Declaration".  Now, this may seem self explanatory and quite normal to some people, but others think that this is rather strange.  "Why would I initialize a variable that I may never use?"  The problem is, that not everyone follows the same coding practices in real life.  Sometimes the compilers help/hurt us in this regard.  Back when I was predominantly working on the mainframe, we were switching from an older version of COBOL to COBOL II.  Ooh.  COBOL.  I can see your eyes glazing over.  Stay with me, there is method to my madness.

The process of conversion was really quite simple.  Recompile.  It wasn't that hard.  However, we discovered a little bit of a problem.  When we did our testing we discovered that we were occasionally getting OC7 (data exception) errors when everything should have been working.  Indeed, running the program multiple times against the same data actually generated different results.  After a lot of head scratching we determined that the problem lay in the fact that the old compiler, by default, initialized variables when they were defined.  COBOL II did not do this by default.  When the application was loaded into memory it would occasionally access memory that had been initialized for some other purpose and the program would work.  Other times, however, it was accessing "garbage" and the program would blow up.  If the original developer had initialized the variables in the fist place we never would have had a problem.

So, we made a small change and everything was perfect.

Almost.  Because of how we were doing the upgrade process I had to baby sit the recompilation of 1900 COBOL programs in 4 different environments (7600 recompiles altogether).  Took almost 48 hours to do it and I got almost no sleep, and all because someone failed to initialize a couple of variables.

Antacids for the PM

PMs, can you imagine the look on the faces of those developers when I said that they needed to supply you with things like estimates and time sheets?  I bet some of them were knocked off their feet!

I mean it's not like you were asking them for anything difficult.  I mean, how hard can it be to create an estimate for a project?  It's not like it's rocket science.  Every developer should be able to do it.  I mean, the National Gun Registry hit it's target.  Well, maybe it didn't.  Well, what about the Denver luggage handling system?  Another fiasco?  OK, the FBIs Trilogy Project, now that was ... a disaster?

Everybody can look into the past and come up with failed estimate, but sometimes, the Project Manager does it to themselves.  This may be a shock for some of you, but sometimes Project Managers are under different pressures than you realize.  Back when I was younger I worked for a consulting company.  My job was to come up with the technical work plan and estimates for the projects the local office undertook.  I would then, using historical data and some darn fine guessing, come up with what the effort would be on the technical staff for the project (technical staff meaning project DBA and internal technical support). 

One of my proudest, and saddest, estimating moments came for a relatively simple business project that had a number of interesting technical complexities.  My estimate of the technical cost was 179 days.  When combined with the other parts of the project it turned out that this simple application was going to cost the client a lot of money.  In an effort to reduce the impact tot he client the scope was shuffled, but this did not reduce the technical effort that needed to be spent, so in a classic PM moment the number of days was arbitrarily reduced to 79.  I am proud/sad to say that this is one of the few times in my life where I was dead on with regard to the effort.

Project Managers, if your team says that something is going to take a certain amount of time, ask them questions, make sure they understand both the problem and their own estimate, but don't arbitrarily change it unless you know for a fact it can be done for less.  The odds are they understand what needs to be done better than you and by changing their estimate you are telling them that you don't trust them.  If you still feel the estimate is high, have them sit down with you and walk you through the estimating process they used.  But, if the numbers still add up to something you don't like, take a tums.

Load Balancing Failures

It shouldn't come as a surprise to anyone when I say that our Production environment is load balanced.  I have mentioned this before and I will be mentioning it again in the future.  But, for those who may have missed my previous tirades, let me explain the impact of load balancing on Session State.

One of the features of ASP.NET is to store information specific to a browser session (aka user) into something called Session State.  Session State is kind of like the junk drawer you have at home where you have batteries, twist ties, plastic spoons, stud finders and assorted other "stuff".  Session State allows you store what you need to store in order to keep track of where the user is in the application and what data you need to save on their behalf.  The next time the user accesses the application the session state is automatically loaded and you're ready to rock.

There are a number of places to store session state:  In Process, Out of Process or in SQL Server. 

In Process means that Session State is going to be stored in the Application Pool running the web site.  So, if the application pool recycles, all session state is going to be lost.  A number of projects currently use this method and are in danger of losing Session State, if they use Session State, as we use Application Pool recycling to solve a number of application issues.  In addition, if, for some reason, BigIP sends the user to a different web server to service the request then the Session State is not going to be present, potentially causing a number of application failures to occur

Out of Process is where the session state is hosted by ASP.NET in a different process, potentially on a different machine.  While somewhat safer than storing it in the same Application Pool, a problem arises if this service needs to be reset whereby the Session State is again lost.  Indeed, if the process is hosted on the same server as the web site, moving the request to another part of the load balanced cluster is going to be a problem as Session State will not be available for the request.  If Session State is stored on a separate machine then the biggest problem is that of durability of the data.  Any problems with the service may wipe out all session state for all machines.

Storing Session State in SQL Server is the slowest method, but is by far the safest method for durability and the best method when utilized in a cluster.  Each request for Session State goes out to SQL Server to ensure that the latest and greatest version of Session State for that user is retrieved and used.

In our environment we have asked people to use SQL Server Session State, and yet, by looking trough the web.config files of a number of projects I've noticed that they have their Session State set to In Process.  If Session State is actively being used, this is a recipe for disaster.  I urge each project team to take a quick look at their web.config files and change it to use SQL Server instead of In Process.  Even if you don't currently use Session State, you may in the future and this will prevent you from having a nasty accident.

In Memoriam

There were a number of comments yesterday from people wondering what the heck I was talking about when I told everyone to take a break.

Recently, Wednesday to be precise, we lost a companion that we had known for more than thirteen years.  When my wife and I first moved into our house we found it to be a large, lonely place.  In order to fill up some of the space we went to the SPCA and picked up a pair of cats, a brother and sister pair.  We named him Spike, because of his spiky hair and we named her Willow.

For thirteen years they were our companions.  Through three kids, hundreds of hair balls conveniently coughed up in the middle of the path to the bathroom, tainted cat food scandals (yes we found a few cans), Spike and Willow were there.  Recently Willow had been having some trouble with her hips.  Old age seemed to setting in quite quickly for her and the doctor recommended a special diet for her kidneys and glucosamine for her joints.  She seemed good, for her, for almost two years, but swiftly went down hill about 10 days ago.  A visit to the vet confirmed the worst:  kidney failure, liver disease, and a host of other problems were manifesting themselves at the same time.  When asked how many months she had, the vet told us "one week".

We made the most of the week with Willow and spent a lot of time petting her and keeping her company, much like she had kept us company for thirteen years.  We soon saw, however, that her time had come to let her go.  The whole family had a good cry on Tuesday night.  Wednesday, a friend of the family helped my wife with the final details and we all had another good cry that night.

I enjoy my work.  I enjoy the people the people I work with, even the ones I yell at a lot.  But I also enjoy other things.  My work does not define who I am, however, just what I do for a living.  Sometimes you need to take a break, step back from the work and look at everything around you.  If you've spent so much time working that you can't remember the last time you truly relaxed, take a break.  If you can't remember the last time you hugged someone or something close to you, take a break. 

Enjoy life.

Take a Break

Today's message is short and simple. Tomorrow we'll explain why.

No matter who you are. No matter what you do. Sometimes you need to stop working and enjoy life.

Take a break from work. Take a break from education. Take a break from stress. Just take a break.

Really Simple Solution

Really Simple Syndication.  RSS.

The concept behind RSS is really quite simple:  users, on their own timetable, download an XML file that contains headlines and/or stories on a particular topic.  For instance, I subscribe to a RSS feed that is created based on the Blog of J.D. Meier.  He is the Project Manager for "security and performance on the patterns & practices team".  His blog contains a variety of interesting topics, but I never actually have to visit his web site to get the latest from him.  I have an RSS reader (in this case Outlook 2007, but there are hundreds of others) that periodically goes out and checks his RSS feed to see if there are any changes.  If there is new stuff it shows up in Outlook.  I subscribe to a variety of blogs and web sites this way:

• Electronic Freedom Foundation
• Mark Russinovich's technical blog covering topics such as Windows troubleshooting, technologies and security.
• MSDN: Canada featured articles
• Scott Berkun, author of The Art of Project Management has a feed talking about innovation and creativity.

While RSS feeds are not new, their use in business environments is relatively new.  There are some businesses that have an RSS feed per major application and they post outages, tips, tricks and, most importantly, changes that are or have occurred to the application.  This way their business area is not surprised Tuesday morning when a completely new version of their favorite application shows up.  Larger project teams can create an RSS feed that contains status reports, meeting updates or even updates on when the implementation party is occurring.

RSS feeds can solve a number of issues, but it is not a hammer that can be used on every nail.  It has specific benefits in specific situations.  Like everything else, it is a tool that you have in your toolbox, but it is not necessarily a tool that you need to use.