Some things should not be "added on"

When building an application there are some things that can be added on afterwards:  new functionality, better graphics and friendlier messages.  These are all things that add more value to the application.

There are some things, however, that should not be added on afterwards:

• Error Handling.  What?  Don't add on error handling afterwards?  No.  It needs to be done at the start.  Now, I know what some of you are saying "But, Don, we'll add this in if we have a problem."  Face it, every new application has problems and if you don't have error handling in at the beginning you are spending needless cycles trying to debug the application and you are causing me to drink Pepto-Bismol as if it were Dr. Pepper.  We recently had to help a project debug their application in the UAT environment, but they had no error handling at all, except the default ASP.NET error handling.  Thank goodness for Avicode, as it helped us pinpoint the problem quickly, just far too late in the development cycle.
• Object Cleanup.  If you create an object, kill the object.  It's simple.  It's so simple that even Project Managers can do it.  By not cleaning up after yourself you raise the potential for memory leaks to happen.  And you know what that means?  Alka-Seltzer, Petpo's cousin. I can't tell you the number of applications in which we recycle the component once it hits a certain limit because it would keep you awake at night.  (Lunesta, another cousin)  Suffice to say that many of our applications are forced to recycle after a certain period of time or when they exceed a certain size. 

The scary thing is that both of these items are considered  best practices for writing code in the first place.  I know the excuses "...the project manager won't let me do this ..." or "...I don't have enough budget to do this ..." or, the one heard most frequently "... I don't have the time to do this ...:.  Very few project managers tell their staff how to code, so the first excuse is just a cop out.  As for the budget, doing these items does not add significantly to the cost of application as it usually makes debugging faster and easier, so the budget excuse is just that, an excuse.  As for the time, if you're short on time, you need to do this as it will help you.

One of the things that many Health Organizations are putting in place is prevention of disease so that there is no need to cure the disease.  Prevention is much more cost effective.  Object Cleanup is prevention, pure and simple.  When someone has symptoms that need to be diagnosed, what does the doctor do?  Perform a seance?  Guess?  Or do they use a tool to help them out?  Ever heard of an MRI or even an X-Ray?  Think of Error Handling as a tool to help you diagnose the disease faster.  It's better than guessing.

So, object cleanup prevents problems and error handling helps diagnose problems.  So, I guess this means that I'll be seeing more applications with these items as an integral part of the overall application or do I need to go back to the medicine cabinet?

Don't Worry About Performance!

There was a statement made a number of years ago in an ISWG (Integration Services Working Group) meeting which, to summarize, said "Don't worry about performance, we'll take care of that".  While that is probably going to be the epitaph of at least one person, I think it is time to set the record straight.

Worry about performance.

OK, now that we've gone 180 degrees it's time to put some parameters around this.

• Don't worry about things over which you have no control.  The speed of a DCOM call is something you have no control over, neither is the time required to create a connection for a connection pool, the time required to retrieve session state, nor the time required to process an HTTP request.
• Do worry about things over which you do have control.  While you can't do anything about the speed of a DCOM call, you can control, to an extent, the number of DCOM calls that you make.  Less chattiness is better.  While you do not have control over the speed of the resources, you have control over how effectively you use those resources.

The UAT and Production environments into which your application will eventually move has a myriad of interconnected pieces that, together, create the environment in which your application will exist.  While you cannot control those resources you control how you use them.  <Al Gore>  Ineffective use of resources and our unwavering dependency on technologies that produce greenhouse gases is threatening our livelihood and our very planet.</Al Gore>  Ineffective use of resources in any ecosystem is bad, whether that ecosystem is our planet, or our clustered production environment.  Infinite resources are not available and never will be, but effective use can be made of existing technology:

• Up until the late 1990's the computers on board the Space Shuttle flight deck were Intel 8086 chips.
• The original Xbox had a 733MHz CPU with 64MB of memory and yet it could (still can) outperform peoples desktops.
• Mission critical systems have been designed with, when compared to modern technology, primitive means.
• The first spreadsheet I used, Visicalc, ran on a 1 MHz processor.

All of these examples show that you can make something run well in limited circumstance, but you have to want to.

Validity vs. Reasonableness

While most of our applications do validity checks on data, not all of them do reasonableness checks.  Let me explain the difference.

Data Validation.  Let us suppose you have a number of fields on the screen:  name, address, birth date, phone number, and spouse's name, spouse's address, spouse's birth date, spouse's phone number and a marriage date.  Data validation would ensure that if there is a birth date, it is a valid date.  So in this case it would check to ensure that all of the date fields are valid.  It would also check to ensure that the phone number follows any one of a number of different standards, but predominantly the fact that it is numeric in nature.  You can also extend data validation to more complex tasks such as determining if the postal code is correct.   In general terms, data validation serves to ensure that a single piece of data is a valid for that data type.

Data Reasonableness.  OK, now that we've gotten the basics out of the way, there are still a number of checks that we can perform.  If there is a marriage date, then the date must be a certain time period after the birth date of both parties.  This is not just a simple "if marriageDate > spouseBirthDate then Happiness()".  We need some additional logic to ensure that even if the data is valid, it also must make sense.  Having data make sense is as important as ensuring that it is valid.

While there are many schools of thought on this, most post secondary training lumps both data validation and data reasonableness together under the "validation" banner.  This, unfortunately, has had the effect, in most cases, of putting data reasonableness checks in the background or has the checks embedded deep within the business logic of the application.  In most cases these checks can be done at the UI level, really quickly and prevent a lot of background processing that clogs the servers.  The other big problem is that some of these reasonableness checks are missed because "I thought the client was going to do that".

Just remember, there is a lot of data checking that needs to be done and reasonableness is just another item in the list.

Weapons of Mass Destruction

America went to war with Iraq because they wanted to find and destroy the Weapons of Mass Destruction.

In many respects that what I do by looking at the way applications are installed, operate and behave when encountering errors:  I'm looking for weapons of mass destruction.

My first IT job was with a construction company and my crowning achievement was an application that accurately allocated work site costs to various job codes in the accounting system on a daily basis.  It was rather tricky due to the fact that the company I worked for was a multinational company and each country, indeed province/state, had different holidays so it needed to take into account when costs should be allocated based on whether or not the previous day had been a holiday or not, in the location where the construction site was located.  Jobs on which 7x24 construction was occurring had other conditions that needed to be met.  All in all it was a masterpiece of software engineering.

Almost.

You see, I was under a tight timeframe for getting this done, as the VP in charge of construction had told the CEO that it would be in place by July 1st.  I only had 4 more weeks to finish the coding and then implement the application in the main production batch jobs.  Time was tight so I did what every rookie (and many seasoned professionals) do when faced with something that is difficult to compute:  I hard coded the answers.  I hard coded the holidays for all jobs sites for the next 18 months inside the application. 

It was simple to do and saved me a lot of trouble, because, you see, I left the company 2 months later, leaving behind a ticking time bomb in their production systems.  In sixteen months things were going to blow up, all because I took the easy way out instead of doing it properly.

Tick, tick, tick, tick ...

Schopenhauer's Law of Entropy

So, just what is Schopenhauer's Law of Entropy?  Simply put, it is this:

If you put a spoonful of sewage in a barrel full of wine, you get sewage

So, what does sewage have to do with programming?  It's not sewage that I'm looking at, but rather the concept behind it.  In IT terms, what Schopenhauer is saying is that no matter how good the overall application, if one part doesn't work the whole application gets tarred with the bad brush. 

It is unfortunate that a single poorly designed, written or executed page can make someone believe that the entire application is poor.  Their perception of the application is what is important, not reality.  Kind of scary, isn't it, when perceptions are more important than reality.  But this is what happens in our business and it is something that we need to understand and do our best to influence.

So what influences this perception?  Well, consider this:  two web applications side by side on your desktop.  You push a button on the left one and you get the ASP.NET error page:  unfriendly, cryptic and somewhat unnerving.  You push a button on the right one and you get an error message in English, that explains there is a problem and that steps are being taken to resolve the issue.  Which one would you perceive to be better written and robust? 

How about another example?  You push a button on the left application and you get an error message that says "Unexpected Error.  Press the OK button".  You push a button on the right application and you get an error message that says "Our search engine is currently experiencing some difficulties and is offline.  Please try again later."  Which one do you perceive to be better?  Which one do you think your business clients will think is better?

It's not just one thing (error message or not) that gives you a feeling of confidence when dealing with an application, it is a multitude of little things.  Making things more personalized helps.  Translating from Geek ("Concurrency error") to English ("Someone else has updated the data before you") helps a lot.  Making it seem that you spent some effort to foolproof the system (i.e. don't make every error number in your application the same error number).

No matter how good the rest of your application, one bad move can create sewage.

Initialize All Variables at Their Points of Declaration

I was reading a book recently called Code Craft - The Practice of Writing Excellent Code and one of the comments struck a particular chord with me as it brought back memories of an upgrade that went horribly wrong.  At least for me.

There was a brief section called "Initialize All Variables at Their Points of Declaration".  Now, this may seem self explanatory and quite normal to some people, but others think that this is rather strange.  "Why would I initialize a variable that I may never use?"  The problem is, that not everyone follows the same coding practices in real life.  Sometimes the compilers help/hurt us in this regard.  Back when I was predominantly working on the mainframe, we were switching from an older version of COBOL to COBOL II.  Ooh.  COBOL.  I can see your eyes glazing over.  Stay with me, there is method to my madness.

The process of conversion was really quite simple.  Recompile.  It wasn't that hard.  However, we discovered a little bit of a problem.  When we did our testing we discovered that we were occasionally getting OC7 (data exception) errors when everything should have been working.  Indeed, running the program multiple times against the same data actually generated different results.  After a lot of head scratching we determined that the problem lay in the fact that the old compiler, by default, initialized variables when they were defined.  COBOL II did not do this by default.  When the application was loaded into memory it would occasionally access memory that had been initialized for some other purpose and the program would work.  Other times, however, it was accessing "garbage" and the program would blow up.  If the original developer had initialized the variables in the fist place we never would have had a problem.

So, we made a small change and everything was perfect.

Almost.  Because of how we were doing the upgrade process I had to baby sit the recompilation of 1900 COBOL programs in 4 different environments (7600 recompiles altogether).  Took almost 48 hours to do it and I got almost no sleep, and all because someone failed to initialize a couple of variables.

Solving the Right Problem

One of the hardest things to do it solve the right problem at the right time. 

When investigating a problem you may end up looking at a wide variety of possible solutions.  Some of these solutions are quick fixes while others require a fair amount of effort to implement.  The question is, which one do you propose?

For a crisis, the quick fix is usually the right choice.  Things need to be resolved quickly and the best solution may not be able to solve the problem fast enough.  As a result the quick fix is usually chosen for Production emergencies and rushed through into Production.  Quick fixes are not meant to be permanent solutions, but in many cases they end up being permanent for a variety of reasons.

In less crisis oriented situations, however, the best solution may actually be the resolution of a deeper, more convoluted problem that is actually the root cause of the issue.  Unfortunately, resolving the root cause of a problem may actually be a problem in and of itself.  There may be significant effort and money that needs to be spent in order to resolve the issue in the manner that it should.  Sometimes the problem is so fundamental to the application that it almost appears that you have to re-write the application to make it work as desired.  If this is the case, is this what you should propose?

As with many things in life, it comes down to a business case:  is the cost of implementing the solution less than the cost of living with the quick fix?  If this were strictly a matter of dollars and cents then the answer would be know right away.  Unfortunately the cost of living with the problem is not easily quantifiable.  How do you measure consumer lack of confidence in terms of cost?  How do you measure consumer satisfaction in terms of cost?  in many cases only the business area affected can even hope to determine the cost.  It is our job to present the facts as we know them, the costs as we know them, and let the business decide the ultimate cost.

Trust

One of the biggest things we need to deal with in applications is trust in other parts of the system.  How much trust do you have in the calling application?  How much trust do you have in the common routines that you call?

For instance, you've created a method that will take an XML stream and use it to update a row in a table.  How much faith do you have that the calling application has sent you the proper types?  Should you check everything to ensure that strings are strings and numbers are numbers?  Do you double check to ensure that dates are actually valid dates and that the time listed actually exists?

I used to work for a manager that insisted that every time your method gets invoked it should double check all of the data being passed in before it did any work: verification was the first thing you did.  Being young and full of myself, I didn't follow that rule because, well, to be honest, I was writing both sides and I knew what I was passing myself!!!  Fast forward a couple of years and someone else is maintaining the code.  Well, they made some changes that didn't follow the rules and, in production, it blew up horribly because the method did not verify that the correct type of data was being passed.  Being on the support side I was called in to troubleshoot and instantly recognized what the problem was and the solution that was required.  A quick compile and test and the application no longer died horribly, but gave a nice, easy to understand error message.

With today's modern languages much of this work is taken care of for you by the development tool during design time as you need to ensure that you are calling with the correct types, or the the compiler won't even compile your application for you.  However, there is a problem when you are using XML or if you are taking in a string and attempting to use it as a numeric value.  This is of particular concern to user interfaces as pretty much everything you retrieve from the UI is a string that you need to convert and use.

A user interface should place no trust in the end user entering in the correct type of data into the text box.  But, how much trust do you place in one piece of code calling another piece of code?  I guess that depends on whether or not you are going to be the person maintaining the code for the lifetime of the application.  If you are, then I guess you can trust the code.  If you aren't then being paranoid may be beneficial.

Batch Processing ... Part Two

I must be honest, I am sometimes quite surprised by the reaction my notes some times invoke.  For instance, the note about batch processing generated a number of "high fives" in the hallway and a couple of 500 word responses.  The response was, as I expected, all over the place with some people telling me I was not in my right mind while other people were saying that this is what they have always believed.  Never one to let the flames of disagreement burn out, I thought I would list my personal rules as to whether or not something should be done in batch.  This is not something you need to follow, but it may help illuminate my comments from the other day.

Interfaces with other systems.  I originally had the word "external" but I replaced it with "other" as different systems internal to an overall application may not be able to handle a continuous stream of data.  For instance, our interface to IMAGIS is done through a file transfer which is done once a day.  To ensure that we get as much as possible into the system we do the transfer close to the cut off time that we have arranged with the IMAGIS team.  In this case the target system is just not designed to handle us sending them information multiple times per day.  Now, it may be the case that there are other ways to communicate with IMAGIS that we have not utilized, but with the current set up, we need to do it on a scheduled, batch basis.

Reports. This seems like a logical item to do in a batch process.  Whether this batch process is done via another tool, such as ReportNet scheduling the report, or whether it is scheduled via Windows Scheduler, reports are good batch residents. However, in my mind a report does not process and create data, it merely reports on the data.  If your "report" creates and stores data then that portion should be separated out and done in an asynchronous manner.  A report, any report, should be able to be generated quickly from data that is already stored in the database.  In addition, in many cases, reports do not need to be run on a scheduled basis, as long as the report can be generated on demand and that it will contain the identical content as if it had been generated on a previous day.  For instance, if reporting on the applications that were approved on July 12th, it would list all approvals, even if one of the application was subsequently denied.

And that's pretty much it.  It's a very short list of things that need to be done in a batch window.  For me it all comes down to this:

It is our job as IT Professionals, however, to not just do what our clients say, but to educate them as to what can be done, to show them new opportunities, and to give them something better than what they had, not just something newer.

Clusters

Clustered servers.  One of the most important reason for having clustered servers is so that in the event that one server fails the other servers will pick up the load and take over for the failed server.  However, there are some things you need to know before everyone goes rushing off to say "Clusters will solve all of our problems."

Applications must be built to function in a cluster.  So, what does this mean?  It means that if you are going to store a file being exchanges with the client, the file needs to be stored in a location that all servers can access.  It means that if session state is stored it needs to use SQL Server as the storage mechanism as this is used across all servers.  Essentially, nothing should be stored on the local machine.  Nothing.

Connections are "sticky", but don't depend on it.  By default the load balancer will try to send the user back to the same server that they were on previously.  However, in the event that the server is out of the cluster or there is a particularly heavy peak load on the server, the load balancer may send the user to another server.  This is expected and desired from a load balancing perspective, so your application better not rely on the user always going back to the same server.

Clusters do not protect against soft failures.  If a server dies, blows up, is incinerated by aliens from a distant galaxy, is cryogenically frozen in a block of nitrogen, or has it's network connection severed, the load balancing software will automatically move users to one of the other servers in the cluster.  However, in the event that the error is a soft error, an error in which the application responds, but does so incorrectly, the load balancer does not know that there is a problem and will continue to send users to that server.  Indeed, if the error is pervasive and actually causes all pages to fail, and fail quickly, the load balancer may be confused to the point where it thinks that the server is not under a large load and that it can handle more connections.

If you keep these things in mind you'll understand that while clusters do help us out a lot, there is some work to ensuring that they can help and that the help is what we are expecting.  Overall, however, the benefits of a clustered environment far outweigh the disadvantages.

Flow Control

Computer languages contain a number of things generically called "flow control".  These construct (words) are what allow a programmer to change the course of the application.  They contain things such as:

• if ... then


• do ... while


• while ... wend


• plenty of other examples

The one thing they have in common is that they have the potential to do something based on a condition.  But you know what?  Every time you execute one of these flow control statements you have the potential of slowing your application down.  You also have something else you need to check.  In addition, if you've chained a number of these statements in a row you must ensure that you are checking them in the proper order.

Essentially, what it comes down to is trying to avoid these statements in the first place.  If you want to get carried away, there are some object oriented purists who believe that you should be able to write your entire application without the use of an if ... then statement.  (My personal belief is that while this is an interesting thought, if ... then can save a lot of extra coding.)

This is also true of any business processes that may be implemented by the project.  Avoiding the use of flow controls makes the process easier to use, easier to follow and easier to understand.  So, whether it is an application or a business process, minimize the use of conditional statements and you'll find that things go faster, smoother and with fewer problems.

Hot Fixes

What is a hot fix?  This question seems to be coming up  more often and I think it needs a bit of discussion in this arena.  Definitions of hot fix that I have seen include:

• A hotfix is code (sometimes called a patch) that fixes a bug in a product. (Source)


• Microsoft's term for a bug fix, which is accomplished by replacing one or more existing files (typically DLLs) in the operating system or application with revised versions. (Source)

I think we can all agree that a hot fix is something that fixes a bug.  The question now arises as to the size of the patch.  The second definition is important in this aspect as it talks about replacing one or more DLLs.  So, a hot fix will fix a bug by replacing an indeterminate number of DLLs.  Darn it, I've used that word again: replacing.  That happens to be the crux of the problem that we are experiencing.

Replacing DLLs does not mean the uninstaling of the entire application and the installation of a new version of the application which has the bug fix inside.  This is simply an install of the application.  A hot fix would take the DLLs that were changed, package those up and install those on affected machines.  This is standard practice used by Microsoft, IBM, Sun, Oracle, Hewlett Packard, PeopleSoft, SAP, Symantec, Trend Micro, Adobe, Electronic Arts, Intuit, AutoDesk Check Point, and, quite literally, millions of other companies.  You don't re-install Windows every time there is a hot fix for Windows.  You don't re-install your anti-virus software every time there is an update to the software.  You don't re-install your entire application because there is a spelling mistake on a page.

If you are asking for a migration to a Shared Environment, and you are essentially asking us to install a new version of the application, don't call it a hot fix, as you are disagreeing with the vast majority of the IT world and the definition that the Deployment Team uses for a hot fix.  A hot fix replaces DLLs.  By packing everything up into a new install for the application you are potentially including other changes in your fix that are not related to the bug you are trying to install.

Deadlocks ... Part 2

So, how do we prevent deadlocks?  Well, there are a number of different ways to go about doing it, so we'll only concentrate on a couple.

Small.  This may seem obvious, but it is one of the most overlooked items in the book.  The smaller the transaction (the fewer rows that have been updated, inserted or deleted) the less likely you are to trample on someone else.  The fewer locks you hold the better off you'll be.  One of the biggest problems in this regard is applications that update a row, even when it doesn't need to be updated.  The application follows a specific path and part of that path says that Row X in Table Y needs to have certain values.  The row is not checked to see if it has those values, it is just automatically updated.  Well, that placed a lock on that row even if the values remained the same.

Fast.  This may also seem obvious, but the shorter the amount of time you hold those locks the less likely you are to have a problem with deadlocks.  One of the classic problems here is that the application does something to the database right away, say updating a table indicating where the user is at in a process, and then does a lot of calculations or a lot of work and then does more updating at the end, interspersed with a smattering of updates / inserts / deletes.  The locks are held from the first update to the final commit and as that length of time increases, the more likely it is that deadlocks will occur.

Application Developer Guide.  OK, this one is a little esoteric, so let me explain.  One of the ways that deadlocks can happen is if one part of the application updates tables in this order - Table A, Table B, Table C - while another part of the application updates tables in the reverse order.  By explaining the order in which tables should be updated in the application's developer guide everyone on the project will know the order that they should be doing things.  Changing the order should be something that is discussed with the DBA prior to being implemented as changing the order may impact more than just deadlocks.

As usual, there are dozens of other things to take into account, but these are some of the items with the biggest payback.

Deadlocks ... Part 1

Deadlocks are an interesting condition within the database.  In very simplistic terms, it occurs when two people are both trying to change data that the other person has already changed.  For instance, Person A has already updated Row 1 in Table 1 and now wants to update Row 2 in Table 2.  However, Person B has already updated Row 2 in Table 2 and is trying to update Row 1 in Table 1.  As you can see, unless someone gives in they could sit there all day.  The database is the arbiter in this case and makes a decision as to which person is going to get the error message when their transaction is canceled.

Now, you may have heard the phrase "Deadlocks are a natural part of application processing and are not a large concern."  While I do not advocate violence, the person who says this should be slapped in order to knock some sense into them.  Deadlocks are not a natural par tof an application.  I previously worked on a web-based system that had, as regularly peaked at over 650 simultaneous users, with over 250 of those being "hard core" users. This is 10 times the size of any application we have currently running.  If we got a single deadlock during the day we had to investigate why the deadlock occurred and determine if the deadlock could be prevented.  We would go for weeks, or even months without a deadlock, but when one occurred we dropped everything and investigated the problem.

Why do we get deadlocks in an application?  Sometimes it is because in one part of an application we update Table 1 and then Table 2, but in other parts we update Table 2 and then Table 1.  This is a disaster waiting to happen.  In other circumstances we have background tasks running that are not properly tuned and they try to update too much at once before committing their data.  This is also another disaster.

There are a number of surprisingly simple and, to most people, obvious strategies to use that will eliminate deadlocks to the "rare" occurrence that they should be.  We'll discuss those tomorrow.

Just Because ...

Just because you can do something, doesn't mean you should.

Geeks like new things.  They like playing with new technology.  Using new tools.  Trying out stuff that they've never done before.  XML is a good example.  XML has a lot of really good useful purposes.  It allows for easier portability of data between disparate systems.  It is, for the most part, human readable.  It is good for configuration files.  Indeed, configuration files are probably one of the most commonly used applications for XML.

There is one thing (okay, probably more, but this topic needs to be short enough to keep your attention) that XML shouldn't be used for:  continuously updated reference tables or codes tables.  For instance, you wouldn't put the conversion rate between U.S. and Canadian dollars into an XML for processing by an application because it would involve daily updates to that file.  You probably wouldn't put the top 10 grossing movies of the week into an XML file either, as that changes quite often as well.  You could, however, put a list of provinces in XML as that normally doesn't change that often.

We have a carefully controlled production environment.  In order for any new files to be moved into Production we first deploy them to UAT, have them tested and then moved into Production.  You can see that by putting volatile data in an XML file you are creating a lot of extra work for you, the Deployment Team and even the users who need to test the changes as every change will create a lot of overhead.

What's the alternative?  A database.  Reference tables that have the potential to be changed on a frequent basis should never be put into XML files, they should be placed into a database with the proper screens in place to change the data.  Putting this sort of data does nothing for your application in terms of flexibility or ease of use.  It was a matter of using a cool new toy when it didn't need to be done.

Everything New is Old Again

You've just rewritten your application using the next version of the development tool and your web app is up to date with all sorts of spiffy new features.  Your team is tired after the prolonged effort to get things running, but it's done and you can relax.

Unfortunately, with today's technology, you've also signed up for a continual cycle of redevelopment and maintenance.  I can hear the mainframe guys saying "Oh, we never have to do that."  Yeah, right.  The longest weekend of my life (well, maybe the second longest) was spent compiling over 1900 COBOL programs, four times, once for each of our different environments (DEVL, SYST, UAT, PROD), all because the version of COBOL changed and made certain programming approaches invalid with the new version of COBOL.

Technology changes, of this there is no doubt.  Businesses that produce that technology will only supply support for a specific period of time.  They need to continually make money to fend off shareholders and supporting old technology is a money sink, not a money generator.   Within Microsoft there is something called Mainstream Support and this is the period when hot fixes are free.  Hot Fixes are available in Extended Support, but only if a support agreement is paid for in advance.  For some Microsoft technologies that we currently use these are some dates for the end of Mainstream support:

• .NET Framework 1.1 - October 14, 2008


• Visual Basic 6 - March 31, 2005


• Visual Studio 2003 - October 14, 2008


• SQL Server 2000 - April 8, 2008


• BizTalk 2002 - July 10, 2007


• BizTalk 2004 - July 14, 2009


• Windows 2000 - June 30, 2005

Take a look at the list and you'll see that for many of your applications, Mainstream Supports end within the next 18 months.  While this may not be an issue for some of you, you should consider upgrading the development tool as failure to do so may result in incompatibilities between your application and newer technologies.

Batch Processing

Batch processing is the mainstay of mainframe processing. This is where you get to line up one, two, dozens of jobs, let them take off and then charge the various departments, divisions, companies, based on the CPU time used for that job. People would schedule their jobs for weird hours of the day because you normally got charged a lower rate for processing at night or during "off peak" processing hours. After all, the mainframe was only a certain size and you had to ensure that your online customers during the day had the processing power available to them for what they needed done.

In the Wintel (Windows/Intel) era, however, batch processing is, quite frankly, antiquated. There are few tasks that actually need to be done in a batch manner as many of them can be done in an asynchronous manner throughout the day. Now, I am not including reporting in this as some reporting needs to be done on a consistent, daily basis and can easily be scheduled (if the report even needs to be run at all!). What I am referring to is setting up Job B to run only after Job A has successfully completed.

To be honest, in 90%+ of the cases there is no good business reason why Job B must run after Job A. What there is a specific business reason for is business process B to follow business process A, for a specific, small set of data. In one of my previous lives I worked in the insurance business. You didn't pay out on the insurance until you determined the persons eligibility. You did not need to determine the eligibility of everyone who applied that day before paying a specific person, you just needed to determine the eligibility of that one person.

This is a completely different mindset than what many "old timers" are used to as we are no longer concerned about trying to schedule things for "off hours". Instead, we are processing the data on an as-needed basis. What does this do? Well, in many cases it eliminates many batch jobs as the majority of the processing has already occurred during the day. It eliminates the need for fancy scheduling between jobs. It eliminates the requirement for a batch window.

This isn't an easy mindset to get rid of. Many customers still operate in this mode as it is easier to understand and is much more familiar territory. It is our job as IT Professionals, however, to not just do what our clients say, but to educate them as to what can be done, to show them new opportunities, and to give them something better than what they had, not just something newer.

Keeping it Simple

In my second year of NAIT one of our instructors decided that we should enter a programming competition put on by ACM.  With more than just a little curiosity we entered the competition.  One of the problems was to print out a multiplication grid, using Roman numerals.  We had a difficult time with this question, but when we finished we had a program that could print a grid of almost any size and convert the numbers into Roman numerals.  It worked, but our instructor was disappointed.  He asked us why we made it so complicated.  All that was asked was to print out the multiplication grid, so why hadn't we just printed the grid with 12 PRINT statements?

Talk about keeping it simple!!!

Blaire Pascal, a French mathematician and philosopher, once wrote "If I had more time, I would have written a shorter letter."  This completely goes against "traditional" productivity measurements in IT that state each programmer should generate x KLOC (thousand lines of code) per month:  if you generate more, you are more productive.  Generating more lines of code goes completely against keeping it simple.  The odds are you are cutting and pasting lots of code, doing things in ten lines when it can be done in one and generally doing your best to add fluff to the application.

The next time you are sitting down and writing a method in the language of your choice, check to see the size of the method.  A mentor a number of years ago had a simple rule:  if you can't see all of the lines of the method on the screen at once, rewrite it.  I have tried to follow this as closely as possible for much of my professional career and have been quite successful.  By, quite frankly, putting in arbitrary limits I have forced myself to think simple.  Sometimes a single method needs to be greater than the height of the screen, but this is an infrequent occurrence.

Keep it simple and you'll be surprised at how productive you really are!

Changes and IT

I always find it really funny during playoffs (baseball, basketball, hockey, football, etc.) when people suddenly adopt these superstitions. You know what I'm talking about. The baseball player who refuses to shave because "it might cause bad luck". Or the hockey player who taps the pads of his goalie in a particular sequence because "it's good luck". Or the developer who always does things in a particular way because he's "always done it that way before and it's worked". People don't like change.

One of the most fascinating things about the IT industry, whether you work on mainframes, minis or PC class servers, is that the technology changes. It doesn't always change quickly but you don't always change the course of an ocean liner quickly either. (Although I bet the Lusitania wished that you could.) The process of making steel hasn't really changed in a long time. Neither has the process for making plate glass. IT? Yeah, it changes.

But you know, change cause problems. Changes require people to learn new things. Changes require people to adapt. Darwin recognized this a long time ago and wrote a book about it, some of which is actually applicable to the IT field. As new technology enters the field it is up to the developer to understand the technology and adapt his personal best practices to the new technology. Yes, you may have done this dozens of times in the past, but the technology has changed and maybe, just maybe, you shouldn't be doing this in the future.

Unlike Darwin's theory, failure to adapt will not cause your extinction, but it will limit what you can work on in the future. Embrace the technology and embrace the change. Learn to take your personal best practices and apply them to the new technology. They worked once and they can work again, but you need to make an effort.

Alone in a sea of web sites

Have you ever been to the mall when it's crowded?  You drive around the parking lot, looking for a spot, but the only thing close to the doors is a handicapped spot.  You look at the vehicle your driving, realize that it is not a handicap and decide to move on.  You eventually find a spot in the same postal code and start the long walk to the door.  On the way you see this little sports car drive into the handicap stall and out pops someone who, other than his driving style, has no handicap.  What he did, and you didn't do, was take the first available spot to park in.

This is, in many ways, how most projects build their installation applications.  (Yeah, I'm wondering how I'm going to connect the two myself, but hang in there for a couple of minutes.)

Many development projects have their own servers.  These servers are isolated in that no one else uses them and the project teams can configure and administer them any way they wish.  Once they are ready for UAT, however, they need to leave this isolated environment and enter the world of the Shared Environments.  On these servers there are multiple web applications and, drum roll please, multiple web sites.  Many setup applications are designed so that they install in the first web site they come across (much like parking in the first parking spot they see).  This is such a bad idea that I shudder when I write about it.

Applications need to understand that they co-exist with many other application.  Indeed, they are running on a server that has at least one web site and probably more.  During the installation process the web application should enumerate the web sites that are available and ask the person installing the application under which web site it should be installed.  Much like asking which directory to install an application into, the web site to install into is just as important, if not more important than the physical location of the files.  If the application just blindly installs itself in a web site the Deployment Team will quite often need to move the web application between web sites and, hopefully, everything moves over properly.  Nothing like crossing your fingers and hoping that something works.

So, please, in the future, change your web application installation programs to enumerate the web sites and ask which web site the application should be installed under. It's either that or I start leaving nasty notes on your windshield.