Batch Processing

Mainframes are built to move huge amounts of data around.  IBM did a fantastic job of building giant batch processing engines.  Granted there are online options, but, to be honest, their bread and butter is huge amounts of data in and out.  Because of that mentality, the fact that the course designers at Universities and older people in the IT field "growing up" with this technology and even the integration of the term "batch processing" into the general lexicon of life, people design systems to include batch processing.

*WRONG*

OK, that may be too strong.  Let's say that the majority of the "reasons" for batch processing are actually replaceable with the phrase "because we've always done it that way".  Batch processing, the processing of huge volumes of data in a scheduled, off prime time manner, is a relic of the mainframe and needs to be replaced with the idea of a batch of one.  This idea of a batch of one is not new, as I`ve even used it on the mainframe.  The user needed certain processing done immediately as opposed to that night and we dynamically created a job, complete with the JCL and necessary parameters, in a low priority job queue and the job was processed when the machine had spare cycles.  It was batch, but a batch of one.

The idea is applicable in today's world.  When I ask people why they are creating a batch job they say something like "the printer is loaded with a certain paper at night so that is the only time we can do this".  *WRONG*  The processing can be done at anytime, the printing can only occur at night.  Instead of doing a lot of processing at night, do the processing in an asynchronous manner throughout the day and do the printing at night.  Split things into two pieces:  processing and printing.  They are two separate functions and do not need to be combined.

Let me rephrase in a more "global consulting" manner:  by separating processes into their unique steps, you create more opportunities to provide value to the business client.  Through a selective process of decoupling the steps in a business process you provide opportunity to inject new business functionality into a more flexible framework with less rework to the existing systems. Isn't that really what the business user wants?  More flexibility?  More value?

Short tempered

Some have said that my comments in the Daily Migration Note are brusque or just plain old rude.  Others have said that I am condescending in my notes.  A number of years ago a project manager actually said he found me "abusive and your attitude repugnant".  (I always keep those good emails so that I can refer to them later.)  Others call me up to see if I am still sane or whether I'm going to go postal.

I am, by my own admission, a strange duck.  Pressure doesn't affect me the same as other people.  I've been in situations where an the vast majority of an organization was sitting on their hands, waiting for me to come up with a solution. I've been the key person involved in resolving a payroll issue that affected tens of thousands of people.  While they are stressful situations, the pressure of the problem, the adrenaline that is produced, all blend together.

And yet, some things make me react quite harshly.  Yes, even quite rudely.  Why?

At a recent event there was an opportunity to evaluate myself based on a number of different characteristics.  Based on my answers to the questions, I was (drum roll please) not a people person.  This may not be a surprise to a bunch of you, maybe all of you.  What the quiz showed me was something that I knew all along:  process and logic are the key factors when I am dealing with work.

So, what does this mean?  Well, if things don't make sense to me I have a tendency of digging my heels in and saying "No".  If there is no logical sequence of events I have a tendency of saying "stop".  If people try to pressure me to do something that I think is illogical, I fight back.

Do you know what actually makes sense to me?  Something that is a logic progression when resolving a problem?  A partnership.  Not a partnership where one or more sides pay lip service to the idea of a partnership so that they can check it off their list of marketing responsibilities (there are many such partnerships amongst major corporations that fall into this category), but a true partnership where each side is working towards a common goal and where all parties are free to bring forth ideas to the table.  No finger pointing.  No "blame game" tactics. 

I have been involved in a number of different groups that were true partnerships between two or more groups and the results are amazing.  With the freedom to make any suggestion about any topic without fear of reprisal the ideas and solutions that come out of such a group are amazing.  There is tremendous difficulty in this as you need to leave your ego at the door and be willing to admit you were wrong.  You need to understand that there is more than one way to skin a cat and that yours may not be as effective.  And, in a true partnership, you don't try to pressure people to follow your lead or your suggestion, you true to influence them and persuade them that your way is appropriate to try.  As soon as you try pressuring people the partnership is gone.  And, in my case, I start resorting to my stock answers of "no" and "you must be kidding".

Partnerships.  Give them a try and you'll be amazed at what you can do.

Debugging

Don't blame the Deployment Team if your application fails and you can't figure out why.

Increasingly we have seen project teams with problems in Production (or UAT for that matter) and they are at a loss to explain what the problem is.  Sometimes there is a cryptic error message in one of the event logs along the lines of "Object reference not set to an instance of an object".  <sarcasm> Now that is a descriptive error message. </sarcasm>

There are a number of things you can do to make your life easier:

• The various .NET languages come with a feature called a Try...Catch block.  Where you try something and if things don't work you catch the exception.  Amazing how that works.  If you haven't put Try...Catch blocks in your code, particularly around any code that is calling a service outside of your control (SHR, EPS, DMS, PRS, etc.), then you score -5 points on the attention to detail rating.
• Did you know that if you catch an exception you can add your own information?  Yes, if you've caught something you can add information such as what you were doing when you got the exception or copies of the data used in the method so that people don't have to guess at what went wrong.  If you just rethrow the exception without adding any information, deduct 5 points from your "value added" score.
• Did you know that there is a Trace class in .NET?  It can do all sorts of things for you, such as allowing you to dump out detailed trace information (that is if you've written detailed trace information in your application) by changing a config file or even dynamically through code.  This is a very handy feature when used in combination with breadcrumbing so that you can see the complete route that an application has taken and, if you've added it, the data used throughout the process.  If you've never used Trace then how do you ever expect to trace anything in Production?

There are lots of things you can do to make your life easier when it comes to resolving problems in Production.  Simple things that can be useful in any environment.  Simple things that you should know how to use and how to use effectively.