Tuesday, July 03, 2007

Batch Processing

Batch processing is the mainstay of mainframe processing. This is where you get to line up one, two, dozens of jobs, let them take off and then charge the various departments, divisions, companies, based on the CPU time used for that job. People would schedule their jobs for weird hours of the day because you normally got charged a lower rate for processing at night or during "off peak" processing hours. After all, the mainframe was only a certain size and you had to ensure that your online customers during the day had the processing power available to them for what they needed done.



In the Wintel (Windows/Intel) era, however, batch processing is, quite frankly, antiquated. There are few tasks that actually need to be done in a batch manner as many of them can be done in an asynchronous manner throughout the day. Now, I am not including reporting in this as some reporting needs to be done on a consistent, daily basis and can easily be scheduled (if the report even needs to be run at all!). What I am referring to is setting up Job B to run only after Job A has successfully completed.



To be honest, in 90%+ of the cases there is no good business reason why Job B must run after Job A. What there is a specific business reason for is business process B to follow business process A, for a specific, small set of data. In one of my previous lives I worked in the insurance business. You didn't pay out on the insurance until you determined the persons eligibility. You did not need to determine the eligibility of everyone who applied that day before paying a specific person, you just needed to determine the eligibility of that one person.



This is a completely different mindset than what many "old timers" are used to as we are no longer concerned about trying to schedule things for "off hours". Instead, we are processing the data on an as-needed basis. What does this do? Well, in many cases it eliminates many batch jobs as the majority of the processing has already occurred during the day. It eliminates the need for fancy scheduling between jobs. It eliminates the requirement for a batch window.


This isn't an easy mindset to get rid of. Many customers still operate in this mode as it is easier to understand and is much more familiar territory. It is our job as IT Professionals, however, to not just do what our clients say, but to educate them as to what can be done, to show them new opportunities, and to give them something better than what they had, not just something newer.

No comments: