In-Depth

DRBC Strategy #3: Remember the Mainframe

If your enterprise data can be deemed as critical, it's bound to be sitting on a mainframe. So, what do you need to know to make sure it's a part of your DRBC plan?

• By Bill Heldman
• 06/22/2006

When it comes to Disaster Recovery and Business Continuity (DRBC), you may have your own Goliath and not even be aware of it. It’s called the mainframe and it’s hairy, ugly and gnarly. Here’s the deal: If you don’t slay it (i.e. bring it under your DRBC plan), it’s going to slay you. Not only will you not be king, you’ll likely be relegated to the position of jester.

Ouch.

OK wiseguy, so what the heck is a mainframe anyway?

Maybe we should start by explaining what a mainframe is, and what it is not.

Mainframe computers got their start back before many of you were a gleam in your papa’s eye. We’re talking 1940’s-50’s. International Business Machines wisely saw that businesses all over the world could benefit from the ideas coming out of the fledgling computer industry. The company invented and patented many new innovations that would do exactly that (including the most popular typewriter ever invented, the IBM Selectric). Also out of that pioneering effort came mainframe computers: hardware and software.

"But you didn’t say anything about HP mainframes running UNIX. What’s the matter? Are you an HP bigot?" Nope, not in the least. Boxes that run Unix are not mainframes; they're mid-size computers. (By the way, your little PCs and 1U-8U servers are called micro-computers). So there is a well-known, long understood differentiation between mainframes, mid-size computers and micros. Never mind that today the micro-community far outpaces the mid or mainframe world. Or does it?

7 Mainframe Truths
Before you get those mainframes under your DRBC plan, there are seven basic things you should know about them:

1. Mainframes are more secure than any box on the planet. You don’t hear about a lot mainframe hacks. Why? Two reasons: Top Secret and RACF. Both of these identity management programs are so tight, lots of government entities aren’t afraid to even stick their mainframes out on the DMZ.
   
   Tip: Mainframes have their own OS. The previous version was called OS/390. Today’s version is z/OS.
   
2. You must thoroughly understand a special mainframe-only language to be able to access the mainframe (provided of course you have the needed privileges). Job Control Language (JCL) is well known and understood by virtually anyone who regularly interacts with the mainframe. Want to get anything going on the mainframe? Better be able to write a JCL script, submit it and monitor it. There are quite a few good books available for those interested in pursuing a better understanding of JCL.
3. You must interact with the mainframe through a program screen customized for you by the system administrators. Time Share Option-Interactive System Productivity Facility (TSO/ISPF) and ROSCOE are but two of the interactivity suites users might have to use in order to perform mainframe tasks. In the mainframe world, it isn’t as simple as opening a command prompt and running some NET commands.
4. Mainframes have their own file types, file systems and storage access methods. Virtual Storage Access Method (VSAM) is one of the most popular, but there are myriad others such as ISAM and QSAM, among others. Like the difference between FAT, FAT32, NTFS and Linux file systems, the mainframe has its own unique flavors you must understand before you can fully interact with the mainframe. And, on top of all that, the mainframe has the ability to run different file systems in different spaces, called LPARs.
5. Mainframes have their own languages and databases for user applications. The most common is a very good IBM product called Customer Information Control System (CICS, pronounced "kicks" by savvy mainframers). This program is installed practically everywhere on every mainframe and handles literally billions of customer transactions every day. (Read that as billions of dollars in revenue.) Most programs that interact with CICS are written in COBOL. There are other programs such as the heavily ensconced and highly respected Natural/Adabas system from Software AG and the more user-friendly and equally popular FOCUS program from Information Builders. In addition to all of this, there may be some FORTRAN and even mainframe Assembler code running. Back in my mainframe programmer days I worked with a more exotic program called MARK IV. Column-based programming -- who can beat it?
   
   Tip: Mainframers call the mainframe "the host."
   
6. Mainframes require a special, separate and usually large operations team to manage them. The Windows Server admins probably aren’t the ones running the big OS/390 box in the back room. Not only are there mainframe systems administrators, there are operators who manage the JCL scripts and back-up operations, security administrators who do nothing but deal with RACF or Top Secret all day, and, of course, there are programmers who write and maintain host-based programs. None of these people will be likely to have a deep understanding or love of server/PC technologies. These people deeply comprehend the immense processing capabilities of the host and probably don’t give a rip about your little PC program. (I’m trying to introduce a flavor of the us-versus-them mentality you’re likely run into when working with mainframers.)
7. PCers must access the host with a special program called a 3270 emulator. You can’t just Telnet into the host and do your work. This has to do with the keyboard layout from the old days of mainframe terminal computing. Jolly Giant Software is one company that has done well with its cheap and effective QWS emulation software. Want to submit a JCL job, or see the output of some JCL? You’ve got to have a 3270 emulator.
   
   Tip: Mainframes have been a TCP/IP player for a long, long time. However, if you’re going to interact with the mainframe via TCP/IP, you’ve got to do things the TCP/IP way. For example, if you want to set up a routine FTP job, you’ve got to understand the mainframe’s FTP command construct, as it may be quite different than the one you’re accustomed to. Hence, the file download you get may not be what you’re expecting. Or, you may not get a download at all. And, though it goes without saying, you won’t natively submit an FTP job to the host. You do that through JCL, of course.

Dinosaurs Among Us, Still

The mainframe will continue on, probably long after you leave.

Here’s the issue: With billions of lines of COBOL code running and a tremendous amount of employees interacting with mainframe systems, it is a nearly-impossible proposition for companies to divest themselves of mainframe computing, even if they wanted to. Most of the people in charge of customer systems will be very reticent to move their systems from something that has run successfully for 30 years to something new, untried and unproven. The "if it ain’t broke, don’t fix it" principle most certainly applies here.

Back to our DRBC conversation, how on earth are you supposed to protect such important and exotic resources? With millions of dollars invested in each mainframe, you probably can’t tell the CIO she needs to by a new mainframe computer for the hot site you’re planning. (See jester note above.)

You have three options:

Fortunately, we had a mainframe computer no longer in heavy use, as it had been replaced by a bigger, newer model. We prepared a project plan to move the old mainframe to the new spot, make sure the appropriate applications were running on it, and the required data was being regularly copied to it. Once done, we held a couple of real-life disaster recovery scenarios to make sure it would be sufficient for our needs.

The result? We got the mainframe moved over (by special movers qualified and bonded to move IBM mainframe gear), the applications set up and the data regularly replicating. This effort took nearly six months. At DR scenario time, we put a handful of users on the machine to see how it would do in a production scenario. We continued to add users until they brought the machine to its knees (computationally speaking). Thus we knew when we had our theoretical limit of the number of users who could successfully interact with the machine. Of course, the number was far less than the big new host could handle.

This translated into productivity lost and the ensuing question: "How many workers can successfully interact with the system at any one time, and how much data can we process? OK, in a DR scenario, we’ll probably need to have three shifts of workers on 24x7, else we’ll get behind in our work." No one liked the idea of downscaled operations, or the fact that people would have to be on shifts. But then again, we were able to 100% replicate the organization’s mainframe operations to a hot site and prove that the cutover would work as planned. At this point, we merely had to deal with the mechanics of user schedules -- a far cry from the original "Hey! We’ve got a mainframe over here we better be thinking about!"

Option 2: Rent a hot site
IBM and others have also thought about the problem of ACME Widgets, Inc. not being able to afford a multi-million dollar mainframe to act as a standby in a hot site. So they have provided the facilities and the managers needed to help you with such an effort.

For a price of course.

This idea has merit because you gain some advantages you may not when attempting to use your own equipment at your own site:

• A rented hot site is much more likely to have updated, newer equipment. And, it’s more likely to be routinely updated.
• Mainframe subject matter experts (SMEs) are made available to help you work through the nuances of application and data transfers.
• Users report to a secure, off-site, building that is probably a good geographic distance from your regular building. (Remember it is underwater, blown up, or in some other way unavailable.) This means the building owners deal with sticky issues like parking, and security badges.

There are two problems with the idea of renting a mainframe hot site:

• Because of the expense, you’re not likely to move all of your DRBC operations to a rented environment. Which means some people will report to one hot site, and the rest to another.
• From a budgeting perspective, someone’s going to have to plan for and allocate the ongoing expense of maintaining a rental hot-site. This represents a budgeting problem in the Operations and Management (O&M) arena. Perhaps most significantly, it means this particular budget item may be cut in lean years. Not cool.

Note: There are only a few SAN companies that will work with mainframe data. If you have a mainframe and you want to jointly host SAN data alongside so-called “open systems” (e.g. Windows, Linux, UNIX) data, check with your vendors first! If you’re going to go the rent-a-hot-site route, be sure the SAN systems work with both open systems and mainframe data.

Option 3: Offload mainframe data and operations with BizTalk
Here’s the larger question: Why do you need a mainframe anyway? Can’t you, with good planning and execution skill, move the applications and data off of the mainframe while simultaneously planning for hot site replication?

The (careful) answer is yes, you can do this. BizTalk is an excellent, mature tool with connectors available for almost all mainframe applications and files. Given the right programming and database teams, enough time and money, you might very definitely pull off this coup.

However, be warned for most mainframe systems, you’re looking at 30 years of various programming and database efforts, including "fixes" to the programs in a whole different language than the original (e.g. Assembler versus COBOL), JCL scripts patched over the years, databases built for various new operations and features, customized CICS screens, and so on. While the mainframe program works, it could be very gnarly once you open the hood. Discovery time will be in the years, not the months. You’ll be whacking at what you think is one serpent head, when in reality you have an 8-headed Hydra on your hands.

And it’s likely there isn’t just one of these systems, there are probably ten or twelve -- all of which consist of hundreds of programs, dozens of databases and scripts, and workarounds that only Charlie, the old midnight-shift guy on the mainframe operations team is aware of.

Meanwhile, an event occurs, the building augers in and your mainframe’s gone.

Shucks, just when you were about ready to assemble the BizTalk cutover team.

The reality is, just like the Eiffel Tower and Hershey’s cocoa, mainframes will not go away anytime soon. Nor should they go away. Yes, Oracle and SQL Server can handle millions of transactions, and you can certainly stack enough servers together to assemble some astonishing computing power. But at the end of the day, the mainframe does something none of the others do: It allows thousands of users to connect to a single source to run a variety of centrally hosted programs and databases, and does so at speeds in which most users can feel comfortable.