Integrating z/OS applications with the rest of the world

Many mainframe applications were built in an era where little integration with other applications was needed. Where integrations were needed, this was mostly done through the exchange of files. For example, for the exchange of information between organizations.

In the 1990s the dominance of the mainframe applications ended and client-server applications emerged. These new applications required more extensive and real-time integrations with existing mainframe applications. In this period many special integration tools and facilities were built to make it possible to integrate z/OS applications and new client-server applications.

In this chapter I will highlight categories of these integration tools that are available on z/OS, from screen-scraping tools to modern integrations supporting the latest REST API interfaces.

File interfaces

The mainframe was designed for batch processing. Therefore integration via files is traditionally well catered for and straightforward.

You can use multiple options to exchange files between applications on z/OS and other platforms.

Network File System

Network File System (NFS) is a common protocol that you can use to create a shared space where you can share files between applications. Although it was originally mostly used with Unix operating systems, it is now built into most other operating systems, including z/OS. NFS solutions however are usually not a preferred option due to security and availability challenges.

FTP

The File Transfer Protocol (FTP) is a common protocol to send files over a TCP/IP network to a receiving party, and it is also supported on z/OS. With FTP a script or program can be written to automatically transfer a file as part of an automated process. FTP can be made very secure with cryptographic facilities.

FTP is built into most operating systems, including z/OS.

Managed File Transfer

Managed file transfer is also a facility to send files over a network, but the “Managed” in the category means a number of additional features are added.

Managed file transfer solutions make file transfers more reliable and manageable. A number of additional operational tasks and security functions related to file exchange are automated. Managed file transfer tools provide enhanced encryption facilities, some form of strong authentications, integration with existing security repositories, handling of failed transfers with resend functionality, reporting of file transfer operations, and more extensive API’s.

On z/OS a number of managed file transfer tools are available as separate products: IBM has Connect:Direct and MQ-FTE, CA/Broadcom has Netmaster file transfer and XCOM, BMC provides Control-M  and there are other less commonly known tools.

Message queueing

Message queuing is a generic manner for applications to communicate with each other in a point-to-point manner. With message queuing applications remain de-coupled, so they are less dependent on each other’s availability and response times. Applications can be running at different times and communicate over networks and systems that may be temporarily down. As we will see in the next section, when using alternative point-to-point protocols like web services, both applications and intermediate infrastructures must be available for successful application communications.

The basic notion of message queuing is that an application sends a message to a queue and another application asynchronously reads messages from that queue and (optionally) responds with another message over a queue. Besides the specific asynchronous nature of message queuing, a big advantage is that it can assure message delivery: messages will not get lost, and when the infrastructure is not available, messages remain stored until they can be delivered.

IBM’s MQSeries, or WebSphere MQ as it is called now, is a separate is one of the most well-known and robust solutions for message queuing available on z/OS.

The open API for messaging called Java Message Service (JMS) is implemented by WebSphere MQ and WebSphere Application Server on z/OS.

Applications using Message Queuing

Web services (SOAP, REST)

Web services is the modern technology that enables applications to communicate over the web protocol HTTP, the protocol we also use for browsing the web.

SOAP and REST are two different types of web services. SOAP is a bit older and exchanges XML messages. XML is more resource intensive because handling XML is a complex operation. REST is more modern and lightweight, and today’s API economy is mostly based on REST APIs.

The benefit of integration with web services is that no special infrastructure is needed for applications to integrate, apart from a capable web application server. Integrations are lightweight and can be very loosely coupled.

The downside of web service is that the HTTP protocol does not guarantee message delivery (as opposed to message queueing as we have seen above). Applications using web services have to implement their own recovery and retry mechanisms to cope with situations where connections are lost.

On z/OS today, most modern versions of application middleware on z/OS, like CICS, IMS, WebSphere Application Server, IDMS, and others support REST and SOAP interfaces.

Applications using Web Services

Enterprise Service Bus

Another form of integration can be achieved through Enterprise Service Bus tools. These tools probably give the widest variety of integration options. They can receive and send service requests over a number of different protocols. They can convert messages from and to many formats. And they can orchestrate complex message interactions between multiple applications. Enterprise Service Bus products in the market are Tibco Substation ES and IBM Integration Bus.

ESB solutions can be implemented on z/OS itself, which than often has the advantage of easier integration with the z/OS application side, but also a non-z/OS platform and integrate with z/OS agent software.

Enterprise Service Bus

Adapters

In many situations it may not be possible to refactor your old mainframe applications. The applications may not be designed properly in a layered manner, middleware technology may have limited options, skills may not be available, or the risk of a changing existing applications is too high. Or there may be other reason you do not want to touch the code.

For these situations, application adapters can help in opening up applications. In general, an adapter converts a proprietary middleware protocol like a CICS, IDMS or IMS API into a more common API or generic protocol, like a Java program, a web service or message queueing interface. Some adapters provide the option of converting a proprietary 3270 screen interface into a neat API through screen scraping.

I will highlight a number of the type of tools here.

Generic functioning of an adapter

CICS Transaction Gateway

CICS Transaction Gateway provides an API for Java and C programs to call a CICS transaction on z/OS.

CICS Transaction Gateway provides only a way to call functionality in CICS, but there is no possibility in this tool to reversely invoke an external program from CICS. CTG is only meant for external programs to call CICS.

CICS Transaction Gateway adapter

IMS Connect

IMS Connect provides a Java API through which you can invoke IMS functions form Java programs. Through IMS Connect you can access IMS transactions as well as data in IMS DB (see section Middleware for z/OS). As such it functions quite similar to CTG, although the native interfaces are of course different.

z/OS Connect

A recent product from IBM is z/OS Connect. This tool converts a REST API into one or more proprietary backend protocols, like a CICS or IMS transaction or call to Db2. Also, z/OS Connect makes it possible to call REST APIs from mainframe applications.

Thus, z/OS Connect provides a bi-directional adapter for REST API through which you expose and call RESTful APIs from existing z/OS programs in CICS, IMS, Db2, WebSphere Application Server and MQ.

z/OS Connect adapter

Screen scraping tools

You may have old legacy applications that are built as a silo, have only 3270 user interfaces and no decent program interfaces.

For this problem, screen scraping tools can be a last resort.

The integration problem of an application silo – refactoring is the ideal solution

A screen scraping tool provides the ability to simulate the interaction of a business user behind a screen, with the old application’s user interface. The screen scraper tool automates the workflow of the end-user by filling in the old application screen programmatically. With these automations such a tool can then aggregate and expose these interactions into higher level services. These higher level services can then be invoked through a modern API, such as a web service by other applications in your organization.

Integration with a screen scraping solution

The big problem with screen-scraping integrations is that you end up with more development artefacts that you need to maintain. Not only do you have the old application to maintain, but now also need to manage the screen scraping middleware and logic.

Screen-scraping should be considered a (very) temporary solution for a serious issue in your application landscape. Such a solution should be replaced by a strategic integration or new application as soon as possible.

Products like HostBridge, Rocket LegaSuite and IBM Host on Demand provide screen scraping facilities.

Legacy integration suites

There are many integration tools on the market that provide one or more of the forms of adapters that I have discussed in the above. For example, GT Software and Oracle Legacy Adapter provide functionality to bridge native z/OS interfaces including screen interfaces to and from modern applications.

Database access via JDBC, ODBC

So far, we have discussed application integration through application interactions – applications calling one another.

Applications on non-z/OS platforms alternatively can directly access data in databases on z/OS through the standard data access protocols ODBC and JDBC. All suppliers of database software for z/OS that I know provide drivers for ODBC and/or JDBC.

Integrating with JDBC and ODBC

From an architectural perspective it is not a preferred solution for integrating applications. Applications should manage their own data and access other applications’ data only through service interfaces, and follow the principle of loosely coupling for application architectures.

Programming languages for z/OS

  • Post category:DBAOTMProgramming
  • Reading time:9 mins read

In this post I will discuss the programming languages you find on z/OS, and what they are generally used for.

COBOL

The COBOL programming language was invented 60 years ago to make programs portable across different computers. The language is best usable for business programs (as opposed to scientific programs).

COBOL is a language that must be compiled into executables, load modules.

       IDENTIFICATION DIVISION.
       PROGRAM-ID.
           COBPROG.
       ENVIRONMENT DIVISION.
       DATA DIVISION.
       PROCEDURE DIVISION.
           DISPLAY "HELLO WORLD".
           STOP RUN.

PL/I

PL/I was developed in the mid-1960s with the aim to create a programming language that could be used for business as well as scientific applications.

Like COBOL, PL/I programs must be compiled into load modules.

   World: Procedure options(main);
          Put List( 'Hello world' );
          End World;

Assembler

Assembler is still around. In the past business applications were developed using Assembler. Nowadays you should not do that anymore. But there are still a lot of legacy assembler programs around on the mainframe.

In the old days, assembler was often used to implement tricks to achieve things that were not possible with the standard operating system, or other programming languages. This practice has created a problematic legacy of very technical programs in many mainframe application portfolios.

The modern stance is that Assembler program should be regarded as severe legacy, because it is no longer maintainable and Assembler program are a risk for operating system and middleware updates.

Furthermore, we find Assembler programs in modifications to the z/OS operating system and middleware.

z/OS offers a number of points where you can customize the behavior of the operating system. These so-called exit-points oftentimes only have interfaces in Assembler. Like application programs in Assembler, z/OS exits in Assembler are a continuity risk. Not only because nobody knows how to program Assembler anymore, but even more so because these exit points make use of interfaces that IBM may (and wishes to) change at any point in the future.

IBM is actively removing Assembler-based exit points and replacing these where needed with configuration parameters.

The bottom line is that you should remove all home-grown Assembler programs from your z/OS installation.

TEST0001 CSECT               
         STM   14,12,12(13) 
         BALR  12,0         
         USING *,12         
         ST    13,SAVE+4     
         LA    13,SAVE       
         WTO   'HELLO WORLD!'
         L     13,SAVE+4     
         LM    14,12,12(13) 
         BR    14           
SAVE     DS    18F           
         END

Java

The language invented by a team from Sun in the 1990s with the goal to develop a language that could run on any device. Support for Java on the mainframe was introduced somewhere in the beginning of the 21st century.

Java programs do not need to be compiled. They are interpreted by a special layer that must be installed in the runtime environment, called the Java Virtual Machine.

The execution is (therefore) far more inefficient than COBOL and PL/I. So inefficient that running it on the mainframe would be very expensive (see section Understanding the cost of software on z/OS, MLC and OTC). To address this IBM invented the concept of zIIP specialty engines (see section Specialty engines), which makes running Java on the mainframe actually extremely cheap.

public class HelloWorld {
   public static void main(String[] args) {
      // Prints "Hello, World" in the terminal window.
      System.out.println("Hello, World");
   }
}

C/C++

The C/C++ programming language was added to z/OS in the 1990s as a more mainstream programming language for mainframe applications and tools.

The process of compiling a C source program into a load module is basically the same as it is for COBOL.

#include <iostream>
using namespace std;

int main() 
{
    cout << "Hello, World!";
    return 0;
}

JCL

JCL is the original “scripting” tool for the mainframe. It is hardly a programming language, although it has been enhanced with several features over time.

JCL looks very quirky because it was design for interpretation by punch card reader, which you can still see very clearly. The main purpose of JCL is to start a program or a sequence of programs.

Many of the quirky features of JCL have very little use in today’s z/OS programming but are maintained for compatibility reasons.

I mentioned before that there can be tens of thousands of batch jobs running on the mainframe. You should realize that mean you will easily have thousands of JCL “programs” as well to run these jobs.

Nevertheless, we could do with a more accessible, more modern alternative.

//JOBNME5  JOB AB123,PRGRMR,NOTIFY=MYUSER1,MSGLEVEL=(1,1),
//       CLASS=1                   
//RUN       EXEC PGM=COBPROG <- PROGRAM TO RUN  
//* PROGRAM WAS PUT IN HERE --v           
//STEPLIB  DD DISP=SHR,DSN=MYUSER1.LOADLIB 
//SYSPRINT DD SYSOUT=*

Rexx

The story goes that the Rexx programming language was created by an IBM developer, Mike Cowlishaw, who was totally fed up with the only available language for scripting at that time, the CLIST language. In one night he is said to have developed Rexx. When he showed it to his colleges next day, they were immediately very enthusiastic.

On z/OS Rexx fulfils the same role a Unix scripts in Unix environments. It is mostly used by system administrators to automated all kinds of administration tasks.

You can run Rexx interactively under TSO/ISPF, but you can also use it in batch jobs.

Rexx is somewhat similar to PHP, I find. It has the same sort of flexibility (and drawbacks).

/* Main program */
say "Hello World"

Unix shell script

z/OS has a Unix part, which is complying to POSIX standards, and hence also support a command shell like any Unix flavor. With the shell scripting language you can automate all kinds of Unix processes.

Shell scripts can also be ran in batch jobs.

#!/bin/bash
echo "Hello World"

SAS

Many z/OS users exploit the SAS language from the company with the same name. SAS is used for ad hoc programs and reporting, besides its analytical capabilities.

On the mainframe SAS is often used to process the measurement data that z/OS generates, and create all kinds of usage and performance reports.

proc ds2 libs=work;
data _null_;

  /* init() - system method */
  method init();
    declare varchar(16) message; /* method (local) scope */
    message = 'Hello World!';
    put message;
  end;
enddata;
run;
quit;

Easytrieve

The programming language Easytrieve from CA/Broadcom you also find regularly in z/OS environments. This language is used by application support staff to create ad-hoc programs, and by advanced end-users to to create business reports from application data. 

Other languages

There are many other languages available on z/OS. But the ones discussed here are the mainstream languages. Languages like Python and R are emerging for analytical applications, JavaScript for use in in Node.js, PHP for web applications. Rocket Software, the company that supports a ported version of Python for z/OS, also have a supported version of PHP and Perl.

Middleware for z/OS – Database management systems

  • Post category:Db2DBAOTM
  • Reading time:3 mins read

In the previous post I started the first part of describing the middleware tools available on z/OS, kicking off with the available application servers of transaction managers.

In this part I will discuss the database management systems that can run on z/OS.

Db2

Db2 for z/OS is the z/OS version of IBM’s well-known relational database management system. It is a regular high-end RDBMS, except that it exploits the sysplex capabilities of z/OS.

IDMS/DB

IDMS/DB is the network database management system com CA/Broadcom. A network database uses special concept to organize data, namely in the form of a network of relationships. Besides some modelling advantages this way of data access can be extremely fast, but as for hierarchical data models like in IMS, it is more difficult to program for it.

IMS/DB

IBM’s IMS/DB is a hierarchical database management system. Data in such a database management system is not structured in this database in tables like in Db2, but in tree-like hierarchies. In Db2 and other relational databases there is the well-known SQL language to access data, in IMS you have a language called DL/I to manipulate data.

The hierarchical data model has some modelling advantages and also data access is extremely fast and efficient. The drawback of it is that it is more complex to program.

Datacom/DB

Datacom /DB is a relational database management system from CA/Broadcom.

ADABAS

ADABAS is Software AG’s database management system. ADABAS organizes and accesses data according to relationships among data fields. The relationships among data fields are expressed by ADABAS files, which consist of data fields and logical records. This data model is called an inverted-list model.

Middleware for z/OS – Application Servers

  • Post category:DBAOTM
  • Reading time:6 mins read

There is a large variety of middleware tools available on z/OS. Some are very similar to the software also available on other platforms, like WebSphere Application Server and Db2, and some are only available on the mainframe, like IMS and IDMS. I will highlight a number of the main middleware tools for z/OS in this chapter.

Application Servers

Application Servers are tools that make it easier to run interactive applications. Today we call these tools Application Servers. On the mainframe these tools were traditionally called Transaction Managers. A small intermezzo to explain the similarities and get acquainted with the terminology.

Applications Servers and Transaction Managers intermezzo

Despite their different name, Application Servers and Transaction Managers achieve the same goal: make it easy to build and run interactive applications. Application Servers gather a set of common functions for these types of applications. These functions include network communications, transaction functionality, features to allow scaling of applications, recovery functions, database connectivity features, logging functionality and much more.

For Java a standard for these functions is created in the Java Enterprise Edition (JEE) standard. The z/OS Transaction Managers all provide a similar set of functions, for multiple programming language like COBOL, PL/I, C/C++ and Java.

With a modern web application server, the user enters a url consisting of the name of a server and an identification of the piece of code on that server. For example, a user types in his browser http://acme.com/fireworks/index.html . In this, acme.com is the server name and fireworks/index.html is the piece of code to execute on that server – called the uri. The application server takes the uri, executes the code and returns a response html page.

The traditional transaction managers work in a similar way. First you must make a connection from your terminal to the transaction manager.

Traditionally you did this by typing something like “LOGON APPLID(CICSABC)”. Then you were connected to the application server and you were presented some screen. Then you type in a transaction code. The transaction code is similar to the uri: it identifies which piece of code to run. The transaction manager executes the code and returns a response screen to the user.

The transaction managers on z/OS nowadays can work in both ways. They still have the traditional interface, which is hardly used for business applications anymore, and they also have a modern web application interface like web application servers.

CICS traditional versus a web application server

CICS traditional versus a web application server

Now let’s have a look at what sort of application servers we have on the mainframe.

WebSphere Application Server

IBM’s WebSphere Application Server (WAS) is an application server for Java programs, complying to the JEE Java application standard. WebSphere was one of the first implementations of a Java application server. It was made also available on z/OS.

Initial implementations of WAS on z/OS were very inefficient and had stability issues. After a redesign and the introduction of speciality engines for Java processing (see section Specialty engines), z/OS has become of very cheap platform for JEE applications.

CICS

CICS is the most popular Transaction Manager on z/OS. It was designed for COBOL and PL1 applications, but nowadays you can also runs Java applications.

IMS

IMS is a transaction manager like CICS, but it also has a database management component. Although less prominent as CICS, quite a number of very large organizations are relying on IMS for their daily core processing.

An interesting fact is that IMS was built for NASA as part of Kennedy’s moon challenge.

IMS has two parts: IMS/TM and IMS/DB. IMS/TM is IMS/Transaction Manager, an application server. IMS/TM is as such functionally similar to CICS. It is also build for COBOL and PL/I, and can now also run Java programs.

IMS/DB is described briefly below.

IDMS

IDMS is a transaction manager and a network database manager, owned by CA/Broadcom.

IDMS, like IMS, also has an application server and a database manager part. IDMS/DC is the transaction manager/application server part. It looks very much like CICS.

IDMS/DB is a network database management system. See below.

ADABAS and NATURAL

NATURAL is Software AG’s fourth generation application development system that allows you to create, modify, read, and protect data that the DBMS manages. You can have online – like CICS – and batch Natural programs.

Natural usually uses ADABAS. Natural is the application server that uses ADABAS as it’s backend database management system.

IDEAL and Datacom

Another combination of application server tools that is quite common on mainframes is Datacom/DB and IDEAL. The products are now owned by CA/Broadcom.

IDEAL is a 4GL programming environment, designed for the relation database management system Datacom/DB. IDEAL generates COBOL, which runs in CICS, and uses Datacom/DB as a backend store. Although originally built for Datacom/DB, it was later also enabled for IBM Db2.

Turning a PDS into a PS with standard tools (for email)

  • Post category:Utilities
  • Reading time:2 mins read

I recently got a question from a collegue. He wanted to transfer an entire PDS in an email to someone else. You can download all the member of the PDS with FTP, zip up all the files and transfer that. But it might be easier to use this trick.

Create a PS from in PDS by using the XMIT command. XMIT is a TSO command that you can use to transfer a dataset to another user, or system, or both.

The trick now to “zip” a PDS with XMIT is to XMIT the PDS to yourself on the same system:

xmit userid.node dsn(dataset) outdsn(outdsn)

This creates the “zipped” PS dataset, that you can send through email. If you download the file to your Mac or PC, makes sure you download it in binary mode.

The receiver can “unzip” the file into a PDS with the accompanying receive command:

RECEIVE INDSN(dataset) DSN(outdataset)

Modern tools for development and operations

  • Post category:DBAOTM
  • Reading time:3 mins read

In the previous section I explained that green screen interfaces still exist for administrative tasks. But even for these kinds of work there are modern tools with contemporary interfaces. z/OS itself and almost all middleware running on z/OS can be managed with web-based tools, Eclipse-based tools for z/OS, or nowadays more and more Visual Studio Code based tools for z/OS. Furthermore almost all administration tasks on z/OS can be invoked from external tools through REST APIs. More and more development and operations functionality will be made available only in this modern kind of tool sets.

The standard Eclipse-based tool for z/OS that you can download for free is called z/OS Explorer. This tool is a desktop client interfacing with z/OS. Many mainframe tools and middleware solutions provide plug-ins for this tool.

z/OS Explorer
Figure – z/OS Explorer

Considering development tools, there are a number of modern options for the mainframe. IBM has developed an Eclipse-based development tool called IBM Developer for z/OS (IDz). The software company Compuware sells a set of tools for mainframe development called Topaz. There are also open source tools and plugins, like the IBM Z Open Editor for Visual Studio.

These modern tools provide a development experience for z/OS applications that is very similar to the experience you have when you develop applications for other platforms, like Java, PHP or Python. Support for developing and debugging mainframe languages such as COBOL and PL/I is supported in the tools, but also Java is supported. As importantly it provides plugins for interacting with middleware such as Db2, MQ, CICS and IMS.

IBM, CA, BMC and other vendors provide many modern tools for the administration of specific middleware in your organization.

Finally, a recent development is the open source project called Zowe. This project is a collaboration of a number of mainframe software vendors and aims to provide an open source software framework for development and operations teams to securely, manage, control, script and develop for z/OS like any other cloud platform.

In separate chapters I will discuss a little bit more on the modern application development and operations architecture tools for z/OS, and on modern monitoring architecture and tools.

Theropod blog on Medium

  • Post category:General
  • Reading time:1 mins read

Thought you might find this blog by IBMers interesting: Theropod.

Let’s hope they can keep up the good work.

Will add this site to the Link Pack Area collection.

Filewatch utility – file triggering

  • Post category:JCLUtilities
  • Reading time:2 mins read

The filewatch utility is not a well known utility, that can be very handy. I have used it especially when I needed process a unix file after it has been copied to a certain directory.

This is the documentation about the filewatch utility:

https://www.ibm.com/support/knowledgecenter/SSRULV_9.3.0/com.ibm.tivoli.itws.doc_9.3/zos/src_man/eqqr1hfszfstrig.htm

Th job below is submitted and then waiting and activated when there is a change in the input directory /yourdirectory/testfileloc

The job itself moves the files in the input directory to a processing directory, starts the next “watch” job and then the processing job for the files in the processing directory.

The setup is clearly to test the mechanism – and not necessarily a model for a production-like situation. In a production situation you might want to use TWS applications etc, but that is up to your application design of course.

I hope this helps.

 //STEP01  EXEC PGM=EQQFLWAT,PARM='/-c wcr -dea 900 -i 20
 //             -r 123 -t 3 -fi /yourdirectory/testfileloc'
 //*
 //* Register and move files
 //RUNSCRPT EXEC PGM=IKJEFT01,REGION=0M
 //STDOUT   DD SYSOUT=*
 //STDERR   DD SYSOUT=*
 //SYSTSPRT DD SYSOUT=*
 //SYSTSIN  DD *
 BPXBATSL SH  -
   mv /yourdirectory/testfileloc* /yourdirectory/testfileloc/processing
 //*
 //* Then kick off separate job
 //* - Next filewatch job
 //*
 //SUBJOB   EXEC PGM=IEBGENER
 //SYSIN    DD DUMMY
 //SYSPRINT DD DUMMY
 //SYSUT1   DD DISP=SHR,DSN=YOURPDS.JCL(EQQFLWAT)
 //SYSUT2   DD SYSOUT=(A,INTRDR)
 //*
 //* - Processing job
 //*
 //SUBJOB   EXEC PGM=IEBGENER
 //SYSIN    DD DUMMY
 //SYSPRINT DD DUMMY
 //SYSUT1   DD DISP=SHR,DSN=YOURPDS.JCL(PROCJOB)
 //SYSUT2   DD SYSOUT=(A,INTRDR)
 //*
 //*

The interface to z/OS and the green screen myth

  • Post category:DBAOTM
  • Reading time:6 mins read

In the previous posts I have shown you many modern technologies available on z/OS. But still when you think of the mainframe, you think of black screen with green characters, which looks cool in the Matrix, but not so much in real life. Where does this green screen imago come from?

In this section I will talk a little bit about the origins of the green screens in mainframe technology. I will also show you that these green screens have become as uncommon to use as a terminal in Unix or command prompt in Windows.

Green screens are for administrators and programmers, not end-users

The green screens on the mainframe are user-interfaces. In the early days programmers created their programs on paper, behind their desks. They then entered the programs on punch cards or paper tape. That were the media that were then fed into the computers, using a special reader device.

Later, in the 1960s, computers with terminal interfaces were build. With the terminals, users could enter programs and data online. This is the period that the green screens originate from. Each computer type had its own terminal technology. Mainframes had technology indicated with a number: the 3270 terminal. These terminals originally worked with green letters on a black background, and could hold as much as 32 lines of 80 characters (or so). We still refer to these 3270 terminals as green screen.

Modern mainframe applications do not use these terminal interfaces anymore. Applications on the mainframe most often do not even have a user-interface anymore. They only expose services, or APIs, exposed to mid-tier or front-end applications. (See modern mainframe application architecture section.)

Today therefore, the need for these green screens is limited. Only special system administration tools and application programming tools still have a low level interface. And even these are being replaced by tools with more modern interface.

System administrators on Windows use the “DOS” command prompt and Unix techies use the Terminal sessions. Similarly, for mainframe techies there is the “green-screen” 3270 terminal.  Actually, the Unix Terminal and Windows Command Prompt are quite rudimentary, compared to the 3270 interface to z/OS.

Green screen application? Technical debt

The days where you had green screen applications are long gone. If you still have them, you should get rid of them.

Most well-architected green-screen applications can be turned into service-oriented applications. The front-end can then be replaced by a modern front-end application.

You may find yourself in the situation where you need to integrate with green-screen applications that have not been so well-designed. I will talk a little bit about that in a separate section Integration with the rest of the world.

In section Application architecture for modern mainframe applications, I describe a reference architecture for modern mainframe applications.

Now, I will describe what tools typically still needs 3270 screens.

TSO

What is the functionality of the command prompt for Windows and the Shell function is for Unix, is the TSO tool for z/OS: a command line interface with which you can fire off commands to the operating system to get things done on the computer.

Like the DOS command prompt and the Unix shell, this is a very powerful, but clumsy interface. To provide a more user-friendly interface IBM built the tool ISPF on top of TSO.

ISPF

I will no go into the abbreviations here. What you need to know is that ISPF is a standard part of z/OS. This tool gives the user, nowadays mostly system administrators, a very powerful interface to z/OS.

The editor in ISPF and the dataset list utility are probably the mostly used functions is ISPF. With the screen-oriented file editor is you can edit the z/OS datasets. The dataset list utility lets you find the datasets on your z/OS system.

ISPF - Data Set list utility
ISPF – Data Set list utility
ISPF Editor
ISPF Editor

ISPF also provides facilities to extend its features through a programming interface. Many tool vendors provide ISPF tools built on these interfaces as part of their tool installation.

SDSF – or equivalent

SDSF is one of the extensions to ISPF that IBM itself provides as a separate product. This product, or one of its equivalents from other vendors, is an essential tool for system administrators of z/OS installations. SDSF allows support staff to operate z/OS and manage the processes running on z/OS, look at output from processes (jobs) and inspect application and system logs. It is somewhat similar to the Task Manager in Windows.

I talk about SDSF here, which is an IBM tool, but there are tools with equivalent functions from other software vendors, such as IOF, SYSVIEW or (E)JES.

Parallel sysplex

One of the most distinguishing features of the z/OS operating system is the way you can cluster z/OS systems in a Parallel Sysplex. Parallel Sysplex, or Sysplex in short, is a feature of z/OS that was built in the 90s that enables extreme scalability and availability.

In the previous post we highlighted the z/OS Unix part. Here we will dive into the z/OS Parallel Syplex.

A cluster of z/OS instances

With Parallel Sysplex you can configure a cluster of z/OS operating system instances. In such a sysplex you can combine the computing power of multiple of z/OS instances on multiple mainframe boxes into a single logical z/OS server.

When you run your application on a sysplex, it actually runs on all the instances of the sysplex. If you need more processing power for your applications in a sysplex, you can add CPUs to the instances, but you can also add a new z/OS system to the sysplex.

This makes a z/OS infrastructure is extremely scalable. Also, a sysplex isolates your applications from failures of software and hardware components. If a system or component in a Parallel Sysplex fails, the software will signal this. The failed part will be isolated while your application continues processing on the surviving instances in the sysplex.

Special sysplex components: the Coupling Facility

For a parallel sysplex configuration, a special piece of software is used: a Coupling Facility. This Coupling Facility functions as shared memory and communication vehicle to all the z/OS members forming a sysplex.

The z/OS operating system and the middleware can share data in the Coupling Facility. The type of data that is shared are the things that members of a cluster should know about each other since they are action on the same data: status information, lock information about resources that are accessed concurrently by the members, and caching of shared data from databases.

A Coupling Facility runs in a dedicated special operating system, in an LPAR of its own, to which even system administrators do not need access. In that sense it is a sort appliance.

A sysplex with Coupling Facilities is depicted below. There are multiple Coupling Facilities to avoid a single point of failure. The members in sysplex connect to the Coupling Facilities. I have not included all the required connections in this picture, as that would become a cluttered view.

A parallel sysplex

Middleware exploits the sysplex functions

Middleware components can make use of the sysplex features provided by z/OS, to create clusters of middleware software.

Db2 can be clustered into so-called Datasharing Group. In a Datasharing Group you can create a database that can process queries on multiple Db2 for z/OS instances on multiple z/OS systems.

Similarly WebSphere MQ can be configured in a Queue Sharing Group, CICS in a CICSPlex, IMS in an IMSPlex and other software like WebSphere Application Server, IDMS, Adabas and other middleware use parallel sysplex functions to build highly available and scalable clusters.

This concept is illustrated in Figure 15. Here you see a cluster setup of CICS and Db2 in a sysplex. Both CICS and Db2 form one logical middleware instance.

A parallel sysplex cluster with Db2 and CICS
A parallel sysplex cluster with Db2 and CICS

You can see the big benefit of parallel sysplex lies in it’s a generic facilties to build scalable and high available clusters of middleware solutions. You can achieve similar solutions on other operating systems, but every middleware component needs to supply its own clustering features to achieve such a scalable and highly available configuration. This often needs additional components and leads to more complex solutions.

How is this different from other clustering technologies?

What is unique about a parallel sysplex is that it is a clustering facility that is part of the operating system.

On other platforms you can build cluster of middleware tools as well, but these are always specific solution and technologies for that piece of middleware. The clustering facilities are part of the middleware. With parallel sysplex, clustering is solved in a central facility, in the operating system of z/OS.

GDPS

An extension to Parallel Sysplex is Geographically Dispersed Parallel Sysplex, GDPS for short.  GDPS provides an additional solution to assure your data remains available in case of failures. With GDPS you can make sure that even in the case of a severe hardware failure, or even a whole data centre outage, your data remains available in a secondary datacentre, with minimal to no disruption of the applications running on z/OS.

In a GDPS configuration, your data is mirrored between storage systems in the two data centres. One site has the primary storage system, the storage system in the other data centre receives a copy of all updates. If the primary storage system, or even data centre fails, GDPS automatically makes the secondary storage device the primary, usually without disrupting any running applications.