Modern mainframe application development

  • Post category:DBAOTM
  • Reading time:9 mins read

In the previous DBAOTM article on DevOps I have introduced the traditional development process, which is often still used in a mainframe environment. In this post I will present a modern approach to development on the mainframe.

Modern development processes for the mainframe

Requirements for the development process have changed. Applications must be built faster and it must be possible to change applications more often and quicker without impacting the quality of the application. In other words, agile development is needed. The only way to address today business needs into modern agile development processes is to automate all build and deploy processes.

A set of principles can then be derived for modern mainframe develops processes.

  • All application artefacts are managed in the (or a) Source Code Management tool.
  • The build processes for all artefact are automated, and can be coherently executed.
  • A build can be deployed in any environment. A build has no environment or organization-specific dependencies.
  • The deployment process for a build is fully automated. Including the fallback procedure. The deployment process is a coherent process for all application artefacts.

These principles need to be supported by tools and processes that are (re)designed for these purposes. Of course this is not something specific to z/OS applications, but is true for any modern IT solution. But with the background I have sketched in the previous section, there is a legacy of development processes and tools to take into account and in many organizations this implies significant technical and organizational changes.

The modern SCM for z/OS

The modern SCM tool for z/OS needs to support all kinds of application artefacts. For the mainframe this means for one thing that not only traditional MVS-type artefacts must be supported, like COBOL programs, COPYBOOKS and JCL, but also Unix type artefacts like Unix scripts and configuration files in z/OS Unix directories. The tools and processes should allow for EBCDIC type artefacts to be created or the z/OS runtime environment, as well as ASCI, Unicode and binary artefacts.

Modern SCM tools that can manage z/OS artefacts, are ISPW from Compuware, RTC form IBM, and a new option nowadays is Git, or GitHub.

Build automation

The modern DevOps process automates the creation of a build. The build process takes the required versions of the application artefacts from the source code management repository and creates a coherent package of these artefacts. This package, also called the build, is deployed in a (test) environment.

The build could be deployed in any runtime environment, even outside your organizations. This principle not only enforces standardization of processes and infrastructure in your IT organization, it also allows any future deployments in yet unknown environments – for example in the Cloud.

The automated build process itself should be callable through some generic API, so it can be integrated into other automated processes when needed.

Build automation on z/OS can be accomplished with a number of tools. Some of these tools are able to handle the z/OS specific needs. IBM has two solutions: Rational build engine and Jazz build engine. Compuware has capabilities in ISPW. As it stands, all these tools still have some gaps to fill in the coverage of the different artefacts that can make up a z/OS application.

Deployment automation

The modern DevOps process for z/OS automates also the deployment of the application build. The deployment process takes all the artefacts in the build, customizes them for the specific runtime environment, for example through the application of naming conventions and runtime aspects, and deploys the artefacts on the different runtime components in an environment.

The automated deployment process itself should be callable through some generic API, so it can be integrated into other automated processes when needed.

The most important deployment tools available on the market for z/OS are IBM’s UrbanCode Deploy and XebiaLabs’ XLDeploy.

Integration in other pipelines

I have indicated above that the DevOps processes described there must be callable, to use the most generic term I can think of. Since we do not just want to automate the individual pieces of a development process, but the entire chain, this requirement is important.

Only a fully automated Development process – a CI/CD pipeline – can provide optimal speed of development. To achieve this, the integration of build and deployment with other processes like infrastructure provisioning, test data provisioning, and testing is key.

Most of the tools mentioned above have API’s or command line interfaces that allow integration with CI/CD orchestration tools like Jenkins, Ansible, and others.

Implications

The agile development process sketched here impacts the way we do other things on the mainframe as well. I will mention a few here.

Full deployments versus delta deployments

The traditional DTAP development process is based on the development of delta’s: you only deploy these things that are changed.

To facilitate agile development in z/OS environments, we need to move to a process that supports full application deployments. What the consequence of this change are is fully clear, but I am convinced the old way of working with delta’s will not give of the speed and flexibility we need today.

Other impact:

  • Phasing in of an application that consists of many more load modules than we have today, while remaining active, needs to be supported in the middleware tools on z/OS.
  • Application may need to become smaller. Traditionally applications are defined relative coarse grained on z/OS. We may need to split up applications into smaller distinguishable, more loosely coupled parts. We might need to reuse some of the microservices architecture goodies.

To facilitate agile development drastic changes in our thinking about mainframe applications is necessary, and in principle no single goodie from the past should be exempt from reconsideration.

Infrastructure provisioning

We have talked about application processes so far, but the agile DevOps process must be supported by the runtime infrastructure. In the DTAP model, runtime environments are static, defined once and gradually changed, when this was functionally needed.

In order to support rapid changes in applications, we must also allow rapid changes in infrastructure. Similar to the build and deploy processes, all infrastructure provisioning must be automated to allow flexible and instant creation and modification of infrastructure for test environments. This also means that environments must rigorously standardized. Definitions of the infrastructure making up the environment must be treated like code, and be managed in a source code management system, where it can be properly versioned. 

Currently the tool support for infrastructure provisioning is very limited. As part of z/OS the tool z/OSMF is provided that allows the creation of provisioning workflows for z/OS technology-specific creation of infrastructure.

Furthermore, there is work ongoing in IBM and other vendors to extend this lower level capability and integrate this is infrastructure provisioning tools like Kubernetes and OpenShift. And also Ansible for z/OS is quickly emerging. Yet, there is still a long way to go but the first steps have been made.

In a future article I will talk a little bit more on infrastructure provisioning.

Please let me know your thoughts. Always happy to hear from you.

Running a MVS or TSO Rexx program from the z/OS Unix environment

  • Post category:Rexx
  • Reading time:2 mins read

You probably know that you can use Rexx programs in z/OS Unix.

What you may not know is that you can also run a TSO or MVS Rexx program from the z/OS Unix environment.

There is a unix command called tso for this. It works as simple as this:

tso –t “exec ‘YOUR.MVS.REXXPDS(TESTREXX)’ EXEC”

will execute your TESTREXX program.

You may want or need to allocate TSO datasets or other datasets in order to execute the Rexx.

You can simply allocate these through the export command.

export ddname=YOUR.MVS.DATASET

You can add these exports to you shell script, or add them to your .profile.

The Unix parts of z/OS

In the previous DBAOTM post I have introduced you to the z/OS operating system, the flagship operating system for the mainframe. In this post I will introduce you into the Unix side that z/OS has been equipped with over the past two decades.

Since the 1990s IBM has added Unix functionality to z/OS. The first extension was z/OS Unix System Services – z/OS Unix in short – and recently we IBM have added z/OS Container Extensions.

z/OS Unix

The Unix part of z/OS is a runtime environment that is an integral part of the z/OS operating system. z/OS Unix is fully Unix (Posix) compliant. z/OS Unix provides an interactive command interface that is called a shell in Unix terminology.

IBM has developed this part in the 1990s to make it easier to port applications from other platforms to z/OS. Many off-the-self and open source middleware and application packages that are available on z/OS make use of z/OS Unix. Examples are IBM’s own WebSphere Application Server and IBM Integration Bus, the banking product BASE24-eps from ACI, and open source tools like PHP and Git.

z/OS Unix has regular files the same as other Unix systems. In the z/OS Unix shell you can use normal Unix commands like ls, cd, more and many more of the standard Unix commands. You can set up a telnet or SSH session with z/OS Unix and do many more things you can also do with other Unix environments.

z/OS Container Extensions

A very recent development on z/OS (it came with z/OS 2.4, end 2019) is the possibility to run Linux Docker Containers on z/OS.

Docker containers are a hardware independent and lightweight manner to run many applications in a virtualized configuration. The Docker technology has been available on x86 platforms for a long time. With Docker containers you get a virtualization solution that does not need a complete virtual machine with an operating system running in it for every application. Instead your application is running in a small container that provides only a limited set of virtualization facilities. Just the things that the application needs. You can run many Docker containers – so applications – on a single real operating system image.

The interesting thing is that in a conceptual way Docker is a quite like z/OS as we have seen it in section Address Spaces are processes. On a z/OS operating system you can run many applications in “Address Spaces”. With Docker you run many container processes on a single real operating system image.

I will talk a bit more about Docker in section Linux in z/OS Container Extensions.

z/OS Address Spaces versus Docker containers

All Unix variants

A small elaboration, as you may already get confused with the Unix on the mainframe. I mentioned Linux for the mainframe, now I talk about z/OS Unix and Linux in Containers.

It is important to understand the difference between z/OS Unix, z/OS Container Extensions and Linux for Z.

z/OS Unix and z/OS Container Extensions are an integral part of z/OS. You get these options with z/OS.

z/OS Unix applications use the Unix flavour that z/OS provides, which is not Linux.

In z/OS Container Extensions you get the option to run applications with the Linux flavour of Unix, and run these in a containerized setup.

Linux for Z is an operating system that is in no way related to z/OS. Applications running on Linux for Z use the Linux flavour, and in an LPAR or Virtual Machine of its own.

I have tried to put all the Unix variants on z/OS in the picture below. You see z/OS Unix, as part of a z/OS operating systems, you see a z/OS container process running a Linux Docker container, and separate from z/OS there is an LPAR running Linux for Z.

The Unix flavours of z/OS and Linux for Z

What’s next

Now we have seen the basics of z/OS, we can turn to the more specialized and specific parts. In the next post I will discuss the unique clustering feature of z/OS, called parallel sysplex.