Best Practice for language definition usage

[mhardin18]

Currently I am working with a customer who has 50k elements in one Endevor system alone. They have about 120 Gen processors that go along with that. Utilizing the language definition feature in zApp is a great way to deal with this issue and consolidate this. However, doing this on a member by member basis is not manageable. For the use case in question, the Gen processors could be narrowed down to a small amount of language definitions. Lets say 5.

So what would be the best way to organize this both from a repository layout, to zApp properties. It would be most beneficial to have this done at the default application settings whenever possible, and then only overrides at an app level as necessary.

[FALLAI-Denis]

Hi,

We ourselves are in the process of disengagement from Endevor to DBB, with source management with Git.

We have chosen to create a technical file per source to build.
This files are created during migration process and contain the informations taken from Endevor at the time of migration:

• System
• Processor
• CCID
• Comment

They are Yaml files validated with a json schema.

Developers must keep technical files up to date. At a minimum change the CCID which is also used in the build and packaging procedures: we filter the sources to build according to the CCID and also to build packages (Artifactory) by CCID.

We modified zAppbuild to use our technical files.

The Endevor processors have been brought back into zAppbuild with a groovy script per language and a configuration file, (language properties file).
I could provide an example for COBOL language.

The modification of a source or the modification of a technical file alone triggers the construction of the source, provided that the CCID carried by the technical file corresponds to the current context, (the CCID is carried by the name of the Git branch).

We made our choices in relation to the habits of developers but also because of change management practices which are based on projects and component-oriented management while Git (and DBB) are based on operation by configurations and incremental development for of an application.

This essential difference between Endevor, which is a component manager, and Git, which is a configuration manager, should not be underestimated.
This was a big problem for us and forced us to heavily modify the way zAppbuild works and use Git in a non-academic way.

To return to the management of build properties, I am not in favor of management based on the distribution of sources in folders, or management based on a source naming plan.
This is too restrictive for developers who prefer to organize themselves in relation to a functional division.
This is why we chose a technical file associated with each source file, with a second extension and placed in the same folder as the source.

[mhardin18]

@FALLAI-Denis
Can you describe your use of CCID within Endevor and why you have (seemingly) kept that paradigm with Git/DBB/zApp ?

I have had very mixed experiences in how organizations are using the CCID field within Endevor. Most often I've seen it to be a driver of the Move processors, But I've also seen it used simply as a tagging system to tie back to operational functions... i.e. These files need to be updated every year for instance.

[FALLAI-Denis]

Hi @mhardin18,

In our context the Endevor CCID is used to identify all the elements that participate in the same change with the aim of promoting and deploying them together.
It is a selection criteria for creating Endevor packages.
It is also a criteria for tracking changes on elements.
1 CCID = 1 evolution or 1 maintenance of an application.

The CCID is a structuring information in the organization of developments and monitoring of changes.
Many internal tools use this information.

We had to maintain this management by CCID in Git because developments and maintenances are not organized in relation to availability/deployment dates.
Several developments/maintenances can be developed in parallel, on the same application, therefore the same Git repository, and each can be deployed independently of the others (as long as they have no elements in common).
It was not possible to create "develop" or "release" consolidation branche in Git due to the fact that each evolution / maintenance has its own life cycle.

We also had to adapt the merge in the "master" branch because it is loaded with elements that were built from a branch dedicated to a CCID, and what was built, tested and deployed on all test environments and production environments must be what we find in the "master" branch.
We therefore make a copy of the elements from the CCID branch on the "master" branch, without merge, (checkout elements), then we consume the commit of the CCID branch with a merge option "-s ours" (which is not a real merge).

All this because of the practices resulting from the use of Endevor (component manager) during more than 30 years which are difficult to replicate in Git (configuration manager).

To use Git in a standard way, we have to change development methods, which was not in our plan to disengage from Endevor (a purely technical project that should have at least an impact on the development teams).

We have therefore adapted the operation of Git so that it is as close as possible to what is practiced with Endevor:

• component promotion and not configuration promotion
• promotion by overwriting not promotion by merge

Note that our developers were just scared when we started talking to them about merge conflict resolution in Git... we were on the verge of refusing to disengage from Endevor and we had to adapt to their way of doing things to work.

[dennis-behm]

A core aim of such a migration project must be not to replicate old development workflows. We have outlined our recommendation in the solution website

Git is not maintaining build configuration attached as metadata to the source, so we need to have a file in the repository to carry the information. Today's implementation of the zAppBuild Language Configurations, requires to map a "member" to its build processor configuration via a languageConfigurationMapping.properties.

As Denis, I am not proposing the specify build parameters via the folder structure. So, no "COBOL_CICS", "COCOL_BATCH" folders, please.

@mhardin18 I understand the hassle with 50k elements within a single mapping file. What about the following enhancement in zAppBuild:

Instead of carrying a single mapping file under application-conf to map all members to its build processor configuration, we can extend this functionality in such a way, that the build framework looks up the mapping in a file within is placed in each source folder, sample:

You could have something like:

<app>/src/cobol
<app>/src/pli
<app>/src/jcl

within each subfolder we add a languageConfigurationMapping.properties. This will lead to more mapping files, but they are represent the mapping for the files inside the subdir.

If these 50k elements could then be grouped by a business functions, like this.

<app>/billing/src/cobol
<app>/billing/src/pli
<app>/billing/src/jcl
<app>/crm/src/cobol 
...

The mapping file will be more manageable.

[mhardin18]

@dennis-behm
I like the idea of a languageConfigurationMapping.properties within a subfolder. This would simplify the overall situation. If one does not exist, go with the defaults. Drive to less application-conf properties and more default or albeit, repository properties.

Certain DevOps admins (formerly Endevor or SCM you like admin) will not like for the developers to be able to alter this file, but that could probably be mitigated by other controls.

I'll have to give this some more thought. Will get back to you after a week of dealing with this exact issue in real-time on a project.

[dennis-behm]

@mhardin18

Please keep in mind that language configurations is one option to override settings in default build configuration.

If a member is not mapped to a language configuration, it is using the defaults from build-conf/.properties files, the applicationDefaultPropFiles configuration or from the language specific defaults in application-conf.
It is not mandatory to map all files to a language configuration; however, mapping each files can be seen as a strategy to statically replicate the previous "type" configuration in the old build system.

See the full documentation at:
https://github.com/IBM/dbb-zappbuild/blob/main/docs/FilePropertyManagement.md#overriding-build-properties-with-dbb-and-zappbuild

[ijmitch]

This whole "Certain DevOps admins (formerly Endevor or SCM you like admin) will not like for the developers to be able to alter this file ..." needs challenging EVERY TIME - it's simply a red-herring since they can change things to their hearts' content, but no change means ANYTHING unless it is accepted in a merge after a pull request. Control comes as part of this deal!

[v1gnesh]

A different perspective is needed when moving to these newer tools for mainframe.
From: "It's not their stuff, they can't change it; so why should they even get to see it."
To: Open, collaborative, but still gated by PRs, CODEOWNERS, and other automated checks.

[ijmitch]

@v1gnesh absolutely this is a change in perspective. I was just considering adding this type of question to this recent blog post but now considering a completely fresh post dedicated to the audit, governance and security questions and not add it to that one.

[ijmitch]

(In my more 'conversational' moments I'd love to use a line like:

you expect the developers to look after code which could change interest rate calculations or could sneak in an advertisement for their brother's restaurant to every account statement (ha-ha!) but you think they can't be exposed to the compiler options?

as a riposte to this 'fear'.)

[mhardin18]

A different perspective is needed when moving to these newer tools for mainframe. From: "It's not their stuff, they can't change it; so why should they even get to see it." To: Open, collaborative, but still gated by PRs, CODEOWNERS, and other automated checks.

This is definitely a challenge. This should be directed more at IBM directly and the entire on-boarding of a migration project to Git/DBB. There are challenges just beyond the technical aspects that I see being missed. Don't give the admiral a corvette without giving them a seasoned corvette captain.