Use unpack to support use of schema plugins as artifacts

Date	May 20 2020	Contacts	Jody Garnett
Status	In progress	Release	3.12
Resources		Ticket #	#4724
Source code	#4701
Funding	GeoCat, GeoNetwork Enterprise and professional services

Overview

Build change to package up both the plugin classes (as a jar) and resources (as a zip). To prevent conflict schema-plugin version numbering changed to match core-geonetwork. Build system changed to unpack the zip artifacts, rather than copy content between folders.

Technical Details:

Initially proposed as a simple change to take advantage public OSGeo repository; this change has shown several limitations of the core-geonetwork build system for schema-plugins:

Use of schema-plugin version 3.7 across different branches of core-geonetwork, combined with transitive dependencies, ensure conflicts across active branches of core-geonetwork.
web app build copies contents folder to folder, rather than make use of schema plugin artifact zip and jar.
web app contains competing build configurations (for both war:war and jetty:run) on the management of schema-plugins leading to build instability

The above limitations result in an unstable build environment.

Schema Plugin Changes

Example pom.xml making use of maven-assembly-plugin and core-geonetwork project.version number from parent:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

 <parent>
    <artifactId>schemas</artifactId>
    <groupId>org.geonetwork-opensource.schemas</groupId>
    <version>3.11.0-SNAPSHOT</version>
  </parent>
  <modelVersion>4.0.0</modelVersion>

  <artifactId>schema-iso19115-3.2018</artifactId>
  <name>GeoNetwork schema plugin for ISO19115-3:2018 standard</name>
  <description>
    ISO/TC 211 industry standard for geospatial metadata providing information about the identification, the extent, the quality, the spatial and temporal aspects, the content, the spatial reference, the portrayal, distribution, and other properties of digital geographic data and services.
  </description>

  <dependencies>
    <dependency>
      <groupId>org.geonetwork-opensource.schemas</groupId>
      <artifactId>schema-core</artifactId>
      <version>${project.version}</version>
    </dependency>
  </dependencies>

  <build>
    <resources>
      <resource>
        <directory>src/main/config/translations</directory>
        <targetPath>META-INF/catalog/locales</targetPath>
      </resource>
      <resource>
        <directory>src/main/resources</directory>
      </resource>
    </resources>
    <plugins>
      <plugin>
        <artifactId>maven-jar-plugin</artifactId>
        <executions>
          <execution>
            <id>test-jar</id>
            <goals>
              <goal>test-jar</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
      <plugin>
        <artifactId>maven-assembly-plugin</artifactId>
        <executions>
          <execution>
            <id>plugin-assembly</id>
            <phase>package</phase>
            <goals><goal>single</goal></goals>
            <inherited>false</inherited>
            <configuration>
             <appendAssemblyId>false</appendAssemblyId>
             <descriptors>
              <descriptor>src/assembly/schema-plugin.xml</descriptor>
             </descriptors>
            </configuration>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
</project>

Example schema-plugin.xml to package plugins folder:

<assembly xmlns="http://maven.apache.org/ASSEMBLY/2.1.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/ASSEMBLY/2.1.0 http://maven.apache.org/xsd/assembly-2.1.0.xsd">
  <id>plugin</id>
  <includeBaseDirectory>false</includeBaseDirectory>
  <formats>
    <format>zip</format>
  </formats>
  <fileSets>
    <fileSet>
      <directory>src/main/plugin/</directory>
      <outputDirectory></outputDirectory>
      <useDefaultExcludes>true</useDefaultExcludes>
    </fileSet>
  </fileSets>
</assembly>

See 4701 schemas/README.md for step by step instructions.

Build changes

As much as possible these build changes respect developers current workflow, using different maven plugins to accomplish the same goal (see web/README.md for details).

mvn jetty:run -Penv-dev

To pick up a change build your schema plugin using mvn install and update the running jetty application with:

mvn process-resources

A fallback option has been provided using the prior technique of copying files between folders:

mvn process-resources -DschemasCopy=true

To include additional schema plugins into schemas/pom.xml:

<profiles>
  <profile>
    <id>schema-iso19139.xyz</id>
    <activation>
    <file>
      <exists>iso19139.xyz</exists>
    </file>
    </activation>
    <modules>
      <module>iso19139.xyz</module>
    </modules>
  </profile>
</profiles>

To include additional schema plugin in web/pom.xml:

<profile>
   <id>schema-iso19115-xyz</id>
   <activation>
     <property><name>schemasCopy</name><value>!true</value></property>
     <file><exists>../schemas/iso19115-3.2018</exists></file>
   </activation>
   <dependencies>
     <dependency>
       <groupId>org.geonetwork-opensource.schemas</groupId>
       <artifactId>schema-iso19139.xyz</artifactId>
       <version>${project.version}</version>
     </dependency>
   </dependencies>
   <build>
     <plugins>
       <plugin>
         <groupId>org.apache.maven.plugins</groupId>
         <artifactId>maven-dependency-plugin</artifactId>
         <executions>
           <execution>
             <id>iso19115-xyz-resources</id>
             <phase>process-resources</phase>
             <goals><goal>unpack</goal></goals>
             <configuration>
               <encoding>UTF-8</encoding>
               <artifactItems>
                 <artifactItem>
                   <groupId>org.geonetwork-opensource.schemas</groupId>
                   <artifactId>schema-iso19115-xyz</artifactId>
                   <type>zip</type>
                   <overWrite>false</overWrite>
                   <outputDirectory>${schema-plugins.dir}</outputDirectory>
                 </artifactItem>
               </artifactItems>
               </artifactItems>
             </configuration>
           </execution>
         </executions>
       </plugin>
     </plugins>
   </build>
 </profile>

This approach automatically builds the associated plugin if the folder iso19139.xyz is present, use of add-schema.sh automates this change, see 4701 schemas/README.md.

As a result of this proposal there is no requirement to build metadata101 plugins as part of a local core-geonetwork build. Automatic activation is provided to preserve current developer workflow, if building a metadata101 module in isolation use of mvn jetty:run -Pschema-iso19115-xyz can be used to try it out.

Proposal Type:

Type: Improvement to existing feature
Module: schema plugins, metadata101, webapp, root

Voting History

This activity was covered by the Bolsena 2020 code sprint, see Bolsena 2020 Online slides for discussion and decisions.

Decision on version numbering:
- geonetwork: major/minor/revision format
- “3.10.4-SNAPSHOT”
Decision on schema-plugin version:
- Change: Use geonetwork version above
- Resolves transitive instability
Decision on schema-plugin packaging:
- publish both jar and zip to osgeo repository
Decision on metadata101/iso19115-3.2018
- Fold into core-geonetwork
- aside: Can be handled as separate pull request
Vote Proposed: TBA