Skip to content

Java Development Environment

Jump to bottom
Johannes Renner edited this page Aug 19, 2020 · 34 revisions

Prerequisites and Preliminaries

Before it is possible to start working on the code, the following preparations need to be done:

Deployment

Install sumaform and start a virtual machine with the current SUSE Manager Server version.

There are instructions if you don't want to use Terraform but those are not recommended for general development.

Git

  • Add your SSH keys to GitHub

  • Clone the repository:

      git clone git@github.com:uyuni-project/uyuni.git
  git checkout <development_branch_name>

Where <development_branch_name> will be by default:

  • master: upstream Uyuni

Or if you configured the SUSE Manager repository as an additional remote:

  • Manager-4.1: currently released major release, latest version (changes will go into the next maintenance update)
  • Manager-4.0: previous released major release, latest version (changes will go into the next maintenance update)

To create a new topic branch and switch to it use the following command:

git checkout -b master-<topic_name>

Use my-great-feature as the topic name if you work on a feature, or bsc1234567 as a reference for bugs.

Configure the development machine

Install Java, ssh, rsync, ivy, ant, junit-ant and obs-to-maven. On openSUSE you can:

sudo zypper in java-11-openjdk-devel openssh rsync apache-ivy ant ant-junit servletapi5

obs-to-maven can be found in systemsmanagement:Uyuni:Utils. Add the repository for your distribution and then install this package.

  • For SUSE related systems, install the package with the zypper command:

      sudo zypper addrepo obs://systemsmanagement:Uyuni:Utils systemsmanagement:uyuni:utils
  sudo zypper install obs-to-maven

  • For other systems, use your preferred package manager. Full list of supported systems.

To resolve dependencies you need to have a login account in SUSE OBS. After that run the following command to store the OBS credentials in your machine:

osc help

Get the project dependencies from the internal Ivy repository:

cd <path_to_spacewalk>/java
ant -f manager-build.xml ivy

Compile a first version of the branding jar:

ant -f manager-build.xml refresh-branding-jar

When you get an error message like Unable to find a javac compiler;, then you probably need to set the environment variable $JAVA_HOME to the path of the Java 11 OpenJDK libraries. For example:

export JAVA_HOME=/usr/lib64/jvm/java-11-openjdk
debian systems

For debian systems, you need to install java-packages, additionally to the pkg listed. Take the rpm, and convert it, install it with alien https://software.opensuse.org/package/javapackages-tools

Deploying Java code or CSS to the Terraform Virtual Machine

Before deploy to test server, run ant checkstyle

cd <path_to_spacewalk>/java
ant -f manager-build.xml checkstyle

Then, you can deploy Java code with:

ant -f manager-build.xml refresh-branding-jar deploy -Ddeploy.host=suma3pg.tf.local  restart-tomcat restart-taskomatic

Running that will compile the sources, generate a build/webapp directory, copy it over to the virtual machine, make some additional configuration and restart relevant services.

You might want to monitor different logfiles doing something like that (in separate xterms):

    tail -f /var/log/tomcat6/catalina.out
    tail -f /var/log/rhn/rhn_taskomatic_daemon.log
    tail -f /var/log/apache2/error_log
    tail -f /var/log/messages

deploy css to server

Make sure your server has dev_only enabled (sumaform) otherwise it will not work. You dont need to build java if you make a modification only on css/frontend code.

All Frontend Prerequisites should be checked before deploy.

cd spacewalk/java
ant -f manager-build.xml refresh-branding-jar deploy-static-resources -Ddeploy.host=mysuma-devhead.tf.local

If you have Javascript/Node.js errors:

cd /spacewalk/web/html/src
yarn install

Optional: running JUnit tests

JUnit tests get automatically triggered whenever there are commits in the Manager or Manager-X.Y branches. Currently we run unit tests on each PR submitted on spacewalk. ( This are like travis notification, and run on our private jenkins with special containers)

You can also run JUnit tests can on your local machine but need an external database (Oracle or Postgres) to be carried out. (Or you use the container) Tests are expected to cleanup after they finish, so they should not change the database state (except from sequence numbers) during normal operations. Nevertheless, bugs and failed runs can insert random stuff, so you are advised to have a snapshot ready in case you have to roll back your database virtual machine.

IMPORTANT: You can easily start a test database by running the following command make -f Makefile.docker dockerrun_pg on java folder. This requires docker. Some hints for docker:

  • after install docker verify if docker agent is running sudo systemctl status docker
  • your user should be in docker group, to avoid runningit as sudo (may require to restart the machine): sudo usermod -aG docker $USER
  • in file '/etc/docker/daemon.json' (create if not exists) add the following property to be able to connect to suse docker images: "insecure-registries": ["registry.mgr.suse.de"]

Instructions to run tests:

  • configure an rhn.conf file in buildconf/test. JUnit-specific minimal samples are provided for Oracle and Postgres in the same directory;
  • copy spacewalk/spacewalk/config/usr/share/rhn/config-defaults to /usr/share/rhn/config-defaults
  • create a directory /var/log/rhn/.
  • create a directory /srv/susemanager with correct privileges in order to let the user, the one who runs the tests, can write temporary files there.
  • optionally, decide a subset of tests to include, a subset of tests to exclude, or both using globs in buildconf/manager-test-excludes and buildconf/manager-test-includes. By default all tests are included, except those that are broken beyond repair;
  • run ant -f manager-build.xml test.

Optional: General Logging

Edit java/code/src/log4j.properties and add the classes you want to get debug output in catalina.out

log4j.logger.com.redhat.rhn.frontend.action.renderers.HttpProxyRenderer=DEBUG
log4j.logger.com.redhat.rhn.manager.setup.SetupWizardManager=DEBUG

Optional: DWR Logging (AJAX)

Please find instructions about how to setup logging with the DWR library [here] (http://directwebremoting.org/dwr/documentation/server/logging.html). This is especially important since DWR will by default be silent about all application exceptions! A setup with log4j as we use it in SUSE Manager can look like this (add this to log4j.properties):

# DWR logging
log4j.logger.org.directwebremoting.log.accessLog=DEBUG

Further, web.xml needs to be edited as well by e.g. adding the following init parameter to the DWR servlet definition to set a specific log level:

<init-param>
    <param-name>accessLogLevel</param-name>
    <param-value>EXCEPTION</param-value>
</init-param>

Alternatively it is possible to enable a general "debug" mode which will set the accessLogLevel to EXCEPTION by default.

You can also make DWR marshal Java exceptions to Javascript for easier debugging through Firebug or similar. To do that add the following to your dwr.xml:

<convert match="java.lang.Exception" converter="exception"/>
<convert match="java.lang.StackTraceElement" converter="bean"/>

Optional: run Cucumber tests locally

Cucumber tests (defined in their own GitHub project) are automatically triggered every few hours. A mail gets sent to the mailing list when those break.

If you want to test them out yourself, read this documentations here:

Optional: profiling

From time to time you might want to check a certain piece of code for performance reasons. The tool we typically use is the open source VisualVM.

Processor-wise, there are two kinds of analysis that you can do with VisualVM:

  • sampling: VisualVM periodically checks what method the VM is executing and makes time percentage estimations based on the number of counts each method gets. This method has relatively low overhead and low accuracy;
  • profiling: VisualVM keeps track of start and end timestamps for every method in the application, computes wall time estimations based on "real" time data. This method has relatively high accuracy but it adds a large overhead, so in the end it might skew results. Be careful interpreting results!

You can connect a VisualVM instance to a Java process:

  • via a local process connection: VisualVM can connect to a running process on the same machine, this is the most efficient way where applicable;
  • via the JMX network protocol: for a process on another machine;

Then launch VisualVM and use File -> Add JMX Connection... to connect it to <SERVER_NAME>:.

If you are using sumaform, JMX access is already configured on port 3333 for Tomcat and port 3334 for Taskomatic otherwise see the official guide for additional information.

Gotchas:

  • always be sure to do your tests using the same VM vendor/version that you will be deploying to as results might be quite different depending on specific implementation/version optimizations;
  • profiling can only be done from a local connection, so VisualVM must run on the same machine in that case;
  • as of May 2017, profiling only works when the process being profiled runs in OpenJDK, which means changing the conditions in our case as we use IBM Java in SUSE Manager;
  • local connections only see Java processes running from the same VM executable that launched VisualVM (you can change that editing config/visualvm.conf), otherwise you will not see them listed;
  • local connections only see Java processes that were launched by the same user, so be sure to use that same user to run VisualVM;
  • sampling has actually low overhead on openjdk 7 (typically some percent), while it is not so cheap on ibm j9 v6 (typically 200%);
  • beware that all time durations and percentages from VisualVM are "wall time", not "CPU time", so they will include disk access delays, network access, etc.;
  • beware that if VisualVM is slowing your process too much results can also be skewed by the very presence of the profiler. At the first run VisualVM will calibrate itself to try to compensate for that, but it won't be 100% accurate so be sure to intepret results correctly;
  • VisualVM (at least the current Linux version, 1.3.5) does NOT currently profile correctly ANY thread that was started BEFORE the profiler was connected. So be sure to use a breakpoint or other mechanism in order to stop execution until you have VisualVM running, THEN start any thread you want to measure;
  • VisualVM (at least the current Linux version, 1.3.5) does NOT currently work correctly unless it is launched from the bin/ directory. So cd bin; ./visualvm will work while ./bin/visualvm won't!
  • VisualVM (at least the current Linux version, 1.3.5) does NOT currently work if it is launched from IBM j9;
  • VisualVM (at least the current Linux version, 1.3.5) does NOT currently support profiling of IBM j9 processes.

Optional: generating XML RPC docs (aka apidoc)

Uncomment the commons-lang dependency in java/buildconf/ivy/ivy-suse.xml.

Simply run the following within the java folder:

ant -f manager-build.xml ivy apidoc-asciidoc

The other apidoc formats can be generated by calling one of the other ant targets: apidoc-singlepage, apidoc-jsp, apidoc-list, apidoc-html or apidoc-docbook.

To get more debug infos from the template processor, add the -debug parameter to the javadoc additionalparam attribute in java/manager-build.xml and call the ant target again.

Clone this wiki locally