This little scripting enables you to run a fully fledged DX environment with minimal footprint on your local machine. It uses docker-compose to start/stop and manage Docker containers. Docker-compose an addon on top of Docker. On Mac OS Docker desktop docker-compose is available out of the box. On other OS, you might need to manually install docker-compose even if you have docker installed already. For installation instructions see: https://docs.docker.com/compose/install/
Start by cloning this repository locally and cd into the dx-docker-compose directory.
All you need to do is to load the HCL DX docker images into your local docker repository and set up your local environment with some environment variables.
The load.sh script expects a path to a directory containing the docker image archives as a command line argument.
NOTE: If you already loaded the DX docker images into a docker repository of your choice, you may skip executing load.sh or load.bat.
Please make sure to update the image names in the dx.properties file appropriately.
Linux/MAC:
cd ./dx-docker-compose
bash load.sh <docker-image-archives-directory>Windows:
cd ./dx-docker-compose
load.bat <docker-image-archives-directory>If the docker compose is not running on local, then DX_HOSTNAME value in set.sh/set.bat needs to be modified accordingly.
Linux/MAC:
cd ./dx-docker-compose
source ./set.shWindows:
cd ./dx-docker-compose
set.batNOTE: The second command is source ./set.sh and not just executing set.sh directly.
If you want to unset your DX docker-compose environment, you can do so by running unset.sh:
Linux/MAC:
cd ./dx-docker-compose
source ./unset.shWindows:
cd ./dx-docker-compose
unset.batAfter setting your environment, you can start the DX docker-compose environment by running. Important is that you need to be using a minimum version 1.27.4 for docker-compose.
docker-compose upThis will start all services defined in dx.yaml and logs will be printed directly go to your bash.
You can stop docker-compose in this situation by pressing CTRL+C.
If your user does not have permission to write to the persistent volumes location (folder dx-docker-compose/volumes) specified in the docker-compose file dx.yaml, you will see errors and the system will not start properly. If necessary, change the permissions of this folder so that the user running the docker process can read from and write to it.
Here are some useful command line arguments to run docker-compose up:
-d, --detach: detached mode--remove-orphans: this cleans up orphaned containers
For more information on startup parameters for docker-compose up, please see https://docs.docker.com/compose/reference/up/.
If you didn't start docker-compose in detached mode, you can stop by pressing CTRL+C.
If you started docker-compose in detached mode, you can stop your environment by issuing
docker-compose stopThis will securely stop all running docker containers. If you want to properly clean up your system and even purge stopped docker containers, you can do so by issuing
docker-compose downIf you want to look at logs for all of the DX services, you can easily do so by running
docker-compose logsThis will show you all system out logs of all services of all running containers (might be quite a lot - see tips and tricks below).
You can also look at CPU, memory and network consumption using
docker statsExample output:
NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
dx-haproxy 0.00% 7.008MiB / 31.21GiB 0.02% 11.1MB / 10.8MB 0B / 0B 9
dx-dam 0.27% 493.4MiB / 31.21GiB 1.54% 263MB / 381MB 8.19kB / 19.5kB 78
dx-dam-db-pool 0.05% 116.8MiB / 31.21GiB 0.37% 586MB / 659MB 0B / 1.44MB 36
dx-ringapi 0.17% 104.2MiB / 31.21GiB 0.33% 1.45MB / 1.2MB 0B / 24.1kB 23
dx-dam-db-node-0 0.07% 63.26MiB / 31.21GiB 0.20% 414MB / 233MB 0B / 15.6MB 14
dx-core 0.80% 2.137GiB / 31.21GiB 6.85% 1.62MB / 6.71MB 436MB / 563MB 375
dx-cc 0.19% 71.73MiB / 31.21GiB 0.22% 7.7kB / 0B 0B / 11.3kB 23
dx-image-processor 0.17% 426.3MiB / 31.21GiB 1.33% 17.5MB / 4.27MB 0B / 23kB 23
To get an overview of running docker-compose services, you can run
docker-compose psExample output:
IMAGE COMMAND CREATED STATUS PORTS NAMES
hcl/dx/haproxy:v1.0.0_20220713-0158 "/bin/bash entrypoin…" 3 days ago Up 3 days 0.0.0.0:80->80/tcp, :::80->80/tcp, 0.0.0.0:443->443/tcp, :::443->443/tcp dx-haproxy
hcl/dx/digital-asset-manager:v1.12.0_20211213-1448 "/bin/bash entrypoin…" 3 days ago Up 3 days 0.0.0.0:80->8081/tcp, :::80->8081/tcp dx-dam
hcl/dx/persistence-connection-pool:v1.13.0_20211213-1457 "/scripts/entrypoint…" 3 days ago Up 3 days (healthy) 0.0.0.0:5432->5432/tcp, :::5432->5432/tcp dx-dam-db-pool
hcl/dx/ringapi:v1.13.0_20211213-1457 "/opt/app/start_all_…" 3 days ago Up 3 days 0.0.0.0:4000->3000/tcp, :::4000->3000/tcp dx-ringapi
hcl/dx/persistence-node:v1.3_20211213-1454 "/start_postgres.sh" 3 days ago Up 3 days (healthy) 0.0.0.0:5433->5432/tcp, :::5433->5432/tcp dx-dam-db-node-0
hcl/dx/core:v95_CF200_20211213-1442 "sh -c /opt/app/entr…" 3 days ago Up 3 days 0.0.0.0:7777->7777/tcp, :::7777->7777/tcp, 0.0.0.0:10020->10020/tcp, :::10020->10020/tcp, 10032/tcp, 0.0.0.0:10033->10033/tcp, :::10033->10033/tcp, 10034-10038/tcp, 0.0.0.0:10039->10039/tcp, :::10039->10039/tcp, 10040/tcp, 0.0.0.0:10041->10041/tcp, :::10041->10041/tcp, 10042/tcp, 0.0.0.0:10200->10200/tcp, :::10200->10200/tcp, 0.0.0.0:10202-10203->10202-10203/tcp, :::10202-10203->10202-10203/tcp, 10201/tcp dx-core
hcl/dx/content-composer:v1.13.0_20211213-1443 "/opt/app/start_all_…" 3 days ago Up 3 days 0.0.0.0:5000->3000/tcp, :::5000->3000/tcp dx-cc
hcl/dx/image-processor:v1.13.0_20211213-1446 "/home/dx_user/start…" 3 days ago Up 3 days 0.0.0.0:3500->8080/tcp, :::3500->8080/tcp dx-image-processor
The performance for local docker volumes on some operating systems like Mac OS and Windows is quite slow. To improve especially the startup time of DX Core, you may choose to remove the persistent volume configuration. To do so, remove the following lines from the docker-compose file:
volumes:
- ./volumes/core/wp_profile:/opt/HCL/wp_profileNOTE: When removing the volumes at all, any change you apply in DX Core will not be persisted. All your changes will be lost as soon as the container is stopped. If you need any specific folders or files been persistent, then please try to reduce it as much as possible. For example by just saving the server config of the WebSphere_Portal JVM to your local volume.
It is possible to enable debugging mode on the WebSphere_Portal JVM to monitor/debug the whole server process.
For that, please make sure to enable port 7777 for the core service and in your firewall. Below you will find an example to expose that port 7777 in the dx.yaml file:
services:
core:
container_name: dx-core
image: ${DX_DOCKER_IMAGE_CORE:?'Missing docker image environment parameter'}
ports:
- "7777:7777" In addition to that it is needed to enable the WebSphere Application Server debugging services in the IBM Integrated Solutions Console (admin console). For details, please check: Debugging Service details
NOTE: Make sure that the changes are persistent for the current dx-core container. A WebSphere_Portal server restart is needed, as soon as the debugging services are enabled!
After debugging is enabled you can use any IDE like Microsoft Visual Studio Code, Eclipse or IBM Rational Application Developer to connect to that remote debugging port. For details, please check: HDX-DEV-300 HCL Digital Experience for Developers (Advanced)
HCL highly recommends to add shared libraries in the folder /opt/HCL/wp_profile/PortalServer/sharedLibrary/ of the dx-core container to avoid performance problems with shared libraries.
The core of a docker-compose environment are its services.
In the case of DX, each of the different DX components (Core, CC, DAM, ...) is a individual docker-compose service.
The services are all described and configured in dx.yaml.
Amongst other configurations, each service has a external port defined.
Inside a docker-compose environment all containers of a particular service are reachable via their service name. If you connect into a docker container running in docker-compose, you'll be able to resolve the service name via dns. You could do so by just pinging the image processor (service name "image-processor") from any other container. See below on how to bash into a docker-compose container.
In the case that you already have a fully configured DX Core (e.g. an on premise installation) up and running, you can choose to configure docker-compose to connect to the on premise environment.
The below mentioned changes in dx.yaml need to be applied to make this work.
NOTE: You will also have to configure your DX Core environment to connect to the services running docker-compose (e.g. configuration of DAM and Content Composer portlets). Please have a look in the official HCL DX Help Center to understand which changes need to be done, if necessary.
Update the Ring API service configuration as described:
-
Disable the
depends_onparameter.ringapi: # depends_on: # - core
-
Update the
PORTAL_HOSTparameter values.
environment:
- PORTAL_HOST=example.comThe result of the changes to the ringapi service should look similar to the snippet below:
ringapi:
# depends_on:
# - dx-core
image: ${DX_DOCKER_IMAGE_RINGAPI:?'Missing docker image environment parameter'}
environment:
- DEBUG=ringapi-server:*
- PORTAL_PORT=10039
- PORTAL_HOST=example.com
ports:
- "4000:3000"
networks:
- defaultUpdate the Content Composer service configuration as described:
environment:
- PORTAL_HOST=example.comdocker-compose up allows you to start only individual services.
To only start the DAM service, you could run
docker-compose up -d damFor more information see https://docs.docker.com/compose/reference/up/
docker-compose stop allows you to stop only individual services.
To only stop the DAM service, you could run
docker-compose stop damFor more information see https://docs.docker.com/compose/reference/down/
To only look at logs for an individual service you can run
docker-compose logs damFor more information see https://docs.docker.com/compose/reference/logs/
To install CC, DAM and DXConnect applications in DX Core and to enable , you can run
Linux/MAC:
cd ./dx-docker-compose
source ./installApps.shWindows:
cd ./dx-docker-compose
installApps.batNOTE: For any change in DX_HOSTNAME it's a must to re-execute installApps.sh / installApps.bat
To access your dx environment, navigate to http://<PORTAL_HOST>/wps/portal
Example: http://example.com/wps/portal
To access dx admin console, navigate to https://<PORTAL_HOST>:10041/ibm/console
Example: https://example.com:10041/ibm/console
To access the ConfigWizard Server Admin console https://<PORTAL_HOST>:10203/ibm/console
Example: https://example.com:10203/ibm/console
To bash into a docker container of a service, you can directly connect using the service name
docker-compose exec dam bashTo connect into a specific container of a service (if there is multiple containers running for a service), you have to look up the name of the container e.g. using docker-compose ps and then run
docker exec -it dx_dam bashTo perform checks to the mounted volumes, you can directly connect using the dx-prereqs-checker container
docker-compose exec prereqs-checker /bin/bash /usr/local/sbin/run_test.shTo display the logs of the check results, run
docker-compose logs prereqs-checkerPlease check the course content:
HDX-DEV-300 HCL Digital Experience for Developers (Advanced)
- Always use
docker-compose upcommand to start your environment - Make sure that the installApps.sh / installApps.bat script is already completed, before accessing the environment. Only then the additional extentions like Content Composer and DAM will be available!
- Try to avoid using volumes (at best don't use volumes at all to get the best speed)
- Avoid restarting the dx-core container for performance reasons (If needed, just stop/start the server directly in the dx-core container)
- Add shared libraries into folder
/opt/HCL/wp_profile/PortalServer/sharedLibrary/of the dx-core container - In the Web-Browser:
- Access your Portal environment with URL:
http://<hostname>/wps/portal. - Don't access the portal over the direct port! (for example:
http://localhost:10039/wps/portal). If you do so, then you might not be able to access addons like the Content Composer or DAM. The whole communication works via the embeded http-proxy! - By default accessing the portal server environment via https is not configured out of the box. Additional steps need to be done to enable SSL.
- Access your Portal environment with URL:
OpenSSL need to be installed, if own self-signed certificates will be used. Installation instructions can be found at Install openSSL.
NOTE:
The ssl folder contains a localhost.pem file that can be used with the haproxy service running on a local environment. The certificate is created for the hostname localhost and it never expires. Please modify the create_certificates scripts for your needs, if you want to use your own certificates.
-
Create own self signed certificates
Navigate to the ssl folder and execute the create_certificates scriptWindows:
create_certificates.bat
Linux/Mac:
./create_certificates.sh
-
Change the haproxy.cfg file the following section:
frontend dx bind :8081
to use:
frontend dx mode http bind :8083 ssl crt /etc/ssl/private/localhost.pem bind :8081 http-request redirect scheme https unless { ssl_fc }
NOTE:
The parameter:http-request redirect scheme https unless { ssl_fc }automatically redirects http requests into https. It can be removed, if you want to allow http requests as well. -
Modify the dx.yaml and change the haproxy service from:
haproxy: image: ${DX_DOCKER_IMAGE_HAPROXY:?'Missing docker image environment parameter'} container_name: dx-haproxy volumes: - ./haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg ports: - 80:8081 networks: - default
to:
haproxy: image: ${DX_DOCKER_IMAGE_HAPROXY:?'Missing docker image environment parameter'} container_name: dx-haproxy volumes: - ./haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg - ./ssl/localhost.pem:/etc/ssl/private/localhost.pem ports: - 80:8081 - 443:8083 networks: - default
-
Run
docker-compose upto start the environment -
Install or update CC, DAM and DXConnect applications in DX Core to use SSL
Linux/MAC:
./installApps_SSL_Enabled.sh
Windows:
installApps_SSL_Enabled.bat
-
Restart the dx-core container