SysGenius-e-Log: How to use Docker Swarm Configs service with WebSphere Liberty Profile

How to use Docker Swarm Configs service with WebSphere Liberty Profile

In order to make your dockerized application portable, you can externalize (Docker container using configuration from outside of Docker image) configuration that changes from one environment to another (from DEV to QA, UAT, Prod etc.). This helps to maintain a generic docker image for your dockerized application and also get rid of most of the bind-mount configuration files and/or environment variables used by your container. Following Docker Swarm services are extremely useful in externalizing the configuration:

Docker Secrets (available in Docker 1.13 and higher version)
Docker Configs (available in Docker 17.06 and higher version)

You can use Docker Secrets to externalize configuration that are confidential in nature, and Docker Configs for general configuration that has potential to be changed from one environment to another.
In this blog post, I will use Dockerized application powered by WebSphere Application Server Liberty Profile (WLP) to show how to use Docker Configs service to externalize server.xml. You can look my other blog - Using Docker Secrets with IBM WebSphere Liberty Profile Application Server, to learn how to use Docker Secrets.

So, here is my server.xml for my WLP application to be used in this post as an example.


<server description="TestWLPApp">

   <featuremanager>

      <feature>javaee-7.0</feature>

      <feature>localConnector-1.0</feature>

      <feature>ejbLite-3.2</feature>

      <feature>jaxrs-2.0</feature>

      <feature>jpa-2.1</feature>

      <feature>jsf-2.2</feature>

      <feature>json-1.0</feature>

      <feature>cdi-1.2</feature>

      <feature>ssl-1.0</feature>

   </featuremanager>

   <include location="/run/secrets/app_enc_key.xml"/>

   <httpendpoint host="*" httpport="9080" httpsport="9443" id="defaultHttpEndpoint"/>

   <ssl clientauthenticationsupported="true" id="defaultSSLConfig" keystoreref="defaultKeyStore"     truststoreref="defaultTrustStore"/>

   <keystore id="defaultKeyStore" location="/run/secrets/keystore.jks" password="{aes}ANGkm5cIca4hoPMh4EUeA4YYqVPAbo4HIqlB9zOCXp1n"/>

   <keystore id="defaultTrustStore" location="/run/secrets/truststore.jks" password="{aes}ANGkm5cIca4hoPMh4EUeA4YYqVPAbo4HIqlB9zOCXp1n"/>

   <applicationmonitor updatetrigger="mbean"/>

   <datasource id="wlpappDS" jndiname="wlpappDS">

      <jdbcdriver libraryref="OracleDBLib"/>

      <properties.oracle password="{aes}AAj/El4TFm/8+9UFzWu5kCtURUiDIV/XKbGY/lT2SVKFij/+H38b11uhjh+Peo/rBA==" url="jdbc:oracle:thin:@192.168.xx.xxx:1752:WLPAPPDB" user="wlpappuser"/>

   </datasource>  

    <library id="OracleDBLib">

       <fileset dir="/apps/wlpapp/shared_lib" includes="ojdbc6-11.2.0.1.0.jar"/>

    </library>

    <webapplication contextRoot="wlpappctx" id="wlpapp" location="/apps/wlpapp/war/wlptest.war" name="wlpapp"/>

</server>

As you can see in above server.xml, the following items were created as Docker Secrets:


<include location="/run/secrets/app_enc_key.xml"/>


<keystore id="defaultKeyStore" location="/run/secrets/keystore.jks" ...


<keystore id="defaultTrustStore" location="/run/secrets/truststore.jks" ...

See, Create Docker Secrets paragraph of Using Docker Secrets with IBM WebSphere Liberty Profile Application Server to create these confidential configuration items.

Once confidential configuration items are created using Docker Secrets, follow the steps below to create general configuration items using Docker Configs.

Connect to Docker UCP using client bundle.
Create configuration item for server.xml using docker config create ...command.
Important: both the client and daemon API must both be at least at version 1.30 to use this command.

$> docker config create dev_wlp_server_config_v1.0 /mnt/nfs/dockershared/wlpapp/server.xml_v1.0 9i5edohyzyrvopuz988caxw4r

Note: here dev_wlp_server_config_v1.0 is configuration item name which gets configuration from /mnt/nfs/dockershared/wlpapp/server.xml_v1.0. I've decided to version the configuration item, so that in future if I need to update the configuration, it becomes easier.

verify that the configuration item created


 $> docker config ls



ID                        NAME                       CREATED        UPDATED

9i5edohyzyrvopuz988caxw4r   dev_wlp_server_config_v1.0     18 seconds ago      18 seconds ago

geuerj6t98d8eeu8nqvvxgtw9   com.docker.license-0       5 days ago     5 days ago

vdzwhpe91iptvuiro654u3lue   com.docker.ucp.config-1    5 days ago     5 days ago

Use configuration item. Below example shows using YAML.

docker-compose.yml


version: "3.3"

services:

   wlpappsrv:

      image: 192.168.56.102/osboxes/wlptest:1.0

      networks:

         - my_hrm_network

      secrets:

         - keystore.jks

         - truststore.jks

         - app_enc_key.xml

      ports:

         - 9080

         - 9443

      configs:

         - source: dev_wlp_server_config_v1.0
           target: /opt/ibm/wlp/usr/servers/defaultServer/server.xml
           mode: 0444

     deploy:

        placement:

           constraints: [node.role == worker]

           mode: replicated

           replicas: 4

           resources:

              limits:

                 memory: 2048M

           restart_policy:

              condition: on-failure

              max_attempts: 3

              window: 6000s

           labels:

              - "com.docker.ucp.mesh.http.9080=external_route=http://mydockertest.com:8080,internal_port=9080"

              - "com.docker.ucp.mesh.http.9443=external_route=sni://mydockertest.com:8443,internal_port=9443"

networks:

   my_hrm_network:

      external:

         name: my_hrm_network

secrets:

   keystore.jks:

      external: true

   truststore.jks:

      external: true

   app_enc_key.xml:

      external: true

configs:
    dev_wlp_server_config_v1.0:

     external: true

Note: if you don't want to create configuration item in advance (step #2 above), you can also specify configuration file in the YAML file itself. Replace external: true in the above example with file: /mnt/nfs/dockershared/wlpapp/server.xml_v1.0

If you want to use use docker service create ... command, instead of YAML file, here is how you can use config


   docker service create \

      --name wlpappsrv \

      --config  source=dev_wlp_server_config_v1.0,target=/opt/ibm/wlp/usr/servers/defaultServer/server.xml,mode=0444 \
 ... \

  192.168.56.102/osboxes/wlptest:1.0

Validate compose file:


$> docker-compose -f docker-compose.yml config

WARNING: Some services (opal) use the 'deploy' key, which will be ignored. Compose does not support 'deploy' configuration - use `docker stack deploy` to deploy to a swarm.
WARNING: Some services (opal) use the 'configs' key, which will be ignored. Compose does not support 'configs' configuration - use `docker stack deploy` to deploy to a swarm.

Deploy the service as a stack:

$> docker stack deploy --compose-file docker-compose.yml dev_WLPAPP

How to refresh/update or rotate configuration

Configuration item created by Docker Configs service is immutable, however, there is a way to rotate configuration. Let's say, you need to update some configuration value in server.xml, like you have to reference to new version of JDBC driver. See the steps below:

Create another configuration item that references to updated server.xml

$>docker config create dev_wlp_server_config_v2.0 \ /mnt/nfs/var/dockershared/dev_PAL/server.xml_v2.0
o4173tet99vuwuz1fma4dqd2j

Update the service so that it references to the newly created configuration item


$>docker service update \

 --config-rm dev_wlp_server_config_v1.0 \

 --config-add  source=dev_wlp_server_config_v2.0,target=/opt/ibm/wlp/usr/servers/defaultServer/server.xml \

 wlpappsrv

[optional] Once the service is fully updated, you can remove the old configuration item:

$> docker config rm dev_wlp_server_config_v1.0

[optional] If you need to see which configuration item is attached to the service, you can run 'docker service inspect <service-name>' command.


$>docker service inspect wlpappsrv



...

"Configs": [

  {

    "File": {

      "Name": "/opt/ibm/wlp/usr/servers/defaultServer/server.xml",

      "UID": "0",

      "GID": "0",

      "Mode": 292

    },

     "ConfigID": "o4173tet99vuwuz1fma4dqd2j",

     "ConfigName": "dev_wlp_server_config_v2.0"

  }

]

...