Showing posts with label Docker Swarm Secrets. Show all posts
Showing posts with label Docker Swarm Secrets. Show all posts

How to use Docker Swarm Configs service with WebSphere Liberty Profile


   In order to make your dockerized application portable, you can externalize (Docker container using configuration from outside of Docker image) configuration that changes from one environment to another (from DEV to QA, UAT, Prod etc.). This helps to maintain a generic docker image for your dockerized application and also get rid of most of the bind-mount configuration files and/or environment variables used by your container. Following Docker Swarm services are extremely useful in externalizing the configuration:
  • Docker Secrets (available in Docker 1.13 and higher version)
  • Docker Configs (available in Docker 17.06 and higher version)
You can use Docker Secrets to externalize configuration that are confidential in nature, and Docker Configs for general configuration that has potential to be changed from one environment to another.
In this blog post, I will use Dockerized application powered by WebSphere Application Server Liberty Profile (WLP) to show how to use Docker Configs service to externalize server.xml. You can look my other blog - Using Docker Secrets with IBM WebSphere Liberty Profile Application Server, to learn how to use Docker Secrets.


So, here is my server.xml for my WLP application to be used in this post as an example.

<server description="TestWLPApp">
   <featuremanager>
      <feature>javaee-7.0</feature>
      <feature>localConnector-1.0</feature>
      <feature>ejbLite-3.2</feature>
      <feature>jaxrs-2.0</feature>
      <feature>jpa-2.1</feature>
      <feature>jsf-2.2</feature>
      <feature>json-1.0</feature>
      <feature>cdi-1.2</feature>
      <feature>ssl-1.0</feature>
   </featuremanager>
   <include location="/run/secrets/app_enc_key.xml"/>
   <httpendpoint host="*" httpport="9080" httpsport="9443" id="defaultHttpEndpoint"/>
   <ssl clientauthenticationsupported="true" id="defaultSSLConfig" keystoreref="defaultKeyStore"     truststoreref="defaultTrustStore"/>
   <keystore id="defaultKeyStore" location="/run/secrets/keystore.jks" password="{aes}ANGkm5cIca4hoPMh4EUeA4YYqVPAbo4HIqlB9zOCXp1n"/>
   <keystore id="defaultTrustStore" location="/run/secrets/truststore.jks" password="{aes}ANGkm5cIca4hoPMh4EUeA4YYqVPAbo4HIqlB9zOCXp1n"/>
   <applicationmonitor updatetrigger="mbean"/>
   <datasource id="wlpappDS" jndiname="wlpappDS">
      <jdbcdriver libraryref="OracleDBLib"/>
      <properties.oracle password="{aes}AAj/El4TFm/8+9UFzWu5kCtURUiDIV/XKbGY/lT2SVKFij/+H38b11uhjh+Peo/rBA==" url="jdbc:oracle:thin:@192.168.xx.xxx:1752:WLPAPPDB" user="wlpappuser"/>
   </datasource>  
    <library id="OracleDBLib">
       <fileset dir="/apps/wlpapp/shared_lib" includes="ojdbc6-11.2.0.1.0.jar"/>
    </library>
    <webapplication contextRoot="wlpappctx" id="wlpapp" location="/apps/wlpapp/war/wlptest.war" name="wlpapp"/>
</server>

As you can see in above server.xml, the following items were created as Docker Secrets:


  • <include location="/run/secrets/app_enc_key.xml"/>

  • <keystore id="defaultKeyStore" location="/run/secrets/keystore.jks" ...

  • <keystore id="defaultTrustStore" location="/run/secrets/truststore.jks" ...


See, Create Docker Secrets paragraph of  Using Docker Secrets with IBM WebSphere Liberty Profile Application Server to create these confidential configuration items.

Once confidential configuration items are created using Docker Secrets, follow the steps below to create general configuration items using Docker Configs.
  1. Connect to Docker UCP using client bundle. 
  2. Create configuration item for server.xml using docker config create ...command.
    Important: both the client and daemon API must both be at least at version 1.30 to use this command.

    $> docker config create dev_wlp_server_config_v1.0 /mnt/nfs/dockershared/wlpapp/server.xml_v1.0

    9i5edohyzyrvopuz988caxw4r

    Note: here dev_wlp_server_config_v1.0 is configuration item name which gets configuration from /mnt/nfs/dockershared/wlpapp/server.xml_v1.0. I've decided to version the configuration item, so that in future if I need to update the configuration, it becomes easier.

  3. verify that the configuration item created

     $> docker config ls

    ID                        NAME                       CREATED        UPDATED
    9i5edohyzyrvopuz988caxw4r dev_wlp_server_config_v1.0 18 seconds ago 18 seconds ago
    geuerj6t98d8eeu8nqvvxgtw9 com.docker.license-0       5 days ago     5 days ago
    vdzwhpe91iptvuiro654u3lue com.docker.ucp.config-1    5 days ago     5 days ago

  4. Use configuration item. Below example shows using YAML.

    docker-compose.yml
    version: "3.3"
    services:
       wlpappsrv: 

          image: 192.168.56.102/osboxes/wlptest:1.0
          networks:
             - my_hrm_network
          secrets:
             - keystore.jks
             - truststore.jks
             - app_enc_key.xml
          ports:
             - 9080
             - 9443
          configs:
             - source: dev_wlp_server_config_v1.0
               target: /opt/ibm/wlp/usr/servers/defaultServer/server.xml
               mode: 0444

         deploy:
            placement:
               constraints: [node.role == worker]
               mode: replicated
               replicas: 4
               resources:
                  limits:
                     memory: 2048M
               restart_policy:
                  condition: on-failure
                  max_attempts: 3
                  window: 6000s
               labels:
                  - "com.docker.ucp.mesh.http.9080=external_route=http://mydockertest.com:8080,internal_port=9080"
                  - "com.docker.ucp.mesh.http.9443=external_route=sni://mydockertest.com:8443,internal_port=9443"
    networks:
       my_hrm_network:
          external:
             name: my_hrm_network
    secrets:
       keystore.jks:
          external: true
       truststore.jks:
          external: true
       app_enc_key.xml:
          external: true
    configs:
        dev_wlp_server_config_v1.0:
         external: true

    Note: if you don't want to create configuration item in advance (step #2 above), you can also specify configuration file in the YAML file itself. Replace external: true in the above example with file: /mnt/nfs/dockershared/wlpapp/server.xml_v1.0

    If you want to use use docker service create ... command, instead of YAML file, here is how you can use config

    docker service create \
     --name wlpappsrv \
     --config  source=dev_wlp_server_config_v1.0,target=/opt/ibm/wlp/usr/servers/defaultServer/server.xml,mode=0444 \
     ... \
     192.168.56.102/osboxes/wlptest:1.0

  5. Validate compose file:
    $> docker-compose -f docker-compose.yml config
    WARNING: Some services (opal) use the 'deploy' key, which will be ignored. Compose does not support 'deploy' configuration - use `docker stack deploy` to deploy to a swarm. WARNING: Some services (opal) use the 'configs' key, which will be ignored. Compose does not support 'configs' configuration - use `docker stack deploy` to deploy to a swarm.

  6. Deploy the service as a stack:
    $> docker stack deploy --compose-file docker-compose.yml dev_WLPAPP


How to refresh/update or rotate configuration


Configuration item created by Docker Configs service is immutable, however, there is a way to rotate configuration. Let's say, you need to update some configuration value in server.xml, like you have to reference to new version of JDBC driver.  See the steps below:
  1. Create another configuration item that references to updated server.xml
    $>docker config create dev_wlp_server_config_v2.0 \
      /mnt/nfs/var/dockershared/dev_PAL/server.xml_v2.0



    o4173tet99vuwuz1fma4dqd2j

  2. Update the service so that it references to the newly created configuration item
    $>docker service update \
     --config-rm dev_wlp_server_config_v1.0 \
     --config-add  source=dev_wlp_server_config_v2.0,target=/opt/ibm/wlp/usr/servers/defaultServer/server.xml
    \
     wlpappsrv

  3. [optional] Once the service is fully updated, you can remove the old configuration item:
    $> docker config rm dev_wlp_server_config_v1.0

  4. [optional] If you need to see which configuration item is attached to the service, you can run 'docker service inspect <service-name>' command.
    $>docker service inspect wlpappsrv

    ...
    "Configs": [
      {
        "File": {
          "Name": "/opt/ibm/wlp/usr/servers/defaultServer/server.xml",
          "UID": "0",
          "GID": "0",
          "Mode": 292
        },
         "ConfigID": "o4173tet99vuwuz1fma4dqd2j",
         "ConfigName": "dev_wlp_server_config_v2.0"
      }
    ]
    ...


For more information about Docker Swarm Configs service, review the following Docker documentations: