DataPower Front-End for WAS Cluster with Load Balancing and SSO

In this blog post, I'm guiding through simple and easy to follow steps to create a Proof of Concept (POC) environment with Datapower (DP) as font-end and  application serving environment hosted on WebSphere Application Server (WAS) based cluster with Single Sign On (SSO) enabled. DP will also load-balance requests to WAS cluster members. For details of DP supported features as front-end, refer to https://www.ibm.com/support/knowledgecenter/en/SSAW57_8.5.5/com.ibm.websphere.nd.multiplatform.doc/ae/cwsv_topology.html

Assumptions:
  1. In order for you to follow the steps outlined below, you have good working knowledge of IBM WAS and DataPower.
  2. You have WAS ND (version 8.5.X), and IBM DataPower Gateway (version 7.5.X) installed and ready to create Application Serving environment.  For your reference, I'm using the following:
    •    WAS 8.5.5.7 Network Deployment.
    •    IBM DataPower Gateway Virtual Edition (Firmware: IDG.7.5.1.0).
In a nutshell, here is what we'll do as part of this exercise:

On WAS side:

  • Create a cluster with two members (application servers).
  • Create security domain and assign scope to the cluster level
  • Create and configure an external trusted realm for the security domain. It is because, we will not have any user registry configured (for Application authentication) and we'll trust identifies authenticated by DP.
  • Enable Application security on the security domain
  • Deploy a sample application on the WAS cluster
  • Map security roles and authorize any user belonging to External Trusted Realm.
  • Configure LTPA, export LTPA key and customize (as we are using External Trusted Realm) it.
  • Export SSL certificate.

On DP side:

  • Import LTPA key, import WAS SSL certificate.
  • Create DP objects (including Crypto, SSL profiles, Password aliase etc).
  • Create Load Balancer Group
  • Create Multi-Protocol Gateway (MPGW) service etc.

Let's start working on WAS side:

Note:
Once you make configuration changes on WAS either using WAS admin console or wsadmin script,
you need to save the changes and synchronise the changes with the nodes (in Network Deployment (ND). Since it's essential part for every change or set of changes, I'm not going to list these two steps after every change below. But you need to make sure to save and synchronise. You can use
the wsadmin Jython script below to save and synchronize the changes with nodes:
# Save the configuration/change
AdminConfig.save()
# Synchronise the changes with the nodes
dmgrObj=AdminControl.queryNames('WebSphere:type=DeploymentManager,*')
AdminControl.invoke(dmgrObj, 'multiSync', '[false]', '[java.lang.Boolean]')

1.0) Create a WAS cluster containing two members. In this example, we will create a WAS cluster named "sso_clus" with two members "ssoSrv1" and "ssoSrv2". For more info on creating WAS cluster, refer to https://www.ibm.com/support/knowledgecenter/SSRMWJ_6.0.0.10/com.ibm.isim.doc/installing/tsk/tsk_ic_ins_was_85_cluster.htm
Note: You may need to change text in italic as per your configuration.
# Using WAS Admin Console: Servers --> Clusters --> WebSphere application server clusters
# Using wsadmin script:
AdminTask.createCluster('[-clusterConfig [-clusterName sso_clus -preferLocal true]]')

# Add 1st cluster member 'ssoSrv1' using default server template
AdminTask.createClusterMember('[-clusterName sso_clus -memberConfig [-memberNode myWASHostNode01 -memberName ssoSrv1 -memberWeight 2 -genUniquePorts true -replicatorEntry false] -firstMember [-templateName default -nodeGroup DefaultNodeGroup -coreGroup DefaultCoreGroup -resourcesScope cluster]]')

# Add  2nd cluster member 'ssoSrv2'
AdminTask.createClusterMember('[-clusterName sso_clus -memberConfig [-memberNode myWASHostNode01 -memberName ssoSrv2 -memberWeight 2 -genUniquePorts true -replicatorEntry false]]')

2.0) Enable SSO (if not already). Here you need to know request(s) from what domain(s) you are supporting, you can have multiple domains defined here. For detail how domain works refer to: https://www.ibm.com/support/knowledgecenter/en/SSAW57_8.5.5/com.ibm.websphere.nd.doc/ae/tsec_msso.html. For this example, we are going to use 'UseDomainFromURL'. See the above URL for details.
# Configure Single Sign On (SSO)
AdminTask.configureSingleSignon('-enable true -requiresSSL true -domainName UseDomainFromURL -attributePropagation true')

# Make sure LTPA is active authentication mechanism:
AdminTask.setActiveAuthMechanism('-authMechanismType LTPA')

3.0) Disable automatic LTPA key generation option as follows:
  Access WAS Admin console and navigate to Security > Global security > LTPA
   > Click on link "Key set groups"
   > Click on link "CellLTPAKeySetGroup"
   > Uncheck "Automatically generate keys"
  Apply, save and synchronise.

4.0) Create security domain. Creating security domain is encouraged rather than modifying the Global security configuration, because, you may have other application serving environment (s) managed under this Cell and they may rely on Global security and any change may affect them.
For this example, we name our new security domain 'mySSOLTPADomain' and we'll use the Global security as template (basically copying the Global security) to create the new security domain and modifying few things on newly created security domain:
Using WAS Admin Console: Security > Global security > Security domains > New...
# Using wsadmin script:
AdminTask.copySecurityDomainFromGlobalSecurity('-securityDomainName mySSOLTPADomain')


5.0) Customise the newly created security domain as per our requirement.
5.1) Since, we will not have any user registry configured on WAS for application authentication, and we'll trust the credential authenticated by DP, we'll create a trusted external realm called 'DatapowerRealm' and add that to newly created security domain. For detail about trusted external realm, Refer to https://www.ibm.com/support/knowledgecenter/en/SSAW57_8.0.0/com.ibm.websphere.nd.doc/info/ae/ae/tsec_sec_realm_trust.html
Using WAS Admin console: Security domains > mySSOLTPADomain > Federated repositories > Trusted authentication realms - inbound > New...
Using wsadmin script:
AdminTask.addTrustedRealms('[-securityDomainName mySSOLTPADomain -communicationType inbound -realmList DatapowerRealm]')
# And only trust the realm(s) that is explicitly indicated (not all)
Using WAS Console: Security domains > mySSOLTPADomain > Federated repositories > Trusted authentication realms - inbound
Using wsadmin script:
AdminTask.configureTrustedRealms('[-communicationType inbound -securityDomainName mySSOLTPADomain -trustAllRealms false]')
Using WAS Admin console: Security domains > mySSOLTPADomain
Using wsadmin script:
AdminTask.configureAppWIMUserRegistry('[-securityDomainName mySSOLTPADomain -verifyRegistry true ]')
# Enable Application Security
Using WAS Admin console: Security domains > mySSOLTPADomain > Application security > Customize for this domain > check Enable Application security
Using wsadmin script:
AdminTask.setAppActiveSecuritySettings('[-securityDomainName mySSOLTPADomain -appSecurityEnabled true]')
# Here, we are also increasing the LTPA timeout
Using WAS Console: Security domains > mySSOLTPADomain > Authentication Mechanism Attributes > Customize for this domain
Using wsadmin script:
AdminTask.setLTPATimeout('[-securityDomainName mySSOLTPADomain -timeout 220 ]')
# And, we are going to assign the scope of this security domain to cluster 'sso_clus' scope
Using WAS Admin console: Security domains > mySSOLTPADomain
Using wsadmin script:
AdminTask.mapResourceToSecurityDomain('[-securityDomainName mySSOLTPADomain -resourceName Cell=:ServerCluster=sso_clus]')

 
6.0) Once you finish setting the security related configuration, you can run the following command to get information related to active security domain.
Using wsadmin script:
AdminTask.getActiveSecuritySettings('-securityDomainName testDomain')

7.0) Deploy the application. For this exercise, we'll be using DefaultApplication that comes with WAS installation. Below command installs the app on sso_clus.
 Note: for clarity, the below command is broken into multiple lines. Run as a single line.
# AdminApp.install('/apps/IBM/WebSphereND/AppServer/installableApps/DefaultApplication.ear',
   '[-distributeApp -appname DefaultSSOApplication -createMBeansForResources -cluster sso_clus -MapModulesToServers [[ "Increment EJB module" Increment.jar,META-INF/ejb-jar.xml WebSphere:cluster=sso_clus ][ "Default Web Application" DefaultWebApplication.war,WEB-INF/web.xml WebSphere:cluster=sso_clus ]]]')
 
8.0) Security role to user/group mapping:
As outlined in the beginning, for this POC , we will map "All Role" to Special subject, which is "All Authenticated in Trusted Realms". Follow the step below using WebSphere Admin console:
Navigate to Applications > Application Types > WebSphere enterprise application
Select DefaultSSOApplication.ear > Security role to user/group mapping.
Select "All Role" and Select "All Authenticated in Trusted Realms" from Map Special Subjects drop down list. and apply, save and synchronise.

9.0) Export LTPA key:
# Using WAS console: Security > Global security > LTPA
# Using wsadmin script:
AdminTask.exportLTPAKeys('[-ltpaKeyFile file:/tmp/Myltpa.key -password Your-Password ]')

10.0) Since, we are using external trusted realm, you need to open (open it in VI like text editor, so that it does not insert any unwanted character) the exported LTPA key and modify the following:
# Original:
com.ibm.websphere.ltpa.Realm=defaultWIMFileBasedRealm
# Updated:
com.ibm.websphere.ltpa.Realm=DatapowerRealm

11.0) Install ODCInfo application. ODCInfo application will be deployed and run on WAS Deployment Manager server. It captures the WAS cluster information and DP will be able to retrieve it dynamically and load balance the traffic. You can download it from IBM Fix Central. Search for "ResourceKit.710" and download the ResourceKit.710.zip. You need to logon using your IBM credentials to download it. For details on ODCInfo application installation and configuration, refer to https://www.ibm.com/support/knowledgecenter/SS9H2Y_7.1.0/com.ibm.dp.doc/lbg_enablingretrievalwlminfo.html
11.1) Extract ResourceKit.710.zip on Deploymen Manager server. In this example, I've extracted it under ~/ResourceKit.
11.2) Installing the OSGi bundle. Basically copy the com.ibm.datapower.odc.osgi.jar to plugins directory of WAS.
Note: my WAS_HOME is /apps/IBM/WebSphere/AppServer

$>cp ~/ResourceKit/AO/com.ibm.datapower.odc.osgi/dist/com.ibm.datapower.odc.osgi.jar /apps/IBM/WebSphere/AppServer/plugins

11.3) Run following command to clear and recreate the OSGi cache. Details: https://www.ibm.com/support/knowledgecenter/en/SSAW57_7.0.0/com.ibm.websphere.nd.doc/info/ae/ae/rxml_osgicfginit_script.html
$>cd /apps/IBM/WebSphere/AppServer/profiles/myDmgrProfile/bin
$>./osgiCfgInit.sh
   OSGi profile cache successfully cleaned for /apps/IBM/WebSphere/AppServer/profiles/myDmgrProfile.
   OSGi server cache successfully cleaned for /apps/IBM/WebSphere/AppServer/profiles/<myDmgrProfile>/servers/dmgr.
# Start the OSGI console (optional):
$>./osgiConsole.sh
#(Optional) run the OSGI diagnostic
diag com.ibm.datapower.odc.osgi
# Verify that a message states: No unresolved constraints.
$>cd /apps/IBM/WebSphere/AppServer/profiles/<myDmgrProfile>/bin
$>./osgiConsole.sh
   osgi> diag com.ibm.datapower.odc.osgi
   websphere@plugins/com.ibm.datapower.odc.osgi.jar [5]
  No unresolved constraints.

11.4) Install ODCInfo Web application and verify:
# ./wsadmin.sh -f script_path/ODCInfoDeploy.jacl dmgr_server_name dmgr_node_name path_to_war_file ODCInfo
$>cd /apps/IBM/WebSphere/AppServer/profiles/myDmgrProfile/bin
$>./wsadmin.sh -f ~/ResourceKit/AO/scripts/ODCInfoDeploy.jacl dmgr myDmgrNode ~/ResourceKit/AO/com.ibm.datapower.odc/ODCInfo_ND61.war ODCInfo
   ...
   ADMA5013I: Application ODCInfo installed successfully.
   ODCInfoDeploy: saving the configuration


# Verify installation:
$>cd /apps/IBM/WebSphere/AppServer/profiles/myDmgrProfile/bin
$>./wsadmin.sh -f ~/ResourceKit/AO/scripts/ODCInfoCheckInstall.jacl myCell01 dmgr ODCInfo
    ...
   ADMA5072I: Distribution status check completed for application ODCInfo.
   Application ODCInfo is deployed successfully.
 
# Start the ODCInfo application:
$>cd /apps/IBM/WebSphere/AppServer/profiles/myDmgrProfile/bin
$>./wsadmin.sh -f ~/ResourceKit/AO/scripts/ODCInfoStart.jacl myCell01 myDmgrNode ODCInfo
   ...
   WASX7303I: The following options are passed to the scripting environment and are available as arguments that are stored in the argv variable: ...
   ODCInfoStart.jacl: ODCInfo started
 
11.5) Verify that the ODCInfo is able to retrieve the cluster info:
   http://<DMGR_IP>:<DMGR_HTTP_PORT>/ODCInfo/ODCInfo?c=sso_clus
Important note:
If you are not able to access the cluster info using the above mentioned URL, it’s most probably because of following reason.
By default ‘dmgr’ server’s both (secured, and non-secured) webcontainer transport ports are assigned as host aliases under ‘admin_host’ virtual host. However by default, ODCInfo uses ‘default_host’ virtual host as it cannot use ‘admin_host’, what you need to do is remove non-secured webcontainer port of dmgr from ‘admin_host’ virtual host, and add it as an alias to ‘default_host’virtual host. Save & synchronise the configuration and restart the Deployment Manager.

12.0) Extract SSL certificate from WAS.For details on SSL key related wsadmin commands, refer to: https://www.ibm.com/support/knowledgecenter/SSAW57_8.5.5/com.ibm.websphere.nd.doc/ae/rxml_atpersonalcert.html#rxml_atpersonalcert__cmd5

# Here is how you can export the Cell or node cert from WAS:
# List available keystores using wsadmin script:
AdminTask.listKeyStores('[-all true -keyStoreUsage SSLKeys ]')

# List personal certificates:
AdminTask.listPersonalCertificates('[-keyStoreName NodeDefaultKeyStore -keyStoreScope (cell):myCell01:(node):myWASNode01 ]')

# Extract the certificates
# Using WAS Admin console: SSL certificate and key management > Key stores and certificates > NodeDefaultKeyStore > Personal certificates > Extract certificate

# Using wsadmin script:
AdminTask.extractCertificate('[-certificateFilePath /tmp/localWASNodeCertKS.crt -base64Encoded true -certificateAlias default -keyStoreName NodeDefaultKeyStore -keyStoreScope (cell):myCell01:(node):myWASNode01 ]')

Steps to be performed on DP side:


1.0) Upload the LTPA key to DP:
 You can use DP CLI scp command. In following example, I'm copying MySSOLTPA.key located at /tmp directory on MyWAShost to Datapower cert: folder.
Note: make sure you are in config mode in Datapower while running scp command.
copy scp://<username>@<MyWAShost>//tmp/MySSOLTPA.key cert:///MySSOLTPA.key
For detail on using scp/sftp commands from DP CLI, refer to: http://www-01.ibm.com/support/docview.wss?uid=swg21250655

2.0) Upload/import SSL cert  (extracted from WAS) to DP. Use CLI option as explained in step #1.0.

3.0) Configure password alias for LTPA key using DP Console.
 Navigate to Objects > Configuration Management > Policy> Password Map Alias
Click "Add" button
 Name: mySSOLTPAPWAlias
Password: <your password>
Note: Make sure this password matches exactly to the password that you used while exporting the    LTPA key from WAS
 Apply and Save Configuration.

4.0) Generate Crypto Key to be used as server identity and server certificate (we will use self-signed for this POC) in front side handler.
Navigate to Administration --> Miscellaneous > Crypto Tools with options:
key type: RSA
Generate Self-Signed Certificate
Generate Key and Certificate Objects
Object Name: 'mySSOCryptoKeyObj' using the Crypto tool
For password alias, you can use the same one that was created for LTPA key or create new one.
For details, refer to: https://www.ibm.com/support/knowledgecenter/SS9H2Y_7.5.0/com.ibm.dp.doc/cryptotool_generatingkeyscertificates.html

Here is the execution result:
Action completed successfully.
     Generated private key in "cert:///mySSOCryptoKeyObj-privkey.pem"
     Generated Certificate Signing Request in "temporary:///mySSOCryptoKeyObj.csr"
     Generated Self-Signed Certificate in "cert:///mySSOCryptoKeyObj-sscert.pem" and exported a copy in "temporary:///mySSOCryptoKeyObj-sscert.pem"
     Generated a Crypto Key object named "mySSOCryptoKeyObj" and a Crypto Certificate object named "mySSOCryptoKeyObj"
 
5.0) Create/Configure Crypto certificate:
Navigate to Objects > Crypto Certificate
Click "Add"
Name: LocalWASCert
File Name: localWASNodeCert.crt
Note: for file name, make sure to select the file that was extracted from WAS and imported to DP.
Password Alias: none

6.0) Create SSL Client Profile:
Objects > Crypto Configuration > SSL Client Profile
Click "Add"
Name: mySSOWASSSLClntProfile
In the Credential section, you don't need Identification credentials as this SSL client profile is only be used to validate certificate presented by WAS during secure back-end communication.
enable "Validate server certificate"
Validation credential click on "+" button and create Crypto Validation Credentials
Name: LocalWASCertValCred
Certificate: LocalWASCert
 
7.0) Create SSL Server Profile.
Navigate to Objects > Crypto Configuration > SSL Server Profile
Click on "Add" button.
Name: myLTPASSOSrvProfile
Since, it's not participating into mutual SSL auth with client, disable "Request client authentication" option.
   In Identification credential, click on "+"
   Create/Configure Identification Credential:
   Name:myDPIdCred
   Crypto Key: mySSOCryptoKeyObj
   Note: mySSOCryptoKeyObj was created as part of the crypto key generation using Crypto tool in previous step.
 
8.0) Create Load Blancer Group
Navigate to Object > Network Settings > Load Balancer Group and Add new LB Group.
For this POC, we name it as "myWASLBG"
Select "on" for "Retrieve Workload Management Information"
Workload Management Retrieval: WebSphere Cell
WebSphere Cell: click on "+" sign and provide name and enter WAS Deployment Manager information
Notes:

  • "Deployment Manager Port number" should be the one that is part of the 'default_host' virtual host in WAS. In this POC, we used the non-secured (http) port.
  • SSL Profile - can be ignored in this screen as the WAS cluster information will be retrieved through non SSL port.

Back on the Load Balancer Group configuration page, the "Workload Management Group Name" should be the actual WAS cluster name. For this POC, we have created "sso_clus" WAS cluster, and will enter it here. Protocol: HTTPS.

Note: protocol in this page, identifies the actual (application related) communication between DP and WAS.
 
9.0) Verify the Load Balancer Group status:
Navigate to Status > IP-Network > Load Balancer
   Make sure your Load Balancer Group listed with WAS cluster members.
 
10.0) Create Multi-Protocol Gateway (MPGW) service, define policy etc... In order to make this simple, for this POC, we use template provided by IBM.
10.1) Navigate to "Blueprint Console" and click on "Patterns" symbol on left hand navigation panel.
10.2) Click on "Web application with Form-based authentication and authorization" link.
10.3) Read the information and instruction carefully. It explains in detail here how the DP policies are setup for this template and how it's going to work once deployed. Once you understand, Click on "Deploy..." button.
It'll prompt for few inputs:
  •    Service name: myFormBasedLTPASSO
  •    Back-end end point details: https://myWASLBG
  •    Note: "myWASLBG" is the name of Load Balancer Group that we created in step #8.0.
  •    SSL Client Profile: mySSOWASSSLClntProfile
  •    Note: "mySSOWASSSLClntProfile" is the name of SSL Client Profile that we created in step #6.0 .
  •    Front-end end point details: choose the IP for front end or leave default and choose port that is available
  •    SSL Server type: Server Profile
  •    SSL Server Profile: myLTPASSOSrvProfile
  •    Note: myLTPASSOSrvProfile is the name of SSL Server Profile that we created in step # 7.
  •    For step 4 (Authenticate with LTPA), and step 5 (Generated LTPA Token),
  •    For LTPA key file: MySSOLTPA.key
  •    For Key file password alias:mySSOLTPAPWAlias
Note:
  • MySSOLTPA.key was exported from WAS, customised, and imported to DP.
  • mySSOLTPAPWAlias was created in step #3.0.
Once all the information entered correctly, click on "Deploy Pattern" button. It may take a minute or so to fully deploy the pattern.

11.0) Once, it is successfully deployed, you'll see the "myFormBasedLTPASSO" under Multi-Protocol Gateway Service list. You need to change few things:
11.1) Navigate to "Multi-Protocol Gateway" and click on the MPGW gateway link just created. Look for the "XML Manager" setting, most probably, by default, the "default" XML Manager has been set.
Click on the '+' sign to create another XML Manager with Load Balancer Group.
Provide the name for XML Manager and select the LB group "myWASLBG" that you've created in step #8.
11.2) Also, by default, DP uses "AAAInfo.xml" as a credential storage. If you need to add/edit user, password or credential mapping information, you can edit this file or use another one.

12.0) Save the configuration and you're ready to test.

Testing:

1.0) Make sure WAS cluster is up and all DP objects and services running.
2.0) Access the page: https://<DP_Front_Side_Handler_IP>:<DP_Front_Side_Handler_Port>/snoop
   you'll be prompted for Username and Password.

   Note: Use one of the usernames and corresponding password listed in AAAInfo.xml.
   Once login is successful, you'll get the snoop page served by one of the WAS cluster member.

3.0) Verify the DP Load Balancer configuration is working properly:
   > Each time, you access the snoop page, look at the "Server name" and "Server Port" and make sure you're hitting all cluster members.

Troubleshooting:

1.0) Your SSO is not working (prompted for credential again) and you see following message in WAS SystemOut.log:

[23/05/17 14:35:14:841 EDT] 000000a0 LTPAServerObj W   SECJ0371W: Validation of the LTPA token failed because the token expired with the following info: Token expiration Date: Tue May 23 10:44:58 EDT 2017, current Date: Tue May 23 14:35:14 EDT 2017 Token attributes:  username=user:<realm>/<user>.. This warning might indicate expected behavior. Please refer to technote at http://www-01.ibm.com/support/docview.wss?uid=swg21594981.

Resolution:
It can be caused by few things, and one of them is time not in synchronized between DP and WAS servers. So, first thing you want to check and make sure is the time setting in DP and WAS server are in sync.

2.0) Looked like your SSO with LTPA is working (because you see your request hitting the WAS server(s), but you can't see intended page and you see the authorisation error in browser and WAS SystemOut.log shows the following:

[23/05/17 15:10:54:474 EDT] 000000a0 WebCollaborat A   SECJ0129E: Authorization failed for user user:<realm>/<user>:<realm> while invoking GET on default_host://snoop, Authorization failed, Not granted any of the required roles: All Role

Resolution:
It is most probably, because the application security has been enabled, but no User role assigned.
On WAS Admin console, navigate to "Applications > Application Types > WebSphere enterprise applications"
Double click on the application link .
Click on "Security role to user/group mapping"
When role mapping page appears, do one of the following based on your requirement:
   1) Map either User or Group for the role.
   2) Map Special Subjects, for the example, since we are trusting the users authenticated by DP and we've created trusted external realm, we'll select option "All Authenticated in Trusted Realms" option.
Save and Synchronise.
And try your test again.

Hope, you'll find this post helpful !!!

1 comment:

  1. I really appreciate information shared above. It’s of great help. If someone want to learn Online (Virtual) instructor lead live training in IBM DATAPOWER, kindly contact us http://www.maxmunus.com/contact
    MaxMunus Offer World Class Virtual Instructor led training on IBM DATAPOWER. We have industry expert trainer. We provide Training Material and Software Support. MaxMunus has successfully conducted 100000+ trainings in India, USA, UK, Australlia, Switzerland, Qatar, Saudi Arabia, Bangladesh, Bahrain and UAE etc.
    For Demo Contact us.
    Saurabh Srivastava
    MaxMunus
    E-mail: saurabh@maxmunus.com
    Skype id: saurabhmaxmunus
    Ph:+91 8553576305 / 080 - 41103383
    http://www.maxmunus.com/


    ReplyDelete