I love reading. I read every day at least around two hours - partly thanks to my commute time to work. I take YRT to work, it's comfortable seating gives me opportunity to indulge in reading while
commuting. Listed below are some of the books that I have read in the past and liked or currently
been reading.
Purna Poudel's blog covering technology (Architecture, DevOps, DevSecOps, Security etc.), travel, and personal experiences.
DataPower Front-End for WAS Cluster with Load Balancing and SSO
In this blog post, I'm guiding through simple and easy to follow steps to create a Proof of Concept (POC) environment with Datapower (DP) as font-end and application serving environment hosted on WebSphere Application Server (WAS) based cluster with Single Sign On (SSO) enabled. DP will also load-balance requests to WAS cluster members. For details of DP supported features as front-end, refer to https://www.ibm.com/support/knowledgecenter/en/SSAW57_8.5.5/com.ibm.websphere.nd.multiplatform.doc/ae/cwsv_topology.html
Assumptions:
On WAS side:
On DP side:
Once you make configuration changes on WAS either using WAS admin console or wsadmin script,
you need to save the changes and synchronise the changes with the nodes (in Network Deployment (ND). Since it's essential part for every change or set of changes, I'm not going to list these two steps after every change below. But you need to make sure to save and synchronise. You can use
the wsadmin Jython script below to save and synchronize the changes with nodes:
1.0) Create a WAS cluster containing two members. In this example, we will create a WAS cluster named "sso_clus" with two members "ssoSrv1" and "ssoSrv2". For more info on creating WAS cluster, refer to https://www.ibm.com/support/knowledgecenter/SSRMWJ_6.0.0.10/com.ibm.isim.doc/installing/tsk/tsk_ic_ins_was_85_cluster.htm
Note: You may need to change text in italic as per your configuration.
2.0) Enable SSO (if not already). Here you need to know request(s) from what domain(s) you are supporting, you can have multiple domains defined here. For detail how domain works refer to: https://www.ibm.com/support/knowledgecenter/en/SSAW57_8.5.5/com.ibm.websphere.nd.doc/ae/tsec_msso.html. For this example, we are going to use 'UseDomainFromURL'. See the above URL for details.
3.0) Disable automatic LTPA key generation option as follows:
Access WAS Admin console and navigate to Security > Global security > LTPA
> Click on link "Key set groups"
> Click on link "CellLTPAKeySetGroup"
> Uncheck "Automatically generate keys"
Apply, save and synchronise.
4.0) Create security domain. Creating security domain is encouraged rather than modifying the Global security configuration, because, you may have other application serving environment (s) managed under this Cell and they may rely on Global security and any change may affect them.
For this example, we name our new security domain 'mySSOLTPADomain' and we'll use the Global security as template (basically copying the Global security) to create the new security domain and modifying few things on newly created security domain:
5.0) Customise the newly created security domain as per our requirement.
5.1) Since, we will not have any user registry configured on WAS for application authentication, and we'll trust the credential authenticated by DP, we'll create a trusted external realm called 'DatapowerRealm' and add that to newly created security domain. For detail about trusted external realm, Refer to https://www.ibm.com/support/knowledgecenter/en/SSAW57_8.0.0/com.ibm.websphere.nd.doc/info/ae/ae/tsec_sec_realm_trust.html
6.0) Once you finish setting the security related configuration, you can run the following command to get information related to active security domain.
7.0) Deploy the application. For this exercise, we'll be using DefaultApplication that comes with WAS installation. Below command installs the app on sso_clus.
Note: for clarity, the below command is broken into multiple lines. Run as a single line.
8.0) Security role to user/group mapping:
As outlined in the beginning, for this POC , we will map "All Role" to Special subject, which is "All Authenticated in Trusted Realms". Follow the step below using WebSphere Admin console:
Navigate to Applications > Application Types > WebSphere enterprise application
Select DefaultSSOApplication.ear > Security role to user/group mapping.
Select "All Role" and Select "All Authenticated in Trusted Realms" from Map Special Subjects drop down list. and apply, save and synchronise.
9.0) Export LTPA key:
10.0) Since, we are using external trusted realm, you need to open (open it in VI like text editor, so that it does not insert any unwanted character) the exported LTPA key and modify the following:
11.0) Install ODCInfo application. ODCInfo application will be deployed and run on WAS Deployment Manager server. It captures the WAS cluster information and DP will be able to retrieve it dynamically and load balance the traffic. You can download it from IBM Fix Central. Search for "ResourceKit.710" and download the ResourceKit.710.zip. You need to logon using your IBM credentials to download it. For details on ODCInfo application installation and configuration, refer to https://www.ibm.com/support/knowledgecenter/SS9H2Y_7.1.0/com.ibm.dp.doc/lbg_enablingretrievalwlminfo.html
11.1) Extract ResourceKit.710.zip on Deploymen Manager server. In this example, I've extracted it under ~/ResourceKit.
11.2) Installing the OSGi bundle. Basically copy the com.ibm.datapower.odc.osgi.jar to plugins directory of WAS.
Note: my WAS_HOME is /apps/IBM/WebSphere/AppServer
11.3) Run following command to clear and recreate the OSGi cache. Details: https://www.ibm.com/support/knowledgecenter/en/SSAW57_7.0.0/com.ibm.websphere.nd.doc/info/ae/ae/rxml_osgicfginit_script.html
11.4) Install ODCInfo Web application and verify:
11.5) Verify that the ODCInfo is able to retrieve the cluster info:
http://<DMGR_IP>:<DMGR_HTTP_PORT>/ODCInfo/ODCInfo?c=sso_clus
Important note:
If you are not able to access the cluster info using the above mentioned URL, it’s most probably because of following reason.
By default ‘dmgr’ server’s both (secured, and non-secured) webcontainer transport ports are assigned as host aliases under ‘admin_host’ virtual host. However by default, ODCInfo uses ‘default_host’ virtual host as it cannot use ‘admin_host’, what you need to do is remove non-secured webcontainer port of dmgr from ‘admin_host’ virtual host, and add it as an alias to ‘default_host’virtual host. Save & synchronise the configuration and restart the Deployment Manager.
12.0) Extract SSL certificate from WAS.For details on SSL key related wsadmin commands, refer to: https://www.ibm.com/support/knowledgecenter/SSAW57_8.5.5/com.ibm.websphere.nd.doc/ae/rxml_atpersonalcert.html#rxml_atpersonalcert__cmd5
1.0) Upload the LTPA key to DP:
You can use DP CLI scp command. In following example, I'm copying MySSOLTPA.key located at /tmp directory on MyWAShost to Datapower cert: folder.
Note: make sure you are in config mode in Datapower while running scp command.
For detail on using scp/sftp commands from DP CLI, refer to: http://www-01.ibm.com/support/docview.wss?uid=swg21250655
2.0) Upload/import SSL cert (extracted from WAS) to DP. Use CLI option as explained in step #1.0.
3.0) Configure password alias for LTPA key using DP Console.
Navigate to Objects > Configuration Management > Policy> Password Map Alias
Click "Add" button
Name: mySSOLTPAPWAlias
Password: <your password>
Note: Make sure this password matches exactly to the password that you used while exporting the LTPA key from WAS
Apply and Save Configuration.
4.0) Generate Crypto Key to be used as server identity and server certificate (we will use self-signed for this POC) in front side handler.
Navigate to Administration --> Miscellaneous > Crypto Tools with options:
key type: RSA
Generate Self-Signed Certificate
Generate Key and Certificate Objects
Object Name: 'mySSOCryptoKeyObj' using the Crypto tool
For password alias, you can use the same one that was created for LTPA key or create new one.
For details, refer to: https://www.ibm.com/support/knowledgecenter/SS9H2Y_7.5.0/com.ibm.dp.doc/cryptotool_generatingkeyscertificates.html
Here is the execution result:
Action completed successfully.
Generated private key in "cert:///mySSOCryptoKeyObj-privkey.pem"
Generated Certificate Signing Request in "temporary:///mySSOCryptoKeyObj.csr"
Generated Self-Signed Certificate in "cert:///mySSOCryptoKeyObj-sscert.pem" and exported a copy in "temporary:///mySSOCryptoKeyObj-sscert.pem"
Generated a Crypto Key object named "mySSOCryptoKeyObj" and a Crypto Certificate object named "mySSOCryptoKeyObj"
5.0) Create/Configure Crypto certificate:
Navigate to Objects > Crypto Certificate
Click "Add"
Name: LocalWASCert
File Name: localWASNodeCert.crt
Note: for file name, make sure to select the file that was extracted from WAS and imported to DP.
Password Alias: none
6.0) Create SSL Client Profile:
Objects > Crypto Configuration > SSL Client Profile
Click "Add"
Name: mySSOWASSSLClntProfile
In the Credential section, you don't need Identification credentials as this SSL client profile is only be used to validate certificate presented by WAS during secure back-end communication.
enable "Validate server certificate"
Validation credential click on "+" button and create Crypto Validation Credentials
Name: LocalWASCertValCred
Certificate: LocalWASCert
7.0) Create SSL Server Profile.
Navigate to Objects > Crypto Configuration > SSL Server Profile
Click on "Add" button.
Name: myLTPASSOSrvProfile
Since, it's not participating into mutual SSL auth with client, disable "Request client authentication" option.
In Identification credential, click on "+"
Create/Configure Identification Credential:
Name:myDPIdCred
Crypto Key: mySSOCryptoKeyObj
Note: mySSOCryptoKeyObj was created as part of the crypto key generation using Crypto tool in previous step.
8.0) Create Load Blancer Group
Navigate to Object > Network Settings > Load Balancer Group and Add new LB Group.
For this POC, we name it as "myWASLBG"
Select "on" for "Retrieve Workload Management Information"
Workload Management Retrieval: WebSphere Cell
WebSphere Cell: click on "+" sign and provide name and enter WAS Deployment Manager information
Notes:
Back on the Load Balancer Group configuration page, the "Workload Management Group Name" should be the actual WAS cluster name. For this POC, we have created "sso_clus" WAS cluster, and will enter it here. Protocol: HTTPS.
Note: protocol in this page, identifies the actual (application related) communication between DP and WAS.
9.0) Verify the Load Balancer Group status:
Navigate to Status > IP-Network > Load Balancer
Make sure your Load Balancer Group listed with WAS cluster members.
10.0) Create Multi-Protocol Gateway (MPGW) service, define policy etc... In order to make this simple, for this POC, we use template provided by IBM.
10.1) Navigate to "Blueprint Console" and click on "Patterns" symbol on left hand navigation panel.
10.2) Click on "Web application with Form-based authentication and authorization" link.
10.3) Read the information and instruction carefully. It explains in detail here how the DP policies are setup for this template and how it's going to work once deployed. Once you understand, Click on "Deploy..." button.
It'll prompt for few inputs:
11.0) Once, it is successfully deployed, you'll see the "myFormBasedLTPASSO" under Multi-Protocol Gateway Service list. You need to change few things:
11.1) Navigate to "Multi-Protocol Gateway" and click on the MPGW gateway link just created. Look for the "XML Manager" setting, most probably, by default, the "default" XML Manager has been set.
Click on the '+' sign to create another XML Manager with Load Balancer Group.
Provide the name for XML Manager and select the LB group "myWASLBG" that you've created in step #8.
11.2) Also, by default, DP uses "AAAInfo.xml" as a credential storage. If you need to add/edit user, password or credential mapping information, you can edit this file or use another one.
12.0) Save the configuration and you're ready to test.
2.0) Access the page: https://<DP_Front_Side_Handler_IP>:<DP_Front_Side_Handler_Port>/snoop
you'll be prompted for Username and Password.
Note: Use one of the usernames and corresponding password listed in AAAInfo.xml.
Once login is successful, you'll get the snoop page served by one of the WAS cluster member.
3.0) Verify the DP Load Balancer configuration is working properly:
> Each time, you access the snoop page, look at the "Server name" and "Server Port" and make sure you're hitting all cluster members.
Resolution:
It can be caused by few things, and one of them is time not in synchronized between DP and WAS servers. So, first thing you want to check and make sure is the time setting in DP and WAS server are in sync.
2.0) Looked like your SSO with LTPA is working (because you see your request hitting the WAS server(s), but you can't see intended page and you see the authorisation error in browser and WAS SystemOut.log shows the following:
Resolution:
It is most probably, because the application security has been enabled, but no User role assigned.
On WAS Admin console, navigate to "Applications > Application Types > WebSphere enterprise applications"
Double click on the application link .
Click on "Security role to user/group mapping"
When role mapping page appears, do one of the following based on your requirement:
1) Map either User or Group for the role.
2) Map Special Subjects, for the example, since we are trusting the users authenticated by DP and we've created trusted external realm, we'll select option "All Authenticated in Trusted Realms" option.
Save and Synchronise.
And try your test again.
Hope, you'll find this post helpful !!!
Assumptions:
- In order for you to follow the steps outlined below, you have good working knowledge of IBM WAS and DataPower.
- You have WAS ND (version 8.5.X), and IBM DataPower Gateway (version 7.5.X) installed and ready to create Application Serving environment. For your reference, I'm using the following:
- WAS 8.5.5.7 Network Deployment.
- IBM DataPower Gateway Virtual Edition (Firmware: IDG.7.5.1.0).
On WAS side:
- Create a cluster with two members (application servers).
- Create security domain and assign scope to the cluster level
- Create and configure an external trusted realm for the security domain. It is because, we will not have any user registry configured (for Application authentication) and we'll trust identifies authenticated by DP.
- Enable Application security on the security domain
- Deploy a sample application on the WAS cluster
- Map security roles and authorize any user belonging to External Trusted Realm.
- Configure LTPA, export LTPA key and customize (as we are using External Trusted Realm) it.
- Export SSL certificate.
On DP side:
- Import LTPA key, import WAS SSL certificate.
- Create DP objects (including Crypto, SSL profiles, Password aliase etc).
- Create Load Balancer Group
- Create Multi-Protocol Gateway (MPGW) service etc.
Let's start working on WAS side:
Note:Once you make configuration changes on WAS either using WAS admin console or wsadmin script,
you need to save the changes and synchronise the changes with the nodes (in Network Deployment (ND). Since it's essential part for every change or set of changes, I'm not going to list these two steps after every change below. But you need to make sure to save and synchronise. You can use
the wsadmin Jython script below to save and synchronize the changes with nodes:
# Save the configuration/change |
1.0) Create a WAS cluster containing two members. In this example, we will create a WAS cluster named "sso_clus" with two members "ssoSrv1" and "ssoSrv2". For more info on creating WAS cluster, refer to https://www.ibm.com/support/knowledgecenter/SSRMWJ_6.0.0.10/com.ibm.isim.doc/installing/tsk/tsk_ic_ins_was_85_cluster.htm
Note: You may need to change text in italic as per your configuration.
# Using WAS Admin Console: Servers --> Clusters --> WebSphere application server clusters
|
2.0) Enable SSO (if not already). Here you need to know request(s) from what domain(s) you are supporting, you can have multiple domains defined here. For detail how domain works refer to: https://www.ibm.com/support/knowledgecenter/en/SSAW57_8.5.5/com.ibm.websphere.nd.doc/ae/tsec_msso.html. For this example, we are going to use 'UseDomainFromURL'. See the above URL for details.
# Configure Single Sign On (SSO) AdminTask.configureSingleSignon('-enable true -requiresSSL true -domainName UseDomainFromURL -attributePropagation true')
|
3.0) Disable automatic LTPA key generation option as follows:
Access WAS Admin console and navigate to Security > Global security > LTPA
> Click on link "Key set groups"
> Click on link "CellLTPAKeySetGroup"
> Uncheck "Automatically generate keys"
Apply, save and synchronise.
4.0) Create security domain. Creating security domain is encouraged rather than modifying the Global security configuration, because, you may have other application serving environment (s) managed under this Cell and they may rely on Global security and any change may affect them.
For this example, we name our new security domain 'mySSOLTPADomain' and we'll use the Global security as template (basically copying the Global security) to create the new security domain and modifying few things on newly created security domain:
# Using WAS Admin Console: Security > Global security > Security domains > New...
|
5.0) Customise the newly created security domain as per our requirement.
5.1) Since, we will not have any user registry configured on WAS for application authentication, and we'll trust the credential authenticated by DP, we'll create a trusted external realm called 'DatapowerRealm' and add that to newly created security domain. For detail about trusted external realm, Refer to https://www.ibm.com/support/knowledgecenter/en/SSAW57_8.0.0/com.ibm.websphere.nd.doc/info/ae/ae/tsec_sec_realm_trust.html
# Using WAS Admin console: Security domains > mySSOLTPADomain > Federated repositories > Trusted authentication realms - inbound > New...# Using wsadmin script:AdminTask.addTrustedRealms('[-securityDomainName mySSOLTPADomain -communicationType inbound -realmList DatapowerRealm]') Using WAS Console: Security domains > mySSOLTPADomain > Federated repositories > Trusted authentication realms - inbound# Using wsadmin script:AdminTask.configureTrustedRealms('[-communicationType inbound -securityDomainName mySSOLTPADomain -trustAllRealms false]') Using WAS Admin console: Security domains > mySSOLTPADomain# Using wsadmin script:AdminTask.configureAppWIMUserRegistry('[-securityDomainName mySSOLTPADomain -verifyRegistry true ]') Using WAS Admin console: Security domains > mySSOLTPADomain > Application security > Customize for this domain > check Enable Application security# Using wsadmin script:AdminTask.setAppActiveSecuritySettings('[-securityDomainName mySSOLTPADomain -appSecurityEnabled true]') Using WAS Console: Security domains > mySSOLTPADomain > Authentication Mechanism Attributes > Customize for this domain# Using wsadmin script:AdminTask.setLTPATimeout('[-securityDomainName mySSOLTPADomain -timeout 220 ]') Using WAS Admin console: Security domains > mySSOLTPADomain# Using wsadmin script:AdminTask.mapResourceToSecurityDomain('[-securityDomainName mySSOLTPADomain -resourceName Cell=:ServerCluster=sso_clus]')
|
6.0) Once you finish setting the security related configuration, you can run the following command to get information related to active security domain.
# Using wsadmin script:AdminTask.getActiveSecuritySettings('-securityDomainName testDomain')
|
7.0) Deploy the application. For this exercise, we'll be using DefaultApplication that comes with WAS installation. Below command installs the app on sso_clus.
Note: for clarity, the below command is broken into multiple lines. Run as a single line.
# AdminApp.install('/apps/IBM/WebSphereND/AppServer/installableApps/DefaultApplication.ear',
|
8.0) Security role to user/group mapping:
As outlined in the beginning, for this POC , we will map "All Role" to Special subject, which is "All Authenticated in Trusted Realms". Follow the step below using WebSphere Admin console:
Navigate to Applications > Application Types > WebSphere enterprise application
Select DefaultSSOApplication.ear > Security role to user/group mapping.
Select "All Role" and Select "All Authenticated in Trusted Realms" from Map Special Subjects drop down list. and apply, save and synchronise.
9.0) Export LTPA key:
# Using WAS console: Security > Global security > LTPA
|
10.0) Since, we are using external trusted realm, you need to open (open it in VI like text editor, so that it does not insert any unwanted character) the exported LTPA key and modify the following:
# Original:
|
11.0) Install ODCInfo application. ODCInfo application will be deployed and run on WAS Deployment Manager server. It captures the WAS cluster information and DP will be able to retrieve it dynamically and load balance the traffic. You can download it from IBM Fix Central. Search for "ResourceKit.710" and download the ResourceKit.710.zip. You need to logon using your IBM credentials to download it. For details on ODCInfo application installation and configuration, refer to https://www.ibm.com/support/knowledgecenter/SS9H2Y_7.1.0/com.ibm.dp.doc/lbg_enablingretrievalwlminfo.html
11.1) Extract ResourceKit.710.zip on Deploymen Manager server. In this example, I've extracted it under ~/ResourceKit.
11.2) Installing the OSGi bundle. Basically copy the com.ibm.datapower.odc.osgi.jar to plugins directory of WAS.
Note: my WAS_HOME is /apps/IBM/WebSphere/AppServer
$>cp ~/ResourceKit/AO/com.ibm.datapower.odc.osgi/dist/com.ibm.datapower.odc.osgi.jar /apps/IBM/WebSphere/AppServer/plugins
|
11.3) Run following command to clear and recreate the OSGi cache. Details: https://www.ibm.com/support/knowledgecenter/en/SSAW57_7.0.0/com.ibm.websphere.nd.doc/info/ae/ae/rxml_osgicfginit_script.html
$>cd /apps/IBM/WebSphere/AppServer/profiles/myDmgrProfile/bin
|
11.4) Install ODCInfo Web application and verify:
# ./wsadmin.sh -f script_path/ODCInfoDeploy.jacl dmgr_server_name dmgr_node_name path_to_war_file ODCInfo
|
11.5) Verify that the ODCInfo is able to retrieve the cluster info:
http://<DMGR_IP>:<DMGR_HTTP_PORT>/ODCInfo/ODCInfo?c=sso_clus
Important note:
If you are not able to access the cluster info using the above mentioned URL, it’s most probably because of following reason.
By default ‘dmgr’ server’s both (secured, and non-secured) webcontainer transport ports are assigned as host aliases under ‘admin_host’ virtual host. However by default, ODCInfo uses ‘default_host’ virtual host as it cannot use ‘admin_host’, what you need to do is remove non-secured webcontainer port of dmgr from ‘admin_host’ virtual host, and add it as an alias to ‘default_host’virtual host. Save & synchronise the configuration and restart the Deployment Manager.
12.0) Extract SSL certificate from WAS.For details on SSL key related wsadmin commands, refer to: https://www.ibm.com/support/knowledgecenter/SSAW57_8.5.5/com.ibm.websphere.nd.doc/ae/rxml_atpersonalcert.html#rxml_atpersonalcert__cmd5
# Here is how you can export the Cell or node cert from WAS: # List personal certificates: # Using wsadmin script:
|
Steps to be performed on DP side:
1.0) Upload the LTPA key to DP:
You can use DP CLI scp command. In following example, I'm copying MySSOLTPA.key located at /tmp directory on MyWAShost to Datapower cert: folder.
Note: make sure you are in config mode in Datapower while running scp command.
copy scp://<username>@<MyWAShost>//tmp/MySSOLTPA.key cert:///MySSOLTPA.key
|
2.0) Upload/import SSL cert (extracted from WAS) to DP. Use CLI option as explained in step #1.0.
3.0) Configure password alias for LTPA key using DP Console.
Navigate to Objects > Configuration Management > Policy> Password Map Alias
Click "Add" button
Name: mySSOLTPAPWAlias
Password: <your password>
Note: Make sure this password matches exactly to the password that you used while exporting the LTPA key from WAS
Apply and Save Configuration.
4.0) Generate Crypto Key to be used as server identity and server certificate (we will use self-signed for this POC) in front side handler.
Navigate to Administration --> Miscellaneous > Crypto Tools with options:
key type: RSA
Generate Self-Signed Certificate
Generate Key and Certificate Objects
Object Name: 'mySSOCryptoKeyObj' using the Crypto tool
For password alias, you can use the same one that was created for LTPA key or create new one.
For details, refer to: https://www.ibm.com/support/knowledgecenter/SS9H2Y_7.5.0/com.ibm.dp.doc/cryptotool_generatingkeyscertificates.html
Here is the execution result:
Action completed successfully.
Generated private key in "cert:///mySSOCryptoKeyObj-privkey.pem"
Generated Certificate Signing Request in "temporary:///mySSOCryptoKeyObj.csr"
Generated Self-Signed Certificate in "cert:///mySSOCryptoKeyObj-sscert.pem" and exported a copy in "temporary:///mySSOCryptoKeyObj-sscert.pem"
Generated a Crypto Key object named "mySSOCryptoKeyObj" and a Crypto Certificate object named "mySSOCryptoKeyObj"
5.0) Create/Configure Crypto certificate:
Navigate to Objects > Crypto Certificate
Click "Add"
Name: LocalWASCert
File Name: localWASNodeCert.crt
Note: for file name, make sure to select the file that was extracted from WAS and imported to DP.
Password Alias: none
6.0) Create SSL Client Profile:
Objects > Crypto Configuration > SSL Client Profile
Click "Add"
Name: mySSOWASSSLClntProfile
In the Credential section, you don't need Identification credentials as this SSL client profile is only be used to validate certificate presented by WAS during secure back-end communication.
enable "Validate server certificate"
Validation credential click on "+" button and create Crypto Validation Credentials
Name: LocalWASCertValCred
Certificate: LocalWASCert
7.0) Create SSL Server Profile.
Navigate to Objects > Crypto Configuration > SSL Server Profile
Click on "Add" button.
Name: myLTPASSOSrvProfile
Since, it's not participating into mutual SSL auth with client, disable "Request client authentication" option.
In Identification credential, click on "+"
Create/Configure Identification Credential:
Name:myDPIdCred
Crypto Key: mySSOCryptoKeyObj
Note: mySSOCryptoKeyObj was created as part of the crypto key generation using Crypto tool in previous step.
8.0) Create Load Blancer Group
Navigate to Object > Network Settings > Load Balancer Group and Add new LB Group.
For this POC, we name it as "myWASLBG"
Select "on" for "Retrieve Workload Management Information"
Workload Management Retrieval: WebSphere Cell
WebSphere Cell: click on "+" sign and provide name and enter WAS Deployment Manager information
Notes:
- "Deployment Manager Port number" should be the one that is part of the 'default_host' virtual host in WAS. In this POC, we used the non-secured (http) port.
- SSL Profile - can be ignored in this screen as the WAS cluster information will be retrieved through non SSL port.
Back on the Load Balancer Group configuration page, the "Workload Management Group Name" should be the actual WAS cluster name. For this POC, we have created "sso_clus" WAS cluster, and will enter it here. Protocol: HTTPS.
Note: protocol in this page, identifies the actual (application related) communication between DP and WAS.
9.0) Verify the Load Balancer Group status:
Navigate to Status > IP-Network > Load Balancer
Make sure your Load Balancer Group listed with WAS cluster members.
10.0) Create Multi-Protocol Gateway (MPGW) service, define policy etc... In order to make this simple, for this POC, we use template provided by IBM.
10.1) Navigate to "Blueprint Console" and click on "Patterns" symbol on left hand navigation panel.
10.2) Click on "Web application with Form-based authentication and authorization" link.
10.3) Read the information and instruction carefully. It explains in detail here how the DP policies are setup for this template and how it's going to work once deployed. Once you understand, Click on "Deploy..." button.
It'll prompt for few inputs:
- Service name: myFormBasedLTPASSO
- Back-end end point details: https://myWASLBG
- Note: "myWASLBG" is the name of Load Balancer Group that we created in step #8.0.
- SSL Client Profile: mySSOWASSSLClntProfile
- Note: "mySSOWASSSLClntProfile" is the name of SSL Client Profile that we created in step #6.0 .
- Front-end end point details: choose the IP for front end or leave default and choose port that is available
- SSL Server type: Server Profile
- SSL Server Profile: myLTPASSOSrvProfile
- Note: myLTPASSOSrvProfile is the name of SSL Server Profile that we created in step # 7.
- For step 4 (Authenticate with LTPA), and step 5 (Generated LTPA Token),
- For LTPA key file: MySSOLTPA.key
- For Key file password alias:mySSOLTPAPWAlias
Note:
- MySSOLTPA.key was exported from WAS, customised, and imported to DP.
- mySSOLTPAPWAlias was created in step #3.0.
Once all the information entered correctly, click on "Deploy Pattern" button. It may take a minute or so to fully deploy the pattern.
11.1) Navigate to "Multi-Protocol Gateway" and click on the MPGW gateway link just created. Look for the "XML Manager" setting, most probably, by default, the "default" XML Manager has been set.
Click on the '+' sign to create another XML Manager with Load Balancer Group.
Provide the name for XML Manager and select the LB group "myWASLBG" that you've created in step #8.
11.2) Also, by default, DP uses "AAAInfo.xml" as a credential storage. If you need to add/edit user, password or credential mapping information, you can edit this file or use another one.
12.0) Save the configuration and you're ready to test.
Testing:
1.0) Make sure WAS cluster is up and all DP objects and services running.2.0) Access the page: https://<DP_Front_Side_Handler_IP>:<DP_Front_Side_Handler_Port>/snoop
you'll be prompted for Username and Password.
Note: Use one of the usernames and corresponding password listed in AAAInfo.xml.
Once login is successful, you'll get the snoop page served by one of the WAS cluster member.
3.0) Verify the DP Load Balancer configuration is working properly:
> Each time, you access the snoop page, look at the "Server name" and "Server Port" and make sure you're hitting all cluster members.
Troubleshooting:
1.0) Your SSO is not working (prompted for credential again) and you see following message in WAS SystemOut.log:
[23/05/17 14:35:14:841 EDT] 000000a0 LTPAServerObj W SECJ0371W: Validation of the LTPA token failed because the token expired with the following info: Token expiration Date: Tue May 23 10:44:58 EDT 2017, current Date: Tue May 23 14:35:14 EDT 2017 Token attributes: username=user:<realm>/<user>.. This warning might indicate expected behavior. Please refer to technote at http://www-01.ibm.com/support/docview.wss?uid=swg21594981.
Resolution:
It can be caused by few things, and one of them is time not in synchronized between DP and WAS servers. So, first thing you want to check and make sure is the time setting in DP and WAS server are in sync.
2.0) Looked like your SSO with LTPA is working (because you see your request hitting the WAS server(s), but you can't see intended page and you see the authorisation error in browser and WAS SystemOut.log shows the following:
[23/05/17 15:10:54:474 EDT] 000000a0 WebCollaborat A SECJ0129E: Authorization failed for user user:<realm>/<user>:<realm> while invoking GET on default_host://snoop, Authorization failed, Not granted any of the required roles: All Role
Resolution:
It is most probably, because the application security has been enabled, but no User role assigned.
On WAS Admin console, navigate to "Applications > Application Types > WebSphere enterprise applications"
Double click on the application link .
Click on "Security role to user/group mapping"
When role mapping page appears, do one of the following based on your requirement:
1) Map either User or Group for the role.
2) Map Special Subjects, for the example, since we are trusting the users authenticated by DP and we've created trusted external realm, we'll select option "All Authenticated in Trusted Realms" option.
Save and Synchronise.
And try your test again.
Hope, you'll find this post helpful !!!
My 2nd Year of 10K Run
What a day - rainy, but beautiful and cheerful! 15,000 Cyclists, Runners and Walkers stormed the Gardiner Expressway and Don Valley Parkway (@TO_DVP) today braving the rain for Ride for Heart event (www.rideforheart.ca). It was still raining in Toronto when we're at start line of 10KM run, but by the time, we started running, mother nature smiled a little bit and slowly stopped pouring. It felt amazingly good running on traffic free Gardiner Expressway while watching downtown Toronto with fellow participants - all happy and cheering. I completed the race in just few seconds less of one hour (my actual time recorded by BIB and provided by www.sportstats.ca was 59 minutes and 23 seconds). This post is to Thank YOU ALL who supported me and contributed for the good cause (Life-Saving research in the field of Heart and Stroke) and my family who waked up early in the morning and went with me to Ontario Place (http://www.ontarioplace.com/) to cheer for me. Because of your help, I was able to raise $500.00 for Heart and Stroke Foundation. Over all, today's event raised about 6 Million Dollar for Heart and Stroke related research. That's really great! Thank you All!
Custom Ant Task IsInList
I had created this Custom Ant Task sometime ago while working on a project where I needed to check whether an item exists in the list. As I did not find any other efficient way to do it using any of the standard Ant tasks, I created one on my own. I'm publishing (see below GitHub project location) this Custom Ant Task source code as an Open Source. Feel free to use/modify/distribute it as per your need or suggest if you have any other better ways to do it.
2) The GitHub project also has Ant build file build.xml to build the project from source code, sample-usage.xml - Ant build file that shows few usage scenarios of 'IsInList' task and README.txt that basically explains how to use it.
How to Use It?
Follow the steps below:
1) Make sure isinlist-<version>.jar file is in your build classpath. You can do it either by adding it into your $ANT_HOME/lib directory or by defining a custom library path like below and making a reference to it.
2) Next, define the "isinlist" task, below is one of the few ways:
3) Use it, see the examples below:
Example 1:
You have a list of items like "ci;Inting.*;release;SystemOut_16.01.23.log;native_stdout.*;native_stderr.*" separated by ";". Here you need to find out whether or not any item starting with "native_stdout.log" exists. In this case you can do lookup using regular expression (isRegEx="true"). In your build file, you'll need to have:
Example 2:
You have a list of items like "ci;Inting.*;release;SystemOut_16.01.23.log;native_stdout.*;native_stderr.*" separated by ";".
Here you need to find out whether an item called "release" exists in the given list. In this case you can use regular lookup, meaning isRegEx="false".
See the sample-usage.xml for complete example and more detail usage scenarios.
You can get/dowload files from GitHub location: https://github.com/pppoudel/customanttasks.
What IsInList contains?
1) It contains one Java source file: com.sysgenius.tools.ant.customtask.IsInList.java
2) The GitHub project also has Ant build file build.xml to build the project from source code, sample-usage.xml - Ant build file that shows few usage scenarios of 'IsInList' task and README.txt that basically explains how to use it.
How to Use It?
Follow the steps below:
1) Make sure isinlist-<version>.jar file is in your build classpath. You can do it either by adding it into your $ANT_HOME/lib directory or by defining a custom library path like below and making a reference to it.
<path id="ant.opt.lib.path"> |
2) Next, define the "isinlist" task, below is one of the few ways:
<typedef classname="com.sysgenius.tools.ant.customtask.IsInList" name="isinlist" classpathref="ant.opt.lib.path"/> |
3) Use it, see the examples below:
Example 1:
You have a list of items like "ci;Inting.*;release;SystemOut_16.01.23.log;native_stdout.*;native_stderr.*" separated by ";". Here you need to find out whether or not any item starting with "native_stdout.log" exists. In this case you can do lookup using regular expression (isRegEx="true"). In your build file, you'll need to have:
<property name="item.list" value="ci;Inting.*;release;SystemOut_16.01.23.log;native_stdout.*;native_stderr.*"/> |
Example 2:
You have a list of items like "ci;Inting.*;release;SystemOut_16.01.23.log;native_stdout.*;native_stderr.*" separated by ";".
Here you need to find out whether an item called "release" exists in the given list. In this case you can use regular lookup, meaning isRegEx="false".
<property name="item.list" value="ci;Inting.*;release;SystemOut_16.01.23.log;native_stdout.*;native_stderr.*"/> |
See the sample-usage.xml for complete example and more detail usage scenarios.
You can get/dowload files from GitHub location: https://github.com/pppoudel/customanttasks.
Making Your Container Deployment Portable
This post is a follow-up and extension of my previous post "Setting up CommandLine Environment for IBM® Bluemix® Container Service".
In this post, I'm further exploring the way of working with containers whether it's locally deployed native Docker or container created with IBM® Bluemix® Container Service. I'm going to show few basic scripting ideas, so that the same docker-compose.yml and other related files can be used no matter whether you are dealing with locally deployed native Docker container(s) or IBM container(s).
One step ahead, here we will be working with multiple containers and employing Docker Compose. I have used basic steps for this exercise from Bluemix tutorial (https://console.ng.bluemix.net/docs/containers/container_compose_intro.html#container_compose_config) and added few steps and logic to do basic automation and make it portable, so that it can be executed the same way independent of environment.
Pre-requisites for this exercise:
As part of this exercise, we will be putting together 'docker-compose.yml' with replaceable variable(s), '.env' file for environment variables with default values, property file 'depl.properties' for environment specific properties, and script file 'autoDeploy.sh' with basic logic that can be executed to manage both native Docker as well as IBM Bluemix containers. We will be creating and linking following two containers.
At the end, we'll also look into few possible issues that you may encounter.
Let's start with creating docker-compose.yml. Compose simplifies the definition and execution of multi-container Docker applications. See Docker Compose documentation for details.
Below is our simple docker-compose.yml which defines two containers <<lets-chat>> and <<lc-mongo>>. As you see a variable has been assigned as a value to 'image' attribute. It is in order to make it portable between native Docker and container to be deployed on IBM Bluemix as the image registry will be different. You can assign variable this way to any attribute as it's value which will be replaced by value of corresponding environment variable.
Now, let's see, where we can define environment variables. Docker supports either defining it through command shell as 'export VAR=VALUE' or defining them in '.env' file (Note: If you are deploying your service using 'docker stack deploy --compose-file docker-compose.yml <service-name>' instead of 'docker-compose up ...' values in the docker-compose.yml may not be replaced by corresponding environment values defined in .env file. See https://github.com/moby/moby/issues/29133). Environment variable defined through 'export VAR=VALUE' takes precedence. See more detail on variable substitution and declaring default environment variables in file.
Below is our '.env' file:
Usually, it is a best practice to define default variables with 'DEV/Development' environment specific values in '.env' file and have mechanism to override those values for higher environment(s). It helps to boost developers' productivity. In order to follow the above mentioned principle, I've defined my local native Docker container specific environment variables in my '.env' file and will have separate property file to define environment variables and their values for other environments (Bluemix in my case for this post).
Below is my property file 'depl.properties' which defines property and their Bluemix specific values:
In this post, I'm further exploring the way of working with containers whether it's locally deployed native Docker or container created with IBM® Bluemix® Container Service. I'm going to show few basic scripting ideas, so that the same docker-compose.yml and other related files can be used no matter whether you are dealing with locally deployed native Docker container(s) or IBM container(s).
One step ahead, here we will be working with multiple containers and employing Docker Compose. I have used basic steps for this exercise from Bluemix tutorial (https://console.ng.bluemix.net/docs/containers/container_compose_intro.html#container_compose_config) and added few steps and logic to do basic automation and make it portable, so that it can be executed the same way independent of environment.
Pre-requisites for this exercise:
- (native) Docker installed and running locally(may be on your laptop/desktop)
- CommandLine environment setup for IBM® Bluemix® Container Service. See previous post "Setting up CommandLine Environment for IBM® Bluemix® Container Service".
- Docker Compose version 1.6.0 or later installed on your laptop/desktop. See installation instruction here
- lets-chat and mongo images are available in your local and Bluemix private registry.
As part of this exercise, we will be putting together 'docker-compose.yml' with replaceable variable(s), '.env' file for environment variables with default values, property file 'depl.properties' for environment specific properties, and script file 'autoDeploy.sh' with basic logic that can be executed to manage both native Docker as well as IBM Bluemix containers. We will be creating and linking following two containers.
- lets-chat (basic chat application)
- mongo (database to store data)
At the end, we'll also look into few possible issues that you may encounter.
Let's start with creating docker-compose.yml. Compose simplifies the definition and execution of multi-container Docker applications. See Docker Compose documentation for details.
Below is our simple docker-compose.yml which defines two containers <<lets-chat>> and <<lc-mongo>>. As you see a variable has been assigned as a value to 'image' attribute. It is in order to make it portable between native Docker and container to be deployed on IBM Bluemix as the image registry will be different. You can assign variable this way to any attribute as it's value which will be replaced by value of corresponding environment variable.
lets-chat:
|
Now, let's see, where we can define environment variables. Docker supports either defining it through command shell as 'export VAR=VALUE' or defining them in '.env' file (Note: If you are deploying your service using 'docker stack deploy --compose-file docker-compose.yml <service-name>' instead of 'docker-compose up ...' values in the docker-compose.yml may not be replaced by corresponding environment values defined in .env file. See https://github.com/moby/moby/issues/29133). Environment variable defined through 'export VAR=VALUE' takes precedence. See more detail on variable substitution and declaring default environment variables in file.
Below is our '.env' file:
# COMPOSE_HTTP_TIMEOUT default value is 60 seconds.
|
Usually, it is a best practice to define default variables with 'DEV/Development' environment specific values in '.env' file and have mechanism to override those values for higher environment(s). It helps to boost developers' productivity. In order to follow the above mentioned principle, I've defined my local native Docker container specific environment variables in my '.env' file and will have separate property file to define environment variables and their values for other environments (Bluemix in my case for this post).
Below is my property file 'depl.properties' which defines property and their Bluemix specific values:
# Define property as |
Now, we need to have a script (logic) that can set appropriate environment variables based on the target environment.
Below is sample (autoDeploy.sh) script:
#!/bin/sh
|
Now, it's time to test the logic above.
First, let's execute the script locally against native Docker.
$> ./autoDeploy.sh -t native
|
As per script execution logic, it first identifies if any container instance of 'lc-mongo' and 'lets-chat', if so, it stops and removes the existing container then creates new one from existing images and starts and checks if they are running successfully. Since '-t native' option passed through command line, it didn't set any environment variable, but Docker Compose used the default environment variables defined in '.env' file.
It is time to test the same against IBM Bluemix Container Service. See below:
$> ./autoDeploy.sh -t bluemix -u abc.def@xyz.com -p xxxxxxxxx
|
As you have noticed, we passed options '-t bluemix -u abc.def@xyz.com -p xxxxxxxxx' while executing the autoDeploy.sh. This enforced script to read properties from 'depl.properties' file and set corresponding environment variables specific for Bluemix. Everything else including docker-compose.yml and .env file remain unchanged.
Note: IPs, username and password masked.In terms of defining properties specific to any environment, in this post, I'm just showing the case for two environments - local native Docker and IBM Bluemix Container Service environment, however,
if you have more environments, you can
define corresponding properties with appropriate prefix, for example:
dev_NAMSPACE=
tst_NAMESPACE=
qa_NAMESPACE=
prd_NAMESPCE=
And while running the build pass relevant container type option like '-t|--conttype dev|tst|qa|prd' then the script should set environment variable appropriately.
Note: You may need to update the logic in the autoDeploy.sh as per your requirement.
There are few other important aspect to remember while trying to make your code/script portable among native Docker and IBM Bluemix Container Services. Few of them are listed below:
- Currently IBM Bluemix Container Service only supports Docker Compose version 1 of the docker-compose.yml file format. Refer https://docs.docker.com/compose/compose-file/compose-file-v1/ for detail.
- IBM Bluemix Container Service may not support all Docker or Docker Compose commands or it has other commands that are not found in native Docker. Meaning in certain situation, you may still need to use the 'cf ic' commands instead of native docker command to perform task specific to IBM Bluemix Container Service. See the Supported Docker commands for IBM Bluemix Container Service plug-in (cf ic). The best way to find what native Docker commands are supported within IBM Bluemix or what 'cf ic' commands are available, just run the 'cf ic --help' and you'll see the list. The commands with '(Docker)' at the end are supported Docker commands.
Finally, let's talk about the possible issue(s) that you may encounter.
Error response from daemon:
|
The above mentioned error was encountered while sending build context to IBM Bluemix Container Service. It was because the 'DOCKER_TLS_VERIFY' was set with empty value. You may encounter this error in any case when you are trying to establish secure connection, but any one of the following environment variables is not set correctly:
DOCKER_HOST
DOCKER_CERT_PATH
DOCKER_TLS_VERIFY
2)
ERROR: for lets-chat HTTPSConnectionPool(host='containers-api.ng.bluemix.net', port=8443): Read timed out. (read timeout=60)
|
You may encounter the above mentioned error while executing 'docker-compse up' when request times out. The default read timeout is 60 sec. You can override this value by either defining it in '.env' file or as environment variable. e.g. 'export COMPOSE_HTTP_TIMEOUT=120'. Refer to https://docs.docker.com/compose/reference/envvars/ for all available environment variables.
That's it for this post. Try and let me know. You can find/get/download all files from GitLab here: https://gitlab.com/pppoudel/public_shared/tree/master/container_autodeploy
Looks like you're really into Docker, see my other related blog posts below:
- Using Docker Secrets with IBM WebSphere Liberty Profile Application Server
- Make your container deployment portable (this post)
- Experience sharing - Dock Datacenter
- Setting up CLI environment for IBM Bluemix
- Quick start with IBM Datapower Gateway Docker Edition
Setting up CommandLine Environment for IBM® Bluemix® Container Service
Last week, I had gone through couple of steps to
create an account (free - trial for 30 days) with IBM® Bluemix®, deploy my IBM Container (Docker based) and access my IBM Container running on Bluemix using Bluemix/CloudFoundry command line tools setup locally on my laptop. I have done it to prepare myself and also it's a kind of POC
for upcoming project work. I've decided to share these steps so that other people in the same situation can benefit from it. Below are steps:
Looks like you're really into Docker, see my other related blog posts below:
- Make sure you have your IBM Container deployed and running on IBM Bluemix. If you don't have one follow the below sub steps:
- Create a free account with IBM Bluemix (https://console.ng.bluemix.net/)
- Once the account is created, you can create an IBM Container. See below quick steps:
- Now it's time to setup the command line tools on your local desktop. I have used the Fedora v24.x running as a virtual machine.
- Download Bluemix_CLI_0.4.6_amd64.tar.gz from http://clis.ng.bluemix.net/ui/home.html and extract it to some directory:
$>tar -xvzf ~/Downloads/Bluemix_CLI_0.4.6_amd64.tar.gz
- Among others files, you'll see 'install_bluemix_cli' executable file under /Bluemix_CLI
$>sudo ./install_bluemix_cli
- Once it's installed, download CloudFoundry tool:
$>sudo wget -O /etc/yum.repos.d/cloudfoundry-cli.repo https://packages.cloudfoundry.org/fedora/cloudfoundry-cli.repo
- Install CloudFoundry CLI:
$>sudo yum install cf-cli
...
Installed:
cf-cli.x86_64 0:6.26.0-1 - Check the version:
$>cf -v
cf version 6.23.1+a70deb3.2017-01-13 -
Install the IBM Bluemix Container Service plug-in (cf ic) to use the native Docker CLI. More details about it can be found here.
$>cf install-plugin https://static-ice.ng.bluemix.net/ibm-containers-linux_x64
-
Verify the plugins:
$>cf plugins
Listing Installed Plugins...
OK
Plugin Name Version Command Name Command Help
IBM-Containers 0.8.964 ic IBM Containers plug-in -
It's time to login to CloudFoundry and run you container command to manage your container.
-
Login to Bluemix/CloudFoundry:
$>cf login -a https://api.ng.bluemix.net
Email> purna.poudel@gmail.com
Password>
Authenticating...
OK
-
Login to Container:
$> cf ic login
Deleting old configuration file...
Retrieving client certificates for IBM Containers...
Storing client certificates in /home/osboxes/.ice/certs/...
Storing client certificates in /home/osboxes/.ice/certs/containers-api.ng.bluemix.net/7b9e7846...
OK
The client certificates were retrieved.
Checking local Docker configuration...
OK
Authenticating with the IBM Containers registry host registry.ng.bluemix.net...
OK
You are authenticated with the IBM Containers registry.
...
-
Login to Bluemix/CloudFoundry:
-
It's time to manage your Container(s) from your desktop using command line
-
Let's check our running Container process(es)
$> cf ic ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
d476a406-ba9 registry.ng.bluemix.net/ibmliberty:webProfile7 "" 2 days ago Running 169.46.21.44:9080->9080/tcp, 169.46.21.44:9443->9443/tcp sysgLibertyCont
-
Let's inspect the running Container
$> cf ic inspect d476a406-ba9
[
{
"BluemixApp": null,
"BluemixServices": null,
"Config": {
"AttachStderr": false,
"AttachStdin": false,
"AttachStdout": false,
"Cmd": [],
"Dns": "",
"Env": [
"logging_password=",
"space_id=7b9e7846-0ec8-41da-83e6-209a02e1b14a",
"logstash_target=logmet.opvis.bluemix.net:9091",
"metrics_target=logmet.opvis.bluemix.net:9095"
],
"Hostname": "instance-002eacfa",
"Image": "registry.ng.bluemix.net/ibmliberty:webProfile7",
"ImageArchitecture": "amd64",
"Labels": {
"doc.url": "/docs/images/docker_image_ibmliberty/ibmliberty_starter.html"
},
"Memory": 256,
"MemorySwap": "",
....
-
Let's check our running Container process(es)
Looks like you're really into Docker, see my other related blog posts below:
- Using Docker Secrets with IBM WebSphere Liberty Profile Application Server
- Make your container deployment portable
- Experience sharing - Dock Datacenter
- Setting up CLI environment for IBM Bluemix (this post)
- Quick start with IBM Datapower Gateway Docker Edition
Subscribe to:
Posts (Atom)