Showing posts with label extendedDocumentRoot. Show all posts
Showing posts with label extendedDocumentRoot. Show all posts

How to Make your WAS Static Content Deployment Shareable

   Make your static content (html, java script, Cascading Style Sheets (CSS) etc.) deployment on IBM WebSphere Application Server (WAS) shareable among different applications by taking advantage of the feature in IBM WAS's application deployment descriptor extension. This is specifically useful if several applications require access to a set of common static files. You can place these files in a directory (can be network file system) and configure each application so that the content deployed outside (external to Web Application Archive (WAR) is accessible to each application.
   Usually, the best practice (for performance reason) is to deploy the static content in a front-end Web Server (Apache, IBM HTTP Server (IHS), nginx etc). In this case, we don't have to worry about the application deployment extension feature of WAS. However, if you don't have front-end Web Server but still want to deploy the static content separately (outside of a WAR directory) or have to deal with some unique requirements like the one I'm going to describe below, you can utilise the application deployment extension feature of WAS.

   Here is what I am dealing with on one of the migration projects (migrating from front-end IHS to DataPower Gateway).
   In the old environment, IHS served the static content, which was deployed under it's DocumentRoot directory. The WAS plug-in for IHS filtered the requests (URLs) and only passed requests for Servlet or JSP to back-end WAS.
   Now, as part of the migration project, DataPower Gateway will act as a Proxy as well as load balancer. Even though, Datapower can cache the content and serve to end users, but there is no easy mechanism to deploy the static content there. So, it is decided that WAS is going to serve the static content along with Servlet and JSPs. However, the static content will be packaged and deployed separately.
   In addition to that, during the transition period, users must be able to access the application the usual way (request passes through the front-end IHS) as well as the new way (request passes through the front-end Datapower Gateway). This means, IHS should continue to serve the static content for the traffic coming through it and only pass Servlet or JSP related request to back-end WAS.
   Here is the requirements in point form:
  1. User traffic can pass through either one of the  available two distinct routes as shown in the diagram below.
  2. For user requests that pass through front-end DataPower Gateway to WAS, both application and static content are served by WAS. 
  3. For user requests that pass through IHS to WAS, static content is served by IHS and application is served by WAS. 
  4. Static content is packaged and deployed separately.

The diagram shows user traffic coming though two different routes. Static contents deployed and served accordingly. 

In order to fulfil the above mentioned requirements, we need to deploy the static content as follows:
  1. Deploy static content under DocumentRoot directory of IHS and configure WAS plug-in for IHS so that IHS serves static content from within the DocumentRoot directory and only passes requests for dynamic content (like Servelt, JSP etc) to back-end WAS.
  2. Deploy static content outside of the WAR directory (can be network file system) that is accessible to WAS web container. This is achieved through proper configuration in application deployment descriptor extension file, namely ibm-web-ext.xmi or ibm-web-ext.xml files.
The main focus of this post is to explain how to do the 2nd option (i.e. configure the application deployment descriptor extension file) above. However, we'll also do a quick overview of the 1st option. And we'll do it using a real example. We'll use DefaultApplication.ear, that ships with WAS and play with it. First however, let's have quick overview of these extensions.

ibm-web-ext.xmi vs ibm-web-ext.xml

  • ibm-web-ext.xmi: IBM extension for Java Enterprise Edition (EE) application deployment descriptor and compatible with pre-Java EE 5 application deployment
  • ibm-web-ext.xml: IBM extension for Java EE application deployment descriptor and compatible with Java EE 5 or later application deployment
For more details on compatibility, refer to supported configurations paragraph from https://www.ibm.com/support/knowledgecenter/en/SSAW57_8.5.5/com.ibm.websphere.nd.doc/ae/tweb_jspengine.html
extendedDocumentRoot, and fileServingEnabled are two important properties in ibm-web-ext.xmi or ibm-web-ext.xml that need to be enabled and configured properly in order to support the file-serving (by Servlet) when an application requires access to files that exist outside of the application web application archive (WAR) directory.

extendedDocumentRoot - specifies one or more directory paths (comma separated if more than one) outside of WAR from which static files and Java ServerPages (JSP) files can be served. For details, see extendedDocumentRoot section from https://www.ibm.com/support/knowledgecenter/SSAW57_8.5.5/com.ibm.websphere.nd.doc/ae/cweb_flserv.html

Note: to serve static files from an extended document root directory, you must enable file serving by assigning 'true' value to fileServingEnabled attribute.
Note: there are some other attributes related to extendedDocumentRoot, however, we are not going to discuss about them in this post. Refer to below URLs for details:
   Now, it's time to go through a real example and understand the concept. As a working example, we take DefaultApplication.ear application that ships with IBM WAS, update ibm-web-ext.xmi file accordingly and deploy its static content outside of the WAR directory. I'll also show an example of WAS plug-in for IHS. DataPower Gateway configuration is out of scope for this blog post.
Note: WAS does not support the modification of deployment descriptor extension parameters through the Administrative Console or wsadmin scripting.
   I've put together an Ant based scripting solution, that can update the file serving and extended document root properties in one or more WAR modules within one or more Enterprise Application Archive (EAR). I'll share the scripting solution as well.

Here are the steps to deploy static content outside of WAR

1. Prepare EAR/WAR file and deploy static content.
Since, DefaultApplication.ear is built as pre-JEE 5 compatible, we'll need to update the  ibm-web-ext.xmi that is located under WEB-INF directory of it's web module DefaultWebApplication.war.

Here is look of DefaultApplication.ear before the changes. As you can see ibm-web-ext.xmi file located under WEB-INF directory has attribute fileServingEnabled="false".
DefaultApplication.ear Archive view

DefaultApplication.ear/DefaultWebApplication.war Archive view




As a part of our update, we'll remove the index.html and banner.gif from DefaultWebApplication.war, update the ibm-web-ext.xmi and deploy the EAR file and static content.

Here is how the updated ibm-web-ext.xmi will look like after the changes:

<?xml version="1.0" encoding="UTF-8"?>
<webappext:WebAppExtension xmi:version="2.0" xmlns:xmi="http://www.omg.org/XMI" 
  xmlns:webappext="webappext.xmi" xmlns:webapplication="webapplication.xmi" 
xmlns:commonext.localtran="commonext.localtran.xmi" 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
xmi:id="WebApp_ID_Ext" reloadInterval="3" reloadingEnabled="true" 
fileServingEnabled="true" directoryBrowsingEnabled="false" 
serveServletsByClassnameEnabled="true" preCompileJSPs="false" 
autoRequestEncoding="false" autoResponseEncoding="false">
  <defaultErrorPage xsi:nil="true"/>
  <additionalClassPath xsi:nil="true"/>
  <webApp href="WEB-INF/web.xml#WebApp_ID"/>
  <extendedServlets xmi:id="ServletExtension_1">
    <extendedServlet href="WEB-INF/web.xml#Servlet_1"/>
  </extendedServlets>
  <extendedServlets xmi:id="ServletExtension_2">
    <markupLanguages xmi:id="MarkupLanguage_1" name="HTML" mimeType="text/html" errorPage="Page_1" defaultPage="Page_2">
      <pages xmi:id="Page_2" name="Hello.page" uri="/HelloHTML.jsp"/>
      <pages xmi:id="Page_1" name="Error.page" uri="/HelloHTMLError.jsp"/>
    </markupLanguages>
    <markupLanguages xmi:id="MarkupLanguage_2" name="VXML" mimeType="text/x-vxml" errorPage="Page_3" defaultPage="Page_4">
      <pages xmi:id="Page_4" name="Hello.page" uri="/HelloVXML.jsp"/>
      <pages xmi:id="Page_3" name="Error.page" uri="/HelloVXMLError.jsp"/>
    </markupLanguages>
    <markupLanguages xmi:id="MarkupLanguage_3" name="WML" mimeType="vnd.wap.wml" errorPage="Page_5" defaultPage="Page_6">
      <pages xmi:id="Page_6" name="Hello.page" uri="/HelloWML.jsp"/>
      <pages xmi:id="Page_5" name="Error.page" uri="/HelloWMLError.jsp"/>
    </markupLanguages>
    <extendedServlet href="WEB-INF/web.xml#Servlet_2"/>
  </extendedServlets>
  <extendedServlets xmi:id="ServletExtension_3">
    <extendedServlet href="WEB-INF/web.xml#Servlet_3"/>
    <localTransaction xmi:id="LocalTransaction_1" unresolvedAction="Rollback"/>
  </extendedServlets>
  <fileServingAttributes name="extendedDocumentRoot" value="/opt/ibm/static_contents/" xmi:id="FileServingAttribute_1"/>
</webappext:WebAppExtension>
As seen (highlighted above), fileServingEnabled is changed to "true" and fileServingAttribute "extendedDocumentRoot" has been added with value "/opt/ibm/static_contents"
Once, the ibm-web-ext.xmi is updated, we remove the static content from WAR file and place them under the /opt/ibm/static_contents directory on the server where WAS web container will be able to access them. Recreate the EAR file and Deploy into WAS. Here is how the deployment of static content and EAR looks on server where WAS is running:

Listing of /opt/ibm/static_content:
$> cd /opt/ibm/static_contents
$> ls -la
drwxrwxr-x 2 wasadmin wasadmin 4096 Sep  7 10:28 .
drwxr-xr-x 7 wasadmin wasadmin 4096 Sep  7 10:27 ..
-rw-r--r-- 1 wasadmin wasadmin 3798 Sep  7 10:28 banner.gif
-rw-r--r-- 1 wasadmin wasadmin  667 Sep  7 10:28 index.html

Listing under Deployed DefaultApplication.ear.ear/DefaultWebApplication.war. As you see, static contents are not there.
$> cd /opt/ibm/WebSphere/AppServer/profiles/AppSrv01/installedApps/ubuntuwas9Cell01/DefaultApplication.ear.ear/DefaultWebApplication.war $> ls -la
drwxr-xr-x 4 wasadmin wasadmin 4096 Sep  7 11:03 .
drwxr-xr-x 5 wasadmin wasadmin 4096 Sep  7 11:03 ..
-rw-r--r-- 1 wasadmin wasadmin  742 Aug 20  2015 auth_error.jsp
-rw-r--r-- 1 wasadmin wasadmin  393 Aug 20  2015 HelloHTMLError.jsp
-rw-r--r-- 1 wasadmin wasadmin  849 Aug 20  2015 HelloHTML.jsp
-rw-r--r-- 1 wasadmin wasadmin  461 Aug 20  2015 HelloVXMLError.jsp
-rw-r--r-- 1 wasadmin wasadmin  365 Aug 20  2015 HelloVXML.jsp
-rw-r--r-- 1 wasadmin wasadmin  525 Aug 20  2015 HelloWMLError.jsp
-rw-r--r-- 1 wasadmin wasadmin  476 Aug 20  2015 HelloWML.jsp
-rw-r--r-- 1 wasadmin wasadmin 3284 Aug 20  2015 HitCount.jsp
drwxr-xr-x 2 wasadmin wasadmin 4096 Sep  7 11:03 META-INF
drwxr-xr-x 4 wasadmin wasadmin 4096 Sep  7 11:03 WEB-INF     

Note: If you want to change the ibm-web-ext.xmi or ibm-web-ext.xml file of pre-deployed/running application manually for testing purpose, you need to do it in two locations:
  1. under <was-install-dir>/profiles/<profile-name>/installedApps/<cell-name>/<app-ear-dir>/<app-war-dir>/WEB-INF
  2. under <was-install-dir>/profiles/<profile-name>/config/cells/<cell-name>/applications/<app-ear-dir>/deployments/<app-dir>/<app-war-dir>/WEB-INF
Also, note that if it is network deployment environment, then as soon as you do synchronisation from master configuration repository, your manual changes will be overwritten.

2. Test and Verify
Let's access our default application using the URL: http://<host>:<port>/index.html. As long as we can see the index page with the listing of Snoop, Hello and Hitcount servlets, it is safe to say that the static content is served from the directory defined by extendedDocRoot property of fileServingAttributes, because our repackaged and redeployed application does not contain any static content. However, there is another (more official) way to do validate it as well.
   We can enable WAS WebContainer Trace to confirm where static content is served from. Here is the trace spec. Detail about the WebContainer trace can be found here.

*=info:com.ibm.ws.webcontainer*=all:com.ibm.wsspi.webcontainer*=all:HTTPChannel=all:GenericBNF=all

You can enable the run-time trace as follows:
Launch and access the WAS administrative console and navigate to:

  • Troubleshooting > Logging and tracing > <server-name> Change log detail levels
  • Select "Runtime" tab 
  • Replace the existing "*=info" with the above mentioned trace spec into "Select components and specify a log detail level. ..." text box
  • Click "OK"
There is no need to restart the server. Once the trace is enabled, type the application URL (http://<host>:<port>/index.html) again on your browser and access it. As soon as the index page is rendedered on the browser, look the trace file for your WAS. You'll see something like below:
[9/7/17 11:14:06:498 EDT] 000000a0 servlet       1 com.ibm.ws.webcontainer.servlet.StaticFileServletWrapperImpl handleRequest   request--->/index.html<---
[9/7/17 11:14:06:498 EDT] 000000a0 SRTServletReq 1 com.ibm.ws.webcontainer.srt.SRTServletRequest getWebAppDispatcherContext
[9/7/17 11:14:06:498 EDT] 000000a0 SRTServletReq 1 com.ibm.ws.webcontainer.srt.SRTServletRequest getAttribute this->com.ibm.ws.webcontainer.srt.SRTServletRequest@178b0ff6:  name --> javax.servlet.include.request_uri
[9/7/17 11:14:06:498 EDT] 000000a0 servlet       1 com.ibm.ws.webcontainer.servlet.StaticFileServletWrapperImpl handleRequest relative uri -->[/index.html]
[9/7/17 11:14:06:498 EDT] 000000a0 servlet       1 com.ibm.ws.webcontainer.servlet.StaticFileServletWrapperImpl handleRequest Request path=[/index.html]
[9/7/17 11:14:06:498 EDT] 000000a0 util          > com.ibm.wsspi.webcontainer.util.FileSystem uriCaseCheck ENTRY file canpath=/opt/ibm/static_contents/index.html, matchString=/index.html, checkWEBINF=true
[9/7/17 11:14:06:498 EDT] 000000a0 util          < com.ibm.wsspi.webcontainer.util.FileSystem uriCaseCheck : result=true RETURN
[9/7/17 11:14:06:499 EDT] 000000a0 servlet       > com.ibm.ws.webcontainer.servlet.FileServletWrapper handleRequest com.ibm.ws.webcontainer.servlet.StaticFileServletWrapperImpl@11621b65 ,request-> com.ibm.ws.webcontainer.srt.SRTServletRequest@178b0ff6 ,response-> com.ibm.ws.webcontainer.srt.SRTServletResponse@d3200815 ENTRY
[9/7/17 11:14:06:499 EDT] 000000a0 SRTServletReq 1 com.ibm.ws.webcontainer.srt.SRTServletRequest getRequestURI  uri --> /index.html
[9/7/17 11:14:06:499 EDT] 000000a0 servlet       1 com.ibm.ws.webcontainer.servlet.FileServletWrapper handleRequest   request--->/index.html<---     

As you can see from the highlighted (in yellow) text above, WAS is able to match and route /index.html to /opt/ibm/static_contents/index.html

Serving Static Content From IHS

Since, it is a very general type of deployment, I'm not going to elaborate this topic in detail, but just highlight few things in the context of our requirements above. 
As per the requirement above, users whose application traffic passes through IHS, the static content is served by IHS.  
Below are the steps:

1) Copy/Deploy static content under directory defined by DocumentRoot in httpd.conf.

Listing of /opt/ibm/HTTPServer/htdocs:
$> cd /opt/ibm/HTTPServer/htdocs
$> ls -la
drwxrwxr-x 2 wasadmin wasadmin 4096 Sep  7 10:28 .
drwxr-xr-x 7 wasadmin wasadmin 4096 Sep  7 10:27 ..
-rw-r--r-- 1 wasadmin wasadmin 3798 Sep  7 10:28 banner.gif
-rw-r--r-- 1 wasadmin wasadmin  667 Sep  7 10:28 index.html

Now, we need to make sure that the WAS plug-in for IHS filters the request correctly so that static content is served directly from IHS.

Fragment of (original) generated plugin-cfg.xml, which passes every request to back-end WAS.
<UriGroup Name="default_host_sso_clus_URIs">
   <Uri AffinityCookie="JSESSIONID" AffinityURLIdentifier="jsessionid" Name="/*"/>
</UriGroup>  

Here is a fragment of modified plugin-cfg.xml, which filters the request.  It enforces IHS to serve the content not explicitly listed in given UriGroup.
<UriGroup Name="default_host_sso_clus_URIs">
   <Uri AffinityCookie="JSESSIONID" AffinityURLIdentifier="jsessionid" Name="/snoop/*"/>
   <Uri AffinityCookie="JSESSIONID" AffinityURLIdentifier="jsessionid" Name="/hello"/>
   <Uri AffinityCookie="JSESSIONID" AffinityURLIdentifier="jsessionid" Name="/hitcount"/>
</UriGroup>

2. Test and Verify

Now, let's access our application. This time, we'll use IHS's host and port. IHS in this case is listening on port 80. So, I access it using http://<host>/index.html
Since, I've enabled the trace in my plugin-cfg.xml file, I can see that the index.html and banner.gif is served from IHS itself. See below:

[Thu Sep 07 14:19:13 2017] 00007774 00009d24 - DETAIL: ws_common: websphereShouldHandleRequest: trying to match a route for: vhost='localhost'; uri='/index.html'
[Thu Sep 07 14:19:13 2017] 00007774 00009d24 - DEBUG: ws_common: websphereShouldHandleRequest: NOT config->odrEnabled(reqInfo(ef699b0))
[Thu Sep 07 14:19:13 2017] 00007774 00009d24 - TRACE: ws_common: webspherePortNumberForMatching: Using logical.
[Thu Sep 07 14:19:13 2017] 00007774 00009d24 - TRACE: ws_common: websphereVhostMatch: Comparing '*:80' to 'localhost:80' in VhostGroup: default_host
[Thu Sep 07 14:19:13 2017] 00007774 00009d24 - TRACE: ws_common: websphereVhostMatch: Found a match '*:80' to 'localhost:80' in VhostGroup: default_host with score 1, exact match 0
[Thu Sep 07 14:19:13 2017] 00007774 00009d24 - TRACE: ws_common: websphereVhostMatch: Comparing '*:9081' to 'localhost:80' in VhostGroup: default_host
[Thu Sep 07 14:19:13 2017] 00007774 00009d24 - TRACE: ws_common: websphereVhostMatch: Comparing '*:9080' to 'localhost:80' in VhostGroup: default_host
[Thu Sep 07 14:19:13 2017] 00007774 00009d24 - TRACE: ws_common: websphereUriMatch: Failed to match: /index.html
[Thu Sep 07 14:19:13 2017] 00007774 00009d24 - DETAIL: ws_common: websphereShouldHandleRequest: No route found  
....
....
[Thu Sep 07 14:19:13 2017] 00007774 0000a368 - DETAIL: ws_common: websphereShouldHandleRequest: trying to match a route for: vhost='localhost'; uri='/banner.gif'
[Thu Sep 07 14:19:13 2017] 00007774 0000a368 - DEBUG: ws_common: websphereShouldHandleRequest: NOT config->odrEnabled(reqInfo(ef679f8))
[Thu Sep 07 14:19:13 2017] 00007774 0000a368 - TRACE: ws_common: webspherePortNumberForMatching: Using logical.
[Thu Sep 07 14:19:13 2017] 00007774 0000a368 - TRACE: ws_common: websphereVhostMatch: Comparing '*:80' to 'localhost:80' in VhostGroup: default_host
[Thu Sep 07 14:19:13 2017] 00007774 0000a368 - TRACE: ws_common: websphereVhostMatch: Found a match '*:80' to 'localhost:80' in VhostGroup: default_host with score 1, exact match 0
[Thu Sep 07 14:19:13 2017] 00007774 0000a368 - TRACE: ws_common: websphereVhostMatch: Comparing '*:9081' to 'localhost:80' in VhostGroup: default_host
[Thu Sep 07 14:19:13 2017] 00007774 0000a368 - TRACE: ws_common: websphereVhostMatch: Comparing '*:9080' to 'localhost:80' in VhostGroup: default_host
[Thu Sep 07 14:19:13 2017] 00007774 0000a368 - TRACE: ws_common: websphereUriMatch: Failed to match: /banner.gif
[Thu Sep 07 14:19:13 2017] 00007774 0000a368 - DETAIL: ws_common: websphereShouldHandleRequest: No route found   
As you can see from the highlighted text above it has failed to match the URI /index.html and /banner.gif  as expected. However, the browser is able to render the page, means the content is served by IHS.

Above, I explained the concept of extended document root and how to use it in certain scenarios. As I mentioned above, you need to update the IBM extension file for application deployment descriptor to utilise the extended document root feature, however, WAS does not support the modification of deployment descriptor extension parameters through the Administrative Console or wsadmin scripting, so you have to do it manually. It is OK to manually update, if it is just a one or two web modules, what if you have 10 or hundreds of web modules? I have put together Ant based script just to do that. You can get everything that is required to run the script from GitLab project here:  https://gitlab.com/pppoudel/public_shared/tree/master/extdocroot
You can run the script either using extDocRootUpdater.sh (Unix) or extDocRootUpdater.cmd (Windows). Included README.txt explains some property values that you need to update as per your environment. Here is the content of the README.txt

Purpose:
Use this script to update EAR/WAR file(s) to enable the file serving and add extendedDocumentRoot feature. 
Script updates ibm-web-ext.xml, and/or ibm-web-ext.xmi file(s) in all Web modules inside each EAR file.

Setup:
Script is written using Apache Ant framework. So, it requires Apache Ant and few additional libraries found under ${basedir}/lib
1) ant-contrib-1.0b4.jar - Refer to https://sourceforge.net/projects/ant-contrib/files/ant-contrib/1.0b3/
2) xmltask-1.16c v1.16.jar - Refer to http://www.oopsconsultancy.com/software/xmltask/

Property file: pkgupdate.properties
Open it and update the value of following properties as necessary:

1) installableapps.loc
It is the location where you put all your EAR/WAR files to be updated.  
installableapps.loc=C:/IBM/installableApps

4) failOnError
If failOnError is true, it stops processing further even one update fails. If you want to continue the processing assign 'false' value.
For example:
failOnError=false

5) ext.doc.root, and append.ctxroot
It is a runtime property. The value will be used to formulate the value for extendedDocumentRoot. Here is how the value is formed.
if 'append.ctxroot=true'; then
extendedDocumentRoot=${ext.doc.root}+<root-context>
Note: root-context value is extracted by script from the application.xml
For example:
ext.doc.root=/opt/ibm/static_contents
and root-context=/myapp
then
extendedDocumentRoot=/opt/ibm/static_contents/myapp

if 'append.ctxroot=false'; then
extendedDocumentRoot=${ext.doc.root}

How to run:
> Set JAVA_HOME and ANT_HOME appropriately in extDocRootUpdater.sh or extDocRootUpdater.cmd then 
> launch command prompt (windows) or Shell (Unix/Linux)
> cd to extdocroot directory
> execute extDocRootUpdater.sh or extDocRootUpdater.cmd     

Hope, you found this blog post useful while deciding how to manage and deploy static content for your WAS based application serving platform.