Helping to make a difference one step at a time!

Helping to make a difference one step at a time!  That's what exactly my two daughters and wife did today. They helped to raise fund for United Way Toronto & York Region and climbed 1,776 steps of the 2017 CN Tower.
Christine (grade 10) and Suzanne (grade 7) not only raised funds for great charitable work that United Way does but also enjoyed climbing the CN Tower, and inspired their fellow classmates. I'm so proud of them. Hard work always pays off! It was amazing to see great smile on their faces after successful climb!!!

How to use WLP passwordUtilities feature for encryption/decryption

Recently one of my colleagues asked if there was an easy way to utilise existing IBM WebSphere Application Server  (WAS) provided library to encrypt and decrypt password (or any sensitive information for that matter) from within the Java application deployed on WebSphere Liberty Profile (WLP). After doing some digging around, I found very useful WAS Liberty feature called "passwordUtilities-1.0". Class "com.ibm.websphere.crypto.PasswordUtil" is part of this feature, which provides set of static methods to deal with text encryption/decryption using default 'xor' or 'aes' algorithms. In this post, I'll walk you through, how to use it in you application.
Note: looks like feature "passwordUtilities-1.0" is available only from WLP version 8.5.5.9. Here is how you can check the version of your installed WLP product:

$> cd /opt/ibm/wlp/bin $> ./productInfo version Product name: WebSphere Application Server Product version: 17.0.0.2 Product edition: BASE_ILAN

If you are using WLP version prior to 8.5.5.9, however, still need to use com.ibm.websphere.crypto.PasswordUtil, there is some work-around, which I'll explain later in this post.
Note: If you need to verify whether or not "passwordUtilities-1.0" feature is available for your version of WLP, you can run "installUtility find passwordUtilities*". The feature is supported, if it appears in search result.

$> cd /opt/ibm/wlp/bin $> ./installUtility find passwordUtilities* Establishing a connection to the configured repositories ... ... Searching assets. This process might take several minutes to complete. feature : passwordUtilities-1.0 : Password Utilities

1) Steps for WLP 8.5.5.9 and Later that Support passwordUtilities-1.0 features

1.1) Make sure "passwordUtilities-1.0" is installed.

$> cd /opt/ibm/wlp/bin $> ./installUtility install passwordutilities-1.0 Establishing a connection to the configured repositories ... This process might take several minutes to complete. ... ... Step 5 of 8: Downloading passwordUtilities-1.0 ... Step 6 of 8: Installing passwordUtilities-1.0 ... ... All assets were successfully installed. Start product validation... Product validation completed successfully.

1.2) Make sure feature "passwordUtilities-1.0" is listed under <featureManager> in the server.xml

<featureManager>    ...    <feature>passwordUtilities-1.0</feature> </featureManager>

Note: If you forgot to install the "passwordUtilities-1.0" (step #1), but declared this feature in server.xml (step #2), you'll see an error message in the messages.log like this [10/7/17 20:56:42:807 BST] 0000001f com.ibm.ws.kernel.feature.internal.FeatureManager  E CWWKF0042E: A feature definition cannot  be found for the  passwordutilities-1.0 feature.  Try running the command, bin/installUtility install passwordutilities-1.0,  to install the feature. ...

1.3) [optional] Define encryption key. 

This step is required only if you are using 'aes' encryption/decryption and want to override the default encryption key. The default encryption key can be overridden by setting the wlp.password.encryption.key property. Property can be defined either in server.xml or bootstrap.properties file. However, as a best security practice, define this property in an external XML or properties file and include it in server.xml or bootstrap.properties accordingly. This ensures that the file containing the key is not included in the server dump or package command output. Here we define it in app_enc_key.xml file and put it outside of WLP configuration directory.

/opt/secrets/app_enc_key.xml

<server>     <variable name="wlp.password.encryption.key" value="replaceM3"/>    </server>

Now, include /opt/secrets/app_enc_key.xml in server.xml

<?xml version="1.0" encoding="UTF-8"?> <server description="new server">     <featureManager>         <feature>webProfile-7.0</feature>         <feature>passwordUtilities-1.0</feature>     </featureManager>     <include location="/opt/secrets/my_enc_key.xml"/>     <httpEndpoint id="defaultHttpEndpoint"                   httpPort="9080"                   httpsPort="9443" host="*"/>     <applicationManager autoExpand="true"/>     <application context-root="pwdencdemo" id="pwdencdemo" location="/opt/ibm/apps/pwdencdemo.war" name="pwdencdemo" type="war"/> </server>

1.4) Use PasswordUtil methods in the application.

Put together your application that has encoding/decoding (xor algorithm based) or more robust (aes algorithm based) encryption/decryption. You can use public static method of com.ibm.websphere.crypto.PasswordUtil class. See example below:

String myplainTxtPwd="t3mp_pwD"; // xor encoding String xorEncodedVal=com.ibm.websphere.crypto.PasswordUtil.passwordEncode(myplainTxtPwd, "xor")); // return xorEncodedVal value: {xor}K2wyLwAvKBs= // aes encryption String aesEncodedVal=com.ibm.websphere.crypto.PasswordUtil.passwordEncode(myplainTxtPwd, "aes")); // return aesEncodedVal value: {aes}AM99D1D9JLThDLt32ZCn3VBj27kuwNp5MorW6Ig/7h5h // xor decoding String xorDecodedVal=com.ibm.websphere.crypto.PasswordUtil.passwordDecode("{xor}K2wyLwAvKBs=")); // return xorDecodedVal value: t3mp_pwD // aes decryption String aesDecodedVal=com.ibm.websphere.crypto.PasswordUtil.passwordDecode("{aes}AM99D1D9JLThDLt32ZCn3VBj27kuwNp5MorW6Ig/7h5h")); // return aesDecodedVal value: t3mp_pwD

For full details of all available methods see Java documentation of  com.ibm.websphere.crypto.PasswordUtil. 

I have created a GitLab project, which contains server.xml and a simple web application pwdencdemo.war, which you can deploy into your WLP server and test the working of encryption/decryption.

2) Steps for WLP 8.5.5.8 and Prior version.

2.1) Define Java library and configure classloader for the application. 

Since "passwordUtilities-1.0" feature is not available in WLP 8.5.5.8 or earlier versions, in order to use public methods of com.ibm.websphere.crypto.PasswordUtil, we need to make sure this class and any other dependent classes are accessible for the application in run time. The easiest way to do it is to define a Java library (that includes com.ibm.ws.crypto.passwordutil_<version>.jar, com.ibm.ws.kernel.service_<version>.jar, and  com.ibm.ws.logging_<version>.jar), include in the application itself, and configure classloader for the application.  These jar files come with WLP product and located under ${wlp.install.dir}/lib directory. See below for example:

<application context-root="pwdencdemo" id="pwdencdemo" location="/opt/ibm/apps/pwdencdemo.war" name="pwdencdemo" type="war">     <classloader>        <privatelibrary>           <fileset dir="${wlp.install.dir}/lib" includes="com.ibm.ws.crypto.passwordutil_*.jar,com.ibm.ws.kernel.service_*.jar,com.ibm.ws.logging_*.jar">           </fileset>        </privatelibrary>     </classloader>  </application>

See IBM Knowledge Center documentation for more detail on configuring Java library.
Note: Make sure your library configuration is correct and all listed jar files are available in the given location, otherwise, you may get java.lang.NoClassDefFoundError: com/ibm/websphere/crypto/PasswordUtil exception.

2.2) Follow steps #1.3 and #1.4 from from the previous section.

Hope, this post will be helpful for your current or next WLP based project. If you are deploying your application in Docker container, visit my blog <<Using Docker Secrets with IBM WebSphere Liberty Profile Application Server>>

Note: If you just need to encode/encrypt password to put into your server.xml or bootstrap.properties, you can use securityUtility tool that comes with WLP product. Below is a simple example:

$> cd /opt/ibm/wlp/bin $> ./securityUtility encode --encoding=aes --key=replaceM3 t3mp_pwD

For more detail on securityUtility , refer to https://www.ibm.com/support/knowledgecenter/en/SSEQTP_8.5.5/com.ibm.websphere.wlp.doc/ae/rwlp_command_securityutil.html


Looks like you're really interested in WebSphere Liberty Profile, see my other related blog posts below:

• Using Docker Secrets with IBM WebSphere Liberty Profile Application Server
• How to use WLP passwordUtilities feature for encryption/decryption (this post)
• How to Create and Configure WebSphere Liberty Cluster End-to-End
• How to Deploy Application in WebSphere Liberty Cluster
• How to Setup Front-End Web Server for WebSphere Liberty Cluster

How to Make your WAS Static Content Deployment Shareable

   Make your static content (html, java script, Cascading Style Sheets (CSS) etc.) deployment on IBM WebSphere Application Server (WAS) shareable among different applications by taking advantage of the feature in IBM WAS's application deployment descriptor extension. This is specifically useful if several applications require access to a set of common static files. You can place these files in a directory (can be network file system) and configure each application so that the content deployed outside (external to Web Application Archive (WAR) is accessible to each application.
   Usually, the best practice (for performance reason) is to deploy the static content in a front-end Web Server (Apache, IBM HTTP Server (IHS), nginx etc). In this case, we don't have to worry about the application deployment extension feature of WAS. However, if you don't have front-end Web Server but still want to deploy the static content separately (outside of a WAR directory) or have to deal with some unique requirements like the one I'm going to describe below, you can utilise the application deployment extension feature of WAS.

   Here is what I am dealing with on one of the migration projects (migrating from front-end IHS to DataPower Gateway).
   In the old environment, IHS served the static content, which was deployed under it's DocumentRoot directory. The WAS plug-in for IHS filtered the requests (URLs) and only passed requests for Servlet or JSP to back-end WAS.
   Now, as part of the migration project, DataPower Gateway will act as a Proxy as well as load balancer. Even though, Datapower can cache the content and serve to end users, but there is no easy mechanism to deploy the static content there. So, it is decided that WAS is going to serve the static content along with Servlet and JSPs. However, the static content will be packaged and deployed separately.
   In addition to that, during the transition period, users must be able to access the application the usual way (request passes through the front-end IHS) as well as the new way (request passes through the front-end Datapower Gateway). This means, IHS should continue to serve the static content for the traffic coming through it and only pass Servlet or JSP related request to back-end WAS.
   Here is the requirements in point form:

1. User traffic can pass through either one of the  available two distinct routes as shown in the diagram below.
2. For user requests that pass through front-end DataPower Gateway to WAS, both application and static content are served by WAS. 
3. For user requests that pass through IHS to WAS, static content is served by IHS and application is served by WAS. 
4. Static content is packaged and deployed separately.

The diagram shows user traffic coming though two different routes. Static contents deployed and served accordingly. 

In order to fulfil the above mentioned requirements, we need to deploy the static content as follows:

1. Deploy static content under DocumentRoot directory of IHS and configure WAS plug-in for IHS so that IHS serves static content from within the DocumentRoot directory and only passes requests for dynamic content (like Servelt, JSP etc) to back-end WAS.
2. Deploy static content outside of the WAR directory (can be network file system) that is accessible to WAS web container. This is achieved through proper configuration in application deployment descriptor extension file, namely ibm-web-ext.xmi or ibm-web-ext.xml files.

The main focus of this post is to explain how to do the 2nd option (i.e. configure the application deployment descriptor extension file) above. However, we'll also do a quick overview of the 1st option. And we'll do it using a real example. We'll use DefaultApplication.ear, that ships with WAS and play with it. First however, let's have quick overview of these extensions.

ibm-web-ext.xmi vs ibm-web-ext.xml

• ibm-web-ext.xmi: IBM extension for Java Enterprise Edition (EE) application deployment descriptor and compatible with pre-Java EE 5 application deployment
• ibm-web-ext.xml: IBM extension for Java EE application deployment descriptor and compatible with Java EE 5 or later application deployment

For more details on compatibility, refer to supported configurations paragraph from https://www.ibm.com/support/knowledgecenter/en/SSAW57_8.5.5/com.ibm.websphere.nd.doc/ae/tweb_jspengine.html
extendedDocumentRoot, and fileServingEnabled are two important properties in ibm-web-ext.xmi or ibm-web-ext.xml that need to be enabled and configured properly in order to support the file-serving (by Servlet) when an application requires access to files that exist outside of the application web application archive (WAR) directory.

extendedDocumentRoot - specifies one or more directory paths (comma separated if more than one) outside of WAR from which static files and Java ServerPages (JSP) files can be served. For details, see extendedDocumentRoot section from https://www.ibm.com/support/knowledgecenter/SSAW57_8.5.5/com.ibm.websphere.nd.doc/ae/cweb_flserv.html

Note: to serve static files from an extended document root directory, you must enable file serving by assigning 'true' value to fileServingEnabled attribute.

Note: there are some other attributes related to extendedDocumentRoot, however, we are not going to discuss about them in this post. Refer to below URLs for details:

• https://www.ibm.com/support/knowledgecenter/SSAW57_8.5.5/com.ibm.websphere.nd.doc/ae/cweb_flserv.html
• https://www.ibm.com/support/knowledgecenter/en/SSAW57_8.5.5/com.ibm.websphere.nd.doc/ae/rweb_jspengine.html 

   Now, it's time to go through a real example and understand the concept. As a working example, we take DefaultApplication.ear application that ships with IBM WAS, update ibm-web-ext.xmi file accordingly and deploy its static content outside of the WAR directory. I'll also show an example of WAS plug-in for IHS. DataPower Gateway configuration is out of scope for this blog post.
Note: WAS does not support the modification of deployment descriptor extension parameters through the Administrative Console or wsadmin scripting.
   I've put together an Ant based scripting solution, that can update the file serving and extended document root properties in one or more WAR modules within one or more Enterprise Application Archive (EAR). I'll share the scripting solution as well.

Here are the steps to deploy static content outside of WAR

1. Prepare EAR/WAR file and deploy static content.
Since, DefaultApplication.ear is built as pre-JEE 5 compatible, we'll need to update the  ibm-web-ext.xmi that is located under WEB-INF directory of it's web module DefaultWebApplication.war.

Here is look of DefaultApplication.ear before the changes. As you can see ibm-web-ext.xmi file located under WEB-INF directory has attribute fileServingEnabled="false".

As a part of our update, we'll remove the index.html and banner.gif from DefaultWebApplication.war, update the ibm-web-ext.xmi and deploy the EAR file and static content.

Here is how the updated ibm-web-ext.xmi will look like after the changes:

<?xml version="1.0" encoding="UTF-8"?> <webappext:WebAppExtension xmi:version="2.0" xmlns:xmi="http://www.omg.org/XMI"    xmlns:webappext="webappext.xmi" xmlns:webapplication="webapplication.xmi"  xmlns:commonext.localtran="commonext.localtran.xmi"  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"  xmi:id="WebApp_ID_Ext" reloadInterval="3" reloadingEnabled="true"  fileServingEnabled="true" directoryBrowsingEnabled="false"  serveServletsByClassnameEnabled="true" preCompileJSPs="false"  autoRequestEncoding="false" autoResponseEncoding="false">   <defaultErrorPage xsi:nil="true"/>   <additionalClassPath xsi:nil="true"/>   <webApp href="WEB-INF/web.xml#WebApp_ID"/>   <extendedServlets xmi:id="ServletExtension_1">     <extendedServlet href="WEB-INF/web.xml#Servlet_1"/>   </extendedServlets>   <extendedServlets xmi:id="ServletExtension_2">     <markupLanguages xmi:id="MarkupLanguage_1" name="HTML" mimeType="text/html" errorPage="Page_1" defaultPage="Page_2">       <pages xmi:id="Page_2" name="Hello.page" uri="/HelloHTML.jsp"/>       <pages xmi:id="Page_1" name="Error.page" uri="/HelloHTMLError.jsp"/>     </markupLanguages>     <markupLanguages xmi:id="MarkupLanguage_2" name="VXML" mimeType="text/x-vxml" errorPage="Page_3" defaultPage="Page_4">       <pages xmi:id="Page_4" name="Hello.page" uri="/HelloVXML.jsp"/>       <pages xmi:id="Page_3" name="Error.page" uri="/HelloVXMLError.jsp"/>     </markupLanguages>     <markupLanguages xmi:id="MarkupLanguage_3" name="WML" mimeType="vnd.wap.wml" errorPage="Page_5" defaultPage="Page_6">       <pages xmi:id="Page_6" name="Hello.page" uri="/HelloWML.jsp"/>       <pages xmi:id="Page_5" name="Error.page" uri="/HelloWMLError.jsp"/>     </markupLanguages>     <extendedServlet href="WEB-INF/web.xml#Servlet_2"/>   </extendedServlets>   <extendedServlets xmi:id="ServletExtension_3">     <extendedServlet href="WEB-INF/web.xml#Servlet_3"/>     <localTransaction xmi:id="LocalTransaction_1" unresolvedAction="Rollback"/>   </extendedServlets>   <fileServingAttributes name="extendedDocumentRoot" value="/opt/ibm/static_contents/" xmi:id="FileServingAttribute_1"/> </webappext:WebAppExtension>

As seen (highlighted above), fileServingEnabled is changed to "true" and fileServingAttribute "extendedDocumentRoot" has been added with value "/opt/ibm/static_contents"

Once, the ibm-web-ext.xmi is updated, we remove the static content from WAR file and place them under the /opt/ibm/static_contents directory on the server where WAS web container will be able to access them. Recreate the EAR file and Deploy into WAS. Here is how the deployment of static content and EAR looks on server where WAS is running:

Listing of /opt/ibm/static_content:

$> cd /opt/ibm/static_contents $> ls -la drwxrwxr-x 2 wasadmin wasadmin 4096 Sep  7 10:28 . drwxr-xr-x 7 wasadmin wasadmin 4096 Sep  7 10:27 .. -rw-r--r-- 1 wasadmin wasadmin 3798 Sep  7 10:28 banner.gif -rw-r--r-- 1 wasadmin wasadmin  667 Sep  7 10:28 index.html

Listing under Deployed DefaultApplication.ear.ear/DefaultWebApplication.war. As you see, static contents are not there.

$> cd /opt/ibm/WebSphere/AppServer/profiles/AppSrv01/installedApps/ubuntuwas9Cell01/DefaultApplication.ear.ear/DefaultWebApplication.war $> ls -la drwxr-xr-x 4 wasadmin wasadmin 4096 Sep  7 11:03 . drwxr-xr-x 5 wasadmin wasadmin 4096 Sep  7 11:03 .. -rw-r--r-- 1 wasadmin wasadmin  742 Aug 20  2015 auth_error.jsp -rw-r--r-- 1 wasadmin wasadmin  393 Aug 20  2015 HelloHTMLError.jsp -rw-r--r-- 1 wasadmin wasadmin  849 Aug 20  2015 HelloHTML.jsp -rw-r--r-- 1 wasadmin wasadmin  461 Aug 20  2015 HelloVXMLError.jsp -rw-r--r-- 1 wasadmin wasadmin  365 Aug 20  2015 HelloVXML.jsp -rw-r--r-- 1 wasadmin wasadmin  525 Aug 20  2015 HelloWMLError.jsp -rw-r--r-- 1 wasadmin wasadmin  476 Aug 20  2015 HelloWML.jsp -rw-r--r-- 1 wasadmin wasadmin 3284 Aug 20  2015 HitCount.jsp drwxr-xr-x 2 wasadmin wasadmin 4096 Sep  7 11:03 META-INF drwxr-xr-x 4 wasadmin wasadmin 4096 Sep  7 11:03 WEB-INF

Note: If you want to change the ibm-web-ext.xmi or ibm-web-ext.xml file of pre-deployed/running application manually for testing purpose, you need to do it in two locations:

1. under <was-install-dir>/profiles/<profile-name>/installedApps/<cell-name>/<app-ear-dir>/<app-war-dir>/WEB-INF
2. under <was-install-dir>/profiles/<profile-name>/config/cells/<cell-name>/applications/<app-ear-dir>/deployments/<app-dir>/<app-war-dir>/WEB-INF

Also, note that if it is network deployment environment, then as soon as you do synchronisation from master configuration repository, your manual changes will be overwritten.

2. Test and Verify
Let's access our default application using the URL: http://<host>:<port>/index.html. As long as we can see the index page with the listing of Snoop, Hello and Hitcount servlets, it is safe to say that the static content is served from the directory defined by extendedDocRoot property of fileServingAttributes, because our repackaged and redeployed application does not contain any static content. However, there is another (more official) way to do validate it as well.
   We can enable WAS WebContainer Trace to confirm where static content is served from. Here is the trace spec. Detail about the WebContainer trace can be found here.

*=info:com.ibm.ws.webcontainer*=all:com.ibm.wsspi.webcontainer*=all:HTTPChannel=all:GenericBNF=all

You can enable the run-time trace as follows:
Launch and access the WAS administrative console and navigate to:

• Troubleshooting > Logging and tracing > <server-name> Change log detail levels
• Select "Runtime" tab 
• Replace the existing "*=info" with the above mentioned trace spec into "Select components and specify a log detail level. ..." text box
• Click "OK"

There is no need to restart the server. Once the trace is enabled, type the application URL (http://<host>:<port>/index.html) again on your browser and access it. As soon as the index page is rendedered on the browser, look the trace file for your WAS. You'll see something like below:

[9/7/17 11:14:06:498 EDT] 000000a0 servlet       1 com.ibm.ws.webcontainer.servlet.StaticFileServletWrapperImpl handleRequest   request--->/index.html<--- [9/7/17 11:14:06:498 EDT] 000000a0 SRTServletReq 1 com.ibm.ws.webcontainer.srt.SRTServletRequest getWebAppDispatcherContext [9/7/17 11:14:06:498 EDT] 000000a0 SRTServletReq 1 com.ibm.ws.webcontainer.srt.SRTServletRequest getAttribute this->com.ibm.ws.webcontainer.srt.SRTServletRequest@178b0ff6:  name --> javax.servlet.include.request_uri [9/7/17 11:14:06:498 EDT] 000000a0 servlet       1 com.ibm.ws.webcontainer.servlet.StaticFileServletWrapperImpl handleRequest relative uri -->[/index.html] [9/7/17 11:14:06:498 EDT] 000000a0 servlet       1 com.ibm.ws.webcontainer.servlet.StaticFileServletWrapperImpl handleRequest Request path=[/index.html] [9/7/17 11:14:06:498 EDT] 000000a0 util          > com.ibm.wsspi.webcontainer.util.FileSystem uriCaseCheck ENTRY file canpath=/opt/ibm/static_contents/index.html, matchString=/index.html, checkWEBINF=true [9/7/17 11:14:06:498 EDT] 000000a0 util          < com.ibm.wsspi.webcontainer.util.FileSystem uriCaseCheck : result=true RETURN [9/7/17 11:14:06:499 EDT] 000000a0 servlet       > com.ibm.ws.webcontainer.servlet.FileServletWrapper handleRequest com.ibm.ws.webcontainer.servlet.StaticFileServletWrapperImpl@11621b65 ,request-> com.ibm.ws.webcontainer.srt.SRTServletRequest@178b0ff6 ,response-> com.ibm.ws.webcontainer.srt.SRTServletResponse@d3200815 ENTRY [9/7/17 11:14:06:499 EDT] 000000a0 SRTServletReq 1 com.ibm.ws.webcontainer.srt.SRTServletRequest getRequestURI  uri --> /index.html [9/7/17 11:14:06:499 EDT] 000000a0 servlet       1 com.ibm.ws.webcontainer.servlet.FileServletWrapper handleRequest   request--->/index.html<---

As you can see from the highlighted (in yellow) text above, WAS is able to match and route /index.html to /opt/ibm/static_contents/index.html

Serving Static Content From IHS

Since, it is a very general type of deployment, I'm not going to elaborate this topic in detail, but just highlight few things in the context of our requirements above. 

As per the requirement above, users whose application traffic passes through IHS, the static content is served by IHS.  

Below are the steps:

1) Copy/Deploy static content under directory defined by DocumentRoot in httpd.conf.

Listing of /opt/ibm/HTTPServer/htdocs:

$> cd /opt/ibm/HTTPServer/htdocs $> ls -la drwxrwxr-x 2 wasadmin wasadmin 4096 Sep  7 10:28 . drwxr-xr-x 7 wasadmin wasadmin 4096 Sep  7 10:27 .. -rw-r--r-- 1 wasadmin wasadmin 3798 Sep  7 10:28 banner.gif -rw-r--r-- 1 wasadmin wasadmin  667 Sep  7 10:28 index.html

Now, we need to make sure that the WAS plug-in for IHS filters the request correctly so that static content is served directly from IHS.

Fragment of (original) generated plugin-cfg.xml, which passes every request to back-end WAS.

<UriGroup Name="default_host_sso_clus_URIs">    <Uri AffinityCookie="JSESSIONID" AffinityURLIdentifier="jsessionid" Name="/*"/> </UriGroup>

Here is a fragment of modified plugin-cfg.xml, which filters the request.  It enforces IHS to serve the content not explicitly listed in given UriGroup.

<UriGroup Name="default_host_sso_clus_URIs">    <Uri AffinityCookie="JSESSIONID" AffinityURLIdentifier="jsessionid" Name="/snoop/*"/>    <Uri AffinityCookie="JSESSIONID" AffinityURLIdentifier="jsessionid" Name="/hello"/>    <Uri AffinityCookie="JSESSIONID" AffinityURLIdentifier="jsessionid" Name="/hitcount"/> </UriGroup>

2. Test and Verify

Now, let's access our application. This time, we'll use IHS's host and port. IHS in this case is listening on port 80. So, I access it using http://<host>/index.html

Since, I've enabled the trace in my plugin-cfg.xml file, I can see that the index.html and banner.gif is served from IHS itself. See below:

[Thu Sep 07 14:19:13 2017] 00007774 00009d24 - DETAIL: ws_common: websphereShouldHandleRequest: trying to match a route for: vhost='localhost'; uri='/index.html' [Thu Sep 07 14:19:13 2017] 00007774 00009d24 - DEBUG: ws_common: websphereShouldHandleRequest: NOT config->odrEnabled(reqInfo(ef699b0)) [Thu Sep 07 14:19:13 2017] 00007774 00009d24 - TRACE: ws_common: webspherePortNumberForMatching: Using logical. [Thu Sep 07 14:19:13 2017] 00007774 00009d24 - TRACE: ws_common: websphereVhostMatch: Comparing '*:80' to 'localhost:80' in VhostGroup: default_host [Thu Sep 07 14:19:13 2017] 00007774 00009d24 - TRACE: ws_common: websphereVhostMatch: Found a match '*:80' to 'localhost:80' in VhostGroup: default_host with score 1, exact match 0 [Thu Sep 07 14:19:13 2017] 00007774 00009d24 - TRACE: ws_common: websphereVhostMatch: Comparing '*:9081' to 'localhost:80' in VhostGroup: default_host [Thu Sep 07 14:19:13 2017] 00007774 00009d24 - TRACE: ws_common: websphereVhostMatch: Comparing '*:9080' to 'localhost:80' in VhostGroup: default_host [Thu Sep 07 14:19:13 2017] 00007774 00009d24 - TRACE: ws_common: websphereUriMatch: Failed to match: /index.html [Thu Sep 07 14:19:13 2017] 00007774 00009d24 - DETAIL: ws_common: websphereShouldHandleRequest: No route found   .... .... [Thu Sep 07 14:19:13 2017] 00007774 0000a368 - DETAIL: ws_common: websphereShouldHandleRequest: trying to match a route for: vhost='localhost'; uri='/banner.gif' [Thu Sep 07 14:19:13 2017] 00007774 0000a368 - DEBUG: ws_common: websphereShouldHandleRequest: NOT config->odrEnabled(reqInfo(ef679f8)) [Thu Sep 07 14:19:13 2017] 00007774 0000a368 - TRACE: ws_common: webspherePortNumberForMatching: Using logical. [Thu Sep 07 14:19:13 2017] 00007774 0000a368 - TRACE: ws_common: websphereVhostMatch: Comparing '*:80' to 'localhost:80' in VhostGroup: default_host [Thu Sep 07 14:19:13 2017] 00007774 0000a368 - TRACE: ws_common: websphereVhostMatch: Found a match '*:80' to 'localhost:80' in VhostGroup: default_host with score 1, exact match 0 [Thu Sep 07 14:19:13 2017] 00007774 0000a368 - TRACE: ws_common: websphereVhostMatch: Comparing '*:9081' to 'localhost:80' in VhostGroup: default_host [Thu Sep 07 14:19:13 2017] 00007774 0000a368 - TRACE: ws_common: websphereVhostMatch: Comparing '*:9080' to 'localhost:80' in VhostGroup: default_host [Thu Sep 07 14:19:13 2017] 00007774 0000a368 - TRACE: ws_common: websphereUriMatch: Failed to match: /banner.gif [Thu Sep 07 14:19:13 2017] 00007774 0000a368 - DETAIL: ws_common: websphereShouldHandleRequest: No route found

As you can see from the highlighted text above it has failed to match the URI /index.html and /banner.gif  as expected. However, the browser is able to render the page, means the content is served by IHS.

Above, I explained the concept of extended document root and how to use it in certain scenarios. As I mentioned above, you need to update the IBM extension file for application deployment descriptor to utilise the extended document root feature, however, WAS does not support the modification of deployment descriptor extension parameters through the Administrative Console or wsadmin scripting, so you have to do it manually. It is OK to manually update, if it is just a one or two web modules, what if you have 10 or hundreds of web modules? I have put together Ant based script just to do that. You can get everything that is required to run the script from GitLab project here:  https://gitlab.com/pppoudel/public_shared/tree/master/extdocroot
You can run the script either using extDocRootUpdater.sh (Unix) or extDocRootUpdater.cmd (Windows). Included README.txt explains some property values that you need to update as per your environment. Here is the content of the README.txt

Purpose: Use this script to update EAR/WAR file(s) to enable the file serving and add extendedDocumentRoot feature.  Script updates ibm-web-ext.xml, and/or ibm-web-ext.xmi file(s) in all Web modules inside each EAR file. Setup: Script is written using Apache Ant framework. So, it requires Apache Ant and few additional libraries found under ${basedir}/lib 1) ant-contrib-1.0b4.jar - Refer to https://sourceforge.net/projects/ant-contrib/files/ant-contrib/1.0b3/ 2) xmltask-1.16c v1.16.jar - Refer to http://www.oopsconsultancy.com/software/xmltask/ Property file: pkgupdate.properties Open it and update the value of following properties as necessary: 1) installableapps.loc It is the location where you put all your EAR/WAR files to be updated.   installableapps.loc=C:/IBM/installableApps 4) failOnError If failOnError is true, it stops processing further even one update fails. If you want to continue the processing assign 'false' value. For example: failOnError=false 5) ext.doc.root, and append.ctxroot It is a runtime property. The value will be used to formulate the value for extendedDocumentRoot. Here is how the value is formed. if 'append.ctxroot=true'; then extendedDocumentRoot=${ext.doc.root}+<root-context> Note: root-context value is extracted by script from the application.xml For example: ext.doc.root=/opt/ibm/static_contents and root-context=/myapp then extendedDocumentRoot=/opt/ibm/static_contents/myapp if 'append.ctxroot=false'; then extendedDocumentRoot=${ext.doc.root} How to run: > Set JAVA_HOME and ANT_HOME appropriately in extDocRootUpdater.sh or extDocRootUpdater.cmd then  > launch command prompt (windows) or Shell (Unix/Linux) > cd to extdocroot directory > execute extDocRootUpdater.sh or extDocRootUpdater.cmd

Hope, you found this blog post useful while deciding how to manage and deploy static content for your WAS based application serving platform.

European Journey aboard Epic

Recently (just two weeks ago), my family and I returned home to Canada from an amazing seven day European (Western Mediterranean Sea) cruise aboard Norwegian Cruise Line (NCL) Epic. No, we didn’t win an all-expenses-paid trip to Europe, but rather paid all expenses out of our own pockets :( Good thing that unlike the 1985 comedy film European Vacation, we came back with good memories, not controversies and conflicts.

Here is how Norwegian Epic looked like.
Picture taken from Cannes' Beach

We planned (for the trip) for a while, almost few months, since we purchased our Norwegian Cruise Line (NCL) Epic ticket from Costco Travel, back in March 2017, and the execution went almost flawlessly, except some minor issues here and there . Most of the work went on to devise a plan for shore excursions, because NCL's official excursion rate is way too expensive. We also planned how to stay safe, and secure our cash while in Italy and Spain. A few Travel books suggested that Barcelona, Rome and Naples are Pickpocketing Hotspots. I bought a Sling Chest Bag from Amazon. I had a hard time finding one, as I needed a sling bag that fits an iPad pro. Fortunately, we didn't have to face any pickpocketing. We found Italian, Spanish and French people very friendly, and helpful.

Note:

• Make sure to check in online through NCL website at least 48 hours before your cruise departure schedule. 
• Reserve your on-board entertainment (they quickly run out of seat). If you want, you can start registering for them few months in advance. 
• Print baggage tags in advance.
• Plan your shore excursions. You can purchase directly from NCL or plan on your own.

In terms of planning for shore excursions, I read travel books, researched on internet (used sites like https://www.rometoolkit.com/), used information available on NCL website as well as Cruise Critic site. My friend and I tried to devise a detailed plan for each city that we were going to go for shore excursions in a separate spread sheet tab with three alternate options. We put together a check list. We reviewed our plan by all members of both families (total seven people) few times and a final time just a week before flying to Barcelona.

In short, here is how our overall travel plan looked like:

• Flying to Barcelona from Toronto. 
• Spend a night in Barcelona. 
• Take cruise 2nd day from Barcelona.
• Take shore excursion in each city where ship docks and arrive back to Barcelona after seven days 
• Fly back to Toronto the same day. 

Below map shows are cruise schedule.

NCL Epic route. Image taken from NCL website.

Flying to Barcelona

We flew on Air Transat. Because of the Baggage Handlers' strike, we had a small delay in Toronto Pearson airport. Good thing, we had two kids belonging to Air Transat Kids club, we were prioritized while boarding the plane in Toronto.

Note: Make sure to register (free) your kid(s) to Kids club, you also get free seat selection and other surprise goodies.

While the service provided by Air Transat was very satisfactory, I didn't like the complementary lunch/dinner sandwich. It was kind of tasteless. However, I enjoyed the complementary glass of wine. Thanks Air Transat! Hope, you'll serve better sandwiches next time :)
Passing through the Spanish custom/immigration was hassle free. I said 'Hola' and handed over our family's passports to the immigration officer, he stamped them and let us go. No baggage delay. Hurrah!

In Barcelona

While we were heading out (of the airport) after collecting our baggage, my friend shouted, "Oh look, you're famous here in Barcelona. Someone is holding your name placard". "Yeah, you guessed it right, my friend", I said and I waved to the person holding my name. I wish I was famous, but it's just that I had pre-purchased (online) airport to hotel taxi/van service from suntransfers.com. They charged EUR 58.87 for a seven seater van. It had good and reliable service which I found as a very satisfactory deal. If we hired a van at the airport, we could have saved around EUR 10.00 or so, but we had to wait in a big line at the airport, negotiate and all that stuff. Our driver was pretty courteous, helpful, friendly and knowledgeable. My friend, with his just-in-time Spanish (I guessed little better than Google translate), was able to acquire a lot of information from the driver.

Finally, we were gliding on the street of great football/soccer city of Barcelona. We were so amazed looking around through our car window, we even didn't notice when we arrived at the hotel. Our driver had to knock us hard to let us know. Our hotel Garbí Mil·lenni (located near the cruise port and centre of the city), was nice. I had booked the junior suite for EUR 189.00.
We were greeted by Marco at reception. He was a nice Spanish fellow with a very business like attitude.

After taking a shower and recharging ourselves, we walked towards the world famous La Rambla street. Yes indeed, we had sometime for walking around and getting ourselves lost as we flew to Barcelona one day ahead of our cruise schedule. Even though, one and half day is not really enough time to visit all the places in the city of Barcelona - a city with rich history and great architecture, it's still better than arriving the same day and rushing to the cruise. So, I recommend anyone taking cruise abroad, get there at least a day or two ahead and avoid last minute rush.

Here, I'm walking on famous La Rambla street.

Special note about La Rambla: My family and I are deeply socked and saddened by the news of  the tragic loss of lives and injuries on La Rambla street caused by the terrorist attack. I’d like to dedicate this post for those innocent souls. I’d like to express my deepest sympathy to all those who have been affected by this terrible act. Read more about La Rambla terrorist attack here.

We had just enough time to tour the city on the Hop On Hop Out (HOHO) bus and we had purchased the ticket online in advance to save money (10% discount if you purchase online) and time. Barcelona has three (green, blue, and red) official HOHO bus routes . The first day, we took red and in the morning of 2nd day, we took blue line tour. HOHO bus seemed like a very convenient option for us;  we could get off at any stop where we wished to take more time to look around and could hop back on at the same stop or another.

Note: Another convenient feature of HOHO bus is ticket validity period. It comes with either 24 or 48 hours option. During that time frame, you can use it any number of times in any of the designated route.

A street view of Barcelona 
(picture taken from the top deck of HOHO bus).

On the 2nd day, since we had to checkout from the hotel before noon and rush to cruise, we thought, we didn't have enough time to walk to La Rambla and take HOHO bus to tour the blue line, we decided to take a taxi instead from our hotel to visit the Antoni Gaudi's famous architectural project Sagrada Familia. The hotel receptionist called the taxi for us and we headed to Sagrada Familia. I paid EUR 16.00 for the trip. Luckily, just after spending sometime at Sagrada Familia, I saw the blue line station right beside the Basilica and we hopped on the HOHO bus and ventured the blue line. 
The blue line along with Sagrada Familia and other historic and architectural objects, include:
-Plaça Catalunya – The main square in the heart of historic city.
-Casa Batlló – The dragon-inspired house designed by Gaudí
-Park Guëll – An unique park from where you can see majestic city views.
-Camp Nou Stadium – The home stadium of FC Barcelona.

Sagrada Família, designed by Antoni Gaudí,
still under construction
(Started 1862 and anticipated completion date of 2026)

Tourist hot spot - La Rambla

Casa Milà/La Pedrera, designed by Antoni Gaudí
and built between 1906 and 1910,
(picture taken from the top deck of HOHO bus)

At around 12:00 PM, we checked out of the hotel, and headed to the cruise. We got a seven seater van and the driver charged us EUR 25.00. Not a bad deal! The driver was cheerful and helpful and became our unofficial guide too. On our way to the cruise, I could see a huge ship from a distance, parked at the port. But, my younger daughter was saying that she could not see it. She worriedly said, “how come I can’t see it?". Later she laughed to herself, she was under impression that the huge structure looking like a multi story building was in fact our ship. Norwegian Epic is one of the largest ships. Third largest? As per Wikipedia. It accommodates more than 4000 people on board. See more details on the NCL site.

We were in a long queue of people on hot summer day in Barcelona, desperate to check into  the cruise to have that so called first time (or may be nth time) cruise experience.  Standing in line was a daunting task, but thanks to my friend's little son, we got some priority treatment for our luggage and ourselves.

Note: make sure to print your NCL Cruise luggage tag (you can find it in one of the pages of your NCL eDocument) before hand and attach/staple it into your baggage handle, so that your baggage is checked in without any issue.  For details, see NCL page - packaging for your journey. NCL staff deliver your checked in baggage to your room later that afternoon.

During check in, each of us got a NCL card (Note: it works like your identity while in the ship as well as your cash/credit card), with our room number on it. Children got the one with corner chipped, meant they're not eligible to enter into the adult entertainment zone or buy alcohol etc. It also worked as room key and and to enable the light switch.

Note: It is important that you have your NCL card with you or near you all times. It may be a good idea to buy waterproof cardholder, especially for children as they go into the water (pool, water slides etc) all the time.

On Board

Once checked in and passed through security, we took the escalator to enter the ship. In the entrance of the ship, we were greeted by cheerful NCL staff. And as we walked inside, we were amazed by the fancy carpeting and glittering chandeliers, and of course people from different corners of the world. We took the elevator up to the 11th floor where our ocean view balcony suite was located.  It was a beautiful suite with twin beds, along with sofa bed and just-in-time use ceiling bed. Just imagine a bed that comes out of the ceiling! Immediately my family started calling dibs on who got to sleep there but in the end I won! :) Yeah, amazing! That night, while sleeping on my ceiling bed, I could see the ocean in peace and in war/wave. Yeah, I have to admit that initially we didn't know about the existence of ceiling bed in our suite. I became furious, because there was not enough beds for a family of four and went hurriedly to customer service. They calmly explained me that the room service would come in the evening to prepare the ceiling and sofa bed.

Note: It is specially important to choose your room carefully while booking your cruise vacation. Location matters. It will be too noisy if it's near to engine floor or theatre or near to other entertainment venues. Also important is to choose which side of the ship you want your cabin, specially if you have ocean view window or balcony. Some info from NCL here. And some valuable insight from cruisecritc here.

The first day, we familiarized ourselves to all the cruise amenities - swimming pool, water slides, sports zone, gym and fitness place, running track, theatre, casinos, restaurants, buffet etc. We had good lunch in garden caffe on the top floor of the ship beside the swimming pool and water slides. I also spent some time in pool and went in the Jacuzzi to relax myself.

The 2nd day, I woke up early in the morning. I wanted to start with Yoga. I went to the balcony; it was very early, just the beginning of dawn and little bit chilly outside, but peaceful. And guess what? I did my Yoga for the first time in the middle of Mediterranean Sea. Below are two pictures taken from balcony that morning. One early morning and another little later. 

Cargo ship seen far.
Picture taken from my balcony, early morning.

Picture taken from balcony - Mediterranean Sea

I was also excited about jogging/running that day and went to deck 7 for jogging, but I was little bit disappointed that the track was only on one side of the ship, and all the life boats hanging there blocked the ocean view. And also because of hot/warm air coming from all big machineries there, made jogging not so pleasant. See some messages from people regarding jogging track here. Anyways, I ran around 20 minutes.
The whole 2nd day, we were on the water as there was no planned docking and most of us did whatever we wanted. We had continental breakfast at Garden Caffe on the top of the ship. The cafeteria was huge and selection of breakfast items was amazing and it operated 24 hours a day. Later, I went to swimming pool for a bit and then went to jacuzzi, there I talked to lady from Netherland and her husband for few minutes and after that spent time under the sun on beach chairs while reading the book I brought with me - The Millionaire Next Door by Thomas J. Stanley Ph.D., William D. Danko. And, What the heck! I was allegedly imagining myself as millionaire while relaxing and reading the book. Once, I'm awaken from my millionaire day dream, I went to the gym with my friend. When I looked out the window while running on the treadmill it felt like I was running on the ocean.
The Evening was total blast that day. We went to the Epic theatre to watch the 'Burn the Floor' performance. Wow, it was amazing. They presented it in a real professional way. As NCL puts it in their website, "...The performances of Burn the Floor are not just hot - they sizzle..." Yeah, they did. Everybody liked it including my daughters.

Dawn in Naples (Italian: Napoli)

What an extraordinary morning in Napoli! I just awoke up and saw the beginning of the twilight through the glass door (obviously, I was peering from my ceiling bed). I rushed to balcony, and in few minutes I saw the majestic sun rise. And after some time, we were nearing to port of Naples.

Sun rise in Naples (picture taken from the balcony of my suite)

City of Napoli from harbour front side
(picture taken from the balcony of my suite)

Today, we were having an early breakfast around 7:15 AM, so that we'd be ready to go out for the excursion. By 8:30 AM, we were ready to get out of the ship.

Note: You need your cruise card to check in and check out. In NCL Epic, the exit deck is usually either deck 6 or deck 4, depending upon the port and their infrastructure. Listen for an announcement to find which deck to exit from.
Note: At Naples, cruise ships dock at Naples Stazione Marittima, which is right in front of the city centre. From here, you can easily walk around the city or take the HOHO bus. Another option is to take the train (Naples Circumvesuviana) and go to Mt. Vesuvius.

We entertained the 2nd option and took the HOHO bus- "City SightSeeing Napali". It was quite a good choice and we really enjoyed the whole trip. The whole Naples city itself looked like a big open museum with it's architecture, arts, monuments and cathedrals.

Note: Just like in Barcelona, Naples standard "City Sightseening" HOHO buses also operate in 3 routes. Line A (Red), Line B (Blue), and Line C (Green). Price: EUR 23.00/adult; EUR 11.5/child. In addition, there is a 4th route (Purple line), which is called "City Sightseeing Hystorical Center - Cathedral" and it's cost is EUR 12.00/person. It operates in small van as it passes through narrow streets neighbourhoods. If you have standard ticket and you also want to take Purple line tour, you can just add EUR 3.00/person.

We ventured through all three tour lines and also the purple one. One of my favourites was the Purple line which passed through city's historic narrow street neighbourhood. Since, big double decker bus would not pass through the narrow street, they'd use smaller fifteen seater van on this route. 

Largo Castello at the Piazza Municipio, Naples
(picture taken from the top deck of HOHO bus)

Guglia of the Immaculate Virgin (1750), Gesu Nuovo square
(picture taken from the top deck of HOHO bus)

Naples, Mt. Vesuvius seen in the background
(Picture taken from the upper deck of HOHO bus. 

One of the historic narrow streets neighbourhood in Naples
(picture taken through the window of HOHO bus)

We returned to the ship after our Naples excursion around 2:30 PM and had fun on the ship. My daughter went for rock climbing, and she successfully touched and ringed the bell. Bravo!
Later in the night, we went again to the Epic theatre to watch a Broadway musical show "Priscilla Queen of the Desert". It featured the journey of three "divas" with hundreds of colourful costumes and headdresses along with disco hits from the 70s and 80s.

Note: Parental guidance is recommended if you're taking children to the show.

Road to Rome

We arrived in Civitavecchia, a port town and comune of the Metropolitan City of Rome on the 3rd day of our cruise trip. We were planning to go to Rome and visit Vatican today, so we were ready and got out by 8:30 AM. We took the free shuttle provided by NCL from the pier to the port gate and from there took a city bus (EUR 2.00 per person) to reach the train station. We got a combined train ticket ( Civitavecchia to Roma Termini (Rome)) plus HOHO bus tickets for Rome city tour at Civitavecchia train station.

Note: Some of the agents working inside Civitavecchia look and act like Train Italia staff, but be careful, they may represent private tour operator.
Note: There are express as well as local commuter train available from Civitavecchia to Rome. For details visit Train Italia web site.

So, we took the train. While going to Rome, the train was jam packed, and not everyone of us got a seat. But we were still happy that we were visiting the historic city of Rome. Once we got out of the Roma Termini, a bunch of street agents (representing tour buses and others) were behind us trying to sell tickets. Since, we had our ticket already purchased, we just moved forward, kind of ignoring them. We had to walk around 7 minutes to reach to our Big Bus Tour bus.

Note: There are multiple HOHO buses operate in Rome. You can find information about all these here. Compare and take the one that best suits you.


We were on the top of a double decker HOHO bus and it seemed like one of the hottest day here in Rome, however, we weren't worried much as we were sight seeing one of the greatest city on earth - Rome; and travel through it's history and visit the places where the ancient Romans lived, worked and socialized. We got a chance to admire the fine arts and architecture attached to Rome's grand buildings and beautiful churches. Compared to Naples, I found that historic monuments and buildings were well preserved/maintained here in Rome. Maybe Rome is richer and gets a bigger budget?

Touring Rome in style? Definitely, next time. 
Rolling Rome on Sageway? 

monument dedicated to King Victor Emmanuel II,
the first king of Italy, in Piazza Venezia
(picture taken from the top deck of HOHO bus)

Rome colosseum

Street of Rome (picture taken from the top deck of HOHO bus)

Road to Vatican. We were entering one of the smallest
but historically significant country of the world.

St. Peter's Basilica and Square, Vatican.

Hoping to see the Pope Francis and get blessings in-person, we spent around 2 hours around Vatican, but he decided to bless us remotely. :) Blessed, we returned back to our HOHO bus to complete the remaining tour of Rome. At around 2:00 PM, we returned back to Roma Termini to take train back to Civitavecchia. We had to ask several people to find out our exact platform, finally, Train Italia's agent helped us figure it out.

Back to the ship, after having a quick late lunch, I went to spend some time in pool and later on in Jacuzzi. That night, we went to watch the musical show "The Epic Beatles" in the Epic Theatre. Someone has posted a footage of the show here in Youtube.
Later on, we decided to go for dinner at the Manhattan dinning room. NCL Epic has a lot of dinning options. See here for details.

Note: If you like Indian dinning, you need to ask for an Indian menu. The main menu does not include Indian dishes.

Touching the Tower in Pisa.

The ship docked in the port of Livorno, Italy at around 7:00 AM and we were out for our excursion at around 9:00 AM. We planned to go to Pisa to see the leaning tower.

Note: There are few option for self excursion here in Livorno.
1. Take train, taxi or tour bus and go to Florence
2. Take train, taxi or tour bus and go to Pisa.
3. Take NCL Shuttle which charges EUR 12.00 and go to downtown Livorno and either take HOHO bus or other form of transportation to go around the city.

We wanted to go to Pisa, and fortunately, found a good deal with a tour bus. We had paid EUR 21.00 per person. The tour bus took us first to downtown Livorno, and from there another bus took us to Pisa.

Note: Here is their website, if you like to get more info or plan ahead for your tour. http://www.livorno.city-sightseeing.it/eng/tour-livorno-pisa.html

Port of Livorno, Italy

Piazza dei Miracoli -
Pisa Cathedral with the Leaning Tower of Pisa

For you to guess.

Hey look, I touched the tower!

Note: There is almost a 20 minutes walk from the bus station to the Piazza dei Micacoli. You pass through the parking lot, cross the rail line and go through the beautiful street with a lot of gift shops. Your bargain skill really helps here. Specially, if you want to buy any gift items directly from the walking street vendor.


We spent around 2 hours in Pisa and took bus back to Livorno at 1:00 PM. We were back to the ship by 2:30 PM for lunch. After lunch my friend and I went to try the rock climbing. You probably guessed it right, we couldn't touch and ring the bell, but nevertheless it was a very good experience.
Later that night, we were supposed to enjoy the Cirque Dreams and Dinner Entertainment. All dressed up, we went to the show, but after inquiry we found that the ticket we purchased "standard seating", might not give us the right experience, and also the show was not what we thought - a full fledged circus kind of show, but it seemed more of an acrobatic performance only. We asked to cancel our booking, and a nice lady coordinating the show agreed to give us full refund. We were little disappointed, but found another interesting dance show that featured dancing with the NCL staff.

Coming to Cannes.

The ship arrived around 8:00 AM in Cannes, France, but didn't exactly dock, it remained offshore and tendered the passengers to the shore . Since we had not reserved it in advance, we had to stay in line of hundreds of people. Fortunately, line moved quickly and we arrived at shore in no time.

Note: You can reserve (free) Tender seat from NCL concierge service few days in advance.
Note: On the shore, there is an information kiosk, wash room, wi-fi etc.

At the information desk, my friend asked some questions in French, but to our amazement, the lady replied in English. Any ways, we got a city map, and we headed to take the trip on Le Petit Train.

Note: Le Petit Train starts from Palais des Festivals, the Hotel Majestic beaches and for one hour goes around the historical part of the city and covers the Croisette Tour - luxury hotels, beautiful beaches, brand name boutiques, lavish yachts, Palm Beach, casinos etc. Cost EUR 10.00 (adult), EUR 5.00 (children)

We were amazed by the beauty of Cannes - clean well maintained streets full of flowers, towering palm trees and sandy beaches. It gave us whole different perspective of Europe - Luxurious, and Lavish life style and beauty. I've never seen so many luxury cars - Bentley, Maserati, Ferrari, Porsche, Mercedes, Rolls Royce, Lamborghini you name it, parked in the small fragment of the street. Same goes to brand name boutique stores one after another. No wonder, this city is called sister city of Beverly Hills. If I had a million dollars, I'd definitely go there again, you know, to spend it in style!  
We spent an hour going around the city on Le Petit train, while inhaling fresh Cannes' air and submerging our eyes onto the beauty of French Riviera.

Cannes. Picture taken from balcony of my suite.

Le Petit Train

Street of Cannes - see the palm trees

(picture taken from the top deck of HOHO bus)

After the tour on Le Petit train, we walked for a while by the sea shore, looking for the perfect place for a blue water beach experience and we found one. Wow! What a pristine and warm water and perfect weather! Beaches in Cannes are really well maintained. They even have a fresh water tap there right on the beach for people to take shower after you are done with salty ocean water. For us, coming from Toronto, where we used to go to fresh water beaches, ocean water was whole new experience. Yeah, it felt salty! I know, I shouldn't have drank the water. But, a big wave passed through my mouth, oh sooo salty!!!

Beach, Cannes, France

Beach, Cannes, France

After spending two hours or so, we returned back to Tender Boat station to get back on the ship. We had to catch all the ship events as well! We tried to kind of balance between shore excursions and ship events and entertainments.
Back on the ship, that day other than our regular activities like going to gym, swim and Jacuzzi, we went to the “Michelle” musical show. Michelle not only sang and touched our heart but also made us laugh with her humor and funny stories.

Path to Palma De Mallorca

Even though, our ship was supposed to dock only around 1:00 PM today, I woke up early morning, went to the gym and when I came back to my room, everyone was ready to go for breakfast. Since, it was our last night before checkout early the next morning, we wanted to do as much as possible. After breakfast, my younger daughter went for Bungee Trampoline jumping and my older daughter went for Rock Climbing again.

Finally, our ship docked at Palma island port - Estacion Maritima 1 around 1:00 o'clock in the afternoon. We had already taken our lunch and were ready to explore the beautiful Palma island.

Note: Ships dock at the port directly in front of the old town of Palma. Around 10-15 minutes walk and you can pass the terminal and reach the street where you can catch a HOHO bus (Cost: EUR 18.00/adult, EUR 9.00/child)or other form of transportation or walk further ahead to the Palma town. However, in order to go to Mallorca island you need transportation.
Note: Each HOHO bus stop has name and number. The one near to port is number 14 (Estacion Maritima). If you want to go to beach, get off the HOHO bus at stop # 6 (Av. gabriel Alomar i Villalonga (porta des camp)

It was a very hot day and we had to line up for about 30 minutes for the HOHO bus. Once, we were on the top of open double decker bus, because of ocean wind, we didn't feel that hot after all.

Palma

Catedral de Santa María de Palma de Mallorca
(picture taken from the top deck of HOHO bus)

We got off the bus at stop #6 and walked to the beach of Palma. Since it was burning hot and we didn't have a beach umbrella, we did a purchase negotiation with a street vendor. He agreed to sell it at EUR 5.00, even though he originally priced it as EUR 15.00, it's because we promised to return it to him once we were done. We put up our umbrella, but it was so hot that we could barely touch the sand with our bare feet, so in no time, we jumped into the ocean water and it felt amazing!

Sea shore street, Palma.

Palma beach, near to HOHO bus stop # 6

Comparing Cannes' beach vs. Palma's one, both were amazing. However, water seemed cleaner in Cannes and a lot of weeds here in Palma. Also, unlike Cannes, no fresh water tap was available for beachgoers here in Palma, so we carried our ocean salt with us to the the ship :(
In the evening, we did our packing (as we had to check out early next morning), attached the baggage label and put our check in baggages in the corridor for NCL staff to pickup.
Later that night, we went to watch the four station concert. It was a really good show. Here, I found some footage of the show in an Youtube video.
After the show, rather than going to sleep, I went to Spice H2O - Hot White Party to get the taste of music and dance under the open sky. Spice H2O is adult only zone for having fun with music, dance and with some energizing cocktail-drink. Here, I found some footage of the show on Youtube.

Note: Get your tag for each luggage from NCL concierge and put your name, room number and other relevant info and attach it to each baggage and put your baggage in the corridor just beside your door. Next day, you'll collect them once you check out.

Saying Goodbye :(

The last day, it felt little sad as we were leaving the luxurious cruise life and going back to our every day life. But we had to do it anyways, so we got our last breakfast and headed for check out around 9:00 AM. I met my room service guy (from Indonesia) in the corridor, and gave him some tips and thanked him for his excellent service.
After checkout, we collected our baggages from baggage carousel and headed out to find a seven seater van to airport. It was a big line (few hundred people were already in line) for taxi. After around 40 minutes our turn came, but there was no van (only taxis were there). Now, we either had to split and take two taxis to the airport or wait for a van. After another twenty minutes, we finally got our van. The Spanish driver this time seemed not as friendly. He charged us more than the standard rate. We also ran into little blunder here. Driver asked us what our airport terminal number was. I checked our Air Transat eTicket and there was no terminal number written for return flight from Barcelona. When I said that there was no terminal number in the ticket, the driver said confidently it was Terminal 1. I countered him saying that we landed in terminal 2 while coming to Barcelona. However, he showed his confidence again and said it was for sure terminal 1. We agreed and he took us to terminal 1. We only found while we were already inside the terminal that we were in wrong terminal and had to go to terminal 2. By that time, our taxi already left and we had to take the airport shuttle bus (Note: takes around 20 minutes from terminal 1 to terminal 2) and go to terminal 2. Since we had enough time until our flight, it did not cause any lasting issues.

Note: Check your ticket carefully. Our Air Transat ticket had terminal info for outbound flight (both Toronto and Barcelona), but for Inbound flight, there was no terminal number for Barcelona airport.

At terminal 2, we found that there was a strike going on by the workers handling luggages, which delayed our flight by almost an hour.

We arrived in Toronto at around 6:00 PM safely. No issues at the airport. We were home in Canada! And by the way, the complimentary food provided by Air Transat was good this time. May be, because it was prepared by Spanish chefs? Anyways, thanks Air Transat!

Overall, the trip was quite fun and became a success story. We met amazing people from around the world, saw European culture, arts and architecture with our own eyes. In terms of the cruise, our family’s favourite would be the sky deck with the pool and water slides and beach chairs. We also equally enjoyed and used the gym, the sports court and the library.

Note: In terms of food, basically, the ship has everything (kind of) and best of all there is 24 hour FOOD! Nothing gets better than that. You can go and eat whenever you like, and eat whatever you like. Each day there is something different to eat and there are always special dinners of a certain theme for example “The Taste of Spain” or “Italian Night” or “Seafood extravaganza.” Regardless of the theme there is something for everyone (kind of) to eat and enjoy no matter what food restrictions you have.

Desert from NCL dinning.

NCL provided overall good services. The room was cleaned twice a day, once around noon - when we were out for the excursion and again in the evening - just before dinner. Room service did quite a good job. During the evening service, they created cute towel animals (towel art) and my daughters really liked them.

Towel (Art) animal created by NCL room service staff for our room.

Personally, both Europe and the cruise were the first time experience for myself and my family, and we really enjoyed it. If I had a chance to go again, I'd like to visit Rome and Naples and get a more closer look of the arts and architecture, but my daughters and wife told me that they would go to Cannes, France instead.