SonarQube issue - Could not find branches



SonarQube allows branch level (for example, if you are using Git, you may want to scan your feature branch and fix identified issues before creating a Pull/Merge Request to the develop branch) analysis. In order to do branch level analysis, you need to specify your branch using SonarQube parameter 'sonar.branch.name' and optionally 'sonar.branch.target' (the name of the branch into which the temporary branch specified with 'sonar.branch.name' will be merged.
However, if SonarQube project does not exist for the given source code repository yet and you are scanning your source code repository for the first time and you specified your branch name using 'sonar.branch.name', you'll get the following error:
[ERROR] Failed to execute goal org.sonarsource.scanner.maven:sonar-maven-plugin:3.6.0.1398:sonar (default-cli) on project web-mywebapps: Could not find branches. A regular analysis is required before creating branches. -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:

As you can see from the error message, it says "...could not find branches. A regular analysis is required...". So, for the first scan whatever branch you are scanning, do not use parameter 'sonar.branch.name' One way to tackle this issue is to write some conditional logic in your SonarQube job. Below is a kind of a pseudo code example. Here the -Dsonar.branch.name property and value is passed to Maven goal only if the variable 'FIRST_TIME_BUILD' is false.
Note: 'FIRST_TIME_BUILD' is not a SonarQube built in variable, I've used it as an example to specify the condition.

<goals>clean install sonar:sonar -Dsonar.host.url=\$SONAR_HOST_URL <% if(!FIRST_BASELINE_BUILD) {%> -Dsonar.branch.name=<%= sonarBranchName%> <%}%> -Dorg.xml.sax.driver=com.sun.org.apache.xerces.internal.parsers.SAXParser -U</goals>

Hope it helps. For details on SonarQube branch analysis, visit https://docs.sonarqube.org/latest/branches/overview/

My First C.N. Tower Climb for United Way

Thank you All who supported me to raise funds for United Way. With your support and contribution, I was able to raise $331 and proudly climbed the CN Tower’s 1,776 steps yesterday. As per United Way, "all funds raised for the 2015 CN Tower Climb for United Way go directly toward the work of United Way Toronto & York Region, supporting a network of roughly 240 agencies across the region. Your time, efforts and generosity are moving people from poverty to possibility, building strong communities, and helping kids be all that they can be. Together, we are creating opportunities for a better life for everyone in our region." 
My wife and I climbed together and it took us 34 minute and 19 seconds. We raised $531  (combined total) together. Below is my T-shirt generously given by United Way!


Winter Festival of Lights in Dufferin Islands

Some beautiful pictures I captured while visiting "Winter Festival of Lights" in around Dufferin Islands (Niagara Falls, Ontario, Canada) on January 23, 2016. If interested see more pictures from official site http://www.wfol.com/the-lights/dufferin-islands






My First 10K Run

   Today, I was one of the runners participating in 10 KM race at beautiful Tommy Thompson Park, Toronto (http://www.tommythompsonpark.ca/) organised by MEC Toronto. The weather was fantastic around 22℃ and I fully enjoyed the race. My wife and two daughters were there from early morning to support and cheer for me. It was the first race in my life, so I was little nervous as well as thrilled and I completed the 10 KM distance in 58 minute 22 seconds. Not bad for me :)






Jenkins Job Chain and How to Resolve Issue with Parameter Passing

You can trigger a dependent (or other) job in Jenkins automatically from your current job. This way you can have multi-step builds or job chain. One of the use case scenarios for this is - let's say you have a parent build that creates a EAR file which relies on the successful completion of the child build(s) which creates some jar files.
In some situation, you also need to pass the parameters and their values while triggering the other build. But what to do, if you pass the parameter when triggering a job, but the parameter is not available in the triggered job?

Here we will examine the these situations and possible resolutions:

1) Triggering the other job without passing the parameter(s):

If you don't need to pass the parameters, then it's easy. You can just use either the <<Build Other Projects>> as a  Post Build Action for your current job or <<Build after other projects are built>> as a Build Triggers for the other job.

Let's say, if I have a build job called  <<build_jars_no_param>>, it compiles source code and creates bunch of Jars files, which will be used by a EAR file created by <<build_ear>> job. So, as soon as the <<build_jars_no_param>> job is completed, I want <<build_ear>> job is kicked off.
In this case, I just need to define following in the Post Build Action of <<build_jars_no_param>> job as follows:
Post Build Action - Build Other Projects

This triggers the <<build_ear>> job once the <<build_jars_no_param>> job is successfully completed. Multiple projects can be specified delimited by comma in the "Projects to build" input field.
Other than building the other projects that have a dependency on the current project, this can also be used to split a long build process into multiple jobs.

2) Triggering the other job with parameter(s) and their value(s)

If you need to pass the parameters along with triggering the other job, you can use Parameterized Trigger Plugin (https://wiki.jenkins.io/display/JENKINS/Parameterized+Trigger+Plugin) and this can be used both as a Pre Steps or as a Post Steps. This call also provides option to block the current job until the completion of the triggered builds.  

However, there is a security related catch with the use of Parameterized Trigger Plugin. This is one of the Plugins affected by fix for SECURITY-170/CVE-2016-3721. After this fix, Jenkins only allows build parameters that have been explicitly defined in a job's configuration. Any other arbitrary parameters added to a build by plugins will not be available by default.
So, your triggered job (with parameters passing), which might have worked in the prior version (the fix was first included in Jenkins versions 1.651.2 and Jenkins 2.3) is not working in the newer version of Jenkins. 
Here are three ways to resolve:
  1. Work-around #1: restore the previous behavior by setting the system property:
    -Dhudson.model.ParametersAction.keepUndefinedParameters=true
    Example: java -Dhudson.model.ParametersAction.keepUndefinedParameters=true -jar jenkins.war
    This could be a security risk, so use it just as a short-term workaround only.

  2. Work-around #2: white-list parameters by setting system property
    -Dhudson.model.ParametersAction.safeParameters=<comma-separated list of safe parameter names>
    Example: java -Dhudson.model.ParametersAction.safeParameters=FOO,BAR,ref_release_number -jar jenkins.war

  3. Convert/define the other job (the job to be triggered from your current job) with parameters using option "This project is parameterized" in the General section of the job definition. 
From the security point of view, the 3rd option is the preferred option. However, if you have legacy jobs and looking for short term work-around before (re) defining a triggered jobs to be a parameterized projects, you can use option #1 or #2. Option #2 is better than #1, because option #1, just blindly restores the previous behavior.   

Let's look through an example:
1) I've a project  called <<build_ear>> which calls <<build_jars_with_params>> job. Below diagram shows the Pre Steps setting of <<build_ear>> project, and as you can see it defines parameter ref_release_number=2.1.0.${BUILD_NUMBER} to be passed to the triggered job.

Pre Steps - defining a job to be triggered from this project.

On the other hand, here is how I'm testing the the value of the passed parameters in the <<build_jars_with_params>> downstream/triggered project.
Triggered job - testing the passed parameter. 


I'm using Jenkins ver. 2.150.1, which means it includes the fix for SECURITY-170, and I don't see the value of the passed parameter. 

Passed parameter is not available in downstream job

Now, let me use the above mentioned resolution options:

1st option (work-around), I'll have to start the Jenkins with '-Dhudson.model.ParametersAction.keepUndefinedParameters=true'option:


And here, the triggered job, shows the value of ${ref_release_number}

Passed parameter (from upstream job) is available in triggered job.

Use 2nd option. Here I'm starting Jenkins with '-Dhudson.model.ParametersAction.safeParameters=ref_release_number' system property so that 'ref_release_number' is considered as a safe parameter.
Define all parameters to be passed to triggered job as safe parameters using system property.

And here, the triggered job, shows the value of ${ref_release_number}
Passed parameter (from upstream job) is available in triggered job.


Now, here is how to use the 3rd (preferred) option. For this, I have to redefine the <<build_jars_with_params>> as a parameterized project and add 'ref_release_number' as an input parameter.

Define triggered job as a parameterized project.

When my triggered (downstream) job is defined as a parameterized project and I define the parameter 'ref_release_number' here, my upstream job can safely pass this parameter when triggering this job and Jenkins will allow it, no more work-around.
So, here I'm starting Jenkins without any system properties:


And my triggered job still correctly displays the value of the paramter passed from upstream job:


You may be interested reading the following Jenkins related blog:
How to Retrieve Info Using Jenkins Remote Access API
Jenkins Pipeline - Few Troubleshooting Tips

How to Retrieve Info Using Jenkins Remote Access API



We can use REST-like Remote Access API in Jenkins to post or retrieve information in XML or JSON format.  General format of the URL is <Jenkins-URL>/<data-context>/api/xml|json.
Below are few examples:

1) Retrieve Build Result.
Below examples shows the build result in JSON format for job 'web-apps-build-pkgvalidator' and build number ''6'. Credential is passed using '--user <user-name>:<password>'
result is piped to 'jq' to pretty print.
Notes:



$> curl -X GET http://localhost:9090/job/web-apps-build-pkgvalidator/6/api/json --user ppoudel:<password> | jq

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  100  1765  100  1765    0     0  14120   0 --:--:-- --:--:-- --:--:-- 16192


{
  "_class": "hudson.maven.MavenModuleSetBuild",
  "actions": [
    {},
    {
      "_class": "hudson.model.CauseAction",
      "causes": [
        {
          "_class": "hudson.model.Cause$UserIdCause",
          "shortDescription": "Started by user Purna Poudel",
          "userId": "ppoudel",
          "userName": "Purna Poudel"
        }
      ]
    },
    {
      "_class": "hudson.plugins.git.util.BuildData",
      "buildsByBranchName": {
        "refs/remotes/origin/release/release-2.0.3": {
          "_class": "hudson.plugins.git.util.Build",
          "buildNumber": 6,
          "buildResult": null,
          "marked": {
            "SHA1": "2c60eff4fef13e9346ae7c6b848efdc2fbf31026",
            "branch": [
              {
                "SHA1": "2c60eff4fef13e9346ae7c6b848efdc2fbf31026",
                "name": "refs/remotes/origin/release/release-2.0.3"
              }
            ]
          },
          "revision": {
            "SHA1": "2c60eff4fef13e9346ae7c6b848efdc2fbf31026",
            "branch": [
              {
                "SHA1": "2c60eff4fef13e9346ae7c6b848efdc2fbf31026",
                "name": "refs/remotes/origin/release/release-2.0.3"
              }
            ]
          }
        }
      },
      "lastBuiltRevision": {
        "SHA1": "2c60eff4fef13e9346ae7c6b848efdc2fbf31026",
        "branch": [
          {
            "SHA1": "2c60eff4fef13e9346ae7c6b848efdc2fbf31026",
            "name": "refs/remotes/origin/release/release-2.0.3"
          }
        ]
      },
      "remoteUrls": [
        "https://pppoudel@bitbucket.org/pppoudel/pkgvalidator.git"
      ],
      "scmName": ""
    },
    {
      "_class": "hudson.plugins.git.GitTagAction"
    },
    {},
    {
      "_class": "hudson.maven.reporters.MavenAggregatedArtifactRecord"
    },
    {},
    {},
    {}
  ],
  "artifacts": [],
  "building": false,
  "description": null,
  "displayName": "#6",
  "duration": 37633,
  "estimatedDuration": 39130,
  "executor": null,
  "fullDisplayName": "web-apps-build-pkgvalidator #6",
  "id": "6",
  "keepLog": false,
  "number": 6,
  "queueId": 150,
  "result": "SUCCESS",
  "timestamp": 1546274196419,
  "url": "http://localhost:8080/job/web-apps-build-pkgvalidator/6/",
  "builtOn": "",
  "changeSet": {
    "_class": "hudson.plugins.git.GitChangeSetList",
    "items": [],
    "kind": "git"
  },
  "culprits": [
    {
      "absoluteUrl": "http://localhost:8080/user/ppoudel",
      "fullName": "Purna Poudel"
    }
  ],
  "mavenArtifacts": {},
  "mavenVersionUsed": "3.5.4"
}

2) Below example shows use of /api/xml with 'xpath' to get just the build status from the build report.

$> curl -X GET http://localhost:9090/job/web-apps-build-pkgvalidator/6/api/xml?xpath=/*/result --user ppoudel:<password>

% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 24 0 24 0 0 102 0 --:--:-- --:--:-- --:--:-- 110


<result>SUCCESS</result>

3) Getting build status using /api/json. The following example shows retrieving job name, build number, build status and timestamp.

$> curl -X GET http://localhost:9090/job/web-apps-build-pkgvalidator/6/api/json?tree=fullDisplayName,number,result,timestamp --user ppoudel:<password>

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   152  100   152    0     0   1216      0 --:--:-- --:--:-- --:--:--  1394


{"_class":"hudson.maven.MavenModuleSetBuild","fullDisplayName":"web-apps-build-pkgvalidator #6","number":6,"result":"SUCCESS","timestamp":1546274196419}

4) Retrieving all jobs under certain view:
Note: here I'm piping the result through 'jq' and 'grep', which is optional.

curl -X GET http://localhost:9090/job/Web/job/mobile-apps/view/mobile-apps/api/json --user ppoudel:<password> | jq | grep name

"name": "mobile-apps-xyzmportal",
"name": "mobile-apps-holportal",
"name": "mobile-apps-tpl",
...
...
"name": "mobile-apps-bcs_jpj",


 5) Retrieving JUnit test summary report:

$> curl http://localhost:9090/job/web-apps-build-pkgvalidator/6/testReport/api/json?tree=failCount,skipCount,totalCount,urlName --user ppoudel:<password>

% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   127  100   127    0     0    451      0 --:--:-- --:--:-- - 579


{"_class":"hudson.maven.reporters.SurefireAggregatedReport","failCount":0,"skipCount":0,"totalCount":20,"urlName":"testReport

6) Below steps can be used to retrieve the SonarQube analysis result using Jenkins' remote REST like API.
6.1) Get the taskId from the build providing build number
Note: http://localhost:9090 is Jenkins server URL.

curl -X GET http://localhost:9090/job/web-apps-build-pkgvalidator_sonarqube/6/api/json --user ppoudel:<password> | jq | grep ceTaskId

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  5182  100  5182    0     0  18507      0 --:--:-- --:--:-- --:--:-- 20811
      "ceTaskId": "AWfXjftAinfFqLzOhqqe",

6.2)  Get the analysisId using taskID:
Note: http://localhost:8000 is SonarQube URL.

$> curl -X GET http://localhost:8000/api/ce/task?id=AWfXjftAinfFqLzOhqqe --user ppoudel:<password> | jq | grep analysisId

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   518  100   518    0     0   2770      0 --:--:-- --:--:-- --:--:--  3029
    "analysisId": "AWfXjgNNVNzkngEjXoPD",

6.3) Get the analysis report using analysisId:

$> curl -X GET http://localhost:8000/api/qualitygates/project_status?analysisId=AWfXjgNNVNzkngEjXoPD --user ppoudel:<password> | jq

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current

                                 Dload  Upload   Total   Spent    Left  Speed
100  1438  100  1438    0     0   4608      0 --:--:-- --:--:-- --:--:--  4841

{
  "projectStatus": {
    "status": "ERROR",
    "conditions": [
      {
        "status": "OK",
        "metricKey": "new_maintainability_rating",
        "comparator": "GT",
        "periodIndex": 1,
        "warningThreshold": "1",
        "actualValue": "1"
      },
      {
        "status": "OK",
        "metricKey": "new_reliability_rating",
        "comparator": "GT",
        "periodIndex": 1,
        "warningThreshold": "1",
        "actualValue": "1"
      },
      {
        "status": "OK",
        "metricKey": "new_security_rating",
        "comparator": "GT",
        "periodIndex": 1,
        "errorThreshold": "1",
        "actualValue": "1"
      },
      {
        "status": "OK",
        "metricKey": "sqale_rating",
        "comparator": "GT",
        "warningThreshold": "3",
        "actualValue": "1"
      },
      {
        "status": "ERROR",
        "metricKey": "security_rating",
        "comparator": "GT",
        "errorThreshold": "1",
        "actualValue": "5"
      },
      {
        "status": "WARN",
        "metricKey": "reliability_rating",
        "comparator": "GT",
        "warningThreshold": "3",
        "actualValue": "5"
      },
      {
        "status": "ERROR",
        "metricKey": "blocker_violations",
        "comparator": "GT",
        "errorThreshold": "0",
        "actualValue": "180"
      },
      {
        "status": "WARN",
        "metricKey": "critical_violations",
        "comparator": "GT",
        "warningThreshold": "0",
        "actualValue": "3806"
      },
      {
        "status": "WARN",
        "metricKey": "major_violations",
        "comparator": "GT",
        "warningThreshold": "0",
        "actualValue": "2878"
      },
      {
        "status": "WARN",
        "metricKey": "coverage",
        "comparator": "LT",
        "warningThreshold": "80",
        "actualValue": "0.1"
      },
      {
        "status": "ERROR",
        "metricKey": "vulnerabilities",
        "comparator": "GT",
        "errorThreshold": "0",
        "actualValue": "107"
      }
    ],
    "periods": [
      {
        "index": 1,
        "mode": "previous_version",
        "date": "2018-12-15T15:11:09-0500",
        "parameter": "1.5.8-SNAPSHOT"
      }
    ],
    "ignoredConditions": false
  }
}

7) If you need to get all the configured projects (paginated output) in SonarQube you can use the following URL:
Note: http://localhost:8000 is SonarQube URL.


# First page:
$> curl -X GET http://localhost:8000/api/components/search?qualifiers=TRK&p=1

8) Or get detail of a specific project:
Note: http://localhost:8000 is SonarQube URL.

$> http://localhost:8000/api/components/show?key=<project-key&gt

For more information on Remote access API visit https://wiki.jenkins.io/display/JENKINS/Remote+access+API

You may be interested reading the following Jenkins related blog:
Jenkins Job Chain and How to Resolve Issue with Parameter Passing
Jenkins Pipeline - Few Troubleshooting Tips

How I achieved my PSM I Certification

I'm writing this post not only to express my joy but also to share the techniques that I employed in order to prepare myself to pass the Professional Scrum Master™ (PSM) I test. I'm extremely delighted to have this certification. But don't get me wrong, employing the knowledge gained, in daily practice is more important than earning a certification!
Just to give you some background, I'm a DevOps specialist and work with different agile teams (using Scrum framework), as part of my daily job. I attended a Agile boot camp just last week, which also gave me a lot of insight on how to use Scrum framework in the real world.

In terms of preparation, I spent around 5 days including 3 days of boot camp. Below is the list of resources, I used to acquire the mastery of PSM I.

  1. Scrum Guide - in my humble opinion, this is the most important source of information. It is short but written very concisely and to the point. I read it just once top to bottom, but spent about four hours reviewing and taking notes (imaging I spent 4 hours and it (the 2017 version) only has 19 pages). I did spend another 15 minutes or so to review it again just before taking my real test. You can download this guide free from www.scrumguides.org
  2. Took free Open Assessments provided by scrum.org. Even though, I was targeting for Scrum Master, I took all available Open Assessments there just to get a bigger picture from the Scrum Master, the Product Owner, the Development Team, and the Scaled Scrum perspectives. I took some of these assessments twice to get close to a 100% passing mark and also took note of individual failed quizzes and reviewed it all again just before taking my real test.
  3.  Big thanks go to Mikhail Lapshin for maintaining such a wonderful site (https://mlapshin.com/index.php/scrum-quizzes/) dedicated to Scrum quizzes. In my opinion you find here sufficient (quantitatively and qualitatively) quizzes. Mikhail also maintains very good Scrum Questions page where he provides detail insight.  
  4. I also reviewed materials from www.volkerdon.com and did their quizzes. I must say that the site has high quality materials and quizzes. I repeated some of the quizzes here twice.
  5. I also downloaded and reviewed The Scrum Master Training Manual (The-Scrum-Master-Training-Manual-Vr1.61.pdf) produced by Management Plaza (https://mplaza.pm). As name suggests, it is written as a manual with nice details. I also noticed that the manual gives too much emphasis on word "Project" (as a noun). My understanding is that Scrum is more product oriented rather than project. Scrum Guide (2017 version) has just a brief touch on project, which states "... Each Sprint may be considered a project with no more than a one-month horizon. Like projects, Sprints are used to accomplish something...". So, make sure you are not confused when reviewing the manual.
  6. If you are a audio visual learner, Scrum Training Series modules from http://www.scrumtrainingseries.com/ are very good. These modules not only explain Scrum in real world like scenarios but also have good set of quizzes. I went through top five modules and did the quizzes. 
  7. Along with the Scrum Guide, I also reviewed the Nexus Guide and got knowledge about the Nexus framework. Nexus is one of the Scrum Scaling frameworks.
  8. I also studied some material (from the internet) to get better understanding on how to do Product Backlog Items (PBIs) ordering/prioritization, PBI work estimation etc. Evidence-Based Management Guide is a good read. I also gained some knowledge on consensus-based estimating technique using Planning Poker®, estimating using Story points etc.
In terms of giving the actual test, I chose the morning time (I heard that the site becomes slower during the day). I woke up Sunday morning around 6:00 AM, prepared myself a cup of coffee (I actually consumed two cups in two hours of time), did around 40 minutes of review before starting the actual test. Once you start the test, time management is very important. I was about to miss few, hadn't I rushed the last 30 minutes. I did mark few questions for later review (you have to manually make a note in separate notepad, as the system does not allow you to mark the question for review), but at the end, did not get enough time to go back and review again. It took me around 58 minutes to finish the test. So, instead of going back and reviewing, I just hit the finish button. I felt that I was confident in my answers, however, I was still nervous for my results.Yes, I got 96.3%.
   Hope this post is helpful to all of you - either considering to get certification or just trying to learn and use Scrum in your daily practice!