Jenkins vs Travis: The War of CI Tools

Automation in the development cycle has often led to saving precious development time. A critical process in the DevOps cycle that sits between development and deployment involves running unit tests to ensure that a new commit hasn’t broken functionality. The process of Continuous Integration (CI) helps in the automation of testing. CI encourages developers to push frequent commits by enabling seamless integration with the central code repository.

It is generally a good idea to integrate this code frequently as it makes the process of debugging easier with a pre-defined set of unit tests. Identifying errors earlier in the development cycle also helps in avoiding rework at a later stage.

How does the CI Process work?

When a developer commits code in their local system and pushes the changes to the central repository, a webhook triggers the whole CI process through a CI tool. The CI server gets the latest version of the code from the repository, builds the application into a temporary environment within the server and runs the tests. If the build or tests fail, an alert is sent to the development team and optionally the commit may be blocked. If the tests pass, the commit is integrated into the central repository.

Prerequisites for CI Development

To enable CI development in your development cycle, there are two prerequisites:

• The use of version control
• A testing framework

In this post, we assume the use of Git through its cloud implementation: GitHub. We also assume a simple unit test on Python to run our tests through a CI tool. In this post, we cover two popular CI tools: Jenkins and TravisCI

Getting Started with TravisCI and Jenkins

TravisCI is a cloud-based CI server. It currently has support only for GitHub repositories. To add other repositories hosted on other servers, add them as submodules. If the source code is public, one can use TravisCI for free.

On the other hand, Jenkins is an open-source CI tool that one needs to host on their server to use. As it is your implementation, you can use it for a repository hosted anywhere, given you can configure the webhooks.

Configuration Settings for TravisCI

In TravisCI, you need to register and connect your GitHub account to see the list of your repositories on GitHub. Enable the required repository for testing from your Travis Settings Page and add a travis.yml file on the root directory of your repository. This configuration file outlines to Travis about the build and the tests that need to be performed on a push. Here is a quick configuration file on GitHub.

language: python
python:
- "2.7.1"

# command to install dependencies
install: "pip install -r requirements.txt"

# command to run tests
script: python tests.py

The first line specifies the environment that Travis needs to set up. Next, you need to specify the versions for which the tests need to be done. The install parameter contains the command needed to set up the development environment. Finally, the script parameter contains the command to run the unit tests. Open source projects like Phpmyadmin have much more feature-rich configuration files.

There are many configuration settings that you can tweak on Travis. For instance, you can set up specific branches to test your code on.

# blocklist
branches:
except:
- hotfix

# safelist
branches:
only:
- master

Here is a list of features that you can customize for your build.

You can also set up an integration with Docker with the services set in the configuration file.

services:
- docker

before_install:
# Custom docker commands to run

Further, you can set up cron jobs (a Linus Utility to run scheduled tasks) too from the settings page of your TravisCI profile.

Configuration Settings for Jenkins

The first step in performing CI operations with Jenkins is to download and run Jenkins. Head to the download page to download the latest stable release of Jenkins.

Once downloaded, you can run Jenkins through the following command:

java -jar jenkins.war --httpPort=8080

Next, proceed to https://localhost:8080 to complete the installation.

In Jenkins, you need to create a text file named Jenkinsfile with the following code:

Jenkinsfile (Declarative Pipeline)
pipeline {
agent { docker { image 'python:2.7.1' } }
stages {
stage('build') {
steps {
sh 'pip install -r requirements.txt'
sh 'python tests.py'
}
}
}
}

This code sets up your test precisely like the TravisCI test that we discussed earlier, by setting up the Python image and then running the dependencies and unit testing scripts, respectively.

Jenkins can integrate with over a thousand community developed plugins. You can search for various plugins available for Jenkins through the plugins page.

To install a plugin, download the plugin file with a .hpi extension and place it in the plugins directory of Jenkins

<jenkinsHome>/plugins/

Once placed, you need to restart Jenkins for the new plugin to take effect. Some useful plugins that you may consider are:

• This server monitoring plugin lets you monitor the usage of your CI server and alert you if it’s approaching full usage
• A compiler warning plugin that sends out warnings when your code fails to compile or build
• A job configuration history plugin helps you track changes that you have made in your Jenkins jobs over time
• If you use a project management tool like JIRA, you may wish to explore the JIRA Jenkins integration plugin

Jenkins vs Travis: The Final Showdown

Let us try and evaluate these two tools based on standard parameters that one may use.

Private Project Support: While Jenkins and TravisCI both support open source projects equally, they bifurcate when it comes to proprietary repositories. Jenkins, as an open-source tool, can be integrated with private repositories as quickly as the open-source counterparts. However, you need to upgrade to a paid version of TravisCI if you would like to build and test your private repositories on GitHub.

Customizability: The next question that one may ask is about the customizability of both tools. Travis generally works well with basic tests. There are various functionalities of Travis that can be tweaked with the configuration file. However, you are limited to only the features that the Travis team provides. Jenkins, on the other hand, is highly customizable. In addition to the default features that it provides, there is a variety of plugins available for your use.

Technical Expertise Required: Travis is a cloud-based implementation of a CI tool that is pretty easy to use with a user-friendly GUI. Jenkins needs a bit more technical expertise to set up and customize. However, once the initial set up is done, it is well worth the effort.

Costs Involved: The costs revolve around the type of your repository. If you manage an open-source repository hosted on GitHub, Travis is free for you to build and test. Though Jenkins is free to use as well, additional hosting costs may not be feasible for a team with a limited budget.

Final Thoughts

The process of testing is relatively similar in these two popular CI tools, though their implementation is very different.

• If you have a repository hosted on GitHub or would like minimal fuss in setting up your testing framework, you should go ahead with Travis CI.
• However, if you have a self-hosted repository and would like complete control over your CI tool, you should go ahead with Jenkins CI.