Most IT companies around the globe currently follow Agile methodologies in their SDLC. Continuous Integration and Continuous Delivery (CI-CD) are basically buzzwords in software development. Automation of build and release activities is quickly becoming a common practice. Jenkins has by far been the most widely used CI-CD automation tool. This is because of the enormous community support and a huge number of plugins it offers.
Many other companies have recently come up with tools to address the requirement for CI-CD Automation and Orchestration. Jenkins is no longer the go-to tool for all CI implementations. CircleCI, TeamCity, TravisCI, Spinnaker, etc. have made their way into the market. More and more teams are opting for these tools to overcome the shortcomings of Jenkins.
This article will compare Jenkins with one of the most promising tools in the same domain, CircleCI.
What is CircleCI?
CircleCI is primarily a cloud-based CI orchestration tool. There is also an Enterprise version which can be setup on one’s own infrastructure. It was founded in 2011 and is based out of San Francisco. This tool helps in automating installation and delivery processes. It is quite simple to configure and maintain.
CircleCI reduces the overhead of having a dedicated server as it is cloud-based. The enterprise version is also low on maintenance. The cloud-based platform offers credit-based plans that are scalable and help in deploying applications faster.
Salient features of CircleCI
- Serves around 30000 clients and is capable of running a million tasks per day.
- Offers performance-based scaling options.
- Incorporates SSH into the build and test runs to debug.
- Enables setting up of parallel builds for faster execution of the process.
- Runs every task as a new container which prevents stale build data causing issues.
- Announces the end of task execution via Email Notification.
- Offers numerous orbs (plugins) that help in connecting the existing tools setup.
- Offers cached third-party configurations and application specifications instead of system deployment.
What is Jenkins?
Jenkins is an open-source automation tool. Initially developed by Hudson, Jenkins later was separated into a new tool and was made open source. It’s extensive community support and contribution has made it the most popular of open-source CI-CD tools. This has also led to the development of over 1500 plugins for various integrations with other tools.
Through the years, Jenkins has gained wide popularity in the DevOps community due to its versatility and huge community support.
Salient features of Jenkins
- With the enormous number of plugins, it can be set up as a simple CI server and also handle CD for complex projects.
- Can be used to connect multiple slave nodes which helps in distributing the workload across platforms.
- Almost any tool can be integrated into Jenkins owing to the enormous number of plugins available in its update center. Most companies who develop tools themselves also release a plugin for Jenkins.
- Owing to its highly distributed nature and huge plugin support, there are a lot of possibilities for what Jenkins can do.
- Ready packages for all OS flavors available on the download center. Java is the only prerequisite it requires.
- Has a very neat and interactive UI which makes configuring projects easy. There is built-in help for most configurations.
CircleCI vs Jenkins
|Build Configuration||– Builds are configured using the circle.yaml file|
– This config will be like any other repo and can be called using CircleCI
– Helps in version control of the configurations and makes sharing of build configs easy
|– Builds are configured through the Jenkins web interface or the Jenkins pipeline-as-a-code groovy syntax|
– Settings are stored in the Jenkins file system on the Jenkins master node
– Build configurations cannot be easily shared as there is no version control integration for storing the config files
|Setup and Maintenance||– No initial setup required as it is primarily cloud-based. Enterprise version is also easy to set up|
– Out-of-the-box solutions available to maintain third-party integrations
– New features will be available to the users as soon as they are released in the cloud version
|– Initial setup can be done using the packages available for respective OSes|
– Initial setup is easy but the configuration becomes a bit tedious
– Maintenance requires a dedicated resource to check compatibility of Jenkins and the plugins installed as jenkins and plugin developments do not happen in sync. Plugins are interdependent causing a huge dependency chain
|Build Environment||Every job will be run in a new container where all the dependencies will be installed by CircleCI||The teams need to maintain the sanity of the build environment as jobs are run on the same server. Maintaining dependencies also becomes a tedious task|
|User Interface||Has an interactive UI and undergoes frequent upgrades||Is heavy and clumsy. It is comparatively slower and has more of a legacy UI|
|Plugin Support||– Offers a decent number of Orbs – plugins that help integrate with various tools|
– Orbs are structured in a standard way and have common best practices making it easy for the users
|– Offers a plethora of plugins integrating almost all tools used as a part of the SDLC|
– Plugins are developed by companies and various community contributors
– They do not have a common structure and thus become difficult for the users
|Performance||– Offers scalable systems to accommodate large build processes and running jobs in parallel|
– Cache dependencies, Docker layers help in speeding up the builds
|– Build nodes need to be scaled in order to scale up Jenkins for faster executions|
– Execution speed depends on the efficiency of the plugins
|Parallelism||Comes with an in-built feature to enable parallelism. This is achieved by running multiple containers at once||Builds jobs can be run in parallel using multi-threading|
|Permissions||– Users can be added via VCS Authentication|
– Permissions can be automatically adopted from VCS
|– Offers matrix-based and role-based permissions.|
– Permissions need to be set up manually and cannot be imported from other security systems. As an alternative, many teams place Jenkins behind their AD or Oauth systems
|Security||– Application level security coupled with runtime isolation using containers provides a secure build environment|
– SOC II and FedRAMP certified
|– Only a single layer of security surrounding the CI fleet|
– Additional security can be manually set up
– Various levels of security for OSS plugins
|Docker Workflow||Built-in support for Docker in Workflow can be accessed by adding services section in circle.yaml||No built-in support for Docker. Plugins can be installed and Docker support can be enabled in the build environment|
|Debugging||Features like SSH and automated DevOps testing features make it easier to debug||Debugging needs manual DevOps testing and integrated team support|
Which tool to use? CircleCI or Jenkins?
CircleCI and Jenkins are both highly capable tools. Which tool to select among these clearly depends on the requirement and resource availability of a particular project. For a basic CI setup, both tools can do fairly well and cater to the requirements.
CircleCI does not have the overhead of initial setup and maintenance. This makes it a go-to choice when the implementation needs to get going in a short span. Additionally, when a company does not have a dedicated resource to maintain the CI environment, the cloud-based platform of CircleCI helps. Parallel execution of builds is another case for which CircleCI can be considered.
Jenkins is an open-source tool. When a company can afford to allocate dedicated servers and manpower to setup and maintain Jenkins, it would surely be the go-to choice. When the workflow has multiple tool integrations, source control other than bitbucket and Github, and when the build uses some highly confidential data which cannot be run over a cloud-provided CI setup, in-house setup of Jenkins can be used.
No matter which CI/CD server is chosen, testing the application’s cross-platform compatibility on real browsers and devices is mandatory. It is the only way to guarantee that the software delivers seamless and consistent UX irrespective of the device and browser used to access them.
Emulators and simulators simply do not offer the real user conditions that software must run within, making the results of any tests run on them inaccurate. Consider testing websites and apps on a real device cloud, preferably one that offers the latest devices, browsers, and OS versions. This applies to both manual and automated testing.
BrowserStack’s real device cloud provides 2000+ real browsers and devices for instant, on-demand testing. It also provides a cloud Selenium grid for selenium automated testing, which can be accelerated by 10X with parallel testing. The cloud also provides integrations with popular CI/CD tools such as Jira, Jenkins, TeamCity, Travis CI, and much more. Additionally, there are in-built debugging tools that let testers identify and resolve bugs immediately. BrowserStack also facilitates Cypress testing on 30+ browser versions with instant, hassle-free parallelization.