Software teams today are tasked with delivering automation across a diverse set of DevOps workflows. They must support a broad portfolio of applications and tools, while also serving the needs of different personas, inlcuding developers, SREs, and QA testers.

The characteristics of open source software very much lend themselves to meeting these challenges, as OSS can be much easier (and cheaper) to integrate with elements in your pipeline than other solutions.

DevOps and SRE teams should be using tools like: Review Board (code review), Artificatory (repo for Open Source projects), Jenkins (CI/CD pipelines) Terraform (infrastructures) and Delphix (data masking protection) to bring automation and speed to development for faster releases.

1. Review Board

Review Board is a web-based collaborative code review tool, available as free software under the MIT License.

An alternative to Rietveld and Gerrit, Review Board integrates with Bazaar, ClearCase, CVS, Git, Mercurial, Perforce, and Subversion.

You can check on a demo of Review board here.

Review Board can be installed on any server running Apache or lighttpd and is free for both personal and commercial use.

There is also an official commercial Review Board hosting service, RBCommons.

Review requests can be posted manually or automatically using either a REST Web API, or a Python script.

2. Artifactory

As the world’s first universal repository, JFrog Artifactory is the mission-critical heart of the JFrog Platform functioning as the single source of truth for all packages, container images and Helm charts, as they move across the entire DevOps pipeline.

3. Jenkins

Jenkins – an open source automation server which enables developers around the world to reliably build, test, and deploy their software.

Jenkins is a free and open source automation server. It helps automate the parts of software development related to building, testing, and deploying, facilitating continuous integration and continuous delivery. It is a server-based system that runs in servlet containers such as Apache Tomcat.

It supports version control tools, including AccuRev, CVS, Subversion, Git, Mercurial, Perforce, ClearCase and RTC, and can execute Apache Ant, Apache Maven and sbt based projects as well as arbitrary shell scripts and Windows batch commands. The creator of Jenkins is Kohsuke Kawaguchi. Released under the MIT License, Jenkins is free software.

Builds can be triggered by various means, for example by commit in a version control system, by scheduling via a cron-like mechanism and by requesting a specific build URL. It can also be triggered after the other builds in the queue have completed. Jenkins functionality can be extended with plugins.

The Jenkins project was originally named Hudson, and was renamed after a dispute with Oracle, which had forked the project and claimed rights to the project name. The Oracle fork, Hudson, continued to be developed for a time before being donated to the Eclipse Foundation. Oracle’s Hudson is no longer maintained and was announced as obsolete in February 2017.

4. Terraform

Terraform is an open-source infrastructure as code software tool created by HashiCorp. It enables users to define and provision a datacenter infrastructure using a high-level configuration language known as Hashicorp Configuration Language (HCL), or optionally JSON.

Terraform supports a number of cloud infrastructure providers such as Amazon Web Services, IBM Cloud (formerly Bluemix), Google Cloud Platform, DigitalOcean, Linode, Microsoft Azure, Oracle Cloud Infrastructure, OVH, Scaleway, VMware vSphere or Open Telekom Cloud as well as OpenNebula and OpenStack.

HashiCorp also supports a Terraform Module Registry launched in 2017 during HashiConf 2017 conferences. In 2019 Terraform introduced the paid version called Terraform Enterprise for larger organizations. Terraform has four major commands: Terraform init, Terraform Plan, Terraform Apply, Terraform Destroy.

With Terraform you can:

Deliver infrastructure as code with Terraform

Write declarative configuration files Collaborate and share configurations Evolve and version your infrastructure Automate provisioning Define infrastructure as code to manage the full lifecycle — create new resources, manage existing ones, and destroy those no longer needed.

Write Declarative Configuration Files

Plan and predict changes Clearly mapped resource dependencies Separation of plan and apply Consistent, repeatable workflow Terraform provides an elegant user experience for operators to safely and predictably make changes to infrastructure.

Plan Changes to Cloud Infrastructure Create reproducible infrastructure Reproducible production, staging, and development environments Shared modules for common infrastructure patterns Combine multiple providers consistently Terraform makes it easy to re-use configurations for similar infrastructure, helping you avoid mistakes and save time.

5. Delphix

What is Delphix?

Delphix is software that runs as a VM on VMware. Delphix manages storage. Delphix looks like a NAS to databases clones using Delphix. Delphix pulls in changes from source databases. Delphix exposes the source database data to clone databases via NFS. Clone databases share all duplicate blocks on Delphix and only modified blocks are kept private to those clones.

Delphix compresses data stored on Delphix and orchestrates access to the data sharing duplicate blocks between multiple clones and keeping modified blocks private to the clones who made the changes. From the source data on Delphix one can spin up clones in a few minutes for almost no storage overhead.

The clone instances run on any hardware that has compatible Oracle software installed and those clone database instances reference the datafiles on Delphix via NFS. Delphix is automated and all operation are enabled via few clicks of mouse in a GUI or can be scripted via a CLI. Delphix is so simple a developer can clone databases in minutes without knowing Oracle or SQL Server. The core value is speed and agility.

Speed up development projects Provide backup and recovery protection eliminating long resource heavy weekly backups Eliminate ETL windows and Reporting database refresh One can give all the members of a development team all their own private copies of the database giving then full copies, thus eliminating bottlenecks caused by sharing a single copy and avoiding bugs caused by using subsets.

One customer estimates eliminating 20% of bugs on prod after shifting from using subsets for development and going to Delphix full virtual databases. Also if each developer has their own copy they can code without being blocked by other developers. One company’s teams were waiting 1 - 2 weeks for developers to make modifications to shared DBs before Delphix.

Even more impressive is that QA can spin up many copies in minutes in parallel instead spending large amount of time building QA databases and then running destructive tests. Some of our customers spent 90+ % of their QA cycle times building QA environments and less than 10% of QA cycle times actually QA’ing. Now with Delphix they have 99% of their QA cycle time actually doing QA!

Largely thanks to QA speedups a few customers have actually reached 5x greater project output. That’s like taking a team of 100 and getting the effective output of 500 people. Unprecedented. A 5x improvement is the high end but 2x project output is an standard improvement.

At the very least we’ve see a minimum 20% improvement. Delphix also acts as efficient backup since after the first full copy Delphix never takes a full copy again but only brings in changes. Dephix keeps weeks of changes meaning if someone or something did a destructive change yesterday on production then they can pull out the data from Delphix before the change.

An cable provider had just installed Delphix 2 days before someone dropped the movies title table on prod meaning none of 26 million customers were unable to rent movies. Of course Dataguard had the drop as well. They would have gone to backup to recover taking 8 hours but since they had Delphix that got the data in minutes.

By doing efficient backups, Delphix frees up many of our customers networks over the weekends because they no longer have to do full copies on the weekend. Delphix can also tag and branch database versions enabling “data control” for databases as one would do source control for code.

5 DevOps Tools You Should Know