A Tale of Two (or more) DevOps Reports

A recent email from IT automation software company Puppet plugged its 2018 State of DevOps Report. Because this topic is important to many companies I write for (and because I read last year’s report), I downloaded a copy. A few things stood out.

First, authorship. Puppet and machine-data software company Splunk present this year’s report. There are three authors from Puppet and one from Splunk. Last year’s report featured two authors from Puppet and three from DevOps Research & Assessment (DORA).

That leads to the question of what happened to DORA. A blog article by Puppet Director of Product Marketing Alanna Brown sheds some light. In it, she thanks DORA for their partnership in previous reports. “Gene Kim and Jez Humble signed on in 2012 to partner with us on our very first survey and Nicole Forsgren joined us in 2014,” she writes. “We’re grateful for all the insights they’ve provided over the years.”

A Q&A posted below that article explains further. A reader who had downloaded a separate “DORA report” had a question. “Are there two different surveys now,” he writes, “or is it one survey and two separate interpretations?” Brown answers that there were now two projects. A co-author of the Puppet report, Brown says they used their own data this year to identify “foundational” DevOps practices and five stages in a DevOps evolution.

Brown indicates that there had been a kind of handoff: “DORA continues the research which Puppet started seven years ago, which examines IT performance impacts.”

What then about the report from DORA? Titled “Accelerate: State of DevOps: Strategies for a New Economy,” co-authored by Forsgren, Humble and Kim, and released in August, it’s a bit shorter than the Puppet report and bears more supporting logos on its cover. Google Cloud is the DORA report’s “Diamond Sponsor,” and 15 other companies are “Gold Sponsors.” By contrast, the Puppet report (not counting Splunk) has six sponsors.

 

Puppet, DORA and DevOps analysts

As Brown notes, the Puppet report focuses on foundational practices and a five-stage evolution. It is especially concerned with helping teams “stuck” in their current DevOps efforts to succeed on a wider scale. The DORA report taps into the views of some 1,900 professionals, but also “represents five years of work surveying over 30,000 technical professionals worldwide.” It aims to assess the impact of cloud adoption, open source software, organizational practices and culture on software delivery performance.

These messages align with a general consensus among analysts. Forrester says that DevOps reached “escape velocity” in 2017 and that discussions with clients shifted from defining terms to implementing at scale. A 2018 report from Gartner placed more than 30 aspects of DevOps on its famed Hype Cycle. That Puppet and DORA were now diverging, while both emphasizing performance, seems like another sign of growing industry maturity.

It also means there’s more to read, but that’s another issue.