In my thirteen year journey of studying high performing IT organisations, I’ve started to see a new and unsettling trend. Whenever I mention ITIL and IT Service Management in presentations and briefings, people in the audience snicker. When I ask why, they roll their eyes, and talk about the shrill, hysterical bureaucrats that suck life out of everyone they touch, doing everything they can to slow the business down, preventing everyone from getting work done.
This is simply not true. In fact, every time I’ll argue that ITSM skill sets are ever more important in a world where there is an ever quickening business tempo.
However, an even more troubling trend is that ITSM practitioners will dismiss emerging movements such as “DevOps,” suggesting that it’s a passing fad.
It is my genuine belief that that patterns and processes that emerge from DevOps are the inevitable outcome of applying Lean principles to the IT value stream. It is an inexorable force that will likely change IT in a manner we haven’t seen since the birth of client-server computing in the 1980s.
More importantly though, ITSM practitioners are uniquely equipped to help in DevOps initiatives, and create value for the business.
The DevOps Movement fits perfectly with ITSM. My goal is to help you become conversant with DevOps and aid you in recognizing the practices when you see them. I hope this article will illustrate how information practitioners can contribute to this exciting organisational journey.
What Is DevOps?
The term “DevOps” typically refers to the emerging professional movement that advocates a collaborative working relationship between Development and IT Operations, resulting in the fast flow of planned work (i.e. high deploy rates), while simultaneously increasing the reliability, stability, resilience and security of the production environment.
Why is it that Development and IT Operations are singled out? Because that is typically the value stream that is between the business (where requirements are defined) and the customer (where value is delivered).
The origins of the DevOps movement is commonly placed around 2009, as the convergence of numerous adjacent and mutually reinforcing movements, most notably the “10 Deploys A Day” presentation given by John Allspaw and Paul Hammond and the Agile system administration movement (Patrick DeBois).
Currently, DevOps is more like a philosophical movement, and does not yet have a precise collection of practices, descriptive or prescriptive (e.g. CMM-I, ITIL, etc.). On the other hand, it is an incredibly vibrant community of practitioners who are interested in replicating the performance outcomes and culture described so vividly by organisations such as Etsy, Amazon, Netflix, Joyent and so forth.
DevOps aims to address a core, chronic conflict that exists in almost every IT organisation. It is so powerful that it practically pre-ordains horrible outcomes, if not abject failure. The problem? The VP of Development is typically measured by feature time to market, which motivates as many changes, as quickly as possible. On the other hand, the VP of IT Operations is typically measured by uptime and availability.
Until very recently, it was impossible to get both desired outcomes of fast time to market and sufficient reliability and stability. Because of these diametrically opposed outcomes (“make changes quickly” vs. “make changes very carefully”), Development and IT Operations were in a state of constant inter-tribal warfare, with ITSM practitioners put right in the middle.
Although many people view DevOps as backlash to ITIL (IT Infrastructure Library) or ITSM, I take a different view. ITIL and ITSM still are best codifications of the business processes that underpin IT Operations, and actually describe many of the capabilities needed into order for IT Operations to support a DevOps-style work stream.
I am part of a team who wrote “The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win”, which codifies the “good to great” transformation we’ve observed these organisations making. Our goal is to create a prescriptive guide that shows how Development, IT Operations and ITSM practitioners can work together to create phenomenal organisational outcomes that none of them could achieve alone.
What are the unpinning principles of DevOps?
In “The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win” we describe the underpinning principles in which all the DevOps patterns can be derived from as “The Three Ways.” They describe the values and philosophies that frame the processes, procedures, practices, as well as the prescriptive steps.
The First Way emphasizes the performance of the entire system, as opposed to the performance of a specific silo of work or department — this as can be as large a division (e.g., Development or IT Operations) or as small as an individual contributor (e.g., a developer, system administrator).
The focus is on all business value streams that are enabled by IT. In other words, it begins when requirements are identified (e.g., by the business or IT), are built in Development, and then transitioned into IT Operations, where the value is then delivered to the customer as a form of a service.
The outcomes of putting the First Way into practice include never passing a known defect to downstream work centers, never allowing local optimization to create global degradation, always seeking to increase flow, and always seeking to achieve profound understanding of the system (as per Deming).
The Second Way is about creating the right to left feedback loops. The goal of almost any process improvement initiative is to shorten and amplify feedback loops so necessary corrections can be continually made.
The outcomes of the Second Way include understanding and responding to all customers, internal and external, shortening and amplifying all feedback loops, and embedding knowledge where we need it.
The Third Way is about creating a culture that fosters at two things: continual experimentation, which requires taking risks and learning from success and failure; and understanding that repetition and practice is the prerequisite to mastery.
We need both of these equally. Experimentation and risk taking are what ensure that we keep pushing to improve, even if it means going deeper into the danger zone than we’ve ever gone. And we need mastery of the skills that can help us retreat out of the danger zone when we’ve gone too far.
The outcomes of the Third Way include allocating time for the improvement of daily work, creating rituals that reward the team for taking risks, and introducing faults into the system to increase resilience.
What Are The Areas Of DevOps?
We divide up the DevOps patterns into four areas:
Area 1: Extend Development into IT Operations:
In this area, we create or extend the continuous integration and release processes from Development into IT Operations, integrating QA and infosec into the work stream, ensuring production readiness of the code and environment, and so forth. The steps include:
- Create the single “repository of truth” containing both the code and environments
- Create the one-step Dev, Test and Production environment build process
- Extend the deployment pipeline processes into production
- Define roles and integrate QA, Infosec, Ops/CAB into Dev workstream
First, we put everything needed to rebuild the service into a common repository from scratch, including both the application and the environment (i.e., operating system, databases, virtualization, and all associated configuration settings).
Next, we will make a one-step environment creation process available at the earliest stages of the Development project. By using a common build process and requiring that Development be responsible for ensuring that the code and the environment work together, we’ll have an unprecedented level of production ready, even at the earliest stages of the development project.
This impacts the ITSM process areas of release, change, and configuration management. The ways that ITSM practitioners can actively integrate into the DevOps value stream includes the following:
- Find the automated infrastructure project (e.g., puppet, chef) that provisions servers for deployment. We can help that team with our release management readiness checklists, security hardening checklists and so forth, integrating them into the automated build process.
- Define pre-authorized changes and deployments, and ensure that production promotions are captured in a trusted system of record that can be reviewed and audited.
- Define changes and deployments that require authorization, such as security functionality that is relied upon to secure systems and data (e.g., user authentication modules). The goal is to ensure that changes that could jeopardize the organisation (e.g., the infamous 2011 Dropbox failure where customers discovered that authentication was disabled for four hours) never occur.
Area 2: Create IT Operations feedback into Development
The steps in this area ensure that information from IT Operations is radiated to Development and the rest of the organisation. IT Operations is where value is created, and this feedback is required in order to make good decisions.
The specific steps in this area include:
- Make all infrastructure data visible
- Make all application data visible
- Modify the incident resolution process and blameless post-mortems
- Monitor the health of the deployment pipelines
The first step overlaps with the ITSM process areas of event management, while the second step requires creating the monitoring infrastructure so that there’s no excuse for developers not to add telemetry to their application (e.g., “since it only requires one line of code, even the laziest developer will instrument their code”).
The third step then enables IT Operations and Development to resolve incidents quickly, by ensuring that all relevant information from the entire application stack is at hand to determine what might have caused the incident, and then to restore service.
ITSM practitioners can help by ensuring that the process areas of event management, as well as incident, problem and knowledge management are modified to incorporate Development.
Area 3: Embed Development into IT Operations
According to the Second Way, the goal of steps in this area is to create knowledge and capabilities where we it is needed, and shorten and amplify feedback loops. A delightful quote that frames this comes from Patrick Lightbody, CEO, BrowserMob. He said, “We found that when we woke up developers at 2am, defects got fixed faster than ever.”
To facilitate creating tribal knowledge within IT Operations and shared accountability for uptime and availability with Development, the steps in this area include:
- Make Dev initially responsible for their own services
- Return problematic services back to Dev
- Integrate Dev into the incident management processes
- Have Dev cross-train Ops
Area 4: Embed IT Operations into Development
This area is the reciprocal of Area 3, and the goal is to create the service design and delivery equivalent of designing for manufacturing (DFM). In plant engineering, DFM recognizes that the primary customer of engineering is the manufacturing personnel, and therefore one of the engineering goals is to parts are designed for easy assembly, minimizing the likelihood of putting on parts on backwards, over-tightening, being damaged during transit or assembly, and so forth.
Similarly, in addition to ensuring that IT Operations needs are integrated into the daily Development processes of design, requirements specification, development and testing, the product and processes are designed with resiliency in mind.
The steps in this area include:
- Embed Ops knowledge and capabilities into Dev
- Design for IT Operations
- Institutionalize IT Operations knowledge
- Break things early and often
This includes embedding or liaising IT Operations resources into Development, creating reusable user stories for the IT Operations staff (including deployment, management of the code in production, etc.), and defining the non-functional requirements that can be used across all projects.
Conclusion
It is my firm belief that ITSM and the DevOps movement are not at odds. Quite to the contrary, they’re a perfect cultural match. As DevOps gains momentum I’m excited by what we can achieve using a winning combination of the two. It is my sincere hope that by reading this article, you’ll better understand what DevOps is, see why it is important and be energized by the possibilities it creates, and generate some ideas of how to put some of these practices into place in the IT organisations you help support.
This article has been contributed by Gene Kim, Author of the book“The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win”.
