Modern History – Changes in the Industry
Introducing— The Full Stack Developer
The “Full Stack Developer”. A fairly recent term which has grown to occupy a huge swath of current developer job descriptions. Whether you’re a JS dev, a .NET or .NET Core dev, a Java dev, an iOS developer working in Swift or Objective-C—or even better, a polyglot with no attachment to any particular language—you are a “full stack developer” if: Your job is to design, implement, deploy, manage, and support complete solutions from initial whiteboard thru to production, including selecting the infrastructure and platform, cloud provider, language, and all the other important project metadata.
These developers have more power and control over their projects and platforms than any group since the beginning of the web. Consider this quote from Rise of the Developer: The New Kingmakers, by Stephen O’Grady:
“The most successful companies today are those that understand the strategic role that developers will play in their success or failure. Not just successful technology companies – virtually every company today needs a developer strategy.
“There’s a reason that ESPN and Sears have rolled out API programs, that companies are being bought not for their products but their people. The reason is that developers are the most valuable resource in business.”
Stephen O’Grady – Rise of the Developer: The New Kingmakers
It’s even common these days for platforms, languages and even whole technology suites to become commoditized, and for the same team to do projects in a variety of languages—even within the same sprint. Neither is it uncommon for IT managers to refer to their tech stack as “alphabet soup” or a “technological octopus”. But how, after 25 years of language and platform standardization, and workforce siloing, did we seem to go full circle back to the “wild west days” of CGI connectors and Perl, where every project uses a different set of languages and tooling?
And an even more important question is this: “What changed to enable this seemingly more chaotic, but often more productive, approach to software development that flies in the face of conventional wisdom?”
There Has to be a Better Way
For many years, the business practice of “siloing” has been a staple of IT organizations around the world. We have had infrastructure folks, developer folks, DBAs, security teams, telecom, and various other groups involved every single project. As a result, the knowledge associated with each part of our projects was siloed as well. Ops was in charge of server and network provisioning, setup, and maintenance—creating each environment by hand from scratch, for Dev to production and everything in between. Developers created code that needed to be built, distributed, and deployed. DBAs used and maintained DB servers (also created and provisioned by Ops) and so on.
As technology has evolved and we’ve collectively cycled through millions of projects, we’ve had time to see just how inefficient and unproductive that “siloed” platform can be. Ops blames Dev for failures and vice versa. Ops is overloaded and has huge budgets because the entire infrastructure is bespoke, crafted to purpose, constructed with very expensive hardware… essentially building pets that need love and tenderness to flourish, and to which people become quite attached. For developers, the compliance and cost headaches has led to missed deadlines, budget overruns and the feeling that Ops is a drag on their velocity. Case in point, in one position I held it took at least 6 weeks to provision and deliver a new VM from the date of request!
This approach adds nearly insurmountable amounts of friction to solutions delivery, crippling our ability to quickly deliver value to the customer. But here’s the rub: the ability to quickly deliver value to the customer is your business. And, actually, “reducing friction and accelerating value to the customer” is one of the more widely used definitions of DevOps (but more on that later)!
In any case, forced to choose between missing deadlines, planning months in advance, or living by the old adage “it’s better to ask forgiveness than permission”, software development managers started using departmental funds to establish what we now call “shadow IT”: unsanctioned on-prem or cloud compute resources that are maintained by the Dev staff themselves, creating their own IT environments. As a side-effect of this evolution, we now have a whole generation of developers with fairly strong operations skillsets. (Take note, here, of the deterioration of the silos as a side effect of people just trying to get their work done. That alone indicates just how ineffective that paradigm really is.)
Enter DevOps
A Quick Definition
Before we move on, let’s take a second and establish a working definition of DevOps that doesn’t involve specific tasks. When most people think of DevOps they think of build servers, delivery pipelines, and all the other specific tasks that DevOps practitioners engage in.
But really, DevOps isn’t a set of tasks. DevOps is a culture. The thought leadership in the DevOps community tends to use a definition that goes something like this: DevOps is a set of best practices and cultural values that reduce friction and accelerate the delivery of value to the customer. Ultimately, DevOps represents the best in all of us, building community and cooperation. When implemented wisely, Ops gets drafted into the process of delivering new features and functionality to customers, giving them a vested interest in the end product rather than just serving as a service provider to Dev! Now, rather than a perpetual fight, Ops works with Dev with the mutual goal of driving the business forward, cohesively. Overall? It becomes everyone’s job to collaborate—and the customer base is the ultimate beneficiary!
Shifting Perspectives
OK, so suddenly we have a new generation of top-notch Dev professionals with experience in everything from infrastructure design to language choice. Now they play a critical role in selecting the tools and resources to use on a project-by-project basis. While that’s great for Dev careers, it does seem run in direct contradiction to the industry “standardization” Kool Aid that we all grew up drinking!
Well, it turns out it wasn’t Kool Aid. It was more like baby formula. Although we really didn’t realize it at the time, our obsession with standardization was really a bottleneck, a sign of our own technological immaturity. We were limiting ourselves, without even knowing it, by being a “Senior C# Developer” or a “Java Dev”. There was so much human capital going to waste that it was, at times, almost painful to participate in our chosen field. But all we had to work with was expensive boutique hardware and the associated cost-based resource allocation that left Dev high and dry when it came to creatively applying their skills to a technical challenge. It took the maturation of not just cloud technology but a merging of cloud, agile, Ops, and Dev to realize that having more than one language or technology in our “full stack”, agile infrastructure (that is, ephemeral “infrastructure as code” and not just agile projects), wasn’t just more effective, it was the difference between winning in the marketplace or not.
Winning, and Not By a Little
Today it is clear that DevOps has become the differentiator. If we continue to look at DevOps as the reduction or removal of friction between developer productivity (production) and customer value (delivery). Customer value is driven by creating new and more advanced features in our applications, but those features need to be pushed to the customer as fast as Dev can produce, test, and release them. And the path through all this is DevOps.
This sort of performance boost even shows up in official industry research. Case in point: according to Dr. Nicole Forsgren, in the State of DevOps Report 2017, the HP LaserJet Firmware team adopted DevOps and increased developer productivity over 3 years—by 700%! The same report also highlighted the following common attributes of the top DevOps implementors and most productive Dev teams in the industry:
Of the 3200 survey responses received, the top teams…
- Deliver more:
- Deploy on-demand, multiple times per day
- Less than an hour from code to production
- Repair sooner:
- MTTR of less than one hour
- Automate more:
- Perform less manual work = higher quality software and operations
- Spend more time developing business value
Performing Under Pressure
So… what happens when you mix developers with operations skills and operations staff with developer skills, in an environment of mutual support and respect, in an atmosphere of enablement? Magic happens! And when you add to that an ephemeral code-based infrastructure, running on commodity hardware, an efficient control plane, and world-class automation tools? That is where we start using word like “delight”. Now, Dev can spin up a new VM, test a few things things, and tear it back down. Now, Ops can set policies that Dev actually follows and Dev can get resources from Ops nearly on-demand—or even via self-service!
Our current mishmash of languages, tooling and platforms would be unmaintainable without something to enable it, and that thing is “automation”… which is a fundamental element of DevOps. Automating everything from notifications, provisioning, testing and deployment to fully load-balanced KPI-based auto-scaling. And when you give Dev access to your infrastructure and Ops the freedom (and budget) to work with developers to create tools and systems to drive that automation… incredible things happen.
And so…
Today, even just basic expectations have evolved to include high uptime, failover, fault-tolerance and all the benefits of enterprise class cloud installations bring. These were once things we used to have to call out in designs and lobby for budget. As the cloud has become dominant our ability to spend fewer resources on producing the infrastructure and more resources on churning out new features has become more important than ever. DevOps has created a more competitive environment for companies and a more enriching and fulfilling workplace for developers and operations staff alike!