What’s one thing that makes developers at the same time more productive, better at their job, and happier? Owning their code in production!

At least that’s it according to Charity Majors, CTO ar Honeycomb, an avid advocate of “fulfilling the promise of CI/CD, always outspoken on what’s on her mind, even when those are unpopular opinions Those may include testing in production or…deploying on Fridays!

If deploying on Friday is something you and your team dread, Charity has some words of wisdom on how to go from a sick, fragile system where everyone is afraid to touch it to a swift, robust system where your deploys aren’t scary, and it’s easier than you think!

“It’s not actually about deploying on Fridays or not. It’s about the fact that shipping software shouldn’t be scary. Are your deploys lengthy, difficult, fragile affairs that most of your engineers are too scared to touch When something goes wrong, does it take several engineers the rest of the day to clean it up?

After you’ve deployed, do you have any real confidence that your changes are working as intended, Or do you usually find problems via customer complaints that trickle in over the next few days instead of promptly via your own telemetry? Well, then, I wouldn’t deploy on Fridays either. But these are all signs of a very sick system We can and should do much better.“

The “D” should stand for “deployment”, not “delivery“

CI/CD revolution is 15 years old, and back then, it was acceptable for it to mean Continuous Integration/Continuous Delivery,  that you should just be ready to deploy anytime. Nowadays, there’s no more excuse; it should mean Continuous Integration/Continuous Deployment. It’s time to fulfill the promise of CI/CD, according to Charity, your code should be deployed to production within 15 minutes after it’s been merged to the main:

“At Honeycomb, we deploy from cron every 15-30 minutes or so. It’s fully automated –  every developer knows that if they merge their code to main, it will be deployed shortly So if they aren’t ready for it to go live, they either need to use feature flags to disable the new code or pause on merging until they’re ready.

The rule isn’t “don’t deploy on Friday” (or any day or time), the rule is “don’t deploy and run.” Whether it’s Monday at 5 pm or Friday at 2 pm, don’t deploy unless you have time to a) watch your code go out, b) fuss around with your instrumentation to validate your changes, and c) trigger a rollback or debug if something looks wonky.“

Fear of production = Technical Debt

Fear of deploys is the single largest source of technical debt in almost every engineering organization she’s ever seen And we know that the cost of fixing bugs goes up exponentially from the moment you write it, so the sooner you can find it, the better.

Losing fear of production does not only benefit companies, it’s the single biggest factor in making you a better and happier developer:

“Most developers have this delay: you write the code, you commit it and at some future point, someone’s going to deploy it with other changes some other engineers did, all at once.
Decoupling the development and deployment severs the link between the two and the idea of code ownership. 

If you know that your code is going out in the next 15 minutes and your code alone, you’re probably going to look it up in production and see if it’s doing what you expected it to do, and if something looks weird If you don’t know when your code is going out and someone else is going to deploy it, that it might be out in days or even weeks, you’re never going to go and look at it.“

Fixing processes is cheaper than hiring new people

Engineers who have experienced that way of working are not willing to go back to “the bad old days”, she continues. Because it’s transformational, it gives software developers back the feeling that they’re in a creative and problem-solving profession (not just shipping something and then waiting on each other for weeks before they see it in production).

“To become a better developer, you have to lose the fear of deployment. Even to become senior engineers, I would say!

Being able to own your code in production, and being responsible for the stuff you put out in the world makes a great engineer. Tightening up those feedback loops is crucial so you’re not just writing your code and forgetting about it, thinking some other teams will take care of it No, you own your code!“

OK, so what’s the path to achieving that It’s actually not technically difficult but socio-technically challenging. She says “Socio-” meaning the challenge is with people. One of them is that improving the development process is often neglected within organizations, and perceived as a waste of time and money. Many companies are OK with spending budgets on hiring, but feel like they’re losing money if they slow down with development for some time to fix the process:

“It’s easy to focus on just shipping features because they’re tied to profit, and it’s easy to feel like you’re wasting time improving the development process.

I’ve seen companies with 800 engineers working on one app. What are they all doing?!
They’re wasting so much time and energy waiting on each other; everything takes so much time, and then they try to fix it by hiring more people. And it just makes everything take even longer – more people waiting on each other.“

Good engineering managers to the rescue

Charity points out that If you haven’t read the Stripe developer report, you haven’t been properly terrified yet. According to the report, developers self-report that they spend 40% of their time waiting for other teams to finish or start and trying to figure out what they should work on next.

Is there hope? There is, and it lies in good engineering managers who are not too far removed from coding themselves but know how to translate their team’s needs to dollars and cents: if it takes two weeks to deploy some code, the company is wasting five engineers’ worth of salary.

Also, don’t feel stressed about fulfilling Charity’s motto, “deploy in 15 minutes or bust”. She admits it herself it’s a bit hyperbolic, but – the closer you get to it, the better your life.