There’s a theme that comes up over and over at our user conferences: releases are hard and sometimes scary. And the stakes only get higher as you scale.
At LaunchDarkly, we’re big fans of helping you create “boring” releases—the kind that will make you yawn they're so uneventful. If your releases felt more like watching a marathon of slasher flicks before, we want to transform them into something that's more like a two-hour documentary on how paint dries.
Today, we'll share a couple stories relayed at our Galaxy and Trajectory conferences last year about making releases uneventful, even as your company is scaling out.
Slay those release dragons
Releases that go wrong can feel catastrophic—much like a dragon breathing fire over your entire environment, according to LaunchDarkly's Director of Developer Relations, Cody De Arkland. If releasing to production fills you with fear and dread, Cody shared some dragon-avoidance strategies:
- Go slowly and sneak around: Test methodically, and move carefully at each stage of the software delivery process.
- Wait for safety: Stick to safe release times and windows.
- Explore easier caves first: Break your software release journey into more manageable parts and target less risky releases.
These are the typical release management approaches and while valid, they “all add friction and hurt our ability to be agile,” said Cody. “They make processes take longer. They put us in the box of specific times. If we miss that time window, we have to wait and reschedule out longer. It just makes it very challenging to move fast and to innovate quickly.”
Risk avoidance starts being prioritized over progress: “We become fixated on this idea of ‘How do I avoid that risk and how do I not have that happen?’ as opposed to just managing the risk in a smoother way.”
Cody can’t promise to help you slay the release dragon, but he does have a couple pieces of advice for shielding against it. Check out his full talk (including a demo) below when you have the time, but in the meantime here are some highlights.
Don’t debug under pressure
Use feature flags to mitigate the more disastrous effects of a feature not working in production. If you can toggle a feature off, you can troubleshoot safely without impacting your users.
Automate and standardize complex rollouts
Now that you’re able to mitigate release troubles without scrambling to roll back, you can use that newfound confidence to explore more complex and sophisticated rollout strategies. You can build conditions for feature flags to determine exactly which type of user you release to first, and use percentage rollouts to automate the gradual release to everyone as you validate the release is successful.
“We've taken this concept of deployment and broken it apart from the idea of how we release changes,” explained Cody. “So I've released a bunch of changes here without having to every time go in and rebuild that image or redeploy that image or delete the pod in Kubernetes, create an interesting load balancer rule, create some weird DNS waiting to send it to the other application. I'm able to control these releases step by step along the way. And if there were problems with any of these, it's easy for me to come in and start turning those features off immediately.”
Advance your release strategies
Let’s go deeper into how to automate safe, efficient releases with LaunchDarkly's Senior Solutions Engineer, Chris Diaz, in Scaling and Standardizing Your Release Process.
Chris shares some strategies he sees LaunchDarkly customers using to exercise even more fine-grained control over their releases:
Parent-child feature flags
“This is a strategy I see a lot with many of my customers where all new features or new changes will be made dependent on a release flag,” Chris explained. “So we can use this release flag, or in this case this parent flag, to control releasing all of these features at the same time, but we still have that separation of each so that if there's a problem with one we can turn off just that one that's causing the problem.”
Progressive rollouts
Canary releases are a game changer for teams wanting to mature their release process and validate new features with relatively low risk. The next level is automating the entire process by scheduling when and by what percentage you want to incrementally increase the audience you’re exposing that new feature to.
Scheduled maintenance windows
“Tonight we're going to be doing a release and we need to turn this feature off for maintenance while that process happens. We can quickly schedule that and just say, ‘Hey, at midnight tonight turn this flag off for an hour and then after that hour has passed, turn it back on.’”
Custom Workflows and templates for repeatable processes
For even more granular control, Chris showed us how to use custom Workflows to describe, for example, a release process involving gradual rollouts to targeted users at specific intervals. “By saving it as a template, that makes it reusable for other team members. So now we can define this process and it can become consistently used across our organization, which helps us to scale across the organization.”
Watch his full talk below:
How Chronosphere releases features at cloud native scale
Alright, so both Cody and Chris’ talks include demos to show you how these processes look in LaunchDarkly, but if you want to get even more tangible, take it from a customer. At a talk relayed during our latest Trajectory conference, Sidney Wijngaarde, software engineer at Chronosphere, shared how they’re using LaunchDarkly to manage their complex release program:
“In a pre-LaunchDarkly world, we had a lot of static configuration for toggling features on and off, it was a pain to manage, and we actually had to have engineers redeploy services to release features to our customers,” Sidney shared. “Now, we can separate the business decision of releasing a feature with the engineering decision of how to develop it. And that's been really powerful for us with releasing.”
Sidney is part of a time series database team at Chronosphere, a cloud-native observability platform, that also uses a feature flag to enable detailed logging and traces in the event of an error in production. This would be costly and unsustainable to run at all times, but “when something's wrong it's nice to have LaunchDarkly to just flip on this verbose debugging for an application, without redeploying and losing all the context and the specific circumstances that caused the issue.”
Want to take a closer look at how LaunchDarkly can help you rein in your releases? Check out a demo.
.webp)
