The Only Constant is Change

Ever since Agile software development practices came to the forefront of the industry in the early 2000s, people have been talking about delivering software in small iterations. There are clear advantages to working that way from an estimation, planning, and software delivery standpoint. One item many organizations struggle with is how to balance their hard won ability to ship software to end-users more frequently with their customer’s capacity and willingness to adopt them.

I offer no answers here; only some observations via an annecdote because, as I hope to show you, so much depends on the type of software you are building and the industry in which your customers operate it. I hope this post resonates with anyone involved in shipping software and positions you to build empathy for your users when it comes to your release schedule.

The Only Constant in the City is Change

Like many people, one thing I appreciate about New York City is that it is constantly evolving. Just when you think you’ve settled into your groove, or found the perfect lunch spot, something shifts. Construction makes it harder to get to, it closes it’s doors, or someone tells you Thai isn’t fashionable any more and that you need to try the new Lithuanian fusion place two blocks over.

These changes are frequent and most of the time small, and if you are aware of them as they happen, you hardly notice them. You just adjust and move on.

I call this being “in stream” with the changes. It’s really no different than using a modern SaaS product like Salesforce or Jira. Small tweaks to the user experience happen constantly and most of the time it’s not a big deal because you are using the software daily.

What happens though if you go out of stream for an extended period of time? What is that experience like?

A Commute You Love to Hate

I have worked in and around New York City for about fifteen years. Compared to some, that’s barely a blip, but at present that makes up more than half of my career. For all of those fifteen years I commuted into the city by train. On a good day that meant about an hour and a half on the train, round trip, each day. Regardless of my direction of travel there was one constatnt every day, Penn Station, right in the heart of Midtown Manhattan.

I eventually grew to hate the grind of it all, but I always loved the feeling of finally getting topside and feeling like I made it. I’m in New York. People write books and make movies about this town, and I get to work here everyday and at least be part of the backdrop.

Like any major transit hub, Penn Station is always in a state of flux. Repairs to one area of the station cause detours for the commuter foot trafic not unlike those found on the American Interstate Highway system.

A change of vendors in the food court can cause long lines as people queue up to try the newest offering. An outage in the MetroCard system can cause you to replan your whole commute that day. Pockets of bewildered tourists who made the unforgiveable mistake of being part of the morning rush become just another transient obstacle in the stream of consciousness that is Penn Station.

When you are “in stream” these changes hardly affect you. You know how to walk through a throng of people. You know which stairwell to go up on platform 15 to avoid a pile up due to some repair or renovation. You know how far ahead to look to see if there is a queue on the subway so you can shoot up a stairwell and get street side because you know you can jaunt across the street to catch the ACE instead.

Being Out of Stream

In the Spring of 2020, due to the pandemic, I stopped commutting into the city for work and I really hadn’t been back until recently. I had been hearing tales about Penn Station and the significant amount of construction going on there from those few I knew who were starting to head back into the office regularly.

When my wife and I decided to go in to the city and see a show, I thought to myself, “No big deal. I’m a pro commuter. I know the city and how to navigate it pretty efficiently”.

Oh, how wrong I was!

I had failed to account for how out of stream I was in regards to changes at Penn Station. The entire “user interface” was different. None of the restaurants or land marks I was familiar with were there. Everything was white washed and boarded up. Informational signs were at a minimum, etc.

Heading up into the main halls from the train platform was like taking the first steps on some bustling alien planet. I couldn’t determine which direction to go to get to 8th Avenue. I just didn’t have my bearings.

After a few minutes I started to piece things together, but I had to remember what things used to be like in order to do so, and it really made me second guess myself.

The delta between myself, a formerly experienced commuter, and a brand new one had narrowed significantly.

I would argue that this is no different then when major changes in software get flicked on like a light switch.

Imagine the disruption to someone’s “real work” when they come back from their one month vacation in Europe and find out they have been reduced from being the company expert to an utter n00b.

If not answers…

As stated in the opening paragraph, I can’t possibly provide the One True Answer for solving this problem, but I can at least provide some things to think about when deciding what is right for your organization.

Change Tolerance

An important factor is working out how tolerant to change your user-base and market are. This can be a very broad spectrum where you have everything from software developers who live and breath constant change, to medical professionals that need to view patient data quickly and without distraction, to tax professionals where the software is only expected to change seasonally.

If you deliver software that targets a market with a high tolerance for change, then you should be focused on how to ensure smooth adoption of those changes. You could integrate something like Pendo or Appcues into your software and go-to-market process. Basically, make it easy to stay “in stream” with the changes you are releasing.

If you are working on the other side of the spectrum, where loss of expertise could cause harm (e.g. Doctor can no longer find the button to view an x-ray), then you should accrue changes and release on a regular schedule with a lot of lead time and advanced preview of the functionality.

When you sell to a market with high seasonality (education, finance, etc) then you will want to try and release big features in the off-season and make lower impact/incremental changes during higher use periods of time. Not only does this cause less stress for your users, but you are giving them time to adopt features that may help them make that busy season a little better each year. That is the type of thing that makes your product stickier than the competition.

Give users a way back

Regardless of how tolerant to change your users are, they may not want to adopt all of your changes right away. Make it easy for them to control their own experience.

In a change tolerant environment, default to the new experience, but give them an easy way to click back to the old experience. Let them persist that decision for some period of time before deprecation happens. Everyone has used software that does this whether it is GMail, Salesforce, or Jira.

If your users won’t tolerate change, then just invert the above and default to the older experience and provide a call out in your application that will let them try out the new experience and opt into it if they so choose.

Very few companies get in trouble for allowing their users to set the pace of feature adoption.

Keep an eye on your go-to-market

Simply put, as much as developers would like to believe otherwise, a feature isn’t ready for adoption the second testing passes and you can deploy it to production.

You want to think about releasing changes holistically and make sure the scope includes things like knowledge base articles (new or updated) and any notifications to end-users that may be necessary in your market. Make sure support is up to speed on new features, update sales demos, etc.

Measure adoption

Similar to a tree falling in the woods, if a developer ships a feature, and a user never interacts with it, does it really exist?

Make sure that you are instrumenting your application to measure adoption of new features. How many users have enabled the feature flag, opted into the new experience, tried the new experience, gone through the guided tour, etc?

Being able to measure adoption can provide you with a lot of signal about where your typical users fall on the Change Tolerance spectrum, or even if the features you are working so hard to ship are providing the kinds of value you expect. If you can’t measure it though, you will never know what you are providing more of, value or harm.

Measuring adoption also allows you to know when to deprecate older features and user experiences. If it is a net new feature, and adoption never grows enough, kill the feature. Killing new features is something no one wants to do, but every line of code you deliver comes with recurring maintencance costs. Continuing to pay those costs when a feature is under utilized is just poor business.

Don’t neglect testing!

Making features opt-in/opt-out means that you are increasing the number of code paths in your software. You need to make sure that you are testing the relevant combinations to drive out as many issues as possible.

You should also set a budget and make sure everyone in Product and Engineering keeps it in mind. You can’t test all the combinations forever, so you need to track how much impact this kind of testing is having on your productivity and use this to help set the deprecation schedule for features you may be replacing with new experiences.

You should still ship frequently!

Some may be thinking at this point that I’m “giving a pass” that would allow teams catering to a change averse market/user-base to shrug off two decades of industry knowledge saying that you should ship your software frequently.

I’m not saying that at all. I’m simply saying that you should structure your path to production so that you can ship as frequently as possible, but be conscious of the impact of doing so has on your users and mitigate that impact.

Teams should ship to production as soon as changes are ready, but they should leverage things like feature flags to hide those changes from users until it’s actually in the users best interest to show them.

There is a lot of power in being able to ship changes frequently because it also means that you can fix defects, address scaling issues, and patch security vulnerabilities in the features that are currently available. It’s not just about shipping new features more quickly, and so you don’t want to lose that benefit.

In Summary

Sometimes even if you are delivering the right things to your users, you can be contributing to their stress levels if you are doing it more frequently than they can bear or the market requires.

You have to ensure you are spending your change capital wisely. Leverage feature flags, guided tours, or even old school release notes and knowledge base articles to help reduce the amount of stress users have when learning new features. Make everything as opt-in or opt-out friendly as possible.

Using appropriate technologies and techniques, it is possible to simultaneously ship frequently and not disrupt your users when doing so.

There is no one-size fits all answer here; it’s about understanding that you need to strike a balance and then finding the right one for your users, product, and market.