What is a release process?
Q&A with our Deputy CTO, Arnaud Sourioux, discussing everything related to our deployment procedures.— 2 April 2021 ago
As promised, we continue to advance forward in this time of transition. Speaking of advances, let’s talk about how we make our technological advances happen with our release process.
Does this introduction seem far-fetched to you? Rest assured, the rest of the article could not be more serious! Today, we have the honor of welcoming Arnaud Sourioux aka Six. (because he has the mental age of a six-year-old? Or because he has six fingers? The mystery remains…). Deputy CTO, he knows everything related to release processes.
When a new update or bug fix appears on your Shadow, we first have an entire process to manage and set up. Our updates typically go through phases of "Insider", "Alpha", and then "Beta" before moving to "Official" and being visible to everyone. Now that you know the basics, let's move on to our question-and-answer session (Q&A), where we will go into more detail on these different versions and give you a lot more explanation on all the rest of the procedure:
Q: How does the release of an update for Shadow work? Has it always been like this?
No, it wasn't always like this. We are changing the way of doing things. 2 years ago, when I arrived at Shadow, we released the updates pretty much ... as soon as they were ready! Then we tried to synchronize the versions between the application, the client, what the user sees, and the services in Shadow. We industrialized a lot and we used tools to help us do that, such as Gitlab.
Then came the JB era, where he decided to only do one production release per month so that each week we could focus on a different aspect of Shadow. For example, week 2 of each month is dedicated to the "client application" part, which is the desktop, box, mobile, and TV version. Week 3 focuses on the components in Shadow: the streamer and all the services that keep Shadow running.
Regarding the release of an update: the first thing to do is to define what features we are going to put in the first version. As for the bugs, we fix them as we go so that all those that have been fixed so far will be included with this version. We decide on our content in advance to be sure to include them.
Q: What are the Insider, Alpha, and Beta versions for?
In the testing phase, we always start by sending our update to Insider. The perk is that we can send them directly the code that a developer has just made, we don't need to make a “real version” for them. Thanks to this, they can access a version just 20 minutes after a developer writes it.
The Alpha is a frozen code (not a test), but we can send it 3 times a day.
For a real Beta version, we have to go through a QA phase. QA has to test everything and tell us if it works, if it is ok for the Beta, or if it is not acceptable enough based on the bugs found.
To go from Beta to Official, it’s the current Beta that is re-tested in more depth. We are much stricter on all the behaviors and bugs found during the testing phases.
These versions are used to ensure that step-by-step, as we move up from Insider to Alpha, to Beta, and to Official, we ensure quality. Since we do not have the same volume of users, the impact is not the same. When we send an update or a patch in Beta, only 10% of users are impacted, and they can choose to switch to Alpha or Official if needed.
Q: How does a new feature go from the Insider version to the official version (concrete steps)?
Each time a new feature is upgraded, its finishes and the user experience must be improved. The requirements are not the same in Alpha as they are in Beta or Official.
For example, today in Alpha we have the arm64 application for Mac. It's in Alpha because there aren't all the features like Quick Menu, Display Management or others that work with it yet. It was able to be available very quickly in Alpha because it’s not representing a problem, but for the Beta version, it requires a minimum of functionalities. And for the Official version, that's another story.
Q: Is there a fix date/day for the releases? Like Friday night because it's Shadow?
There is what we talked about earlier, the weeks of release depending on which component we are going to update. The higher the scope (Official> Beta> Alpha> Insider), the more it will be released at the start of the week.
For the Official version, we try to do that on Monday or Tuesday. For Beta, we allow ourselves until Thursday. And for Alpha, we do it whenever we want. Even if we have very fast feedback, we still need time to fix problems before the upcoming weekend because this is where we have the most connected users, hence our constraints in terms of the days of the week. When we release the update on Monday or Tuesday, it allows us to make fixes before Friday, which allows our users to have a Beta or an Official version that is relatively stable or free from the bugs found in the meantime.
Q: What is the biggest challenge the team faces in launching a new update?
The more we have teams that are involved in developing a feature, the more difficult it is going to be. Because it means that we will have to work more together and synchronize, and it will make things more complicated: we have to agree on how to do it.
Today, there are a lot of features that seem simple but actually aren’t. When we have a feature on the Quick Menu, we need that at the moment when the user clicks on the Quick Menu, it must be sent to the Shadow by the local application. For example, when you change the resolution on the Quick Menu, it sends the information to the renderer which transfers it to the streamer. The streamer applies the new resolution and confirms to the renderer as well as to the Quick Menu that it is correctly applied. This whole flow that seems very simple as a user is ultimately not that simple.
In addition, when we add the virtualization / orchestration part - For example on the dynamic shutdown, it takes even more time because all the components must be able to talk to each other for things to go well.
Q: Why is it so complicated?
What’s complicated is that we start from a product specification that is a need for the user, that we have to transcribe it technically, and make sure that we know how to do everything. Sometimes you don't know how to do it and you have to figure out how to do it. How long it will take. And make sure with everyone that you can do it a certain way. The point is not to pile up things that kind of work on their own but not with the rest, or not with the next feature. The goal is to find lasting solutions.
Why are there all these steps and why can it take a long time? So that it can be as qualitative as possible, because users are more and more demanding. When the product first launched, our members knew they were testing something that got started, in the “I made my product in my garage” vibe. Now, when you buy Shadow it’s like when you buy a subscription to anything else, so you expect a similar quality.
Before, we only had Insider / Beta / Official and we figured we needed the Alpha version. We were asked for the same quality in Beta as in Official, which was not possible without an intermediate step. 15 insiders didn’t allow us to reach 10% of users in Beta right after in a correct way. Even now when we have bugs in Official, it’s because we sent it to Beta and that we didn’t get any feedback (because one or 2 people must have had it in Beta) and when we go to Official, the impact is not the same: the 2 people can turn into 100, 200, 500 or 2000 people on the first night. The consequences can be dramatic.
The worst that can happen to us with a bug we put in Official is that the user no longer has access to Shadow. That's why we try to pay as much attention as possible to the front phases.
When you update to Beta or Official, it is depending on the number of users affected that you decide what to do next. Depending on the bug, we study it, find a solution, check that we can resolve it before sending a patch to everyone who is impacted (and not just to the person in question). If we go back to the previous version, there is no longer the bug but that means that we are still stuck somewhere.
Q: Do you deserve a raise? And what are your last words (for this article of course)?
Our teams are the ones that deserve the most. They're the ones doing the job, we just organize everything that’s around.
I think we've really made a lot of progress in 2 years. Because the team is bigger, because we did things differently, because we have new people with us. At times we are less agile and slower but we have gained in quality. That's why we wanted to show that we were actually still agile and able to do great things by releasing the dual screen on the Box V1 for Christmas (available in Beta). It was as important to our users as it was to some of our staff members!