Twitch wants the streamers who spend hours just chatting and gaming on the platform to stay well-informed about the features it’s in the process of test-driving.
To that end, the company is launching a new homepage for the experimental features that it tests out, aiming to make that experience more transparent and to give streamers a head’s up about what’s in the pipeline. The new experiments page will list the experiments running on Twitch on a monthly basis and these tests will drop off of the list once they run their course or become integrated into the core experience.
“Sometimes I think we have been guilty of providing less clarity on what we’re doing because we’re doing a lot of things when we experiment and it can be kind of hard to always keep the community abreast of it,” Twitch Chief Product Officer Tom Verrilli told TechCrunch.
“But I think they very rightly expect more of us, and so what we’ve been trying to do with ‘here’s the full roadmap, and here’s the monthly page,’ is just kind of put all that out there and share with them what we’re doing and why.”
Many social platforms test out new features and other product tweaks with random swaths of their users. When platforms aren’t explicit about what’s in the works, users often still spot new features and publicize them, but generally these kind of tests come and go quietly.
Occasionally a social platform’s test trials a major change or is particularly disliked and drums up some attention, giving companies insight into what users think in the process. The most infamous recent example of this is Instagram’s full screen feed test last year, which was rolled back with a full apology from Adam Mosseri after Kim Kardashian rallied users against the new design.
Twitch has a long history of running tests like these and many make it into the core product, though some aren’t meant to. A month-long Twitch test for pinned chat, which let channels highlight announcements and other key messages, turned into a wide feature release in November, two months after the experiment ran. Another test last year explored a channel surfing feature called the channel switcher that let viewers easily and quickly hop between streams without hitting a ton of pre-roll ads.
Verrilli explained that what might be less obvious to users is that Twitch runs three different types of experiment, each with different goals. What Twitch calls “learn-not-launch” experiments are designed to test a particular mechanic and see if users like how it works. Those experiments are never intended to make it into the final product — a fact that can be confusing for users who understandably assume that any test feature could get a full release.
Another category of experiment is what Verrilli calls “genuine tests” where the company is confident in the problem that needs to be solved but doesn’t know exactly what the solution should look like. “So you would roll something out to 20% people and see if this solves the problem, and if it doesn’t, you develop an alternative solution to the same problem,” Verrilli said.
The third category is a more traditional phased rollout, where Twitch understands a problem and is confident in a solution but still needs to move slowly so it doesn’t disrupt streamers’ earnings or their audiences with too much change too quickly.
“So because all of those things look the same to streamers and end users, having a place where we can provide clarity around when we’re just testing something versus when we’re rolling something out, tends to be really important,” Verrilli said.
“Otherwise, people see an experiment somewhere, they screenshot it, they throw it on Twitter, and then all of a sudden it becomes truth — and as far as I can tell, nothing on Twitter ever is.”
No more pogoing.
Introducing a new experiment – the Channel Switcher. pic.twitter.com/WrB33TH6Mq
— Twitch (@Twitch) June 30, 2022
Some companies (looking at you Meta) aren’t very transparent with this process and just run a bunch of tests behind the scenes, but Twitch is already pretty communicative about what it’s playing around with, tweeting out tests in progress. Now, that information will be collected in one place, which is a useful change for streamers who rely on the platform and are invested in even small changes that might affect how their communities work.
In a new explainer on experiments, Twitch notes that it doesn’t always tell users that they’re part of a new feature’s test group, but when it’s able to they may receive an email or notification or see a small lab flask icon on the platform. Twitch also explains that occasionally active experiments won’t show up on the new page at all because informing the community of what’s being tested might confound the results.
On the new experiments page, Twitch lists its February and March tests, which are heavy on analytics and giving streamers insight into how to optimize to increase their views. The analytics-focused experiments include tag impression analytics (useful insight into which tags help discovery), a new “research” tab that will help streamers figure out when to schedule their streams and a new analytics tool that shows streamers how their channels are discovered from viewers already browsing within Twitch.
Beyond analytics, Twitch is also running a five-week test for viewer milestone cues, which will recognize viewers who opt in when they hit engagement milestones, like watching a certain number of streams in a row. And on mobile, Twitch is running a test for a new chat mode that features transparent bubbles overlaid over the stream.
If a social media platform like Instagram can get away with pissing off its entire user base here and there, a creator-centric platform like Twitch needs to be more careful. Twitch is the still the gold standard for long-form livestreaming, but YouTube Gaming is waiting in the wings, even if it hasn’t yet broken through. And while just about everyone posts content on a social app like Instagram, Twitch revolves much more around the big name streamers that draw viewers back week after week. Keeping streamers happy and well-informed is mission critical for the platform’s success — a fact that drives Twitch to stay in constant communication with the communities at its core.
