I admit that I have never watched the show Black Mirror, but after listening to a recent episode of The Product Experience podcast, The Black Mirror Test, featuring guest Roisi Proven, I may find some time for it.
Why was a product management podcast talking about Black Mirror? Great question. Specifically, Roisi was discussing what she calls the Black Mirror Test. A brainstorming exercise that she thinks every product team should do.
What is the Black Mirror Test?
Here is the gist. Take any of your products and try to imagine what is the worst, most horrible thing that an unethical or malicious person could do with them. Work through the details of the steps it would take for this to occur. Then you have found your True South.
The Black Mirror Test is like Ethical Hacking but doesn’t necessarily require a coder to reach your product dystopia.
Roisi uses some great examples of products with potential issues to illustrate:
- Superhuman, the invite-only paid email simplification service, got a lot of negative press when it was discovered they automatically track location along with ‘read’ notifications. The fear is stalkers and other bad actors can figure out where recipients currently are through this mechanism. If anyone on the product team had been the victim of stalking in the past, this would have been an obvious problem when the feature was conceived. It didn’t matter many other popular email tools also use pixel tracking. The CEO eventually decided to keep but disable the feature by default.
- Badi helps people find potential roommates through intelligent personal profile matching. Sounds great if you are looking for a roommate but it can also be maliciously used to promote racism and other profiling. She highlights this is not a theoretical risk buy pointing out that in the US Airbnb has been shown to foster similar, unintended, profiling.
I also started thinking a bit about noteworthy incidents that may have been predicted through this exercise. Three immediate issues I thought about were the leveraging of social media to manipulate US elections, Strava heatmaps exposing military bases, and Fortnite V-bucks used for money laundering.
Facebook Social Accounts
Facebook and other social media companies have built tremendous networks of people to help them stay virtually connected based on family, work relationships, friendships, and special interests. Facebook has long battled fake accounts that distort reality by amplifying good and bad interests. This damages social credibility of individuals and the platform itself that has a long term effect on their ability to generate ad revenue.
This was all just noise to many until the 2016 US Presidential election, exposed a massive coordinated attack by Russia to influence and disrupt the political process. This event, its scope, or significant real-world implications showcases the massive harm bad actors can do on social media.
Strava Heatmaps
Early 2018, Strava faced a very public problem with its heatmapping feature. This feature is designed to let athletes quickly identify locations for popular biking, running, and other activities. As a runner and frequent traveler, I have used these heatmaps to quickly find places to run in foreign locations. The problem with this feature is that military personnel were tracking themselves on secretive military bases around the world and making heatmaps publicly available in otherwise dark locations like Afghanistan.
Fortnite V-Bucks
Fortnite is the wildly popular online game my son along and 250 million other people play, including celebrities and professional gamers. V-Bucks is their virtual in-game currency used to buy special outfits, weapons, and other items in the game. You can earn V-Bucks by playing or buy V-Bucks with US$. People that earn a lot of V-Bucks will sometimes resell those at a discount on third party sites. Virtual currency is a great feature for many video games to use to simplify rewarding gameplay and minimizing friction for players to buy add-ons to enhance their playing experience.
The problem, it turns out, is that V-Bucks are also a fantastic way to launder money that are the proceeds of credit card fraud and other real-world criminal activity. Cybersecurity researchers found hundreds of thousands of dollars laundered in just a 60 day period. It is unclear what Fortnite is doing to curtail this practice now but security experts have suggested many approaches that could help.
What ties these applications together is that while theoretically possible to predict these abuses of their applications, not enough was done to prevent or prepare for it to occur. The Black Mirror Test is a tool that product teams can use to identify potential risks and more effectively manage them.
By all accounts, this type of product analysis is new. As such, there are not standard, best practices that have been created for how to run a Black Mirror Test. Roisi suggests a format for workshops she is planning. Here are a few of those with my added modifications.
- Format should be a brainstorming workshop with cross-functional team members. I suggest you include legal, HR, and others not typically involved in product team activity to gain broader perspectives.
- Use and outside facilitator that can help shift your internally oriented mindset and keep the discussion moving toward something actionable.
- Start by brainstorming potential negative events and issues. The worst of these become your True South events.
- Take each concern and work through the things that would need to happen for it to occur. Roisi says that often, the steps are a logical progression but then there is gap which is the unknown final trigger for things to go wrong.
Once you have discovered your True South, you can consider all future development against that. Simply ask yourself:
Does this feature move us closer to our True South?
If no, then proceed. If yes, modify your plans or at a minimum make a decision to proceed armed with the understanding of the risk you are undertaking.
Conclusion
True South events are hard to imagine until they occur. However, if we as product teams make it a practice to attempt to find potential risks early, we will put ourselves in the best possible position to mitigate them. The Black Mirror Test is a great idea worth exploring further to help all of us create the best possible products.
I suggest you take a listen to the excellent podcast from the folks at Mind The Product to hear about this directly in Roisi’s words.
I am going to explore this more on my own and look forward to future discussions on this topic. Good luck finding and avoiding your True South!