At Microsoft, a small, tight-knit team works long hours – often sorting through downright offensive content – to help ensure that the company’s booming Xbox LIVE service is safe, non-offensive and fun for all.
REDMOND, Wash. – Aug. 2, 2011 – In an otherwise typical Microsoft hallway, a black curtain stretches across the doorway to a large room. The whiteboard next to it offers this ambiguous, if not curiosity-inducing, explanation: “Please do not disturb. Sensitive material behind curtain.”
Behind the black curtain is a unique team of Microsoft employees. Their existence is not widely known, and probably for good reason – if you have a close encounter with a member of Xbox LIVE’s Policy and Enforcement team, chances are you’re on the wrong end of right.
Hackers, cheaters, phishers, account thieves, game code modifiers, communication abuser – they help police it all, including actual crimes in some rare instances. The team is there to help make sure Xbox LIVE is safe, non-offensive and fun for all users.
“If you’re playing a game on Xbox LIVE, and somebody snipes you from across the map and you drop the F-bomb, we’re not going to ban you – not for the occasional slip. We focus on the really bad stuff,” says Boris Erickson, Xbox LIVE Enforcement Unicorn Ninja. Yes, that is his actual job title.
Adds Erickson: “We are not here to be the arbiters of all speech. But there are certainly some kinds of communication on Xbox LIVE that crosses a line – racism, homophobia, sexism, offensive comments about nationalities, and more.”
Day in and day out, the inboxes of Erickson and his fellow enforcers are piled high with stacks of complaints about offensive behavior, speech, and materials. They dutifully sort through it all and decide what’s next. That could be requiring a user to remove an offensive word or phrase from their profile to – in the more egregious cases – outright banning users.
“Or, as we like to say, inviting them to not be our customer,” Erickson says. “These are paid subscriptions we’re taking away, so we want to make sure we’re doing exactly the right thing.”
All Xbox LIVE users agree to a code of conduct when subscribing to the entertainment service. But, as Xbox LIVE tops 35 million users – and, as it incorporates an ever-widening range of entertainment, gaming, and communication features – it’s a given that there will be opportunists and rule-breakers, Erickson says.
But the team’s director, Stephen Toulouse (known widely by his Microsoft e-mail alias, Stepto), says despite Xbox LIVE’s explosive growth over the last several years, the number of complaints his team handles has remained tiny in proportion to the growing number of people who use the service.
“Looking at the stats, the cross-section of bad apples we deal with every day is small – typically less than one percent of the overall population,” Toulouse says. “The user complaint volume has tended to stay relatively flat compared to the line of new users. What that says to me is that our efforts are having an impact, and also that we’re broadening our audience. We’re bringing in different people that want to experience different things on Xbox LIVE, not just gaming, and at the end of the day that’s going to improve everything.”
The Explosion of Xbox LIVE
When Toulouse joined Xbox LIVE in 2007, the entertainment service had not yet reached one million users online at the same time.
“Enforcement was literally done by one guy with a spreadsheet who would go through the complaints once a week,” Toulouse says.
Though it took years to hit the one million user mark, it took one year to hit two million concurrent users.
“We knew Xbox LIVE was going to explode,” Toulouse says. “We knew we were on the cusp of something huge, especially when we saw how many people came into the service with the launch of Halo 3.”
The folks at Xbox LIVE, including Toulouse, wanted to stay ahead of the game. He slowly started assembling a team, and they started designing a tool to help the team effectively police the growing community of users. The result was a software program called Vulcan to help enforcers handle and escalate complaints.
“It was designed on cocktail napkins, then coded and designed to allow people who do complaint investigations to do so in an efficient and accurate way,” Erickson says.
Enforcers are now using a brand-new version of this tool, called “Vulcan 2,” which makes sorting through complaints even faster. In fact, because all enforcers are experienced gamers, they also often use an Xbox controller to navigate their work.
Say one gamer is offering to sell cheating services, or another user in a multiplayer online game is spouting racial epithets into his or her microphone, or yet another registered an offensive gamertag. Enforcement agents will find out about it either via a complaint sent by another Xbox user or by experiencing it firsthand.
“The enforcement agents also play games,” Erickson says. “Part of what we pay them for is to be out there in the community, listening for threats, looking for vulnerabilities, and reporting back to us.”
There are a handful of enforcements the team hands out ranging from a 24-hour ban to the most serious – voting an Xbox LIVE user “off the island” for good.
Apart from being gamers, agents are “steeped and stewed” in Internet culture, as well as being experts in slang, acronyms, and more. Erickson says some of them can actually write in “l33t,” (pronounced “leet”) a hacker pidgin language that incorporates abbreviations and numbers in an attempt to bypass profanity filters.
“We always appreciate having a diversity of knowledge,” Erickson says of the team. “Everybody kind of brings their own little history to the table, and can interpret content in the way the rest of us can’t.”
Toulouse says such diversity is key, though every member of the team shares a common goal.
“They are absolutely passionate about safety on Xbox LIVE,” he says. “I personally believe that when you buy your Xbox LIVE subscription, you are getting us ‘free in the box.’ Microsoft has invested in us, and we are invested in trying to make sure the experience is good.”
“We Have a Great Community”
Along with Toulouse and Erickson, Jason Coon and Andreas Holbrook round out the management team, which includes numerous enforcement agents who work one of three shifts during the day to maximize coverage (together, they cover 18 hours a day, seven days a week, 365 days a year). Coon manages the agents, and Holbrook works with outside companies and law enforcement agencies on deeper investigations.
The team is tight-knit, primarily because of the kind of content and situations they deal with each day.
“There’s a sort of gallows mentality, because we do have to deal with some pretty bad stuff during the course of our day,” Erickson says. “We talk openly and frankly about it and the effect it has on all of us. You can’t help but need to talk after being exposed to the worst of the worst day in and day out.”
Sometimes this includes interacting with courts, law enforcement agencies, and other agencies. In one recent case, that included the National Center for Missing and Exploited Children (MCMEC).
“That, of course, pushes a lot of buttons,” Erickson says. “We make sure people get the space and time they need after something particularly bad – to have a talk, or go outside and take a walk.”
The team also has some go-to “palette cleansers” to brighten things up: LOLcats, those cute and funny pictures of animals adorned with irreverent and witty (and almost exclusively grammatically incorrect) captions.
“Sites like ‘I Can Has Cheezburger’ and ‘The Daily Squee’ are frequently called upon around here,” Erickson says. “They do a lot to help. You sort of need that disconnection from the offensive content sometimes.”
As Xbox LIVE continues to expand, so too does the enforcement team and the effectiveness of their tool Vulcan. But Erickson doubts there will ever be a time that enforcement is totally automated.
“Most of the decisions need human eyes to keep it real, though we are moving into a realm where we’re applying more automation to the process,” Erickson says.
And what of this team whose sole mission it is to deal with “the worst of the worst” – what has it done for their views on humanity?
“I’ve learned that the vast majority of people on our service are out there having fun. We have a great community,” Toulouse says. “To the extent that we do see bad behavior, it’s often tied to the belief that they’re anonymous, they won’t get caught, and we’re not looking. The vast majority of people are out there are trying to be excellent to each other.”
Despite having seen the worst of people, Erickson, too, is still optimistic about people.
“The reality of working in the wild, wild west of the internet is that most people just want to be creative, and to use our products in social ways and to connect to people. And for the ones that don’t, well, that just requires a bit of tweaking. We’re slowly crumbling the nexuses of bad behavior.”
Learn more about, or subscribe to, Xbox LIVE.
Read the Xbox LIVE code of conduct.