Just how good is Mike Trout, really? Who are the five best starting pitchers in baseball? If you had to put a number on Aaron Judge's power, between 1 and 99, what would you come up with? It's the sort of stuff that late-night dorm-room debates are made of -- but for Ramone Russell, it's just part of his 9-to-5.
Russell is a game developer and online community manager at PlayStation's San Diego Studio, and each year he and his team are responsible for completing one of baseball's most sacred tasks: rating every player that appears in MLB the Show, from future Hall of Famers to the 26th man. Providing a ruthlessly honest assessment of some of the best athletes on Earth is, well, not easy, so Russell was kind enough to give us a peek behind the curtain at how it all works -- just in time for Friday night's MLB The Show Players Tournament, in which a bracket of four big leaguers battles it out to determine who's the best gamer in baseball. (It will stream live at 8 p.m. ET on YouTube, Twitch, Facebook and MLB.com, not to mention the MLB and MLB the Show Twitter accounts.)
First thing's first: Before you pass along your litany of complaints, please know that you're not just arguing against a fellow sports fan, or even just people who do this for a living. You're arguing against math.
Take Max Scherzer, for example. His velocity rating (81) and particular repertoire (four-seam fastball, slider, changeup, curveball, cutter)? That comes straight from Statcast. Home run rate, walk rate, strikeout rate -- it's all based directly on advanced metrics, so that there's very little room for guesswork. And the process repeats for hitters: The San Diego team takes a three-year average -- weighted evenly, 33 percent/33 percent/33 percent -- to ensure they have a proper sample size, then lets everything from speed to arm strength to power to plate discipline be determined by what the numbers tell them, and how far above average in that skill a given player is.
"Very rarely does the human element get involved into the player ratings that we're making," Russell said. "If you go to Baseball Savant, they break down everything, so we're all very cognizant of the information that we have now."
There are exceptions, of course, areas where a value judgment is inevitable -- both pitchers and hitters are given a clutch rating, which offers them a slight boost when there are runners in scoring position. Not even baseball's biggest statheads are immune from stuff like this:
But for the most part, Russell says, "we're rarely scratching our heads, trying to figure out how do we take X and translate it into Y."
That begs the question, though: If their process is so precise, so thoroughly rooted in data, how do they handle someone like, say, Vlad Guerrero Jr. -- a generational prospect for whom Major League data is scant?
Enter the ratings czar. His real name is Luis Martinez, a member of San Diego's live content team, but he's described in almost mythical tones: office full of old Baseball Prospectus annuals, day after day spent poring over scouting reports and advanced metrics, an encyclopedic recall for seemingly every baseball number under the sun. He is, in short, the man behind the man -- every piece of every player's rating runs through him, and in the case of someone like Guerrero Jr. or Fernando Tatís Jr., the team relies on his expertise a bit more heavily to project that player's skills to the Majors. Of course, generational hype or not, they remain pretty tough critics.
"We always would rather undershoot than overshoot," Russell said. "I think Aaron Judge's rating was in the 50s or 60s when he came up. We never want to overrate anybody because if we overrate, we have to back down. But if we underrate it, we can always improve upon them."
In all of Russell's years working on the game, few people have dared to even try to lobby Martinez to make some sort of change -- and none of them have emerged from that argument victorious.
"No, that's never happened," Russell said.
"Never, never, never happened. It's kind of like a person who doesn't cook trying to tell the 20-year, three Michelin star chef how to make a dish. If you walk into his office, you better have 'War and Peace' with you, otherwise he's going to talk you under the table because he's been looking at these numbers every day, all day, all year round."
Of course, that doesn't stop big leaguers from ... let's just say "making their voices heard."
"Oh, yeah," Russell said with a laugh. "Everybody's faster than they think they are."
Now, to be fair: 1) It's only natural to be defensive about your chosen profession, especially when you're an athlete set to be featured in a nationally renowned video game, and 2) MLB the Show's rating system can look a little wonky to the uninitiated. 60 is a league-average rating, which means numbers that may seem mediocre -- say, a 75 -- are actually pretty great. Trout, for example, has a fielding rating of 85, and he can do stuff like this.
Which, really, makes a certain amount of sense. Everything about the way San Diego approaches player ratings -- looking at a three-year average, using 60 as a baseline, refusing to heap praise on unproven prospects -- underscores the fact that baseball is, fundamentally, a game of failure. After all, even the greatest of all-time will fail seven out of ten times -- which is why Russell thinks Trout might be the single most underrated player in this year's edition of MLB the Show.
"When it's all said and done, if he stays healthy, he's probably going to go down as the best position player to ever play the game," Russell said. "So everybody expects him to be 99 across the board, but that's just not how it works, right? Like, he's fast, but he's not super fast. He has a decent arm, but he doesn't have a cannon. He hits for power, he hits for contact, but he also strikes out a little bit too much."
Baseball players, being baseball players, can wrap their heads around that. They've experienced a lifetime of trudging back to the dugout after a strikeout or asking for a fresh ball after a home run, and once the development team explains the rating scale -- and shows them the underlying metrics -- they always come around.
"Luis does not hand out diamonds [the game's designation for the best players at each position] all willy nilly," Russell explained. "You have to really perform at a high level for a sustained amount of time to be able to be very highly rated in our game. When you look at it that way, somebody has the 80 power rating and we go, well, 60 is average. And then they'll go okay, so that makes sense."