But the pair couldn't follow up Alex Cobb's masterpiece, which saw the right-hander flirt with a no-hitter through 7 1/3 innings, and the Rays came away with a loss after jumping out to a 4-0 lead for the second consecutive night.
"These guys have been money all year," said Rays manager Joe Maddon of the bullpen pair. "It just didn't work tonight. I'm not going to stand here and point fingers at them. They've been outstanding. Give the Yankees some credit."
The Yankees rallied from a 4-2 deficit in the ninth, effectively erasing all of Cobb's work. The loss officially eliminated the Rays from the American League East race.
McGee entered the game to pitch the ninth, but he hit leadoff batter Chase Headley in the chin with a 96-mph fastball.
"I'm not going to lie, I was a little unsettled," McGee said. "I rarely hit batters."
After Yankees trainers tended to Headley, Austin Romine entered the game as a pinch-runner and Ichiro Suzuki doubled to left to put runners at second and third with no outs.
McGee struck out Zelous Wheeler before Chris Young stepped to the plate and delivered a walk-off three-run homer.
"He was cheating pretty far on the first pitch," McGee said. "I didn't want to go too far in. I wanted to stay with my strength up and away, like I did with the hitter before. And he just got on top of it and got enough of it."
Young's home run was the first allowed by McGee this season.
"It's amazing," Young said. "Any time you're able to come through in a big situation like that, especially with a couple guys getting hit before that happened, I think you'd be a fool to say that you [didn't] have a little extra adrenaline flowing. Not that I think anything's intentional, but it just naturally gets the juices flowing a little bit. To be able to come through in that situation, it felt really good to do that for the team."
Cobb allowed just one baserunner through six innings, which came via a dropped fly ball by sure-handed center fielder Kevin Kiermaier. He walked two batters in the seventh and survived a 10-pitch at-bat by Young in the sixth to carry a no-hitter into the eighth inning.
"He was really good," Maddon said. "He was really, really good."
Despite Cobb's pitch count, the Rays' manager planned to leave Cobb in the game "until he allowed a hit."
Stephen Drew popped out to first to start the eighth, leaving Cobb five outs from joining Matt Garza, who threw the only no-hitter in Rays history.
"Tonight was just everything clicking for me," Cobb said. "Not much else to say."
But Young ended the drama when he drilled an 0-1 pitch into right-center field for a double.
Cobb noted he was "pounding him in pretty good earlier in the game."
"A few at-bats, changeups and the fastballs in," Cobb said. "And he was putting good swings on them, even the ball he popped up to left field. He put a good swing on it. I jammed him a little bit.
"... Just kind of thought he was just covering one side of the plate only. After the curveball, I thought I could sneak a back-door fastball in there. Obviously, he put a good swing on it."
Cobb had thrown 102 pitches by that point, prompting Maddon to bring in Boxberger. Shortly after Cobb exited to the cheers of the 32,627 in attendance, Boxberger surrendered a two-run homer to pinch-hitter Martin Prado, which cut the Rays' lead to 4-2.
"Boxy had to get two outs, I felt pretty good about that," Maddon said. "And then the way it was set up for Jake [being] 4-2, Headley and then Suzuki, and then a pinch-hitter for Drew, whatever, it was set up well."
Had Cobb's masterpiece come to fruition, it would have been the fifth no-hitter this season, joining the likes of Josh Beckett, Clayton Kershaw, Tim Lincecum and a Phillies combined effort.
Not since 1958, when Hoyt Wilhelm no-hit the Yankees in Baltimore, have the Yankees been no-hit by a pitcher throwing a nine-inning complete game.
Cobb entered the game on a roll, allowing two runs or fewer in his previous 11 starts, which was the longest active streak in the Major Leagues and a franchise record.
Drew's drive to center that Kiermaier dropped was the only ball the Yankees hit anywhere close to hard until Young's double.
While Cobb excelled on the mound, Yunel Escobar took care of all the offense with two home runs and four RBIs.
The Rays mounted a two-out rally in the fourth that started when Wil Myers legged out an infield hit and Matt Joyce followed with a single to center. Escobar then launched a 2-1 Michael Pineda slider into the left-field stands for a 3-0 Rays lead.
Escobar's homer extended the Rays' streak of consecutive games in which they've homered at Yankee Stadium to 13.
Escobar added a solo shot in the seventh to push the Rays' lead to 4-0.