ST. PETERSBURG -- The Rays’ offense battled back to score two runs off Yankees closer Aroldis Chapman, but New York was able to respond with five runs off the Tampa Bay bullpen in the top of the 10th inning to take the series opener, 8-4, on Thursday at Tropicana Field.

It’s the second consecutive day that the bullpen has allowed five or more runs in the ninth inning or later. Overall, Rays pitching has allowed 17 runs in the last two games, compared to 17 runs combined in the six games prior to that.

• Box score

“We talked about it last night. The bullpen and that last inning has been a little bit of a challenge here as of late, and it really was no different today,” said Rays manager Kevin Cash. “Oliver [Drake] and Emilio [Pagan] both have done a really good job for us. Tonight, it just didn’t go our way.”

After being held quiet through the first eight innings of the game, the Rays came alive in the ninth against Chapman, who had not allowed a run to Tampa Bay in 19 1/3 career frames.

Chapman started the inning with a four-pitch walk to Nate Lowe, and the Rays were able to make him pay. Joey Wendle delivered with a one-out RBI double to cut the deficit to 3-2. Lowe just barely beat the throw on the close play at the plate, and the call of safe was confirmed after the Yankees challenged. Wendle then came around to score on two wild pitches by Chapman, tying the game at 3.

With Chapman struggling with his command, Guillermo Heredia followed with a two-out single to start the rally back up. Yandy Diaz singled and Tommy Pham drew a walk to load the bases, but Chapman was able to strike out Austin Meadows swinging to end the threat.

In the 10th inning, however, Drake issued back-to-back walks before Brett Gardner reached on a bunt single to load the bases. DJ LeMahieu, who is hitting .480 with runners in scoring position this season, delivered a two-run single to give the Yankees a 5-3 lead.

“Extremely frustrating,” Drake said, when asked about the walks. “When you do that, you don’t really give your team a chance putting two guys on for free. That can come back to hurt you. Unfortunately it did. Our offense did a heck of a job battling back there off Chapman. Really frustrating to do that.”

Cash added: “They capitalize. That’s how it’s felt this season for sure against them. It’s the sign of a really good team. They’ve got a knack for coming up and getting the big hit. When they need one, they get two. When they need two, they get four. It’s a good team.”

Gary Sanchez later capped off the five-run 10th inning with a three-run blast off Pagan that went a projected 461 feet, per Statcast, extending the lead to 8-3.

“Sanchez was just sitting on a slider,” Pagan said. “It wasn’t a terrible pitch. Probably caught too much of the inner half of the plate, but it was a slider down and it had good velocity to it. Just one of those things that he was sitting on it and he’s a really good hitter, obviously.”

Despite the two runs in the ninth inning off Chapman and another run in the 10th, the Rays have scored just eight in the last four games against New York.

Tampa Bay is 2-8 against New York this season and 11-18 against the Yankees dating back to last season. The Rays have been outscored 60-29 by the Yankees, who have a 7 1/2-game lead on Tampa Bay in the American League East race.

The Rays have three more chances against the Yankees before the All-Star break, but the offense needs to show up in order for Tampa Bay to gain ground in the division.

“We’re playing well, but we need to score more runs,” Diaz said. “And that’s what needs to happen.”