Nick Hundley, making his Orioles debut, singled home J.J. Hardy from second base with two outs in the 10th inning to lift Baltimore to a 7-6 win at Miller Park.
"It was a ballgame [where] nothing went right," Brewers manager Ron Roenicke said.
The Brewers retired the first two Oriole batters in both the ninth and 10th innings, but then allowed a pair of runs in the ninth that sent the game into extra innings and another run in the 10th to lose for the seventh time in 10 games.
"It was a tough day all around," Roenicke said. "The bullpen didn't throw as well as they had been, which is going to happen. But we had plenty of chances offensively to get guys in."
It was a frustrating loss for the Brewers, but starter Kyle Lohse said the team would recover quickly.
"It was tough, but this team will bounce back," Lohse said. "We'll forget about it. We'll do what we've got to do. It's always tough when you have the lead like that, but you don't go perfect for the whole year holding the lead. Those things happen."
Milwaukee went 3-for-15 with runners in scoring position but still led, 6-4, with two outs and no one on base in the ninth, thanks to seven extra-base hits and by plating three runs on walks in the first four innings.
The wheels fell off for the Brewers beginning in the bottom of the eighth, when they loaded the bases with one out but failed to score. Pinch-hitter Rickie Weeks struck out, but Roenicke insisted after the game he had been hit by a pitch during the at-bat. Jean Segura then grounded out to end the inning.
Francisco Rodriguez, who converted 17 of his first 18 save opportunities but has been less sharp lately, came in to close the game and retired the first two batters. He was facing Jonathan Schoop, the Orioles No. 8 hitter who came into the game batting .215.
Schoop had homered in the seventh off Lohse, then hammered the first pitch from Rodriguez for another homer to bring the Orioles within 6-5.
Delmon Young pinch-hit and singled before being replaced by speedy pinch-runner David Lough. Nick Markakis then doubled to center field and Lough scored the tying run with ease.
In just a couple of minutes, the Orioles had pulled into a tie off Rodriguez. In his last seven outings, the Brewers closer has allowed seven earned runs in seven innings.
In the bottom half, the Brewers had a great chance to win the game. They had runners on second and third with one out against Darren O'Day (1-1) but again failed to score with both the Orioles infield and outfield playing in.
Mark Reynolds hit a soft liner to shortstop J.J. Hardy, who easily doubled pinch-runner Elian Herrera off third to end the threat. Herrera broke home when the ball hit Reynolds' bat just as he was instructed to do.
"If there are runners at second and third, we're always going on contact," Roenicke said. "The one thing that's going to hurt you is a line drive and that's what happened. That's not [Herrera's] fault at all. He was doing what he was told to do."
In the 10th, Rob Wooten (1-2) retired the first two Orioles before allowing a double to Hardy that hit off the glove of left-fielder Khris Davis. Hundley then hit a sharp single to left and Hardy scored without a play to give Baltimore a 7-6 lead.
Zach Britton pitched the bottom of the 10th for his third save.
Davis and Lyle Overbay hit back-to-back homers in the sixth for Milwaukee. It was Davis' third home run in his past seven games. He is hitting .407 (11-for-27) with five RBIs in those games after struggling to open the season.
"I've been working on some things," Davis said. "I had a discussion with skipper and he reminded me of some things: That I belong here and I should be confident. Hearing that is a big boost for me."
Orioles starter Chris Tillman allowed six runs on seven hits and four walks while striking out seven over 5 2/3 innings. Three of the four walks by Tillman scored and the Brewers had six extra-base hits against the right-hander.
Tillman said he needs to work on his consistency.
"When you are consistent in your delivery, on the mound you are able to get on a roll," he said. "And I'm not able to carry it over. Just comes down to my mechanics, my delivery. It's close. But you got to be able to execute."
Lohse allowed four runs on nine hits in 6 2/3 innings, but didn't walk a batter and struck out five.