The Reds (15-16) took three of four in the series and pulled within 5 1/2 games of the Brewers (21-11), who came to Cincinnati with baseball's best record and had not lost a road series all season.
"I know it's early, but it's important we don't let [the Brewers] get so far ahead that it becomes an insurmountable climb," Reds manager Bryan Price said. "Eventually it's not going to be 'early in the season.' We need to send a message that we're in this thing."
The Reds delivered that message in the bottom of the 10th on Sunday.
Milwaukee reliever Tyler Thornburg (3-1) walked Chris Heisey on four two-out pitches to set up the decisive scenario.
"I'm not necessarily known for my eagle eye at the plate," said Heisey, who has 65 career walks and 280 strikeouts. "That walk turned out to be a big one."
Frazier got ahead in the count 2-0 before sending a liner skipping to the left-field corner. Milwaukee outfielder Khris Davis played the ball off the wall as Heisey raced around the bases and was waved home by Reds third base coach Steve Smith.
"Once it hit the wall, I was waving [Heisey] around like I was the third base coach," Frazier said.
Smith relayed Frazier's call and Heisey arrived at home plate well ahead of the throw.
"On a ball in the gap or down the line with two out, I was expecting to be sent in that situation," Heisey said.
The Reds continue to right the ship after a 3-8 start and can look forward to reinforcements. All-Star closer Aroldis Chapman is scheduled to return from the disabled list this week.
"Our best ball is ahead of us," said relief pitcher Sam LeCure (1-1), who struck out Milwaukee's Carlos Gomez with runners at second and third in the top of the 10th. "This is a nice springboard, a chance to get on a roll."
Brandon Phillips finally got the Reds on a roll in the eighth inning, tying the score at 3 with a solo home run.
Phillips, who received his 2013 Rawlings Gold Glove Award during a pregame ceremony, crushed the second pitch he saw from Milwaukee reliever Brandon Kintzler 425 feet, the ball bouncing off the grass beyond the center field fence.
"You can tell our offense is coming around a bit," Phillips said. "The balls are just finding holes right now."
Cincinnati's hitters had taken some heat from fans earlier this season when the Reds wasted some fine pitching performances.
"The haters are going to hate, and we'll just keep on laughing at them," Phillips said.
Unlike the hitters, there has been little ebb and flow from Cincinnati's consistently excellent starting pitchers.
Reds starter Alfredo Simon held the Brewers relatively in check Sunday, allowing three runs over seven innings to leave his season ERA at 1.99.
The Brewers scored off Simon via solo homers by Scooter Gennett and Davis. The other run came when Gomez led off the third inning with a double and scored on Jonathan Lucroy's RBI single up the middle.
Kyle Lohse, Milwaukee's veteran right-hander, allowed two runs over 6 1/3 innings. Lohse, 35, was making his 16th career start against the Reds and 23rd career start at Great American Ball Park.
"It was kind of a weird outing for me," Lohse said. "They did a good job of mixing up their approach. They were super aggressive early, so I tried to use that. Then they take -- they made me work. It was just a battle."
Brayan Pena got Cincinnati on the board when he led off the third inning with a home run to right. Phillips doubled to left in the fourth inning and came home when Frazier ended a seven-pitch battle with Lohse by belting a high sacrifice fly to right.
"A near-.500 record isn't what we're all about," Price said. "We're better than that."