The Braves loaded the bases in the top of the ninth on a single by Freddie Freeman, his third hit of the game, a two-out double by pinch-hitter Ryan Doumit and an intentional walk to Evan Gattis.
Cardinals closer Trevor Rosenthal, who was looking for his fourth save in as many games, ran the count full to Jordan Schafer, who was hitting .111 at the time. Schafer fouled off a couple of pitches and then took a borderline pitch, which was called ball four to bring in the tying run.
Carlos Martinez was brought in to face Ramiro Pena. He threw a wild pitch, which scored Doumit. Craig Kimbrel pitched a scoreless ninth inning and the Braves averted the sweep before another sellout crowd of 44,278 at Busch Stadium.
Cardinals manager Mike Matheny used Rosenthal for the fourth day in a row.
"It came down to one pitch right there which maybe could have been called [a strike]," Matheny said. "We were one pitch away. He's a tough kid and he wanted the ball today. As soon as he got to two outs, it's his game. Today it just did not work out."
Rosenthal felt the full-count pitch to Shaefer could have been called a game-ending strike. But home-plate umpire Eric Cooper called it a ball, and what would have been a season-high fifth consecutive Cardinals victory turned into a loss.
"I thought it was a good pitch," Rosenthal said. "It was one of those pitches that could have gone either way. I've gotten that pitch for a strike before and it's been called a ball before. It was right there."
Rosenthal said he felt comfortable going out for a fourth straight day. He had retired the side in order, with two strikeouts in each inning, in the prior two games of the series.
"Physically I felt good and mentally I was ready to go," Rosenthal said. "No one feels worse than me, walking in the run that eventually loses the game. But you have to bounce back. There will be another opportunity.. You just have to learn from it."
Freeman, who had a homer, two singles and three RBIs, said Rosenthal may not have been as sharp as he was in the previous two games.
"He can be a little tired," Freeman said. "He left a couple of pitches out. Obviously, Doumit ambushes him as a pinch-hitter, that's cool to see. He left some pitches over and we were able to jump on his mistakes today."
The Cardinals got a two-out bases-clearing double from Kolten Wong in the second inning and added a run on a sacrifice fly by Matt Adams in the fifth to take a 4-2 lead.
Atlanta had solo home runs from Freeman in the first and Justin Upton in the fourth against Jaime Garcia, who was making his first start in a year and a day since he suffered a left shoulder strain and eventually had surgery to repair labrum and rotator cuff in late May last year.
Garcia allowed only one baserunner besides the solo homers, on a hit batsman, through five innings. He gave up two runs in the sixth on a single by Jason Heyward, a double by Upton and a two-run single by Freeman. But he recovered, making a nice defensive play to start an inning-ending double play in the sixth and then pitched a perfect seventh.
He was in position for the victory by virtue of a misplay by Schafer in center field in the bottom of the seventh. Shaefer lost a two-out routine fly ball off the bat of Jhonny Peralta in the sun. Yadier Molina came home all the way from first base to give the Cardinals a 5-4 lead.
The gift run put Garcia in position to get that long-awaited victory.
"Overall, I thought I was all right," Garcia said. "When you're facing Major League hitters and you make mistakes, you pay the consequences. Today that was the case. I made a few mistakes and they were able to put good swings on them. I felt good physically and was excited to go back out there and give us a chance to win."
Garcia threw an efficient 83 pitches and did not walk a batter in his seven full innings.
"He threw the ball well, very well," Matheny said of Garcia. "The solo homers were some hard hit balls, but for the most part he kept the ball down, stayed out of the middle of the plate and got ahead on the count. He made some very good pitches. The way he threw today, he is going to be a nice addition to what we've already got."
As tough as the loss was for the Cardinals, it was just as welcome a win for the Braves, who averted the sweep.
"We grinded it out today," Braves manager Fredi Gonzales said. "We were at the end of this road trip and everybody is tired of losing and we go out and play a grind-out battle royale there at the end. We just kept getting good at-bats, and it's nice to win this one. Maybe this will get us going in the right direction."