The victory was the fourth in a row for the Cardinals, which ties a season high.
Wong was a catalyst as he had the two bunt singles, a stolen base, two runs scored and an aggressive baserunning play, which resulted in the Cardinals' first run.
"I want people to know I can do things with my feet," Wong said. "That's what I'm here for."
Atlanta starter Aaron Harang breezed through the first three innings before Wong opened the fourth with his first bunt single.
"From Day 1 of baseball, I know the bunt was a big part of my game," Wong said. "Whether it's a base hit or getting the guy over, I know that's my job. Sometimes it's hard to find times to bunt, but I will sneak one in here once in a while."
Wong would advance to third on a hit by Matt Holliday and he eventually scored on Yadier Molina's fly ball, which was caught in short right field by second baseman Tyler Pastornicky.
"It was a tough play to read," Wong said of Molina's fly ball. "If the right fielder had called him off, I don't know if I would have went."
That scored the first run of the game for the Cardinals and tied the game after Atlanta had scored an unearned run in the second inning.
Wong and his speed was in the middle of a two-run sixth inning which gave the Cardinals the lead for good. Matt Carpenter was hit by a pitch to open the inning and Wong followed with another bunt single between the pitcher's mound and first base.
Carpenter moved to third on a fly ball by Holliday and Wong promptly stole second base, his fifth steal of the season without being caught and his second in two games since being called up from Triple-A Memphis.
Evan Gattis' throw went into center field as Carpenter scored and Wong went to third. Wong did not stay there long as he scored on an RBI single by Allen Craig.
Wong has four hits in his two games since being recalled from Triple-A. He was optioned April 27 and hit .344 in 15 games with Memphis.
"I realized that I wasn't doing too much with my feet," Wong said. "I wanted to make sure that speed was the main element I was working on when I came back. You know things are going right when you get two bunts in one game. It felt good to come in and help the team get two wins right away."
Speed was instrumental in an insurance run in the seventh inning.
"Speedster" Matt Adams opened the inning with a triple, his second of the season which ties him for the team lead with Bourjos, and eventually scored on a safety squeeze, which resulted in the second of two Bourjos bunt hits.
It was an interesting offensive attack with the four bunt singles, three other singles and the triple by Adams.
"Today was one of the best representations of what speed can do for us," Cardinal manager Mike Matheny said. "The guys got on base and created some havoc. There are days when you are facing a tough pitcher like Harang where you have to figure out a way to manufacture runs, and that is what we did today. It was fun to watch the guys use their gifts."
The aggressive play was also shown in the field. A key play in the game, which led to the ejection of Atlanta manager Fredi Gonzalez, came in the fifth inning when Molina bounced out from behind the plate to field a bunt by Harang which was ruled fair. Molina threw to third to retire Andrelton Simmons, who had led off the fifth with a double. Matt Carpenter then threw to first to complete the double play.
"It is great seeing the aggressiveness whether it was Yadi coming out of the box on the bunt or on the bases," Matheny said. "I love that style of play. We want to keep the throttle down."
The beneficiary of the aggressive play on offense and defense was Shelby Miller, who has won his last six decisions. Miller went a strong seven innings, his longest outing of the season. He allowed just five hits and the one unearned run while striking out seven.
"Shelby had a great game plan going in and did a great job of executing today," Matheny said. "He's got to stick with his bread and butter which is the deception in the four-seam fastball. Today he added the best breaking ball we have seen from him."
Miller threw 101 pitches, 72 of them for strikes.
"I did a good job of keeping the hitters off-balance and getting ahead on the count," Miller said. "My fastball command was a lot better today. I tried to be as efficient as possible. Yadi did a good job behind the plate and the defense did a good job. There is still room for improvement, but I felt comfortable and it was an all-around good day."
Carlos Martinez came in to start the eighth inning. He gave up a walk and a single to Jason Heyward, but struck out Justin Upton (Justin and B.J. Upton combined for six strikeouts in the game). Randy Choate then induced a one-pitch, inning-ending double play off the bat of Freddie Freeman. Trevor Rosenthal earned his 13th save of the season, retiring the side in order in the ninth for his third save in the last three games.
Rosenthal put an exclamation point on the win by striking out Chris Johnson and B.J. Upton on six pitches to finish the game. It was Upton's fourth strikeout in four plate appearances.