It was Goldschmidt's single up the middle off Victor Marte that gave the D-backs the lead, but the game wasn't decided until Heath Bell retired Pete Kozma 4 hours and 53 minutes after the first pitch.
"It was an outstanding baseball game," D-backs manager Kirk Gibson said. "We would have like to have won it earlier. They're a good team, but we didn't give in."
Neither did the Cardinals, who battled back from deficits of 4-1 and 6-3 to force extra innings.
The D-backs would not have made it all the way to the 14th inning if not for a yeoman effort by reliever Josh Collmenter.
Collmenter entered the game in the 10th and tossed four shutout innings, throwing 68 pitches to pick up the win.
"It was unbelievable," Goldschmidt said of Collmenter's performance. "It was fun watching it. They had guys on base; there were a lot of big pitches there. Especially when you're on the road, because you're one pitch away from losing the game.
"He did a great job. It was reminiscent of the third game of the year when we played these guys and he kind of did the same thing."
The game to which Goldschmidt referred took place in Phoenix, as the D-backs beat the Cardinals in 16 innings. In that game, Collmenter worked five innings of relief.
"Colly's performance, outstanding for four innings," Gibson said. "He's just got ice in his veins."
The D-backs finally were able to break through in the 14th against Marte.
Cody Ross drew a walk to lead off the 14th, but he was erased when Parra bunted into a fielder's choice. Didi Gregorius then drew a walk to set the stage for Goldschmidt.
Marte jumped ahead 0-2, but then threw a pair of balls in the dirt before Goldschmidt lined a 95-mph fastball back up the middle.
"He made a couple of good pitches and then a couple of split-fingers that luckily I was able to lay off," Goldschmidt said. "And then fastball away, I was able to get the bat on the ball and get it through the infield."
The hit increased Goldschmidt's average with runners in scoring position to .451, and he raised his National League-leading RBI total to 49.
"That's just Goldy. He's been on a roll for us this year and we've kind of ridden him," Gibson said. "They got into a situation where they had to pitch to him."
Said Goldschmidt: "It felt near the end of the year last year, and really this year. [I] just try to almost pretend like nobody's on base and go with the same approach, and it's worked. But there's always adjustments, so we'll see what happens the rest of the year."
The D-backs still had to get three outs in the bottom half of the inning before they could breathe a sigh of relief.
Closer Heath Bell, the team's only reliever left in the bullpen, came on and immediately gave up a double to Yadier Molina to start the 14th. Bell managed to settle in, and he struck out David Freese, got John Jay to ground to short and fanned Kozma to notch his 10th save.
"Yeah, those are tough," Freese said of the loss. "You kind of feel it a little more if you don't pull these out. But we're going to come back. We got a late report time tomorrow, [we'll] get some rest, come back tomorrow ready to go."
After competing against them for 14 innings, Freese knows the D-backs will come to play Wednesday as well.
"You want to win these games, especially these long nights," Freese said. "But gotta hand it to those guys over there. Playing third, you can hear them; they want to win. They want to win every inning. And they took care of business tonight."
D-backs left-hander Tyler Skaggs, who was called up Tuesday and pitched in place of the injured Brandon McCarthy, was long gone by the time the game was decided. Skaggs allowed five runs on six hits over 5 2/3 innings.
"Tyler's tough," Freese said. "He's a young guy, but knows exactly what he wants to do out there, especially against a tough lineup."