Home-plate umpire Jerry Layne booted Kazmir when the lefty let loose a few words in response to Layne's call of ball four instead of strike three on Jesus Aguilar.
Dan Otero's efficiency and a couple of timely hits helped Oakland pick up the pieces and turn a scramble into another victory.
Otero (4-0) pitched a career-high 3 2/3 innings in relief of Kazmir, getting Oakland's bullpen through the fifth.
Kazmir took out his frustrations on a trash can, but he marveled at the work Otero and Oakland's bullpen put together, allowing just one run in 7 2/3 innings.
"That's something that needs to take notice," Kazmir said of the bullpen's work. "It was incredible how they stepped up and just threw zeros up there. They attacked the strike zone, got quick outs."
No outs came quicker than Otero's first two.
Otero got Mike Aviles to hit into an inning-ending double play on his first pitch.
"We saw him get tossed by the home-plate umpire, and started scrambling around," Otero said. "I didn't know who they were going to bring in. I got moving around a little bit in case the phone was for me, and it was, so I had to go in and try to get outs."
Just 31 pitches later, he had gotten through another three frames.
Otero gave up four hits, but Oakland turned two more inning-ending double plays in the third and fifth innings to keep Otero on course.
Otero's efficiency was impressive, but he said it was merely a byproduct of trying to be aggressive.
"That's the last thing on my mind, trying to be economical with my pitches," Otero said. "I'm just trying to throw strikes and get outs, first-pitch strike and then try to get the guy out."
With Kazmir banished so early, the Indians had an opportunity to get back in the series, but Tribe manager Terry Francona said Otero was what kept them from doing so.
"Otero really did a good job for them," Francona said. "They did some good things, but we're at a point where we're an inning into the game and we're in their bullpen.
"But he was so good. He got the first-pitch double play. Next inning, got the double play. He kind of let them get their bullpen semi in order. He really bailed them out."
With Otero working quickly, Oakland's hitters also wasted no time in backing him up.
In the third, the heart of the order made Josh Tomlin (2-1) pay for a couple of bad pitches.
Trailing 1-0, Josh Donaldson hit a hard single to bring Coco Crisp home and tie things up, and Brandon Moss followed him with a two-run homer to put Oakland up.
"As far as the team goes, they responded terrific," manager Bob Melvin said.
In the seventh, Donaldson and Moss put the game out of reach.
Donaldson slugged a shot to the top of the wall in left-center field with two runners on, and on the bounce, the ball made it almost halfway back to the infield before it landed.
The play was ruled a triple, but Layne ordered a review of the play to see if the ball was a home run or if it remained in play.
The play stood as called, so Donaldson was on third with a two-run triple.
Donaldson is convinced the ball was a home run, given the ball's trajectory off the wall.
"I think so," he said. "Of course, I'm going to be a little biased. But what are you going to do?"
The A's still got Donaldson's run when Moss followed up with a double.
"I'm glad Moss got the hit right after him," Melvin said. "With everything else going on in the game, that was a tough one to swallow at the time."
Once Otero's night ended, Fernando Abad (one inning), Luke Gregerson (two innings, three strikeouts) and Sean Doolittle (one inning, two strikeouts) got Oakland the rest of the way.
Abad ran into some trouble after giving up back-to-back singles to open the sixth, but Cleveland got only a run out of it.
"The bullpen covered the whole game, which was pretty unbelievable," Melvin said. "[Doing it with] all of them going out there knowing it might be more than what they're used to doing."