While Williams' struggles were present, the Angels' defense, which now has a league-leading 72 errors, did not provide much assistance.
Alberto Callaspo, Chris Iannetta and Mark Trumbo all committed throwing errors that led to runs, and although he was not charged with an error, relief pitcher Garrett Richards threw a ball into center field on a pickoff attempt.
"They factored in, no doubt," Angels manager Mike Scioscia said. "Even though we weren't swinging the bat, we could have maybe given ourselves a chance to stay a little close and then things could happen."
The Angels did take two of three games from Oakland and are 7-2-1 in their last 10 series.
"Overall, a good series," Trumbo said. "Today wasn't that great, but bigger picture, we did do a pretty good job against a good team."
While Sunday's game was a bit of a debacle, Scioscia and Williams each feel the right-hander is getting back into a rhythm.
"I thought he threw the ball well," Scioscia said. "If you look at the line score, it doesn't look really healthy, but I thought Jerome threw the ball very well. He had both sides of the plate going, just not a lot of luck."
Williams also felt like he threw well and said he used the All-Star break to find himself.
"The last three starts before the break, I wasn't my normal self," Williams said. "I was getting out of whack. I think during the All-Star break I kind of figured it out, got back to where I needed to be."
Couple the unearned runs with A's ace Bartolo Colon on the mound, and it was not a good recipe for the Angels.
The 40-year-old All-Star, who won the American League Cy Young Award with the Angels in 2005, threw his 12th career shutout and 35th complete game while limiting his former team to four hits.
"Pretty amazing," Oakland manager Bob Melvin said. "He was sick today on top of it. You saw the velocity. The velocity was down all day, so he did it a little bit differently today with more movement and less power. And still got it done."
Oakland got on the scoreboard when Chris Young led off the third inning with a double and Eric Sogard followed with his second home run of the season, giving the A's their first lead of the series.
"It was supposed to be down and away, it was a sinker," Williams said. "I just cut it off, it came back in the zone and he put a good swing on it, hit it out. I think that's probably one of the only mistakes I made and hitters will take advantage of those mistakes. He took advantage of it."
The Athletics added to that lead with a pair of runs in the fifth. Sogard led off the inning with a single, but the damage started when Coco Crisp bunted for a single.
Instead of just eating the ball on the likely single, third baseman Callaspo -- who has had his share of throwing woes this season -- made an errant throw to first base, allowing Sogard to score and Crisp to advance two bases to third.
Four batters later, Brandon Moss singled to left field, driving in Crisp and extending the lead to 4-0.
The Angels' defensive miscues continued in the sixth when Iannetta's throwing error allowed Josh Reddick -- who had stolen third base -- to score. The very next batter, Crisp, hit a ground ball to Trumbo at first base, but his throw to the plate was errant, allowing another run to score.
William Boor is an associate reporter for MLB.com.