NEW YORK -- Back on the other side of the city where his precocious career began, Matt Harvey spent four innings Wednesday night continuing to look more and more like the Harvey of old. Now, Harvey and the Orioles are hopeful any version can return to the mound soon.

A right knee injury was the reason Harvey was pulled from a 10-3 loss to the Yankees at Yankee Stadium after four innings and 63 pitches, only a few batters removed from the end of his career-best 21 2/3-inning scoreless streak. Harvey said he initially felt a “tweak” in the knee while fielding Gleyber Torres’ comebacker in the second, then felt the discomfort resurface while covering first base on Joey Gallo’s fielder’s choice in the fourth. Harvey finished the inning, but he didn’t come back out for the fifth, giving way to César Valdez with the O’s leading, 3-2.

• Box score

Speaking after the game, Harvey and Orioles manager Brandon Hyde both said they hoped Harvey could make his next start, which would come on extra rest because of two upcoming off-days. The O's could also push him back, as they don’t need a fifth starter until Aug. 14 in Boston.

“Going forward in the outing, I felt I couldn’t quite push off the rubber and stay in my mechanics the way I needed to, especially against a lineup like that,” Harvey said. “It was frustrating, because I felt so good early in the outing and I wanted to keep us in the game longer and keep the bullpen from being used.”

With Harvey sidelined, the Orioles' lead evaporated on DJ LeMahieu’s RBI single in the fifth. Then, LeMahieu put the Yankees ahead, 4-3, with another RBI single off Paul Fry in the seventh, starting a five-run rally that broke the game open. That brought a lopsided end to a rubber game that started with Harvey and Jameson Taillon dueling.

Pitching with an early two-run lead, Harvey faced the minimum over three innings and retired 10 of the first 11 batters he faced before Anthony Rizzo hit a one-out solo homer in the fourth. Harvey then gave up another run on Torres’ RBI single four batters later.

By that point, the Orioles had padded their lead via Ryan Mountcastle’s 19th homer of the season, his third in six games. But Baltimore didn't score again vs. Taillon, who struck out 10 over 6 1/3 strong innings, and three New York relievers.

Harvey’s early results were the latest in a resurgent stretch since the All-Star break for the veteran right-hander. He’s held opponents to two runs in 22 1/3 innings in the second half after pitching to a 7.70 ERA in the first half.

“It’s disappointing, because he was throwing the ball so well,” Hyde said of Harvey's knee ailment. “He was in the same groove he was in over his last few starts.”

If Harvey can't return by next week, it leaves the Orioles with a hole in their rotation, at least for the short term, even if Bruce Zimmermann can work his way back from left biceps tendinitis by the middle of the month. The easy solution would be to bring back left-hander Alexander Wells, who was optioned after his Tuesday start, and replaced with Keegan Akin.

The O's could bring back Dean Kremer, despite his continued struggles for Triple-A Norfolk, or perhaps go splashier and promote Mike Baumann, their No. 7 prospect per MLB Pipeline. Recently acquired left-hander Ryan Hartman is another option, though he is on the injured list while undergoing intake screening.

But there is at least a decent chance such juggling won’t be necessary. The schedule allows the Orioles to give Harvey up to nine days of rest before his next turn, if necessary. Anything longer would require an IL stint (though the O's may opt for one anyway to clear short-term roster space for bullpen depth).

Whatever transpires, the Orioles' hope is it doesn’t disrupt the strides Harvey has made since the All-Star break and the stability it’s brought to their pitching staff. His scoreless streak was the longest active in the Majors before it ended and the longest by an O's starter since Kevin Gausman also had a 21 2/3-inning streak in 2016.

“I definitely do,” Harvey said, when asked if he thinks he’ll make his next start. “I don’t think it’s serious at all.”