Late goals from Seydou Sano and Yacine Brahimi condemned the champions to defeat in Doha after Al Ain's Kaku Romero and Sofiane Rahimi had cancelled out an opening double from former Real Madrid striker Joselu.
"We needed this win," said Joselu. "I'm happy I scored but what's important is the team. We lost our last game in the league so today we must show we are a team that can play together and we played well, we scored four goals."
Joselu put Al-Gharafa ahead when he redirected Sano's shot past Khalid Eisa in first-half injury time and he doubled the lead three minutes after the restart with a close-range header.
Kaku pulled one back from the penalty spot in the 56th minute and Rahimi bent his strike beyond Sergio Rico to level 10 minutes later, only for Al-Gharafa to rally late in the game.
Sano pounced from close quarters to restore the hosts' lead in the 72nd and Brahimi sealed victory four minutes later to give Al-Gharafa their first win and leave Al Ain on one point from two games.
Al-Hilal had few difficulties at home against Al Shorta as Jorge Jesus' side picked up their second win to join Al-Ahli, who defeated Al-Wasl on Monday, on top of the West Asian standings after two rounds of the league phase.
Two goals in four first-half minutes put Al Hilal in control, Michael Leonardo netting in the 11th when he pounced after Ahmed Basil saved Sergej Milinkovic-Savic's attempt.
Aleksandar Mitrovic doubled the lead with a header from six yards out as Renan Lodi swung the ball into the penalty area following a quickly taken corner.
Salem Al-Dawsari hit his side's third two minutes into the second half before goals from Nasser Al-Dawsari and Mohammed Kanno put the result beyond doubt.
The top eight finishers in the 12-team western and eastern divisions of the league phase will progress to the last 16, which will be played on a home-and-away basis in March.
Teams advancing to the quarter-finals will participate in a centralised tournament in Saudi Arabia to determine the champions, with the final to be played on May 4th.
