The Argentine saved two penalties, including one that had to be retaken, before Girona's Uruguayan striker Cristhian Stuani scored from the spot himself in the 99th minute to seal all three points.
While Girona, who are 11th, put on a much-improved performance in the second half against sixth-placed Bilbao to snap a four-match winless LaLiga run, Michel's side still looked far from their best.
"Thank God, or Gazzaniga! I'm happy for the three points but we have to improve more. I liked the second half as we made Athletic back off," Michel told reporters.
"With Stuani, it's a sure thing and I knew he would score."
Bilbao were not so clinical as Alex Berenguer's penalty was saved in the first half and after the break Inaki Williams' effort was also stopped but had to be retaken because Gazzaniga's feet were off the line. Ander Herrera stepped up for the retake but his effort was also saved by the Argentine.
"I didn't believe the one he (Herrera) took again because I thought, two yes, but not three. Gazzaniga made three incredible interventions that allowed us to stay alive. I don't know what would have happened with the 1-2."
Girona, who had never before qualified for the European Cup in their 94-year history, secured a place in the Champions League by finishing third in LaLiga last season four years after being promoted to the top flight.
They now sit mid-table with 12 points from nine matches.
"It hasn't been a bad start but if you look at the expectations it seems that we're missing something," Michel said.
