Coventry were leading 1-0 at the 87-minute mark and looking much the better side when substitute Djed Spence slid home an equaliser.
Johnson, who had come on after 17 minutes as a replacement for the injured Wilson Odobert, fired past Ben Wilson five minutes later to seal the great escape.
The result came as a big relief to manager Ange Postecoglou, who made headlines before the match with the assertion that he always won a trophy in his second season at a club and had fielded a much-changed lineup.
In the night's other third-round clash, Brighton beat Premier League rivals Wolverhampton Wanderers 3-2 on the south coast.
Until the late goals, Coventry had looked headed for a deserved upset.
Brandon Thomas-Asante swept the hosts ahead in the 63rd minute, with Tottenham back-up goalkeeper Fraser Forster looking shaky in his first appearance of the campaign.
Tottenham had gone through the first half with more possession but without a single shot on goal and left regulars Son Heung-min, James Maddison and Dejan Kulusevski on the bench.
Jack Rudoni tested Forster in the opening minute, forcing a save, and had another good opportunity after 38 when Destiny Udogie made a crucial block.
Tottenham defender Ben Davies saved a certain goal from Haji Wright in the second half after a glaring misjudgement by Forster who had raced out to clear and left his goal completely exposed.
Kulusevski came on in the 74th for Timo Werner and provided the through ball for Spence while Son got the nod in the 62nd with Maddison as Postecoglou recognised the need for more creativity.
"It was a tough game. Credit to Coventry, I thought they were outstanding," said the Australian of last season's FA Cup semi-finalists.
"It was a cup tie and they really took it to us. We had to just hang in there somehow and we did."
The win came after two losses in the league, including a 1-0 home defeat to Arsenal on Sunday.
"I don't think there's an easy or quick fix to where we want to get to and one result doesn't change that," said Postecoglou when asked about the significance of the result.
"Tonight, at least in the end there, we showed what we have been missing in the first four games, that little bit of spirit to find our way to win.
"We've just lacked that little bit of an edge and hopefully tonight gives the boys a bit of belief."
At Brighton, Carlos Baleba opened his account with the Seagulls on 14 minutes and Simon Adingra shot low for the second goal in the 31st.
Goncalo Guedes pulled one back for the visitors a minute before the break with a deflected side-footed shot before Ferdi Kadioglu made it 3-1 in the 84th on the rebound.
Tommy Doyle then lashed home off the underside of the bar in stoppage time for a nervous finish.
The remaining third-round ties will be played next week, with holders Liverpool at home to Premier League rivals West Ham United on September 25th.
