When the Black Stars failed to clear a low cross, the visitors pounced with Serbia-based Milson pushing the ball past goalkeeper Lawrence Ati-Zigi from close range on 93 minutes.
Home supporters hurled dozens of plastic bottles onto the pitch after the final whistle to express their anger at the Group F matchday one result.
It was the first loss suffered by four-time African champions Ghana at the Baba Yara Stadium since 2000.
Ghana included six Premier League players in the starting lineup as they sought to atone for embarrassing group-stage exits from the 2022 and 2024 Cup of Nations tournaments.
But a notable absentee was 120-cap midfielder Andre Ayew, 34. 'Dede' was overlooked by coach Otto Addo as he is currently clubless after being released by French Ligue 1 outfit Le Havre.
The Black Stars, who last won the competition in 1982, had more shots at goal and possession and passed more accurately, but could not break down the Angolan defence.
Angola reached the 2024 Cup of Nations quarter-finals before bowing out to eventual runners-up Nigeria, and share the Group F lead with Sudan, who beat Niger 1-0 on Wednesday.
In matchday two, former champions Sudan are away to Angola on Monday and Niger host Ghana the same day.
Another unexpected result was Burundi winning 3-2 away to Malawi in Group L in Lilongwe with Mokono Eldhino snatching the 87th-minute match-winner.
Malawi twice equalised before suffering a loss that immediately puts them on the back foot in a group completed by 2022 champions Senegal and Burkina Faso, who meet on Friday.
Faouzi Benzarti, a 74-year-old starting his fifth stint as Tunisia coach, had to wait 98 minutes before Ferjani Sassi netted to deliver a 1-0 Group A victory over Madagascar in Rades.
French second-tier forward Louis Mafouta scored twice as the Central African Republic - one of 10 countries yet to qualify for the finals - overcame Lesotho 3-1 in El Jadida.
The Group B fixture was switched to the Moroccan coastal city because the national stadium in Bangui does not meet international standards.
There are 10 qualifiers set for Friday with title-holders the Ivory Coast and former winners the Democratic Republic of Congo, South Africa, Morocco, Egypt, Zambia and Senegal debuting.
