Soudal Quick-Step's Magnier pipped Israel-Premier's Ethan Vernon in a bunch finish in Newark, helped by his team-mate and former world champion Julian Alaphilippe.
"I'm super-happy again to take the victory," said Magnier, who won stage one on Tuesday. "It was again a super day with the team.
"First we controlled the race to be sure it finished with a bunch sprint, and then they put me in a really good position for the final.
"It was really fast with the tailwind, and in the end I still had the punch to finish off the work of the whole team."
The top five riders in the overall standings are unchanged, with Israel-Premier rider Williams still leading by 16 seconds from Oscar Onley.
The fifth stage takes place on Saturday, with the race ending in Felixstowe on the east coast the following day.
Stage four
1. Paul Magnier (FRA/SOQ) 3hr 11min 54sec, 2. Ethan Vernon (GBR/IPT) at same time, 3. Erlend Blikra (NOR/UXM) s.t, 4. Noah Hobbs (GBR/CGF) s.t. 5. Matevz Govekar (SVN/TBV) s.t.
Overall standings
1. Stephen Williams (GBR/IPT) 14hr 50min 47sec, 2. Oscar Onley (GBR/DFP) at 16 sec, 3. Mark Donovan (GBR/Q36) at 40, 4. Joe Blackmore (GBR/IPT) at 41, 5. Tom Donnenwirth (FRA/DAD) at 41
