South Africa will rue their poor discipline and a slow start that allowed France to build an early lead, while a missed conversion from Manie Libbok late in the game also proved crucial.
The defeat opens the door for New Zealand to top the Southern Hemisphere pool in the competition, while France could well have sealed top spot in the Northern Hemisphere group.
It was a frustrating first half for South Africa, who were as much their own worst enemies as the French were.
The home side dominated possession and, when South Africa did get their hands on the ball, they looked dangerous, but poor execution let them down.
After lock Lood de Jager, no stranger to disciplinary issues, received a yellow card for a dangerous tackle, France went 14-0 ahead thanks to tries from winger Aaron Grandidier-Nkanang and fullback Theo Attissogbe.
It was a deserved lead for the hosts, who applied pressure and were patient enough to wait for Springbok errors to provide opportunities.
The visitors struck back through a try from a driving maul by Jan-Hendrik Wessels, who, along with props Gerhard Steenekamp and Thomas du Toit, had initially been named on the bench but was drafted into the starting XV just before kick-off by coach Rassie Erasmus.
An excellent grubber kick from Antoine Dupont was collected by winger Louis Bielle-Biarrey, who scored in the corner to extend the French lead again.
Home scrumhalf Maxime Lucu received a yellow card for a high tackle on Sacha Feinberg-Mngomezulu, and the Springboks thought they had scored a try on the stroke of halftime through winger Kurt-Lee Arendse, but he was ruled offside at the lineout.
The Springboks' cause was not helped when number eight Jasper Wiese was penalised for an illegal clean-out on Dupont, with his yellow card upgraded to a 20-minute red card following a bunker review.
France then lost another player to the sin bin when Grandidier-Nkanang was shown a yellow card for a deliberate knock-on.
Cobus Reinach burrowed over from close range to reduce the deficit to seven points.
South Africa won a succession of scrum penalties near the French line and, after the umpteenth collapse by the home side while already under a warning, Italian referee Andrea Piardi inexplicably declined to penalise them again.
At the next scrum, closer to halfway, he did show a yellow card to Regis Montagne, a decision that made his previous call on the French five-metre line all the more peculiar.
Malcolm Marx barged over from a driving maul as the Springboks' power game proved too much for the French, but Libbok missed the conversion and France led 21-19 with seven minutes remaining.
France clung on with some heroic defence and celebrated wildly at the final whistle, gaining a measure of revenge for their defeat in the 2023 Rugby World Cup quarter-finals at the same venue.