Get those celebrities out of here! I want the Riviera all to myself

Driving east to west on the Cote d'Azur, Adrian Mourby discovers a peaceful route to the beach

It was one of those early azure mornings in summer that promise so much. We had arrived in Villefranche-sur-Mer from the airport. No one was up. Even the water in the harbour was asleep. Last night the Beautiful People would have been partying on Quai Amiral Courbet but all the Italianate cafés and bars had swept up and stacked their chairs away. John and I had an entire French town to ourselves. The stillness of the moment was delicious.

It was one of those early azure mornings in summer that promise so much. We had arrived in Villefranche-sur-Mer from the airport. No one was up. Even the water in the harbour was asleep. Last night the Beautiful People would have been partying on Quai Amiral Courbet but all the Italianate cafés and bars had swept up and stacked their chairs away. John and I had an entire French town to ourselves. The stillness of the moment was delicious.

John, my 11-year-old son, had been reluctant to return to France. He had hated the noise of Paris, but here, as we sat with our feet dangling in the lukewarm Mediterranean, I could see he was just as taken with this place. Villefranche-sur-Mer is an unusual town. Constructed on a steep slope, it has built its houses out over the harbour's medieval back streets so that roads, such as the 13th-century rue Obscure and the 14th-century Portal de Robert, run like tunnels beneath the town. Jean Cocteau used this warren as the underworld in his 1950 film Orphée. Devoid of people, these streets feel weird – it was as if we had stumbled into the Middle Ages. Swishing our sandals in the harbour waters was a much better way to welcome the morning.

The sound of a single motorboat had been drawing nearer, its prow cutting a perfect V through the still grey waters. Now it arrived and a tanned young man disembarked with a basket. He had clearly been sent by the Beautiful People out on one of those big white yachts to buy that morning's croissants. We followed him. It was time for breakfast.

I had decided to spend our first day motoring west along the coast, all the way to St Tropez. The names on the gazetteer read like the script of some sexy black-and-white film or a novel by F Scott Fitzgerald: Cap Ferrat of the Rothschilds, St Paul de Vence of the Colombe d'Or, Antibes with its Picassos, Cannes boasting of Orson Welles and Sophia Loren, and St Tropez itself, a town synonymous with Bardot and Colette.

We had hired a car at the airport and soon got used to the roads. The drive to St-Jean Cap Ferrat was easy. This small isthmus overlooking the Baie des Fourmis is bristling with pine-forests and the homes of the outrageously rich. There was no way we were going to be able to afford even coffee at the Grand Hotel du Cap, so I took us to the Villa Ile de France, built by Baroness de Rothschild in 1905 and given to the nation in 1934.

Baroness Ephrussi famously greeted her guests dressed as Marie-Antoinette and the villa is equally stylish. Florentine on the outside and Arabian Nights on the inside, it has a glorious glass-panelled salon d'hiver overlooking the sea, which has been turned into a café. Boy, did the Baroness take her tea with style.

After a snack we roamed the grounds, which are divided into seven gardens, each designed around a theme – Spanish, Florentine, Japanese. Released from the confines of his seat belt, John played in the elaborate water-course that bisects the Baroness's lawn, but soon we found the garden was being set for a wedding reception, so I bundled him back into the car and we headed off to Cannes.

We passed close to Antibes, which I must say tempted me. This harbour town was fortified by Vauban, Cardinal Mazarin's military genius, and in 1946 another genius, Pablo Picasso, was resident in the Chateau Grimaldi, using part of its castle as his studio. He left everything he had completed – 150 paintings, sketches and ceramic works – to the town, which is why the Musée Picasso is there.

But my son was having none of this, so we headed for Cannes. After all, there are very few French seaside towns that can claim to have Arnold Schwarzenegger's palm prints in the pavement. After comparing hand sizes with Gérard, Arnie and Samuel L Jackson outside the celebrated Palais des Festivals, we walked round the old town and found a restaurant overlooking the public beaches where I could eat oysters and John managed to wangle fish, chips and ketchup out of the generous proprietor. Cannes lacks charm. It is fashionable, noisy and expensive, and the best beaches off Boulevard de la Croisette are private, but it was worth stopping for and we certainly continued on our journey well fed.

The Riviera proper has always ended at Cannes. To get to St Tropez we had to use the A8, a busy inland motorway that can get very congested at weekends when just about everyone wants to be seen posing in the ultimate sybaritic playground. St Tropez was neglected by the beautiful people for many years. Because of a curve in the coastline, it faces unfashionably north. Whereas Villefranche, Cap Ferrat, Antibes and Cannes are all in the same 40km stretch, St Trop (as the French call it) is isolated 70km west of Cannes. Still, it's worth a visit. We had heard horror stories about traffic snarl-ups in the town on days when up to 100,000 visitors pile in, but I was relieved to discover that the stamping ground of Bardot, Françoise Sagan, Joan Collins and Elton John had invested in a huge new underground car park from which it is possible to walk through the market and down to the harbourside.

It was late afternoon by now and yachts were mooring for night, their open-plan aftdecks crammed in bumper to bumper alongside Quai Jean Jaurès. The harbourside stores were full to bursting – as were the cafés – so John and I decided to go for a swim. This was supposed to be his holiday too, and so far my desire for a dolce vita pilgrimage had ridden roughshod over his for some seaside fun. So we followed a few other people with towels under their arms past the fortified headland and round to the Plage de la Fontanette.

The bustle of St Tropez soon dropped away as we passed blank, pastel-painted houses that fronted on to the Med and it was easy to see how 50 years ago this was just a fishing village, the inspiration for Post-Impressionists such as Matisse, Paul Signac and Bonnard. It was not until Roger Vadim shot Et Dieu créa la femme, with his wife, Brigitte Bardot, in St Tropez that the town became synonymous with wealth and rampant hedonism. We did not make it quite as far Pampelonne Beach, where Bardot celebrated her retirement from films in 1974, but we did manage a refreshing dip that cost us nothing at all.

Then we wandered back into town to eat gambas – killer prawns with attitude – overlooking a sparkling sea. It was one of those days that could not possibly disappoint – and it didn't.

The Facts

Getting there

Adrian Mourby and his son John stayed at the Cannes Villa Francia courtesy of Perfect Places (0870 366 7562; www.perfectplacesonline.co.uk), which offers a week in a family apartment for £963 in July. The price includes a Dover to Calais ferry crossing for a car and up to five passengers.

Easyjet (0870 600 0000; www.easyjet.com) offers return flights to Nice from around £80.

A week's car hire with Holiday Autos (0870 400 0010; www.holiday autos.co.uk) can start from £109.

Further information

French Tourist board (09068 244123, calls cost 60p per minute; www.franceguide.com