Still life: Philip-Lorca diCorcia's breathtaking and outlandish fashion photography for 'W' magazine

Here he talks psychology, surrealism and serendipity with Edward Helmore

In 1997, the celebrated New York artist Philip-Lorca diCorcia received an invitation to shoot for the fashion magazine W. Well known in art circles for a brand of surrealistic imagery both acutely controlled and dependent on improvisation, the artist collaborated with W's creative director Dennis Freedman for the next 13 years, pulling off 11 shoots in locations from Cuba to Los Angeles, Bangkok to Cairo.

Two dozen of the images are currently on display at David Zwirner in New York, and the entire collection of 150 images will be published in ELEVEN: W Stories 1997-2008, out next month. Collectively, they form a survey of a moment in fashion photography that unleashed it from its primary purpose – to create a show-and-tell for clothes – and allowed it to tell almost entirely random stories, often with dark psychological undertones. If every magazine has a photographic template – Guy Bourdin and Helmut Newton for French Vogue, for instance, Irving Penn and Richard Avedon for the US edition – diCorcia established W's art-fashion credentials. The series came to an end in 2008, a year before W was set back on a more conventionally commercial path.

The model Nadja Auermann is found in a Bangkok sweatshop; a male model who later turns out to be a hustler showers in front of prim Park Avenue women; the designer Marc Jacobs sits on a bed as a man sleeps; a woman in a wedding dress climbs a stepladder in Sao Paulo; three Thai drag queens pose in sailor suits behind plate glass; a woman (the supermodel Kristen McMenamy) stands in a glass house in LosAngeles.

Fantasy scenarios are nothing new to fashion, but these are distant cousins to the model-with-happy-tribesmen template. In many, fashion seems to have been forgotten altogether. The psychological story, however, is always apparent – "being a little obvious is not a bad thing in this particular realm," the artist remarks, surveying an image of a woman with a blank expression standing in a suburban house and garden – the quintessential spiritual void. "The environment is meant to reflect a psychology – in this case she's a prisoner in a sterile world. She didn't need much motivation – I think she was that."

He continues: "I think of these people as characters and every character has a psychology. I don't photograph models in the usual way, with all that prancing and jumping, so inevitably they ask me what to do. I don't want them standing there looking glum; I want them to understand the narrative arc of what we're trying to get at even if we don't understand it ourselves."

There are echoes of Caravaggio and Goya, painters who used light for symbolic meaning, in diCorcia's central (and much-immitated) innovation; mixing ambient and artificial light in still photographs. After receiving an MBA in fine art from Yale in 1979, he began to establish a reputation as a photographer.

Initially diCorcia placed his friends within fictional tableaux. Then, using lights secreted in the pavement, he began illuminating anonymous strangers against populous street scenes. The sense of detachment which is key to his work was sustained through each of his early series – Hustlers, Streetwork, Heads, A Storybook Life, and Lucky Thirteen. "I did the first series and missed it when I wasn't doing it. Most people don't pay attention, but I found it fascinating standing there on the street."

Despite the technical control of the photographs, it is surprising how much of diCorcia and Freedman's decade of adventure fashion was left to chance. As the artist explains, "We were always supposed to have some kind of concrete programme, but the concrete programme is only good til the first day you get there."

Some of the strongest shots in the book, such as the society ladies lunching with the hustler in the Windows on the World restaurant atop the Twin Towers, came about through luck. "We'd built a set in a studio and after a couple of days I was getting claustrophobic and wanted to break out. Someone knew the manager of Windows on the World so we went there."

There's a certain decadence to the images. W is now a different animal and its personnel have dispersed. Commercial magazines, now highly focused on the bottom-line, no longer have the freedom from advertisers or the budgets to indulge these exercises. The introduction of digital photography has replaced paper and chemicals, but perhaps something has been lost in the process. Hyper-reality has in a sense become the norm, and not necessarily for the better in diCorcia's view.

"One of the things about [traditional] photography is that it happens in a fraction of a second. I cannot predict and don't really know what I've done." With the image instantly under review on a computer screen, there's been a shift in the emphasis and perhaps romanticisim, irrespective of the tricks employed in digital post-production.

It's is perhaps remarkable that diCorcia and Freedman were permitted to roam so widely and free for so long.

"I held them to ransom. This is the way it has to be or I'm not doing it," the artist recalls of their initial understanding. "And Dennis knew that – what's the point of doing something that just looks like somebody else?"

Philip-Lorca diCorcia: Eleven W Stories 1997-2008, edited by Dennis Freedman, is published by Freedman Damiani in April 2011