The Rational Optimist, By Matt Ridley<br />Pandora's Seed, By Spencer Wells

Is it a cheap shot to point out that Matt Ridley, scourge of greens and author of The Rational Optimist: How prosperity evolves, is also the Matt Ridley whose optimistic chairmanship of Northern Rock steered it to bankruptcy in 2008? I don't think so. Because both book and bank display what might best be described as irrational optimism - a blind belief that what has worked before will always work, regardless of a changing environment, whether financial or planetary.

Ridley looks keen to become chief cheerleader for a cornucopian view of the world. Even in a world of limited resources, he believes, humans should always triumph because, well, that is what we have been doing for the past 10,000 years. "The twenty-first century will be a magnificent time to be alive," he concludes his book. "Dare to be an optimist."

His book coincides with publication of geneticist Spencer Wells's Pandora's Seed: The unforeseen cost of civilization, which shares a similar time-frame but reaches a very different conclusion. Ridley argues that we humans diverged from our ancestors when we discovered how to work together. Individuals specialised and traded for a collective purpose. This was part of life in tribal hunter-gatherer societies, but reached fruition among the first agriculturalists. Farmers quickly organised themselves into cities and then states to manage irrigation, store and trade produce, and run armies to defend their lands.

Industrialisation and globalised trade took all this to a new level, giving us the power over nature's resources to sustain approaching seven billion people – people who are healthier, better fed and happier than ever before. Ridley accepts that there are challenges ahead for Homo sapiens. He identifies "two great pessimisms" – climate change and the fate of Africa. But he sees no reason why, properly managed, our hotter world cannot remain a better world, nor why Africa cannot join in. He sees no tipping points, no Malthusian traps, no limits to growth, no laws of diminishing returns and no need to sound the retreat from our conquest of nature.

Ridley may be right. But he was wrong about Northern Rock, which required a government bail-out. And there is no inter-planetary lender of last resort for planet Earth. The human project is not "too big to fail".

Wells also sees a transformation in our species when we took up farming. But his charting of our genetic history since then is less triumphalist. In our DNA, we largely remain hunter-gatherers, and may ultimately be doomed by the growing mismatch between our genes and the world we have made. Farming allowed our species to proliferate, but it was far from being a free lunch. Farming was harder than hunting. When we domesticated animals, we domesticated a range of killer diseases as well. Though more numerous, we became shorter and sicker, and died younger. Also, those organised societies brought organised religion and state control. The need to cooperate often left us in thrall to despots.

Meanwhile, says Wells, our old genes increasingly get their own back. We have a sweet tooth because that put us off rotting food. Sweet meant safe. But now it just makes us fat. While our genes yearn for the wide open spaces, the noise and stress of modern life is driving us demented. Mental illness will be the second biggest cause of death and disability by 2020. Contrast this with Ridley's assertion that we are the happiest people ever, and that richer people in richer countries are generally the happiest of all.

These two compelling studies share some themes. Both men argue that we got where we are today through our social skills and ability to innovate and spread the word about what works, whether round the hunters' camp fire or the internet. Both display some optimism about our ability to adapt, even benefit from, climate change. Wells suggests that the switchback climate as the planet lurched out of the last ice age forced us to cultivate plants we would once have simply harvested from nature. Man-made climate change too may "provide us with an opportunity in crisis", he says.

While Ridley sees us heading onward and upward, Wells has lost confidence. The next adaptive challenge we face is to dump "growth, expansion and consumption" in favour of downsizing to "a lifestyle with long-term sustainability".

Has Wells chickened out, or is Ridley the irrational dreamer? Ridley's view is often blinkered. His grand historical sweep revels in the success of past civilisations, but their demise is glossed over in a sentence. At one point, he describes how an "endless series of empires" in Mesopotamia have failed, among them "Sumerians, Babylonians, Persians, Roman, Mongol, Ottoman, British, Saddamite, Bushite... Each empire was the product of trading wealth and was itself the eventual cause of that wealth's destruction." He averts his eyes and moves on.

But this is the issue. No doubt each of those empires contained their own Matt Ridleys, the rational optimist of their day who said all was well. Jared Diamond, in his book Collapse, described how, as he put it, "societies choose to fail". Consumed by rational optimism, they could not see the end coming and failed to prevent it.

Ridley would say that each failed civilisation was replaced by something better – and ultimately by us. But today we have one global society, whose specialisation and trading networks sustain unprecedented numbers of people. Good for us. We are now one. But that leaves us uniquely vulnerable if we, too, choose to fail. There is nowhere else for us to go: nobody to bail us out. Our planet, our species, has no lender of last resort. It would be curtains.

I like optimism. Ridley is right to stress that the doomsters have been proved wrong before. The world has got through a period of maximum population growth without hundreds of millions starving. Family sizes round the world are falling without any compulsion, so we can see Malthus was wrong. We are living longer, healthier, more interesting and often freer and more prosperous lives.

We need optimists to gird us for tasks ahead and prevent us from subsiding into the paralysing pessimism that grips many environmentalists. But those tasks remain. When optimists like Ridley end up telling us all is well as we hurtle towards the cliff, then they are the enemies of the future. And they are certainly not being rational.

Fred Pearce's latest book is 'Peoplequake' (Eden Project Books)