Old Scotland took the high road. New Scotland is upwardly mobile

Since the 1980s Scottish society has been transformed at unprecedented speed, and Glasgow is dubbed the 'Capital of Cool'. But the 300-year relationship with England isn't about to end.

During the debates on devolution in the 1990s the then Secretary of State for Scotland, George Robertson, declared confidently and memorably: "Devolution will kill nationalism stone dead." Not only has the Labour Party so far failed to shoot the nationalist fox but last week the leader of the party in Scotland, Wendy Alexander, to the consternation not only of the Prime Minister but of her unionist allies in the Holyrood Parliament, asserted the urgent need for a referendum on the issue of Scottish independence. To say the least, this remarkable juxtaposition of the conflicting responses of Robertson and Alexander, a mere decade or so apart, raises major questions for the historian, not only on the reasons why Scottish devolution has evolved in this way but what the implications might be for the future of the union itself.

The first puzzle is to consider why Robertson's prediction of the inevitable death of nationalism has gone so badly wrong. Instead of the demise of the SNP it is now the governing party of Scotland, committed to the independence of the nation from the rest of the United Kingdom. Devolution itself is a useful starting point in the process of explanation. The main architects of that constitutional change, John Smith and Donald Dewar, asserted that devolution was the "settled will" of the Scottish people. However, the chances of it being a one-off constitutional settlement, perpetually set in stone, were unlikely. Instead, devolution became a process not an event.

There were at least three reasons for this. First, there was an intrinsic imbalance in the new arrangements. Scotland, Wales and Northern Ireland had their assemblies but the biggest partner of all, England, did not. Devolution has, on the whole, served Scotland well, but there is a growing sense of unease south of the border that some of its financial aspects are unfair to the English. There is much unfinished business here. Second, the strength of Scottish identity and the weakening of Britishness, together with the expectations unleashed by devolution have focused even greater attention on the Holyrood Parliament as an agent of major social and economic improvement in Scotland. Third, especially in the first years of the devolved administration and the fiasco over the overspend on the Parliament building, together with the very modest impact of the Executive's policies, there were increasing demands for more powers as the panacea for mediocre performance. In essence, therefore, Pandora's box was opened and there was little possibility that it would easily be shut again.

In a sense, however, all of these challenges could have been addressed within a strictly unionist framework. After all, it had been Tony Blair's Labour government that had delivered devolution in the first place, and even the Scottish Tories had gracefully admitted the defeat of their own aspirations and promised to accept the constitutional will of the people. Ironically, however, the savaging of the Conservatives in Scotland and their destruction in elections for the Westminster Parliament was crucial to the continued rise and influence of the SNP. After the years of Thatcherite rule in the 1980s, the Tories were utterly discredited in Scotland and look likely to remain so for some time to come. The opportunity was there for the SNP to become the main opposition to Labour, a dynamic electoral force that attracted voters of many different political views, including a substantial minority who in no way were committed to the party's raison d'être of independence for Scotland.

But to explain the remarkable changes in Scottish politics over the past decade solely in terms of party manoeuvring would be to miss out a very great deal of a fascinating story. In truth, Scottish society since the 1980s has been transformed at a speed and to a degree unprecedented since the Industrial Revolution. The "old" Scotland of heavy industry, lower living standards than most of the rest of the UK, and the "cringe" associated with decades of remorseless economic decline evaporated. Within less than a generation the nation reinvented itself as a post-modern economy based on financial and other services, oil and gas, tourism, light engineering, knowledge-based industry, and public employment. The old centres of manufacturing are no more. Glasgow has become the second biggest shopping centre after London and is dubbed "Capital of Cool" by National Geographic magazine. Dundee, once famous for jute and notorious for slums, now has an international reputation for bioscience – with one of its universities rated fourth in the world for life science research by The Scientist.

Along with this material renaissance has come a cultural exuberance with a new-found vitality in fields as varied as literature, art, music, history writing, theatre, architecture and much else besides. To deny that the nation still faces major challenges on public health, alcohol consumption, drugs and related criminality would, of course, be myopic in the extreme. But the transformation of many lives has instilled a much greater sense of national confidence, which in itself must influence political aspiration and expectation.

This is especially so because of the social effects that the economic revolution has brought about. The demand for professional and skilled labour in the new economy, together with the vast expansion of tertiary education and the ever-great need for formal credentials and certification, has resulted in very high levels of upward social mobility. Studies by Edinburgh University sociologists demonstrate that almost two-thirds of adults of working age at the 2001 census have moved to a higher social class from their parents. Indeed, nearly half of Scottish men born between 1937 and 1966 have been upwardly mobile as measured by census descriptors and criteria. All of this need not cause long-term political change but it must at least provide a context for shifting allegiances, new loyalties and higher aspirations.

The key question then follows: will these fundamental societal changes provide the impetus towards a recovery of Scotland's ancient sovereignty or can they be eventually contained and satisfied within the union state? Many people think the days of the union are numbered. Hardly a week passes without the international media contacting my colleagues and myself with questions as to why this 300-year relationship may be nearing its end. After all, they say, Scotland is currently governed by a party committed to ending the union; surely the next and more dramatic step cannot be far distant? Then there is the masterly performance of Alex Salmond since taking office as First Minister a year ago. He is the supreme and dominant political figure in Scotland. His government has pursued headline-grabbing initiatives at an unrelenting pace in health, education and justice as well as robustly defending Scottish interests against that other Scot in 10 Downing Street. The SNP administration has demonstrated that it can govern, and govern well.

At dinner recently, after a lecture to senior civil servants in Scotland, I asked those at my table about the managerial competence of the SNP Cabinet. To a man (all were men) they admitted that, somewhat to their surprise, the current crop of ministers were decidedly more talented than those who had gone before. And then last week there came, apparently, the ultimate accolade. The leader of the Labour Party in Scotland carried out a dramatic volte-face and demanded, as soon as possible, a referendum on the SNP's core policy of independence. When to this is added growing irritation about Scotland being "subsidised" by English taxpayers in order to introduce health and educational improvement denied the population south of the border, the future of the union might indeed seem fragile.

Personally, I am not convinced that divorce is likely in the short or even the medium term. It should be remembered that while the SNP narrowly won a victory in the 2007 election, the party attracted less than a third of the vote. Its tally was 32.9 per cent in the constituency contests and 31 per cent on the list vote. Essentially, therefore, the SNP was the largest minority in that election with a total vote nowhere near a majority – and also including those who did not support the primary policy of independence. To win a referendum on independence, a large majority is needed in advance of the poll. Presently, there is no evidence of this, with support for independence varying between 20 per cent and 40 per cent in most polls taken since the establishment of the Scottish Parliament. This is why Wendy Alexander deserves credit for her recent political acrobatics. She may not have improved her personal relations with the Prime Minister, but calling the SNP's bluff must count as a shrewd move. If an independence referendum was called this year in Scotland, the SNP would face massive defeat – with huge implications for its future credibility as a political force.

Tom Devine is the Sir William Fraser Professor of Scottish history and palaeography at the University of Edinburgh. His latest edited work, 'Scotland and the Union, 1707 to 2007', is published by Edinburgh University Press on 14 May