Friday, 3 May 2013

A Picture of Pathways to Impact

This picture went into my recent ESRC application and it seemed a shame to just go to waste there (so to speak) ...

....It was drawn by Tereza Proch√°zkov√°, who is a Masters student of Service Design, a course run by Hazel White at the Duncan of Jordanstone College of Art & Design. I had asked Tereza to interpret my 'Case for Support' so that I could demonstrate to the ESRC how I would try to work with Tereza to produce some reports using words and pictures. The point is not to make superificial or simplistic arguments about complicated topics. Rather, the idea is that the production of drawings forces you to decide what the key points of a document are (perhaps in a stronger way than an abstract or set of bullet points would make you choose). The interpretation of my reports by someone else also allows me to check if I have managed to get my point across to an audience that doesn't understand the issues in the same way (see also
The idea came from the CIPFA annual conference in Scotland in 2013. White’s students contributed to her lecture/ workshop on ‘Service Design’ at the event and Tereza also summarised my lecture (on complex policymaking systems) using text (found here, compare with my lecture/ blog post here) and pictures. I don't *think* that it appealed to me simply because the picture of me was flattering. 


Monday, 29 April 2013

Using the Internet for Political Research (POL9RM 30.4.13)

Since I am giving a lecture on using the internet for political research (for POL9RM), it seems appropriate to upload it to your actual internet. This is, depending on how you view these things, either an excessively long or good-value blog. We might settle on ‘generous’.

The lecture will be split into two basic issues:

1. What type of information should you seek, and from where?

2. What are the best (or, at least, most used) tools to use to source that information?

Types of Information

Before we examine how best to use the internet for political research, we should consider why you (as a former first year student) may have been advised *not* to rely on the internet to produce essays. Why would there be such an objection to using freely available information produced by such a wide range of people? Why, perhaps, would people criticise you for favouring the democratic, as opposed to the elite, production of knowledge? The answer is not that your older lecturers - who still remember the days of punch cards, rolodexes, hard copy journals and the need to speak to someone about borrowing books - resent the fact that you can get information so quickly without going to a library or even leaving your home. It is also not a form of group closure, in which we protect and promote our own people, methods and types of information.  It is not that lecturers associate undergraduate student internet research with a small bunch of people copy-and-pasting from Wikipedia into their essays (although this does happen, sometimes).

The more important explanation is that, in a world where the amount of information seems infinite, it may be increasingly difficult to identify the information to which we should pay most or least attention.  We need to identify some sort of hierarchy of important and/ or reliable information. This is a skill like any other. Your successful development of this skill will be reflected in your grades, since it will relate to the willingness of your essay markers to accept the information you present, according to the way in which you account for your information.

One way to address this issue is to consider why lecturers tend to treat books and/ or journal articles as the gold standard of information - at least as secondary sources, before you are expected to do your own research using primary sources such as government documents.

Possible answers (though not always useful answers) include:

1. They are peer reviewed. Lecturers may recommend journals that only allow publication after a number of relevant academics have commented on the work, often anonymously. The number of peer reviewers may vary from 1 to 6; in my experience an article is usually read by 2 or 3 anonymous reviewers. Their evaluation of journals may be linked partly to the reputations that some journals have in terms of academic rigour, linked to the need for scholars to anticipate and address critical reviews before having their work accepted.

2. They are competitive and much work is rejected. Some journals have an acceptance rate below 10%, which helps them develop an image of prestige and cutting edge research. (You can get a rough idea of acceptance rates in politics and IR here - ). Some people also put some faith in proxy measures of journal reputations, such as their ‘impact factors’ (although this is a problematic faith). It is also an increasing trend to evaluate the status of individual scholars according to the extent to which the publication is cited by other scholars (see below on Google scholar). Being well cited is a proxy used increasingly to gauge respect for the information.

3. They may be based on a long period of scholarly research. This varies from discipline to discipline. Consider, for example, the historical research produced over years after painstaking attention to thousands of documents.

4. The research may be theoretically informed. Being theoretically informed means being aware of the general implications of individual pieces of information. In part, this focus on theory is based on one role of scientific research: to draw lessons from sets of single cases to produce insights that may apply to many or all cases. A focus on theory is a focus on generalisation – something that is difficult to do if we rely only on information that is produced in very particular circumstances.

5. The research may be methodologically sophisticated. Most research is required to reach some sort of level of sophistication. For example, a quantitative survey requires a certain (large) number of responses to be considered statistically significant (in other words, for us to conclude that the results could not have happened by chance). It is then subject to a series of statistical techniques (which you may learn, using programmes such as SPSS) to explore the associations between variables. Qualitative research may be judged on different criteria, but there is a similar requirement that the conduct of the research meets certain professional criteria. The data may then be subject to further techniques to gauge its meaning and significance. It is often a condition of journal article acceptance that the scholars set out clearly their methods (and often provide a copy of the data for others to use).

6. The research may be empirically rich. Much research is based on surveys of thousands of people or qualitative interviews of dozens or hundreds. The data may be combined with documentary and historical analysis to produce a wealth of information. That information may be compared with information from other studies, to help accumulate knowledge within particular fields.

7. The authors may be meticulous when they identify the source of their information. One aim of the scholarly text is to show the reader where they got their information, to allow the reader to follow up, confirm and/ or read further. Most of you will have noticed the attention that we pay to your referencing style and bibliographies (hopefully most of us are looking for a consistent style rather than a particular style, although be suspicious of people who don’t agree that Harvard is best). This is because, when people make empirical or theoretical claims, the understanding is that they show us the information on which they based their claims (or, at least, they give us the option to follow up their work).

It is according to these kinds of criteria that we may judge other sources of information. In many cases, the information may be useful even if it does not live up to many or any of these criteria. For example, it is legitimate to use newspaper stories and commentary pieces, particularly if they provide much-needed timely information and the source is seen as reliable (indeed, although some newspapers now suffer poor reputations, we can still identify a tradition of fact/ source checking as a routine part of information gathering – partly, but not exclusively, because journalists are generally proud of their reputations and newspaper managers do not want to be sued). However, we would then have to consider the trade off against the academic ‘gold standard’ (is the information likely to be theoretically informed and based on a sophisticated method?) and consider the extent to which the trade is appropriate. In many cases, this just comes down to a mix of sources – student essays could benefit from immediate sources but those sources may not be an alternative to a more comprehensive review of the relevant literature.

However, this is not to say that academic information is unproblematic. We may be meticulous when we catalogue our sources of information, but that information may still be of varying value. For example, quantitative work may be limited by the availability of information provided by other actors (such as governments) and qualitative work may be limited by access to the right sources of information and simple things like the ability of interviewees to recall or provide an honest recollection of relevant information. The more difficult task, then, is to consider in more depth how people access and present information and how we might compare and critically analyse those sources to produce what we consider to be an accurate or valuable overall assessment of the available information.

Search Tools

We might divide that search broadly intro two categories: primary and secondary sources. I will a focus on Scottish politics to tailor the advice.

Secondary sources

The starting point for most students is likely to be a secondary source: you look through the existing academic literature for your information. Those texts analyse things like government documents, and you get your information about those documents indirectly, through a secondary source.

For me, the best way to start a search for secondary sources is to use Google Scholar ( You enter a small number of search terms and it produces a list of materials to consult. Most Universities also have the ability (particularly if you search on campus) to link the article access directly to the search. Your results are likely to reflect your search terms. For example, a search for ‘Scottish politics’ reveals a list of general texts, while a search for referendums in Scotland produces more specific texts. They tend to be organised according to the extent to which they have been cited elsewhere (which often produces a tendency for older materials to be listed first).

A good rule of thumb is that you should use Scholar more as you progress in your studies. The ‘further reading’ section of a textbook, or list of readings in a course guide, is essential when you are an early undergraduate. When you become an advanced undergraduate, and start to plan to write a relatively independent piece of work, you are expected to do your own searches for the relevant literature. This will require you to think carefully about your research question and the keywords you will have to use to get the most out of the search. You may also need to think about the sources of offer from Google scholar. There is now a wide availability of journals and books, but how do you prioritise and/ or determine the quality of the information?

Primary Sources

An independent project will also prompt you to seek primary sources of information, including government and parliamentary sources. This requires a bit more thought, since you are unlikely to get useful information unless you have first thought about what your research problem is and how you intend to address it. In other words, you think about what you want to know, what are the most appropriate methods to get the right information, and *then* do these sorts of searches.

Government and parliamentary sources

It is quite amusing to look through the Burnham et al (2008) 2nd edition of ‘Research Methods in Politics’ because it still talks about CD ROMS. These were growing in popularity when I was a student, but you may never have used a physical, round, disk to secure information (and may never have to). Instead, sources of government and parliamentary information tend to be available online directly from them (or, if you are that way inclined, through sites that claim to provide documents that governments don’t want you to see). For example the Scottish Government has a fairly extensive site ( which allows you to search using key terms. This tends to be less straightforward than a Google Scholar search, forcing you to be much clearer in your mind about what sort of information you want (since you will have to be fairly specific in your search unless you want to sift through a tonne of information). Similarly, the Scottish Parliament ( provides a written record of virtually anything that any MSP has said in a committee or plenary discussion since 1999, as well as a full record of written and oral evidence to committees. Again, there is too much information to browse, so you first need to think about what you want to know.


Lexis Nexis (which can be accessed through library resources) is a key archive of UK newspapers and can track the Scottish papers to some extent (it is OK for Scottish papers like the Scotsman, Herald, Daily Record and Press & Journal, but it does not store Scottish editions of the UK papers like the Daily Mail). The archive varies, with papers like The Times providing the longest stretches of data (the Scotsman has also begun to archive its really old material). Again, to make the search manageable (below, say, 1000 stories), you need to be very specific about what you want. For example, a search for Scotland AND referendum AND independence will produce thousands of stories which will take you days to get through (unless you focus on a short space of time).


Blogs are a minefield. Consider the extent to which they meet the gold standard criteria I outlined above. Some of them might do, but how would you know? I would treat blogs in the way that I might treat newspapers: you might trust them if they have a good reputation (but, then, whose opinions do you trust on reputations?). You should also expect biased, and often highly biased, opinions. That means that they can be a good source of information, but you might ask yourself if you can rely on a blog on its own, or as something to be compared with one or more sources. Interestingly (for me at least), along with my co-author on the 2nd ed of Scottish Politics, Neil McGarvey, I had to come up with a list of websites to check out. The idea is that they would be relatively useful, but is this list (below) particularly reliable? Or, are they simply the ones that came to mind at the time?



Twitter and other things

I tend to use Twitter as an alternative to the TV, as an additional source of entertainment. However, it is possible to sign up to a wide range of news and party sites and to use twitter as an alternative to reading newspapers page by page. This provides you with a new source of bias, but perhaps no more problematic than sticking with a paper like the Daily Mail. You might even simply follow specific lists (such as academics on twitter) or, if you are feeling particularly lazy, just look at the list of people/ organisations I follow and piggyback on that.  There are things that are particularly useful, such as the LSE blog sites and accounts such as ‘Writing For Research’ which you might find more useful if you progress to postgraduate work.

And finally ..

Here is a list of websites that Neil and I produced for our book. It is likely to be a bit scattergun and biased (for example, we don’t list small party websites), and some will already be out of date, but there may be some sources there that you wouldn’t otherwise consider.


(from the chapters of the forthcoming 2nd ed. of ‘Scottish Politics’ by Paul Cairney and Neil McGarvey)

The idea here is that these websites might get you started if you don’t want to rely on a scattergun search engine search. It is not a particularly well-thought-out list, so be careful!


Scottish Parliament
Report of the Consultative Steering Group 
The Economic and Social Research Council ‘Devolution and Constitutional Change’ research programme  and ‘The Future of the UK and Scotland’   
University of London’s Constitution Unit    
UK Politics page

Paul Cairney’s Blog:




Scottish Conservative Party  
Scottish Green Party  
Scottish Labour Party  
Scottish Liberal-Democrats
Scottish National Party



 UCL Constitution Unit -


Scottish Government

UK Cabinet Office 
Scottish Ministerial Code 



Convention of Scottish Local Authorities  
Friends of the Earth Scotland     
Scottish Council for Development and Industry
Scottish Trades Union Congress
CBI Scotland
Scottish Council for Voluntary Organizations 

NFU Scotland -

Scotch Whisky Association -

Educational Institute of Scotland -


Centre for Scottish Public Policy  
Centre for Public Policy for Regions

For SPICe summaries of all Scottish Government bills see


Cabinet Office
Scotland Office
Scotland Europa
Government EU Office
British/Irish Council


Her Majesty’s Treasury


Scotland Act 2012 Sewel Motion - Scottish Parliament Official Report 18.4.12 

Scottish Government’s Council of Economic Advisers Fiscal Commission Working Group -

Edinburgh University Blog -



Maybe you have read this far to see what we can say about Wikipedia. My view is that I am sometimes pleasantly surprised about what I see on some of those pages. However, you will find very few academics that will trust your information if your source is Wikipedia (partly because it is difficult to know who is providing the information, how they got it, how well they cite that information, and how easy the information is to edit and manipulate). Maybe a good rule of thumb is that you look at it for a short cut to information, but that you do not rely on it as your definitive source.