Investigative Research

Investigative research is a name given to a collection of research techniques and methods used by researchers (including journalists, social scientists and others). It is intended to unearth secret, hidden or obscure information that can build a more comprehensive picture of the issue under investigation.

Orientation

Jack Douglas advocated ‘investigative’ social research in his 1976 book. He sums up the approach as follows: ‘conflict is the reality of life, suspicion is the guiding principle’. Lee criticises this by noting the potential for the scepticism necessary to ‘harden into cynicism and a contempt for those studied’.

Access

Access to information is a major problem for all researchers, but is felt particularly by investigative researchers who are more likely to come up against refusals and to challenge them. Powerful organisations often attempt to limit access by a variety of techniques. These do not always involve threats of violence as in this example:

If you try and inspect them, I will personally break your legs (Chairman of the company publishing Burke’s Peerage to journalists investigating lack of company documentation submitted to company’s House)

The Royal Ulster Constabulary was a notoriously closed organisation fr researchers and its policy on research issued after the 1994 IRA ceasefire made its instrumental view

We welcome requests... to conduct research which may prove to be of benefit to the force.

However, powerful organisations may allow social researchers access for a varety of reasons. Kevin williams writes:

In spite of the difficulties - and these are many and real - the powerful can be more open and co-operative than many social scientists believe. They are often prepared to discuss matters and in many cases welcome the chance to place their views on the record. their motives are mixed. they can emanate from a desire to correct what they see as misconceptions of their role and work. ... Talking to a researcher appears to be one of the few channels of communication they have with the public. The powerful also talk to the researcher to counter challenges from other interests within their institution. Powerful institutions are not monolithic. a large number of interests exist inside institutions... which are in a state of flux and change. Such a situation can work in the researcher’s favour.

Covert methods

Gunter Walraff is a German investigative journalist who specialises in going undercover to reveal abuses of power. His work is excoriated by corporate lobby groups such as the West German Employers association:

When [Walraff] describes an industry on the basis of his ‘research’, his writing is characterised by a consistent scale of social values which could by fashioned only by a conscious ideologist of class struggle. Each of Walraff’s publications reaches us as a hatefilled social-political campaign aimed at strengthening the machinations of class struggle. His purpose is to arouse among workers by hand and brain a class-consciousness which they will ultimately use to destroy the social system. his methods of investigation and documentation must be categorically condemned; the logical consequence of his point of view is that the end justifies any means and that all sense of responsibility is lost.

Resources

• Australian Centerfor Investigative Journalism Investigative Research, University of Technology Sydney.
• BBC Investigative Research on the Net, BBC Training course.
• Dave Winer How investigative research happens in the blogosphere, Scripting News,Sunday, January 11, 2009.
• Zilliox, Jr., Larry (2006) The Opposition Research Handbook: A Guide to Political Investigations, 3rd Edition, ISBN: 0-9718740-1-8
• Investigative Research Specialists LLC The Opposition Research Training Blog: Tips and Resources for Opposition Researchers
• Investigative Research Specialists LLC Research Books For Professionals

Invisible or Deep Web

• Invisible or Deep Web: What it is, How to find it, and Its inherent ambiguity UC Berkeley - Teaching Library Internet Workshops
• Wright, Alex 'Exploring a 'Deep Web' That Google Can’t Grasp, New York Times 2009-02-22.
• Michael K. |last=Bergman | title = The Deep Web: Surfacing Hidden Value | journal = The Journal of Electronic Publishing | year = 2001 | month = August | volume = 7 | issue = 1 | url = http://quod.lib.umich.edu/cgi/t/text/text-idx?c=jep;view=text;rgn=main;idno=3336451.0007.104 |doi=10.3998/3336451.0007.104
• Garcia, Frank Business and Marketing on the Internet Masthead volume 9 issue = 1 January 1996
• Lesk, Michael How much information is there in the world?

Sriram Raghavan, Hector Garcia-Molina Crawling the Hidden Web Stanford Digital Libraries Technical Report, 2000

• Ntoulas, Alexandros, Petros Zerfos, and Junghoo Cho Downloading Hidden Web Content UCLA Computer Science 2005
• Luciano, Barbosa and Juliana Freire An Adaptive Crawler for Locating Hidden-Web Entry Points WWW Conference 2007, 2007
• Luciano, Barbosa and Juliana Freire Searching for Hidden-Web Databases. WebDB 2005, 2005
• Jayant, Madhavan, David Ko, Łucja Kot, Vignesh Ganapathy, Alex Rasmussen, Alon Halevy Google’s Deep-Web Crawl VLDB Endowment, ACM, 2008

Cohen Laura Internet Tutorials: The Deep Web

• Barker, Joe (Jan 2004). Invisible Web: What it is, Why it exists, How to find it, and its inherent ambiguity UC Berkeley - Teaching Library Internet Workshops.
• Gruchawka, Steve (June 2006). How-To Guide to the Deep Web TechDeepWeb.com, http://TechDeepWeb.com
• Hamilton, Nigel (2003). The Mechanics of a Deep Net Metasearch Engine - 12th World Wide Web Conference poster.
• Template:Cite conference
• Bin, He, Patel, Mitesh; Zhang, Zhen; Chang, Kevin Chen-Chuan Accessing the Deep Web: A Survey Communications of the ACM (CACM), 94–101, 2007 May volume 50 issue 2 doi=10.1145/1230819.1241670+
• Panagiotis G. |last=Ipeirotis |coauthors=Gravano, Luis; Sahami, Mehran | year = 2001 | title = Probe, Count, and Classify: Categorizing Hidden-Web Databases | booktitle = Proceedings of the 2001 ACM SIGMOD International Conference on Management of Data | pages = 67-78 | url = http://qprober.cs.columbia.edu/publications/sigmod2001.pdf%7Cformat=PDF
• King |first=John D. |coauthors= Li, Yuefeng; Tao, Daniel; Nayak, Richi | title = Mining World Knowledge for Analysis of Search Engine Content | journal = Web Intelligence and Agent Systems: an International Journal | pages = 233–253 | year = 2007 | month = November | volume = 5 | issue = 3 | url = http://sky.fit.qut.edu.au/~kingj2/downloads/king07mining.pdf%7Cformat=PDF
• McCown |first=Frank |coauthors= Liu, Xiaoming; Nelson, Michael L.; Zubair, Mohammad | title = Search Engine Coverage of the OAI-PMH Corpus | journal = IEEE Internet Computing | pages = 66–73 | year = 2006 | month = Mar/Apr | volume = 10 | issue = 2 | url = http://library.lanl.gov/cgi-bin/getfile?LA-UR-05-9158.pdf |format=PDF| doi = 10.1109/MIC.2006.41
• Price |first=Gary |authorlink=Gary Price |coauthors=Sherman, Chris |title=The Invisible Web : Uncovering Information Sources Search Engines Can't See |year=2001 |month=July |publisher=CyberAge Books |location= |isbn=0-910965-51-X
• Shestakov, Denis (June 2008). Search Interfaces on the Web: Querying and Characterizing. TUCS Doctoral Dissertations 104, University of Turku
• Wright, Alex (Mar 2004). In Search of the Deep Web, Salon.com, http://www.salon.com/tech/feature/2004/03/09/deep_web/
• Firms Push for a More Searchable Federal Web - article in the Washington Post, Thursday, December 11, 2008; Page D01 by Peter Whoriskey.
• DeepDyve A research search engine to access content in the Deep Web
• DeepPeep a search engine project to "discover the hidden web"

Further Reading

• Cohen, S. (1988) ‘The last seminar’ in Against Criminology, New Brunswick, NJ: Transaction books.
• Lee, R. (1993) Doing Research on sensitive topics, London: Sage.
• Peter Phillips, August 14, 2003 'Inside Bohemian Grove US Elites Celebrate Patriarchy, Racism and Class Privilege', CounterPunch http://www.counterpunch.org/phillips08142003.html
• Phillips, P, A Relative Advantage: Sociology of the San Francisco Bohemian Club A Doctoral Dissertation (1994) http://libweb.sonoma.edu/regional/faculty/phillips/bohemianindex.html
• Scott, J. (1990) A matter of record, Cambridge: Polity.
• Simpson, C ‘Scholars Perfect Psychological Warfare Techniques’ Excerpt from Science of Coercion: Communication Research and Psychological Warfare, 1945-1960 by Christopher Simpson (New York: Oxford University Press, 1996), pp. 48-51: http://www.cia-on-campus.org/social/simpson.html
• Walraff, G. (1978) Walraff: the Undesirable Journalist, London: Pluto.
• Williams, K. (1989). Researching the powerful: problems and possibilities of social research. Contemporary Crises, 13(3), 252-274.

See Also

• Power Structure Research | Social Network Analysis |