Can you help ONA do a better job?

I attended a fascinating roundtable last week on ‘open-source intelligence’ at the Office of National Assessments (ONA), a body whose mandate to ‘provide all-source assessments on international political, strategic and economic developments to the Prime Minister’.  As a recent US report describes the topic:

Open source information (OSINT) is derived from newspapers, journals, radio and television, and the Internet. Intelligence analysts have long used such information to supplement classified data, but systematically collecting open source information has not been a priority of the U.S. Intelligence Community. In recent years, given changes in the international environment, there have been calls, from Congress and the 9/11 Commission among others, for a more intense and focused investment in open source collection and analysis. However, some still emphasize that the primary business of intelligence continues to be obtaining and analyzing secrets.

As a tech-savvy bunch of readers, let me ask the question: if you were in the business of using publicly available data to fulfill ONA’s mandate, are there any novel or unusual sources that you would draw upon? This includes sources that would provide new information, as well as those that might help sift the wheat from the chaff.

(xposted @ Core)

This entry was posted in Uncategorized. Bookmark the permalink.

7 Responses to Can you help ONA do a better job?

  1. Anthony says:

    Decades ago historian AJP Taylor thought trained historians and journalists could dig out most of the information of interest to foreign governments from newspapers, magazines, telephone books, timetables and other sources in the public domain. A CIA experiment in the 1950s proved just that: a team of 5 Yale undergraduates managed to work out the entire US Army order of battle along with accurate appraisals of naval and air strength, all from documents in the public domain.

    This suggests the futility of much of what passes for espionage. The other sensible option is that suggested by Lord Egremont, secretary to Harold Macmillan: ‘Much better if the Russians saw Cabinet minutes twice a week. Prevent all that f***ing dangerous guesswork’

  2. John Quiggin says:

    Henry Stimson was right (modulo some obvious social changes) when he said Gentlemen don’t read other gentlemen’s mail.

    The crucial point is not to find better open sources but to abandon the futile search for covert sources with positive information content. Standard game theoretic reasoning shows that, under realistic conditions, no such source can exist.

  3. peter says:

    The key challenge for foreign intelligence agencies has never been finding information, but discerning intent. In the 1970s, CIA obtained a Soviet plan to over-run western Europe using only conventional weapons, and to aim to do so too quickly for the US to respond effectively. But was the plan genuine, or was it mis-information? And if genuine, when were the Soviets planning to execute it?

    Discerning intent means understanding the mind of your enemies (or potential enemies) and this requires deep knowledge of, and even sympathy for, their culture, society and history. Robert McNamara, in the film “The Fog of War”, argues that the US never had this deep understanding of the culture and society of Vietnam. Arguably, the West does not currently have this deep understanding of Islamic societies and culture.

  4. Kevin Cox says:

    ONA should not be about “secrets” in fact I would be surprised if it was. ONA is a different animal from what we normally think about as “intelligence” and if it isn’t then it has strayed from its mandate.

    I would think they should work closely with the people who are in the business of classification of which the main ones are the search researchers. I would be attempting to become a beta tester for Google. More importantly I would support and collaborate with companies in Australia that are in the search business. For example FunnelWeb is a company in Australia that ONA should support by asking them the same question. FunnelWeb can then apply its technologies to “the web”.

    There are researchers in Australia who are looking at other methods and I would be attempting to support and work with those researchers. For example, anyone working on graph theory and the properties of networks in its broadest sense would be the areas where insights into “what is out there” are likely to give interesting results. This research is really getting machines to help do assessments.

    However, the best source of untapped information for ONA is the government’s own data. ONA should be 100% behind Open Government 2.0 and in particular behind opening up data to the general population. Like all government the best use of resources is not in “doing things yourself” but in empowering the population to do the work for you.

    For example, if ONA worked on getting tax information available for researchers like yourself then that would achieve more than it trolling through data itself.

    So my advice to ONA if they want to draw on “novel and unusual sources” is to

    1. Work on the mechanisms that will allow citizens, as well as themselves, to do assessments by helping others get access to information.

    2. Support and encourage research in the areas of machine classifications and knowledge discovery.

  5. This reminds me of a story in Timothy Garton Ash’s book, The File. After the fall of communism in East Germany, he was able to get the file on him kept by East German intelligence when he had lived there. They thought he was working for MI6, and were trying to work out what he was sending back to London. In fact he was working for The Spectator magazine under a pseudonym, and all the East Germans needed to do find out what he was sending back to London was buy it.

  6. NikolasTM says:

    Thanks for an explanation, the easier, the better …

  7. Leon says:

    Some kind of natural language processing technology might help — like Columbia Newsblaster, which tries to provide coherent summaries of news stories from multiple sources.

Comments are closed.