|
Reuser's Information Services
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Next course... ||| To begin... ||| Flexibility / modularity ||| How to compose a course from the modules ||| exercises ||| Kinds of courses ||| Difficulty level ||| Interactivity ||| Handouts ||| Hardware ||| Contact |||
|
Open Source Intelligence Training
curriculum Arno H.P. Reuser (a@reuser.biz) Version 6.2 published 30 June 2011
In conjunction with IHS Jane's Strategic Advisory Services we have developed a special OSINT Methods and Techniques Training Programme derived in part from my curriculum, with added modules on analyses techniques. The training aims to enable open source researchers and analists to more effectively collect and analyse information in order to produce better quality intelligence reports. FFigure: Training location in Washington D.C.
The JSAS/RIS OSINT Methods and Techniques course is a modularised approach to identifying overarching methods, best practices, considerations, challenges and tools available to open source intelligence analysts. This course provides an overview of the current OSINT landscape and demonstrates how to best access and analyse the most appropriate resources to fulfil OSINT requirements.
Availability The course is available for open registration. Alternatively, the course can be adapted to specific client needs to be conducted for a closed group of participants from a single organisation or group at the client's premises. Methodology The workshop is highly interactive consisting of theory mixed with practical exercises, both individual and group. The desired outcome of each module will be clearly defined at the beginning of the module, and evaluated at the end of the module to make sure participants are on track. Participants will have full access to the Internet using laptops (take your own if you wish) for exercises and notes. All lessons, modules, exercises (and answers) will be available in handouts for later study. Please note that tools, websites, sources and software used during the course are available on the Internet or elsewhere free of any charge. There is no need for participants to buy software/sources/tools after the course. The operating system and office software are standard products found in most environments today (Microsoft). Teachers Principal teachers will be Terry Pattar (IHS Janes) and Arno Reuser (RIS). Intended audience The programme is aimed at: Course programme, registration and fees Here is a link to the full text of the extensive OSINT Course Programme. Here is a link to the registration forms and information regarding bookings. Enquiries For enquiries and further information, please contact Lynne Samuel at Lynne.Samuel@ihsjanes.com (Lynne.Samuel@janes.com) or call: +44 (0)20 8700 3772 . Training schedule Please find below the OSINT Methods & Techniques Training Programme schedule for 2010. If the below does not work, please click here to get the course schedule. edited on 5feb11 by ar
Is it possible to enter a query in a popular Internet search engine and not get 10 million hits but just 200 or less, most of them relevant to the question? Is it possible to make sense of the bewildering array of new information channels such as RCC, Blogs, Podcasts etc. and know or at least understand their place and function in finding just that piece of information we are looking for? Information search strategies for information professionals and beginners are much more complicated and involve much more activities, competencies and techniques than simply 'searching'. From analyses of the patrons information need, breakdown of the information problem, up to reporting about the search results and presenting the result in a orderly manner. A course on search strategies should actually by called something like Information Capabilities Course. The OSINT training course is designed to be completely modular, and addresses (almost) all capabilities and techniques necessary in todays information environment. Since not all clients - except information professionals - really need knowledge of every single module, the course is setup to be modular and flexible, based on many years of practical experience in librarianship and Open Source Intelligence by the tutor while teaching to an wide variety of international users, ranging from military intelligence personel to information professionals working in intelligence.
Since most clients have some knowledge about the art of information achieved through some course or simply acquired on their own, the difference on knowledge levels with clients is typically substantial. Also, organisations are all different, and not all parts of the course apply to everybody.The curriculum is therefore designed for maximum flexibility. It is up to the customer to decide on
Some courses are well suited to be followed on an standalone basis, like \textit {Safe(r) searching for searchers} or \textit {Webpage design in HTML}, but others logically belong to eachother in order to give it some meaning. A Search Strategy Seminar is therefore completely negotionable for the user. A base package may be enhanced by other courses to reflect the particular interest of the client.
A course can be composed based on available attendants and time and off course depending on the profile of interest of the client. Best thing to do is to mail these data to the author for a proposal. For the curious, here are some examples. For higher management and budget holders, a good choice could be Open Source Intelligence on page to get a feel for what OSINT is an why it is needed in an organisation. For attendants working in an information environment wanting a general 1 day course, a good choice would be to start with important Internet backgrounds and services and continue with the entire group 2 module. A two day course, including exercises, may consist of Important Internet backgrounds}, followed by all modules from module group 2 searching & finding, then to finish off with Safe(r) searching to address issues of security, Validating information to address the issue of reliability and validity of information, Browsers to know what browser to use and how, and Tools for Searchers to give the attendants food for (further) thought.
Each course is, depending on the circumstances, mixed with exercises. The course finishes with a class-room test where attendants have to find the answers to questions by applying what they have learned.
In general, there are three kinds of courses:
The difficulty level depends on the audience and customers requirements. I so desired, theory can be shortened and practical exercises extended. In general, some experience with searching is expected. For the exercises, some (day to day) experience with desktop computing in general is required.
The courses are interactive. Attendants ARE expected to actively participate in the proceedings.
Handouts will be provided, as well as some online tools to be used for searching. All material used will be available after the courseday, online, at http://www.reuser.biz/ where a suitable path will be created to store all informtin pertaining to the particular course, inluding the agenda. All material will be provided in XML, HTML or PDF.
As far as facilities is concerned, it basically depends on the type of course choosen and the type of curriculum composed by the client.
Please contact us if you have any questions: Arno H.P. ReuserAddress : De Wetstraat 16, 2332 XT Leiden, The Netherlands Phone :+31-(0)71 572 77 57 GMT-1 ; mobile +31 (0)6 3812 7715 GMT+1 Net: mail: a ATSIGN reuser DOT biz ; www: http://www.reuser.biz
Content: The OSINT module is devoted to the phenomenon of OSINT itself. It discusses what precisely OSINT is, it's place in the overall Intelligence Cycle, its functions and roles. In order to fully understand this, the phenomenon of 'Intelligence' and how that is interpreted will be highlighted. The OSINT module discusses the significance of Open Source Intelligence for the enterprise organisation (which are not necessarily intel organisations but can be commercial as well), its place in intel organisations, its place in the Intelligence Cycle and cooperation with other INT's, problems and challenges. The main focus is the issue of what the mother organization expects OSINT to do, what does the organization expect OSINT NOT to do, where can OSINT be successfull from the viewpoint of the organization, what is the future expectation? Participants will make notes and develop a clear understanding of the clients Wish List.
Content: Based on the clients 'wish list', attention will be paid to how to organise an OSINT department to meet the needs of the customer. The module will discuss the functional and technical requirements, how to set it up, general OSINT policy, personnel, training, financing and budgetting and all the supporting issues required. The exercise will have participants in groups write down an OSINT model based on the discussion in this module, describing all aspects of an OSINT department.
Content: This module highlights very important characteristics of the Internet essential for the serious searcher to get a better understanding of the nature of the Internet and especially its shortcomings and drawbacks. The issue of privacy and anonymity while searching is addressed, the organisation of the Internet, addressing protocols, who 'owns' the Internert, why the Internet may be InterNOT, manipulation, a short history, the coverage of the Internet in terms of connected users and countries, and the importance and relevance of several services and tools (HTTP, FTP, NNTP, Telnet, Traceroute, Ping, etc.) some of which will be exercised.
Content: This module will give a detailed overview of safety aspects of todays searching that are considered to be essential knowledge before starting to use the Internet. A wide range of issues will be addressed, ranging from a general overview of dangers treathening the searcher to e-mail identification, reverse IP engineering, phishing techniques and how to identify these, protecting the searcher's machine from intruders, password hacking and HTTP traffic intercept. Many tools will be discussed and used that should be part of the searcher's day to day toolbox.
Content: Are there any ways of determining the validity or reliability of information found on the Net? This module will discuss the issue of reliability of Internet information and will teach the attendants to be critical when judging information found on the Net, and a checklist to use when validating information.
Content: Which are the better or best search engines and are there any tools out there to assist the searcher in achieving better results? This module will give an overview of the main line of Internet search engines and discuss what things to look for when selecting a favourite search engine, differences between search engines, criteria, the functionalities offered by search engines that are really useable for intelligence, wich extensions to load or which add-onns to install, the overlap between search engines, which ones are the most reliable and which ones are not, is Google the only one or are there any alternatives that should be taken into consideration.
Content: Since browsers are the main instruments to use when searching, what tips and techniques are there to use a browser more efficiently? What's the best / worst browser and which one is best for me? What are the most important Add-ons and extensions to use to be able to work much faster? This module will show the tricks of the trade of the three main browsers used today.
Content: Presents three case studies of solving a particular information problem using either the Internet or a combination of commercial information providers plus the Internet. Each case puts everything that is discussed in module group two in practice in order to show attendants exactly how all theory comes back in real-life practice.
Content: Some tools for searchers usefull during searching or for keeping up to date with developments in the field. The exercise will give participants the opportunity to study the New Repertorium to make efficient use of this tool later during the course. The modules hereafter will teach attendants a systematic methodology of finding information via on the Internet as well as conventional libraries. The systematic approach consists of five steps (one module for each) that logically belong together, in that order.
Content: This module will present a general layout of the search process, loaded with tips, things to do and things NOT to do. Intended to give the attendants a backbone for the remainder of the modules in this group.
Content: How do we translate an information need into something with which we can effectively use a search engine? How do we know when to stop searching? This module teaches the attendants to first analyse the question or information problem at hand before going online by applying a Building Block analyses. Discusses types of questions and a universal method of analyses of the question by breaking down the question into several components or concepts. The result will be an anwerable question, a clear indication of knowing when to stop searching and a Quick Reference Sheet of the components of problem reconstruction.
Content: Is there some systematic way into which information worldwide is organised? Or do we simply make use of whats out there and hope for the best? This module will give an overview of the organisation of information, how information in general is organised worldwide, addressing printed sources, online sources and (free) Internet sources covered in more detail in Module Group 3. A 'landscape' of information will be presented listing primary sources, secondary sources and tertiary sources in the domain of Internet, commercial information providers and traditional information providers. In order to find answers to questions, searchers need to know wich type of source offers which kind of information.
Content: Do we want millions of hits of low relevance, or do we want just a couple of hundred hits with most of them of high relevance? Simply entering terms into a search engines often leads to unexpected results, if at all. Are there any ways to do ''smart'' queries in search engines? This module will teach attendants what search engines have to offer to construct really smart(er) queries; how to search with Boolean Logic; how to use search delimiters; and a basic understanding . the module addresses Internet search engines as well as commercial information providers. Most search engines have some sort of retrieval language that - once known - that makes it possible to enter pretty smart queries that lead to much better and fewer results. Commercial vendors use search languages with almost endless possibilities. This module will teach Boolean Logic which is the underlying technique used in almost every retrieval language today. It will teach attendants how to construct queries the smart way, to achieve much better results than normally. Also techniques how to change queries to get fewer or more results, techniques how to increase the relevance of the results.
Content: How to choose the correct keywords to search with and how to construct keywords to get the most out of the database being searched. This module will teach in total ten more general purpose tactics to improve search results by discussing the ''Semantic Table of Ten'' : which are ten semantic laws to consider when formulating search terms. Each ''law'' can greatly enhance or reduce search effectiveness, and is essential for each searcher.
Content: Are there any general-purpose approaches to searching, or, universal search strategies that maybe applied anywhere? this module will discuss the four main search strategies (as opposed to search tactics, which is of a slightly different level), when, and how to use them. A general methodology will be presented and exercised that is most usefull for any information problem, where attendants are urged to change their mindsets and adopt a more systematic way of searching instead of randomness. Particular attention will be paid to things that can go wrong and how to prevent these. Each of the four strategies will be discussed, explained, and during the exercises compared to eachother to found out wich strategy is the best given a certain information problem.
Content: How is a library is organised and is it usefull for intelligence? how to make the best possible use of library services, differences between types of libraries; the system of national bibliographies and when and how to use these; interlibrary loan systems to get the info. Several types of online catalogues will be researched to assess their value for intelligence. The worlds most important libraries will be discussed and assessed for relevanct for intelligence.
Content: when Internet search engines are for free, is there still need for premium content providers such as Lexis-Nexis? How can these expensive services be used to actually save money? This module discusses commercial information vendors (like Factiva, Dialog, Lexis-Nexis, Jane's Information Group), some reasons why to search commercial vendors when the Internet instead is for free, and how to set up a search. While the interfaces and organisation and off course the coverage of the main vendors are completely different, there are still many similarities and it is possible to lay out a general approach to setting up a decent search in any of the vendors. This course will discuss these similarities and will learn how to handle a search in a commercial vendor, paying attention to the tricks of the trade.
Content: Multimedia is consists of audio, maps, video, film, satellite imagery etc. This module will discuss the significance of each type, it role in intelligence, and tools and search engines best suited to retrieve these formats of information. The exercises will show which search engines or tools to use for each multimedia type of information.
Content: Will address what country information is available, how to retrieve country information from a variety of sources, using mainly free, but also fee-based sources. Country information is mostly the basis of any analyses, and in this module the different aspects of 'country information' (health, order of battle, government, demography etc.) will be defined, and for each apect, how to find and retrieve the best possible information.
Content: The sheer number of 'news' on the Net is overwhelming, and finding out which 'news' is really news and which news is just a copy can be a daunting task. This module will concentrate on how to find out how to use the least amount of tools and efforts to get the most out of the news. The system of news production will be addressed and attendtion will be paid to the main providers, the aggregators and most important news search engines. The exercises are concerned with examening the several news content providers and comparing their relevancy.
Content: Rich Site Service or RDF Site Service or Real Simple Syndication are some of the names for the same thing: a fully automatic technique to get news and other information downloaded every so few minutes and orderley arranged in any way. RSS is can be very convenient to get ahead of the news, but due to the overwhelming amount of channels available, it can also be time consuming. This module will discuss what exactly RSS is, why we need it, the significance of RSS and how most effectively make use of its services. What channels are out there and how can these be searched? What search tools are there for searching RSS channels and RSS contributions?
Content: UseNet is one of the oldest services on the Internet and today is about 75.000 discussion groups on about every conceivable subject or issue. This module will discuss the phenomenon of UseNet or Newsnet as it is sometimes called, and what it looks like. Then, an analyses will be made of which discussion groups are of interest; the reliability of the items published; and how to utilize Usenet in daily work. Pros and cons, do's and dont's of Usenet for intelligence will be addressed. The exercises will compare information found in Usenet with other sources to judge the relevancy of Usenet.
Content: A companion to UseNet, Listserv discussion lists send all contributions to its subscribers. There are thousands of different Listserv discussion lists about many different subjects and issues. Further to Usenet, a listserv typically also has a database that may be querried, and other functionality. This module will explain how to find out which Listserv discussion lists there are, how to find them, how to search a listserv database, and how to suscribe, unsubscribe and change settings. The exercises are devoted to how to use a listserv efficiently with do's and dont's.
Content: The Deep Web or Hidden Web is the part of the Internet that search engines cannot see, for one reason or another. The Deep Web is also about 500 times larger than the entire 'surface' web. This module will discuss this phenomenon and the reasons why such large parts of the Internet cannot be searched. Special tools and techniques will then be addressed and explained on how to circumvent the problem. The exercises will show and use tools to dig into the Deep Web and retrieve information impossible to find through a normal Internet search engine.
Content: Patents are a very special kind of information that is difficult to use and search, but can contain a wealth of information. In this module the significance and system of worldwide patenting is explained, what kind of information may be found in patents and how to retrieve them. Databases and tools will be discussed and used during the exercises to search for patents and to judge the relevancy for a particular information problem.
Content: Searching for information based on a particular publication to find relevant information that is newer in time. Citation searching and snowballing (the inverse technique) is still an important tool to find (mainly) scholarly information and this module will explain the technique and when and how to use these. Tools (free and fee-based) will be used during the exercises.
Content: Retrieving information is one thing, locally storing and organizing the retrieved information such that the info can also be found back on a enterprise intranet is another thing. This module will explain - drawing heavily from the library sciences on subject indexing - how to store and organize information quickly with valuable lessons learned and usefull tips to prvent fatal errors. More specifically, the fundamentals of designing a taxonomy are explained, the principles of building a keyword index based on the taxonomy, and the art of Tagging documents. All three are aimed at the work floor and very practical, with exercises clearly showing participants what can go wrong and how to do it the proper way.
Content: This module explains how to built a personal source book holding all important sources, catagorized in ant way, with descriptions and notes and off course links to the Internet. During searching, the searcher will find many interesting and important sources and links that need to be organised and arranged somehow so they can be useed the next time without having to do the work all over again. The technique learned in this module is completely independent of any computersystem or platform, and is also fully independent of software versions. Thus, the Book of Sources can be used anywhere anytime. The Book of Sources will be used during the entire course.
Content: The Internet is cluttered by many different kinds and types of formats in which information is presented, where in any typical enterprise intranet only a few of these formats are supported. Thus, if the searcher is not carefull, there is a big chance that retrieved information once downloaded turns out to be unreadable. In this module participants will get an overview of the most commonly used formats used on the Internet, how to recognise them and how to reformat them in a format understood by the enterprise intranet. Particpiants will learn how to handle tools to dearchive Internet archived material and how to demime e-mail attachments. Discusses text document formats such as RTF, DOC, XML, HTML, ASCII, PDF, etc., video formats like WAV, AVI, etc), archive formats (ZIP, ARJ, RAR) and MIME formats used in email and email attachments.
Content: Quit often, not just one document is relevant, but the entire page with all links on it is relevant. But downloading all those links by hand is a cumbersome task that takes a lot of time, since the downloaded links all need to be relinked by hand. This module will discuss and practice Offline HTML Webbrowsers, tools that will download complete websites or parts thereof, for offline viewing on the enterprise intranet using Teleport Pro or Webcopier.
Content: This module will teach technical information professionals some techniques and tips on how to design a website (in HTML) the smart(er) way. . The module focuses on the technical backgrounds and tools (thus, not on shodesign) available in any environment, regardless setup. Attention is given to how to organise content for web publishing; how to design and use Cascading Style Sheets (CSS); and the basics of Javascript for information workers. The module will show how a basic website consists of all three elements (HTML, CSS and Javascript).
Content: this module will teach the technical information professionals how to use PERL to organise information on a file system and how to program scripts in PERL to maintain websites. Specifically, it will teach attendants scripts to extract information from text files and reorganise that information in another way, for instance into HTML clickable websites to access information more efficiently. Although programming can never be learned in just a couple of hours, the attendants will after this module have a basic understanding of the applicability and complexity of programming as a bases for further exploration.
Content: XML is a much smarter and more efficient way of storing and desribing information than ''good old'' HTML XML also has more flexibility. Once the basics are mastered, the user will from than on not write in any other format than XML, also due to its flexibility. This module will teach the attendant how use XML, how to create XMl documents and how to design / write XSLt programs to transform XML into for instance HTML for the Web. The handon exercise will use ready made files to practice with.
Content: Vendors offer structured information enriched with metadata and provided in online databases with a highly sofisticated search language and often tools for analyses of search results. When used to the search language and design characteristics of the database, searching vendors can give much quicker results and at the same time much more relevant results than searching the free-for-use Internet. Sometimes, the searcher can process queries and get results that are impossible to get via an Internet search engine. This module will thus explain the What & How of commercial information providers and show the difference in results, time, costs and relevance with searching for the same subjects on the free Internet. Databases from Dialog Corporation are used as the prime example and testcase.
Content: Searching is an iterative process that demands heuristic decisions based on the original question and the results of the query. After the first step, the results are most likely not precisely what we want. Thus, every step is logically aimed at improving search results from the previous step one way or the other. This module will teach participants techniques how to improve the results of a search query. It will teach how to increase the number of relevant results if that is too low, and it will teach how to decrease the number of results if that is too high. The module will start with a list of common errors typical for searching.
Content: A warm welcome to the training and general kickoff of the course. Introduction of the teacher(s): who is he? Introductions of the participants and introduction to the programme: its outline, goals and desired outcome. Also: a (technical) keyboard test : make sure the keys on the keyboard matches the definitions, or, can you type /, ? [ : ; \ @ # $ & etc.
Content: Wrapup of the OSINT Course, closing remarks, comments of the participants, further action, evaluation. |
| page uploaded |
© Reuser's Information Services 2005-2011, Leiden, The Netherlands.
Please contact Arno H.P. Reuser (info ATSIGN reuser DOT biz) for further information. Registered KvK 2731-2325 Leiden |