Threat Intelligence - Evaluating Sources | Optiv
May 16, 2014
One of the highest concerns faced by the intelligence analyst lies in knowing if the assessment they are making hits the mark. As the analyst struggles with determining the applicability and reliability of collected data, the timeframe in delivering intelligence results to stakeholders is extended, which lessens the usefulness of the results.
The usefulness of intelligence lies within the analysis and timely dissemination of collected data, and if the information is not judged or judged incorrectly, the impact it could have on the operations component it supports could be useless, or even detrimental. How do we address the need for reliable, timely, and applicable collection?
As we have seen in industry over the past couple of years, there is no shortage of available information that can contribute to intelligence analysis - threat and vulnerability feeds, dissemination of indicators of compromise via Mandiant’s OpenIOC or Mitre’s Structured Threat Information eXpression (STIX), open-source information sharing platforms and threat lists, etc. Combined, these sourcesprovide an enormous amount of collectable data that can be used in intelligence products or applied to signatures and rules in security appliances. The problem lies within the validity of the information available and whether or not ingesting said information is applicable to your environment.
Sources of intelligence information must be evaluated for their usefulness, fidelity and validity. Authenticating sources and classifying them by their reliability will allow for the analyst to make sound judgments and assessments of the data collected. The tables below provide an outline that analysts can use to grade sources and intelligence information.
Source Reliability Matrix
No doubt about the source’s authenticity or trustworthiness. History of complete reliability.
Minor doubts. History of mostly valid information.
Doubts. Provided valid information in the past.
Not Usually Reliable
Significant doubts. Provided valid information in the past.
Lacks authenticity, trustworthiness and competency. History of invalid information.
Cannot Be Judged
Insufficient information to evaluate reliability. May or may not be reliable.
Information Reliability Matrix
Logical, consistent with other relevant information, confirmed by independent sources.
Logical, consistent with other relevant information, not confirmed.
Reasonably logical, agrees with some relevant information, not confirmed.
Not logical but possible, no other information on the subject, not confirmed.
Not logical, contradicted by other relevant information.
Cannot Be Judged
The validity of the information cannot be determined.
Concerning the above tables, it can be assumed that one of the many sources of information that an intelligence team will utilize is in-house Technical Intelligence (TECHINT) from the security appliances on the network. The intelligence information collected from these sources can be deemed either A-1 to B-3, depending on the level of signature and rule tuning and the analysis of false-positives.
TECHINT is the collection of information about or via technological platforms. This data will be derived from the in-house technical resources, such as firewalls, proxies, intrusion detection and prevention systems (IDPS) and the entity’s Security Information and Event Management (SIEM) system, as well as other technologies used by the information technology group.
To be able to grade your sources and the information they provide, you must track and score them. By doing this over time, you will better be able to grade them and then come to a decision of whether to continue reviewing, evaluating and incorporating said information into your finished products. Just as you would tune out a rule that continues to provide false-positives in your SIEM, so you should with invalid sources of intelligence information.
The intelligence staff must be trusted to provide dependable and timely analysis. Basing assessments off of unreliable or invalid source information will likely lead to a decrease in the security of the organization’s information assets.
Intelligence Disciplines & Sources of Information
Intelligence information derived from human sources.
Data gathered from various communications mediums.
Geospatial intelligence concerning terrain and imagery.
Technical information collected from internal sources and platforms.
Intelligence information collected via publicly available resources.
The above table is provided to detail other disciplines within intelligence information collection where sources and information provided need to be graded for reliability and truthfulness.