As part of an Intelligence Advanced Research Projects Activity (IARPA) effort, AVI teamed with the Johns Hopkins University Applied Physics Laboratory (APL) to assist in designing and building an agile ATN experimentation environment.  ATN was built to host experimentation scenarios designed to evaluate technology systems under test (SUTs) while providing consistency and valid scientific method throughout the experimentation environment. Our team helped to design and develop the experimentation network command and control (C2) environment to support experimentation operations, designed and developed ATN C2 interfaces to SUT technologies and assisted in the design of measurement methods, scoring & metrics scorecards.

The Branching Interactive Graphic Stories for Cybersecurity Education (BIGSCE) project, sponsored by the National Science Foundation (NSF), enabled us to evaluate how to use branching, interactive web comics to teach middle school students about cyber security. Branching stories provide opportunities to explore cause and effect, and middle school students found the comic book style highly engaging. The project leveraged Comic-BEE, AVI’s cyber security education technology based on research sponsored by the Department of Homeland Security (DHS) Science and Technology Directorate, Cyber Security Division (DHS S&T/CSD).

BIGSCE informed the development of new approaches to teaching and learning cyber security: branching graphic stories can incorporate culturally relevant pedagogy, and therefore have the potential to reach a broader and more diverse body of students. The goal of BIGSCE was to explore how this innovative technology can enhance cyber security education, with the following aims:

1. Attract a broad and diverse workforce to the field of cyber security, starting in middle school.
2. Use expert-created comics to evaluate a learner’s knowledge, skill, and ability in the area of cyber security.
3. Deepen students’ understanding of cyber security challenges and consequences using a comic creation activity. BIGSCE will also contribute new understanding of how to effectively use branching, graphic stories as a learning and assessment tool for middle school students.

AVI partnered with Stony Brook University, led by Dr. Lori Scarlatos, Associate Professor, Department of Technology & Society. The free workshops were held at the Town of North Hempstead’s “Yes We Can Community Center” in New Cassel, New York.

The BIGSCE workshops covered topic such as:

• Privacy and keeping safe on the Internet
• The nature of cyber crimes
• Cyber security professionals, and how they solve those crimes
• Heroes in cyber security and STEM (science, technology, engineering, mathematics)
• Fact and fiction in television and movies about cyber security
• How to prepare for a career in cyber security

See the video with some of our workshop participants! 

“Branching Interactive Graphic Stories for Cybersecurity Education” (BIGSCE) was sponsored by the National Science Foundation, Award Number 1623131.

Camus (Cuh-moo) is a proof of concept that enables cyber impact assessment by modeling and automatically mapping cyber assets to the missions and users that depend on them.

Hybrid Analysis Mapping

Two methods for analyzing software security risks are dynamic application security testing (DAST), an outside-in perspective, and static application security testing (SAST), an inside-out perspective. Both have shortfalls. DAST findings do not give insight into the root cause, making remediation time-consuming. SAST tools give you full breadth, but warn of weaknesses that are not exploitable.

Correlating the results of SAST and DAST can overcome these individual challenges. AVI engaged in a DHS-funded Phase II SBIR program entitled Code Ray: Software Assurance Risk Management Framework for Hybrid Analysis Mapping, to produce a hybrid analysis method that can be incorporated into Code Dx and into the DHS SWAMP.

The end result of this R&D program (1) improved speed, accuracy and confidence in detection of vulnerabilities by cross-mapping and normalizing the output of hybrid techniques – dynamic analysis, dynamic tracing, static analysis, and static contextual analysis; (2) enhanced prioritization and mitigation of vulnerabilities by providing both the run-time context for those vulnerabilities and their mapping to security standards; and (3) improved the rapid comprehension and assessment of risks associated with vulnerabilities by delivering results in a simplified risk-management framework.

Common Weakness Enumeration Visualization

CWE-Vis is a resource made available to the software assurance community to explore and learn about weaknesses in software. It provides visualizations and interactions with MITRE’s Common Weakness Enumeration (CWE), an international standard created to define a dictionary of software weaknesses. The CWE greatly facilitates the discussion and collaboration in efforts revolving around the always-present quest to improve software systems. CWE-Vis helps users familiarize themselves with the CWE through visual means. High-level CWE structures and detailed relationship graphs are displayed visually helping users gain a better and quicker understanding of the CWE. Powerful filtering and searching is made available to allow users to focus in on the details of how weaknesses can be manifested in source code and ways to avoid and fix them.

The Curriculum to Broaden Participation in Cybersecurity for Middle School Teachers and Students (CyberMiSTS) project builds on BIGSCE, our prior NSF-funded research into cybersecurity education.  In CyberMiSTS, we are developing and disseminating a curriculum and resources for giving middle schools educators the knowledge, skills, and tools the need to teach cybersecurity.

During the CyberMiSTS workshop, educators  will gain insights from cybersecurity researchers and educators, and learn to use a variety of interactive instructional approaches for teaching cybersecurity, including Comic-BEE, Applied Visions Inc. web application for creating branching web comics aligned with a lesson plan. With basic curriculum, resources, and guidance from the CyberMiSTS team, each educator will create a customized lesson plan for cybersecurity and course materials targeted at their own students.

Educators are expected to use their lesson plan and course materials in their own classrooms to teach their students about cybersecurity, and share aggregate and anonymized classroom data with the CyberMiSTS team.

CyberMiSTS has the following research aims:

1. Integrate recent research-based understanding of cybersecurity into a curriculum that is accessible to teachers and their middle school students.
2. Encourage participation of a broad and diverse set of students in the field of cybersecurity by showing them how human relations play an important role in cybersecurity.
3. Identify what educators need to successfully teach cybersecurity in a middle school classroom and, based on this, develop guidelines and resources for disseminating this teacher instruction widely.

CyberMiSTS is funded by an award from the National Science Foundation, under the Secure & Trustworthy Cyberspace: Education (SaTC: EDU) Program, Award Abstract #1821757.

Applied Visions Inc. partnered with a team from Stony Brook University led by Dr. Lori Scarlatos, Associate Professor of Educational Technology, Department of Technology & Society, College of Engineering & Applied Sciences.

As part of an Air Force Research Lab research project, AVI worked to identify specific types of mission data needed to support the AFRL Cyber Experimentation Environment, determined and helped to implement the practical mechanisms for harvesting such data from a military exercise (the InterTEC Cyber Event), and developed mission-relevant datasets, based on data harvested from the ICE Exercise, to support a mission-centric testing environment for the CEE.

Systems administrators in large data centers often do not know the mission of the machines they manage; their strategy is, “Let’s pull the plug and see who calls.”

The problem is similar for computer security analysts – they do not know the cyber capabilities, mission, or users that depend on a machine that is attacked.

To assess the impact of a compromised or degraded cyber asset, analysts need to know:

• • Who relies on an attacked asset for their job?
  • What social network will be disrupted if an asset is compromised?
  • What organizational mission is impacted by the loss of an asset?
  • What other assets or cyber capabilities depend on an attacked asset?
  • Where is the attacked asset located in the physical or logical network?

This lack of contextual information makes it impossible to effectively prioritize cyber events or assess the impact of attacks. Because organizations do not have accurate mappings between mission tasks and the network assets on which they depend, decision makers cannot accurately assess mission readiness of their network assets or interpret operational impact of network failures or cyber attacks. Security analysts cannot report operational impact of cyber incidents or defensive countermeasures. Managers are unable to accurately assess mission or business risks associated with cyber incidents, or identify cascading operational effects from the degradation or loss of cyber assets.

A current and accurately map of the dependencies between mission tasks, information needed to perform those tasks, and the cyber assets that provide and communicate that information.

The Defense Advanced Research Projects Agency (DARPA) created a National Cyber Range (NCR) to protect and defend the nation’s critical information systems. The NCR revolutionized the state of the art for large-scale cyber testing and provided fully automated range management and test management suites to test and validate leap-ahead cyber research technologies and systems.

Goal Directed Task Analysis

The purpose of the NetDemon project was to identify the information real decision makers actually need, and how they use it. The goal was to identify that information in order to evaluate the environment in which these decision makers operate to determine how well the current technology meets their needs. Future designs can then be created that take those needs into account from the beginning, rather than add in critical features and integration later. To accomplish this, the NetDemon project documented specific information that was required for the operators to do their jobs, and how they used that information to address particular issues or inform decisions.

The first part of the NetDemon project was to determine the current decision model used by the client, and determine if the existing data sources and capabilities were adequate. In the second part of the project, we developed an idealized, future decision model that helped the client reach Information Dominance goals, and described the data necessary to support it. Once the model was defined, we identified technology insertion opportunities that provided new data sources and technologies to support the decision process, and to automate part of it.

DARPA’s Scalable Network Monitoring (SNM) program focused on the development and testing of gateway malicious activity detection devices. AVI teamed with Johns Hopkins University Applied Physics Lab (JHU/APL) to design, develop, and operate a cyber test environment that provides the capability to generate a sufficient volume of both benign and malicious activity to maintain full-line speed (from 1 Gbps to 100 Gbps) at the network gateway.

The Defense Advanced Research Projects Agency (DARPA) has created a Cyber Defense Program, responsible for developing the core computing and networking technologies required to protect DoD’s information, information infrastructure, and mission-critical information systems.

The Cyber Defense Program is developing network traffic monitoring techniques with performance and scalability orders of magnitude better than conventional approaches.

The focus of DARPA’s Scalable Network Monitoring (SNM) program is the development and testing of gateway malicious activity detection devices. Secure Decisions has teamed with Johns Hopkins University Applied Physics Lab (JHU/APL) to design, develop, and operate a cyber test environment, that provides the capability to generate a sufficient volume of both benign and malicious activity to maintain full-line speed (from 1 Gbps to 100 Gbps) at the network gateway.

Network engineers and analysts built a test network environment simulating a secure DoD network gateway and also designed test procedures, program metrics, and a corpus of malicious activity for use within the test environment. Through execution of a series of test scenarios, scoring of those results, and analysis of the performance of the systems under test, the SNM program will provide innovative network defense capabilities for the nation.

Applied Visions Inc. has built an SNM Status Board System for monitoring test execution. This system provides the ability to monitor the test network and display the performance of systems under test. The Status Board console is a Graphical User Interface consisting of six different information monitoring and management panels:

Traffic Panel

• Shows the quantity of throughput flowing through both the high bandwidth switches in both directions, for a total of four charts.

SUT Assessment Panel

• Displays the latest test results. It consists of three different regions: Activity Coverage, Cumulative Activity, and Current Assessment. The Activity Coverage and Cumulative Activity areas are both interactive charts, providing the user the ability to drill down or aggregate data of particular interest

Other panels include:

Management Panel

• • Provides the ability to monitor switch status, manage switch state, configure switch monitoring polling intervals, and manage and refresh data.

Scenario Panel

• • Allows the user the ability to add, modify, and delete scenarios related to a test event.

Reporting Panel

• • Generates reports from test event data that support traffic measurements and SUT assessments.

Systems Event Panel

• • Displays information related to the internal operation of the Status Board system.

Creating interactive visualizations and educational games for skill acquisition is hard, requiring subject matter expertise and interactive design skills. Instructional goals and specific subject matter details must be integrated with other information to develop interactive scenarios. To provide a more efficient turnaround of training games or visualizations, we are developing ShortCut, a web-based collaborative framework for rapid development of serious games. ShortCut’s goal is to facilitate knowledge exchange between subject matter experts and game designers, thereby streamlining the creation of serious games.  Using an interactive, web-based knowledge elicitation, ShortCut allows the instructor or subject matter expert to describe in detail the subject matter, and it also captures additional information needed by interaction and game designers, such as learning objectives and descriptions of real-world situations using the skill or knowledge.

The result of a research program funded by the Air Force Research Laboratory Human Effectiveness Directorate, Warfighter Readiness Research Division (AFRL/HEA) to develop new and exciting concepts for training the next generation of cyber warriors, SimBLEND facilitates  blending the delivery of computer based training (CBT) materials with interactive visualizations and serious games to create a web-based environment where learning is fun and skills can be practiced immediately. To demonstrate SimBLEND, we created browser-based, CND-themed casual games that focused on low-level cyber security concepts and tools. These games are simple to use and can be delivered as stand-alone games or integrated into a Learning Management System or other training environment.

SimBLEND was sponsored by Air Force Research Laboratory Contract number: FA8650-08-C6858.

AVI has developed several professional partnerships and teamed with leaders in the Software Assurance community.

The SWAMP is a forward-looking effort to improve the state of secure software development, particularly for open source projects. Developed by Morgridge and funded by DHS it was an effort that has offered its initial capability since early 2014. It is no longer operational.

Network Defense Visualization

VIAssist is a visual analysis platform to help network security analysts protect their networks. It was designed after a comprehensive cognitive task analysis of network defenders. It provides visual tools for the evaluation of network flow and security data. VIAssist presents multiple, coordinated views – highlighted or filtered data in one view is reflected in all other views – to provide different visual perspectives of the data. These views transform network data into a collection of interactive visualizations that make it easier to analyze data, to see patterns and trends, and to identify risks and actionable information.

The goal of WhyViz was to conduct a study of the effectiveness of visualizations on cyber operator performance during the early stages of incident handling, which require defense analysts to review an endless alert queue of cyber event data to identify, record, and report suspicious behavior or cyber events of interest. Most cyber security visualizations are currently used for historical analyses, however, as part of the WhyViz project, Secure Decisions developed visualizations to support the real-time, in situ processing as it is actually being performed by cyber operators to facilitate event detection and preliminary event analysis.

In Phase I of this Small Business Innovation Research (SBIR), we applied knowledge elicitation (KE) methods to define specific examples of cognitive work that occur in the early stages of incident handling that have the potential for being enhanced (faster, more accurate, more complete) through the use of visualizations by the cyber operator. This cognitive work took the form of the operator seeking to answer specific analytic questions using the available data, in a severely time-constrained work environment. A KE with domain practitioners revealed that cyber operators regularly ask fundamental analytical questions that cut across specific tasks and roles. Our research identified the type of information operators need to answer these questions, the visualization concepts that represent that information in a visual form that can be rapidly comprehended and acted on, and methods to transform raw cyber sensor data into a form that can be used to populate the visualizations. In Phase II of the project (in progress) we will conduct an experiment to objectively evaluate the effects of these visualizations on operator performance.

In Phase II of the project, we conducted an experiment to objectively evaluate the effects of these visualizations on cyber defense analyst performance.

Contract No: FA8650-16-C-6711
DISTRIBUTION STATEMENT A.  Approved for public release:  distribution is unlimited.  88ABW Cleared 02/08/2017; 88ABW-2017-0518.

Wireless Threat Visual Analytics

Critical infrastructures can be vulnerable to cyber attacks through 802.11 wireless networks. WildCAT is a prototype system that enables integration of physical and cyber security resources in order to detect and immediately respond to wireless threats. It is designed to use physical security vehicles as a platform to collect wireless activity and send the data via a cellular network to an analysis center.