Jan-April 2024, Sponsored by NSA Government
An Intelligence Analyst analyzes network traffic to identify threats, investigate attacks, and document findings in a repository to inform best practices for future reference.
This project addresses analysts’ challenges by integrating AI into their workflow, ensuring they maintain control while improving efficiency.
Target Digital Network Analyst: Investigates and analyses networks for security threats.
Tradecraft: Refers to the techniques, and methods to solve an investigative case.
Tradecraft Hub: It is a repository that stores all the Tradecraft reports of cases.
Lacks the modern interface
Closed platform with less third-party integration
Needs specialized training to use
Lacks query recording, making sharing difficult
Real-time monitoring and log analysis
Optimizing an LLM with repository data.
Offers deep insights into trends and threats.
Can integrate insights into other platforms.
Interviewed 6 target digital network analyst from NSA to understand their day to day and the challenges that they have faced:
“Analysts face challenges in keeping up with rapidly changing tools and data.”
“Often take notes, but sharing notes is rare, & when they do, it’s relevant info. for the team”
“Documentation of important queries which gave direction to the investigation”
“Interruption of work flow due to not knowing which data base to investigate on”
“Copying query and data to new spreadsheet tabs to document the process.”
“Often rely on coders for help write query, especially when they are new to the field.”
Switching multiple tool disrupt workflow.
Learning challenges from frequent software updates.
Keeping track of which query gave what out put.
Unstructured document hinder search.
Adjustable widget for quick access.
Query can be generated with natural language.
Analysts can log key queries in real time.
Fine-Tuning LLM with repository Data.
Analysts have multiple tools but need a unified solution to integrate them. Their current tools lacks cross-platform compatibility.
##
I used the user journey method, interviewing 7 analysts to identify pain points and gain insights into their ambiguous, situational tasks.
Identified 3 main problem areas from their user journey: reliance on multiple tools, inefficient collaboration, and disorganized documentation.
##
The interview identified three key problem areas in their current user journey, which are then elaborated upon.
A widget that integrates with all user tools.
Tool that track steps for mentor-mentee collaboration.
Tool that compile key info. in a standardized format.
A widget for managing individual tasks that integrates with all the tools users work on.
A tool that can record investigative steps that can help mentors to supervise and collaborate with mentees effectively.
A “document query feature” to help keep track of important queries and found data.
Compile the important info into a standardized document and can be shared with the team.
I used whiteboarding to foster brainstorming, enabling free-flowing discussions and idea generation. It also helped with visual project planning, mapping tasks, timelines, and dependency.
Leverage AI to enhance the mentoring process and automate documentation
A repository that documents the analyst’s investigative journey sequentially. It can be accessed by all team members at any point of the day via the “Spine” icon in the upper right corner of the window.
AI learns from the Senior TDNA’s input to further optimize queries and identify outliers. Implement AI-driven suggestions for datasets to initiate the query process, with flexibility for TDNAs to incorporate their own preferences.
Ability to adjust the time, date, and area range in the prompt by toggling the slider. Those will get substituted in a prompt and once the analyst is satisfied with the query, they can paste it into the analytic software.
Spine enabling senior analysts to track and assess the performance of multiple analysts simultaneously. It also provides an efficient way to identify tradecraft tendencies and detect outlier queries.
Since “Spine” is a complex visualization, we developed a set of identifiers for queries of different origin, importance and quality, as well as for datasets.
Better Collaboration: Standardized documentation improves knowledge sharing and teamwork.
Increased Efficiency: Analysts save 30-40% of time on manual documentation.
Smarter Decisions: Organized documentation supports better security operations.
Presented the project to NSA
