There are at least four major methods for extracting data from a CUAS system. Which one you choose will depend on your use cases as well as your technical ability and relationship with the vendor. Most users will utilize the vendor’s user interface but even these users may want to understand what other data is available. Why? These systems are responsible for making life safety decisions. Validating the information provided by the user interface is desirable before an event occurs. If the system ends up as evidence post-event the validity of the UI’s data will certainly be examined then.
The four common CUAS data sources are:
- Vendor log files
- Vendor API
- Vendor UI
- Standard or proprietary C2 or integration layer
Vendor log files
For decades, log files have been the definitive source of truth when examining computer systems. They are often immutable and detailed sources of information about every aspect of the system’s behavior. And, for the same decades, making sense of the log files has required subject matter experts and, when done at scale, often expensive tooling.
It is important to understand that log files are often considered “internal use only”. The vendor creating the files has the resources to make sense of the data as well as internal processes and documentation to keep up when formats, units of measure, and sources change and evolve. External users of log files should expect a steep learning curve, particularly if the vendor is not engaged in the data discovery effort.
It is particularly important to understand how data is generated, the reference models (if appropriate), units of measure, and other contextual information. (See COMPARING CUAS TRACK DATA for examples.) False assumptions at this stage may fatally affect later analysis.
And, being “internal use only” there is no common log format. On a recent exercise we wrote custom parsers for each vendor’s log file format.
Logs are a poor mechanism for real time integration and analysis but they are arguably the best source of information for post-event analysis.
Vendor API
API’s have been the standard way to access or integrate with computer systems for years. In some recent discussions, I incorrectly assumed that everyone was familiar with the term and its significance. Occasionally I forget that I am way down in the weeds at times.
An API is:
"a set of functions and procedures allowing the creation of applications that access the features or data of an operating system, application, or other service."
In other words, it is a well defined, formal mechanism for requesting information from another system. Emphasis on well defined and formal. Vendors who publish APIs for their systems want users to engage via this mechanism and so invest a great deal of effort in making the process as efficient as possible. In particular, they are not subject to rapid change, poorly defined units of measure, and lack of structure. APIs exist to hide all of the ugly internals (as much as possible) and to make the data as easy to use as possible.
Please note – “hide the ugly internals”. The API will present the data, and the underlying systems, in a very controlled manner. Identifying issues with the system via the API is likely to be much more difficult than via log files.
For a CUAS test and evaluation exercise, the vendor’s API is likely to be the best source of real time visibility into the system’s behavior. However:
- If there are ten vendors, the data collection team will need to implement support for ten different APIs.
- Data preservation will be the responsibility of the data collection team. The same API call will likely return different results seconds apart and the previous state information is likely extremely transient
- The network bandwidth required to support some CUAS APIs is significant, more than might be available in a remote location
For real time situational awareness and operational uses, the API is certainly the best way to go.
Vendor User Interface
As mentioned, this is the most common method for engaging with a CUAS system. It is also the furthest from the truth, the raw data. Many decisions about data interpretation and presentation are made by the vendor prior to presenting it to the user, all intended to make the system as easy to use as possible. “Easy to use” generally implied “hide as much unnecessary information as possible.” What looks like a smooth curve of a CUAS track may actually be only three detection events connected by a mathematically smoothed curve. Sufficient for the operator, less so for investigators.
That said, the UI is a particularly rich source of information and the only source that captures the user’s experience. A screen recording of the vendor’s UI during evaluation events would likely provide valuable context.
C2 or Integration Layer
The gold standard for integrating CUAS systems or for monitoring them in real time is a C2 or integration layer such as the USAF’s Medusa, the Army’s FAAD C2, or Aunduril’s proprietary Lattice Platform. Unfortunately, these implementations are generally limited to DOD systems. The integration efforts required exceed the potential benefits for vendors targeting customers other than the DOD. A lighter weight (effort, complexity, bandwidth) integration standard might well benefit vendors and customers alike.
Conclusions
CUAS systems generate enormous volumes of data, much of it of no interest to many people. And for most people, the way to obtain data from CUAS systems will be obvious. But everyone should be aware of the fact that the easier it is to obtain data, the further you are away from the unvarnished truth. Conversely, working with very clean data, such as from a C2 layer, will often eliminate a lot of data collection and analysis issues.