Software and Automation Coding in UAS Reports

01

The question

UAS operations increasingly rely on software-mediated autonomy and command-and-control, and a review of ASRS-based UAS safety reporting identifies software and automation as recurring contributing factors that warrant further examination (Davis & Majumdar, 2026). My question was bounded. I asked whether the share of reports coded with the contributing factor “Software and Automation” differs across regulatory regime (Part 107, Section 44809 recreational operations, and public aircraft operations) and across UAS control mode (manual, waypoint, autonomous, transitioning, and unknown or not coded). Then I asked how large those differences appear once uncertainty is represented.

02

The data

ASRS is NASA's voluntary, confidential, non-punitive safety-reporting system. It accepts reports from UAS operators on events such as equipment issues, lost link and flyaways, uncontrolled descent, airspace incursions, miscommunication, procedural issues, and human error (FAA Safety Team, 2021; National Aeronautics and Space Administration, n.d.). I exported reports from ASRS Database Online as a CSV filtered to the three UAS regulatory regimes, which gave 497 reports. Each row is one report, identified by its accession number (ACN), so the analysis compares report-level proportions, not aircraft, operators, flights, or incident rates.

The outcome comes from a field ASRS already codes. A report scores 1 when its multi-label Contributing Factors / Situations field contains Software and Automation, and 0 otherwise. That keeps the outcome reproducible, but it also ties the analysis to ASRS coding practices. Of the 497 reports, 160 (32.2%) carry the code.

03

Method

I wrote the workflow as one commented R script that runs top to bottom from the raw export. It imports the CSV, cleans and recodes regulatory regime and control mode, derives the binary outcome, computes 95% exact binomial (Clopper-Pearson) confidence intervals for each subgroup, and draws a two-panel figure with ggplot2. A key design decision was to keep the report as the unit of analysis while turning a multi-label contributing-factor field into a reproducible binary outcome. I documented how the script handles the ASRS export's two-row header and addressed an R plotting deprecation warning.

Figure 1Software and Automation coding

04

What the figure shows

Software and automation coding appears more common in autonomous and transitioning control modes than in manual control, while regulatory categories appear less differentiated in this export than control-mode categories. In this dataset, control-mode categories show more descriptive variation than regulatory categories and warrant follow-up investigation. The pattern does not establish that transitions between manual and automated control cause safety events.

The confidence intervals help show which subgroup estimates are relatively precise and visually distinct, but I do not treat them as a substitute for a formal significance test.

05

Design decisions I would defend

I named the question type first. Leek and Peng (2015) argue that a common error in data analysis is mistaking the type of question being asked. This one is descriptive and exploratory, so the figure compares patterns with their uncertainty, and predictive modeling is positioned as future work.

The data follow Wickham's (2014) tidy-data logic, one report per row with a derived binary outcome. The multi-label ASRS field becomes one analytic variable, and the report identifier stays the unit of analysis.

Each grouping gets its own panel. Gelman and Unwin (2013) distinguish exploratory from presentation graphics and caution against overloading a single display, so regulatory regime and control mode sit side by side instead of in one crowded chart.

I chose a familiar graph form. The dot-and-whisker plot with intervals and sample sizes follows the reader-centered guidelines of Gillan et al. (1998), which include understanding the reader's task, using familiar graph forms, placing labels close to what they describe, and reducing perceptual effort. Those principles account for the horizontal group labels, the shared percent scale, the visible uncertainty, and the absence of a legend.

I made the uncertainty explicit. Cumming et al. (2007) describe error bars as useful sensemaking aids when their meaning is stated, which is why the caption says the whiskers are 95% exact binomial confidence intervals. Across these choices, the principle is the same, fitting the method to the question and making uncertainty visible.

06

Limits and next steps

ASRS is voluntary and de-identified, so the export captures reported events, not underlying incidence. The figure supports claims about patterns within this ASRS dataset and does not reflect the prevalence of software and automation problems across all UAS operations. Several control-mode subgroups hold fewer than 50 reports, which is why their intervals are wide. Inferential modeling, such as logistic regression or features drawn from the report narratives, is saved for future work with a larger dataset.

RepositoryR scriptASRS Database Online