A Needle in an Underwater Haystack: Machine Learning and the U.S. Navy’s Hunt for Submarines
Submarines are becoming more numerous and quiet as an era of Great Power competition heats up. How is the U.S. Navy adopting machine learning to combat this new threat?
During the Cold War, the U.S. Navy was at the forefront of employing some of the earliest machine learning and computer assisted search systems to hunt for Soviet submarines. These programs used parametric descriptions of how submarines tended to operate to predict the future location of target submarines and assign search forces in an optimum manner. These systems were wildly successful, compared to those that did not utilize computational assistance.[i]
However, the collapse of the Soviet Union confined the Russian submarine fleet to port and led to a drastic reduction in defense spending in the former NATO countries. The last five years, however, have witnessed powers such as Russia and China dramatically expand their submarine operations and refocused the U.S. military on great power competition.[ii] The Navy is once again turning to machine learning and computer assisted search systems to hunt down foreign submarines and sift through the massive amounts of information generated by modern ships, submarines, and aircraft.[iii]
Hunting submarines poses two distinct problems for search forces. First, planners must make use of all information at their disposal to predict where the target might go in order to maximize the chance that the limited number of ships and aircraft find their target. Second, intelligence analysts must analyze staggering amounts of data to glean clues to where a submarine might be. To get a sense of how much data is generated by modern systems, consider that a submarine hunting airplane churns out roughly 900 GB of information on a typical flight.[iv]
Two additional strategic pressures impact the Navy’s submarine hunters. First submarines are becoming more numerous and increasingly stealthy as technology advances and various countries build out their fleets. Second, the massive amount of data produced by modern sensor systems is threatening to inundate the intelligence analysis systems used to support submarine search operations.
To respond, the U.S. Navy is upgrading its legacy submarine hunting operational planning platform.[v] The Navy is also rolling-out a system known is Minotaur, which fuses and analyzes sensor data.[vi] Minotaur algorithms sift through hours of sensor data and identify information patterns that match “high interest” signals from past missions. These “flagged” data packages are then forwarded to a human intelligence analyst for review and analysis.
While new systems bring new capabilities and efficiency, the pathway forward is not completely clear. For example, investment in system upgrades have focused on installing upgraded software on warships rather than on the more connected shore-based fleet headquarters. This is a shortsighted move for two reasons. First, Navy manpower is low, meaning that many warships have insufficient numbers of trained crewmembers. Second, fleet headquarters are better prepared to fuse information from many disparate sources and to provide detailed analysis. The Navy ought to focus more of their investments to upgrade its shore-based rather than ship-base systems.
The Navy is wisely turning to proven computer systems to increase the effectiveness of its limited number of warships and aircraft. Indeed, some observers have suggested that improved information fusion and big data analytics will render the ocean “transparent” and make submarines as readily detectable as surface ships.[vii],[viii] However, significant challenges remain for the American sailors.
First, government contracting moves so slowly that many systems are obsolete by the time they are fielded. Will the Navy be able to buy computer systems and software upgrades quickly enough to remain competitive? Second, the U.S. military enjoyed five decades of driving the aerospace, defense, and high technology sectors given it was the largest source of investment and revenue. However, consumers and private sector investment have become the sources of cash and demand that drive the tech sector today. Will the notoriously socially conscious tech culture in Silicon Valley be willing to build weapons systems?
The U.S. Navy is headed “back to the future” in leveraging proven operations analysis and machine learning systems. Whether the sailors are able to keep up with an evolving threat and a broken acquisition system is an open question. Until then, the secret game of cat and mouse beneath the waves will continue.
[i] Daniel H. Wagner, “Naval Tactical Decision Aids,” (Monterey: Naval Postgraduate School, September 1989), p. II-61.
[ii] Eric Schmitt, “Russian Bolsters its Submarine Fleet, and Tensions with U.S. Rise, New York Times, (April 20, 2016), https://www.nytimes.com/2016/04/21/world/europe/russia-bolsters-submarine-fleet-and-tensions-with-us-rise.html.
[iii] Michael Glynn, “Information Management in Next Generation Anti-submarine Warfare,” Center for International Maritime Security, (June 1, 2016), http://cimsec.org/information-management-next-generation-anti-submarine-warfar/25614.
[iv] Ibid.
[v] “U.S. Navy Fact File – AN/UYQ-100 Undersea Warfare Decision Support System (USW-DSS), U.S. Navy, (January 24, 2017), https://www.navy.mil/navydata/fact_display.asp?cid=2100&tid=324&ct=2.
[vi] William Matthews, “Navy’s Minotaur System is a Step Toward Automated Data Analysis,” Seapower Magazine, (May 18, 2016), http://seapowermagazine.org/stories/20160518-data.html.
[vii] James Holmes, “U.S. Navy’s Worst Nightmare: Submarines may no Longer be Stealthy,” The National Interest, (June 13, 2015), http://nationalinterest.org/feature/us-navys-worst-nightmare-submarines-may-no-longer-be-13103.
[viii] Bryan Clark, “The Emerging Era in Undersea Warfare,” (Washington, D.C.: Center for Strategic and Budgetary Analysis, January 22, 2015), http://csbaonline.org/publications/2015/01/undersea-warfare/.
Mike – great paper on a topic of increasing importance. The Department of Defense has numerous resources at its disposal, but technological innovation and the associated manpower requirements are far from the top of that list. Rather than invest in collection AND analysis capabilities on-board surface ship combatants that have both limited manpower and internet band-with, the Navy should use its deployed assets (both sea going and airborne) as information collecting nodes that feed data to a centralized shore location for analysis. This would allow the Navy to focus its scarce intellectual and technological resources in one location. In all likelihood this data is not capable of immediately changing the course of a dynamic battle. It, however, may be able to influence the tide of a longer war. With that in mind, the analytic capabilities belong at a strategic level, with information collection at the tactical level. As to private sector aid in combating this problem, given Google’s reluctance to support similar objectives with machine learning regarding reducing collateral damage with drone strikes – I am not optimistic.
I know that I’m biased, but I’ve told you several times personally that I think submarines pose the greatest asymmetric threat to national security, so I’m glad that the DoD has amped up the game in combating the threat. I like your idea of having shored-based control stations for the program, but I worry that said stations will feed information to the warfighters inefficiently, I.E. instead of sending a ship or P-8 in the Pacific information only about subs in its theater, sending a one-size-fits-all report that isn’t as user friendly. It will be interesting to see how this goes.
Mike – really enjoyed reading about submarine hunting. Regarding the questions you’ve posed, I have two comments. First, the Navy will need to motivate more young men and women with coding skills to join the US military. A possible approach to this could be to partner with an AI technology company and offer subsidized student loans for students who are interested in a career/building skills in AI. Second, this is currently a big data problem. It requires 900 GB/flight to sweep the ocean and I’m assuming a decent amount of that data is noise. However, to play devil’s advocate, I do see some advantage to processing the data at sea. By historizing the data and then sending the data packs to shore for human analysis, I assume the US military is sacrificing on response time. Could the US Navy eventually have unmanned ships that automatically respond to high interest signals? Just my two cents.
This is definitely a complex problem and I can see how machine learning would be helpful. The piece about how private sector investment has become the sources of cash and demand that drive the tech sector today and the reluctance of a socially conscious Silicon Valley to build war weapons was especially interesting.
The article does a great job at placing the organization in question in a very rich context both in terms of its external challenges and motivations and in terms of its internal drivers. For instance, the point about the Navy being slow to field a rapidly evolving technology at the risk of it becoming obsolete was very insightful. what remains unclear to me however is the extent to which submarines are developing to evade imaging all together. This I imagine is a major factor in assessing the validity of an machine learning solution to the problem.
Great article Mike – though I am quite interested and curious to know how, upon application of ML, the system is trained for a real target. I am asuming that with advancments in technology, what are the typical responses from the targets to deliberately release noise (maybe by generating signals that are no indicators of its position etc.) and how does US Navy trains its systems to identify such objects in real time.
Very interesting and unusual topic, thanks for writing!
I particularly am intrigued about your question, “Will the notoriously socially conscious tech culture in Silicon Valley be willing to build weapons systems?” . As we’ve seen recently, companies have evaded contracts to which they -or their employees- oppose. It is yet to be seen how much this ‘tech-world’ pressure can handicap the U.S.’ stategic defense capabilites. And, is this trend ever going to change? Might a catastrophic event change the tide?
Dear Mike,
Thanks for the article on an interesting and high stakes application of ML. Two things from me
How far up the decision making chain can ML move? Analysis of data from numerous sensors to identify potential subs is obviously an ideal application of ML. This could progress to prioritising those threats, integrating with other assets to create a search plan, and actually taking control of these assets. This is highly relevant as the US Navy develops new surface and sub-surface drones to track submarines. Can the US Navy adapt its decision making structure to integrate this, and where does the person manage the system? As machine learning relies on historic data to make predictions about the future, how quickly can it adapt to changing circumstances: new tacts, new guises, new decoys, new platforms? This is where working alongside a human may yield better performance.
To address your second point, I think the US Government needs to ask itself how it improve its processes to better engage with technology. Defence procurement is notorious for being inefficient and ineffective. How can it improve this situation, and would this attract more technology companies?