Is This the Start of MilitaryGPT?

By: Mulan Teresa

Edited by: Mir Mustaufiq Rahman & Thuth Htet San

Today, the utilization of AI into daily life is becoming increasingly more common. For some students, using AI could determine the difference between a pass or a fail. But what happens when AI is used in the military and interferes with life or death? 

CEO of OpenAI Sam Altman has signed an agreement with the Department of War (DOW) – better known as the Pentagon – to deploy AI models on defense-classified cloud networks. They received immediate backlash; the idea of the AI militarization in the US, especially in this tense political climate, left the public concerned about its ethical usage. This pressured OpenAI to release a statement that highlights their agreement with the US government. 

Within the agreement, it highlights three main red lines: no domestic mass surveillance, no autonomous weapons, and no automated high-risk decisions. 

To implement these red lines, OpenAI emphasized that its AI systems will be a cloud-only deployment. In simple terms, this means that all operations related to the AI will be controlled through a central cloud by OpenAI. Therefore, individual decisions made by the Pentagon aren't possible. This precaution is to ensure that these AI systems will not be provided with “guardrails off." 

Additionally, the agreement highlights that the Pentagon should only use the AI system for ‘lawful purposes’ that are consistent with the current and applicable US law, operational requirements, and well-established safety and oversight protocols. 

This system also extends to intelligence activities, where the Pentagon would have to comply with existing laws regarding national security. CEO Altman says that “It is important for companies to work with the DOW as long as it is going to comply with legal protections.” 

At first glance, OpenAI’s released set of agreements with the Pentagon seems to ensure the safe and ethical usage of AI in the military. However, the agreement highlights what the AI models are prohibited from doing– but it fails to mention how exactly OpenAI would be involved in the military. The agreement highlighted three vague red-lines the Pentagon should avoid, but specific military usage, such as targeting and strike decisions, detention or watchlists, and intel that triggers operations, is still possible with the usage of these AI models. 

The scope and limitations of AI usage are not clearly defined. 

Initially, Anthropic AI was offered the same deal. However, they refused to move forward due to concerns regarding domestic mass surveillance and human responsibility. The deal was picked up within hours by OpenAI, despite the two companies sharing the same red lines. 

One of the main reasons OpenAI decided to take up this deal is that it negotiated by building technical safeguards into the Pentagon’s systems, which involves running the models only through cloud networks. 

Despite having the same concerns, Anthropic’s dealbreaker was regarding the “lawful use” of AI. Anthropic was unwilling to go through the deal because of how AI would be legally unsound and pose unpredictable danger, especially when militarized. 

This is despite Anthropic’s role as the first AI company to utilize its models for militarization, such as intelligence analysis, simulations, mission planning, and cyber operations. These are different from fully autonomous weapons, as they ultimately augment human capability and replace them—otherwise known as decision support systems. Anthropic CEO Amodei emphasizes that current AI systems are not reliable enough to be incorporated in fully autonomous weapons—or in other words, does not believe they should be used in direct combat. In regard to surveillance, Amodei is concerned that AI can assemble a comprehensive profile of citizens from scattered data points. 

So why was OpenAI so quick to seal this deal? 

The term “all lawful use” is non-specific; the agreement itself kept repeating how OpenAI would comply with the law, without ever using exact wording to what laws would be followed. Additionally, specific terms were also left vague and ambiguous. For example, the agreement stated that their AI model will not be used to “assume high-stakes decisions," but there are no specifications on what is considered high-stakes. This creates a responsibility gap: the human blames the data, and the data provider blames the human.

Laws on the use of AI are also very new, and very likely to have loopholes lawmakers did not think about as AI constantly evolves.

For the most part, the agreement published by OpenAI itself rejects exact wording and is filled with decorative phrases. The lack of legal vocabulary also leads to a lack of accountability.  For instance, the recent missile strikes on Iran used AI built into the Maven Smart System for targeting—including the targeting of a school that resulted in the deaths of at least 170 people.

Using AI does not make up for the moral deficiencies of the government or the military the same way it doesn’t for students. Yet, the militarization of AI persists through the controversies and uncertainties. The law can punish a human for killing a person, but can we punish AI that kills? If not, then we have officially entered the era of the “perfect crime.”