The recently released National Institute of Standards and Technology (NIST) Concept Paper on Securing AI COSAIS is a first good step, especially in how it breaks down enterprise use cases from co-pilots to agents. However, AI is such a rapidly evolving field that these controls also need to constantly change, and so will security requirements.
Enterprises run into challenges not just because AI creates a new threat, but because it exposes an underlying risk. For example, an employee may have inadvertent access to financial documents from years ago, but they are not going to manually search through them. However, asking co-pilot “tell me about all the compensation data you have” will quickly process thousands of these documents for sensitive information.
On the other hand, as enterprises fine-tune LLMs with their own data, access control and data safety become more important. It’s also imperative that LLMs don’t get too narrowly trained on certain kind of data that they start to make security related mistakes.
To mitigate the first risk listed by NIST, (AI exposing an underlying risk), when bringing in any kind of co-pilot or Agent even when there is no new access given, a comprehensive testing is needed before the use case is rolled out. The testing should check for known existing sensitive data accesses.
Similarly, the fine-tuning issue in the second use case can be addressed only by checking for over-fitting. So, while NIST lays out the use case correctly, it should add a stage for safety checks due to over-fitting on private data.
Traditionally, AI researchers have concentrated on performance issue due to over-fitting, but not security or safety. Looks like NIST also didn’t consider this factor.
NIST is definitely on the right track in enumerating use cases and risks. But how we go about addressing these concerns will require more thinking, shared insights from the industry and time to allow the industry to develop.
NIST is inviting feedback from the industry asking for input as to whether or not the proposed use cases as to how the overlays should be prioritized, and release a draft to the public of the first overlay in 2026. It’s not clear yet that enumerating a to-do list may suffice as was the case with traditional software, since AI is evolving so fast, but this is moving in the right direction.
