Our Vision at AGIACC
Our Vision
Capable AI needs enforceable boundaries. Our vision is to build trustworthiness into the computing stack, from hardware to applications.
AGIACC’s long-term vision is trustworthiness-native AI infrastructure: systems whose authority, memory access and failure behavior are governed by explicit, testable rules. We connect hardware and new devices, system software, and application frameworks, SDKs, platforms and services.
For developers, the goal is practical: give an AI system the access it needs, constrain what a faulty or compromised component can affect, and understand the protection and resource costs. Our focus is system security and execution safety, alongside the model evaluation, human oversight and domain-specific assurance a deployment requires.
When AI can act, its environment matters#
An agent can read confidential documents, execute code, invoke services and request changes to operational systems. Every connection creates a responsibility: who authorizes the action, which resources it can reach, and what happens when something goes wrong.
Published incidents and security research show how those responsibilities cross application, runtime and device boundaries.
Vendor account
Replit: a destructive database change
Replit acknowledged that its agent deleted data from an app's database while development changes could affect production. The user fully restored the database through rollback.
Design implication
Separate development authority from production, and make recovery a tested part of the system.
Provider security report
Claude Cowork: data left through an approved domain
Anthropic described a disclosed attack in which a malicious workspace file induced uploads using an attacker's API key. The VM boundary held, but files reached the attacker's account through an allowed domain. Anthropic reported a fix.
Design implication
Authorize the identity and operation behind a request, as well as its network destination.
Research demonstration
LeftoverLocals: shared GPU state
Trail of Bits recovered another process's GPU local-memory contents on affected configurations, including an LLM demonstration. The attacker required co-resident GPU code execution; the result does not describe every GPU or current deployment.
Design implication
Extend data ownership, state clearing and isolation into the devices that execute a workload.
Dates refer to source publications. Design implications are AGIACC’s interpretation of the documented cases.
The industry is investing in integrated protection#
Recent architecture announcements connect security mechanisms to the software and hardware that make them usable.
Apple Memory Integrity Enforcement
Apple introduced a defense combining hardware memory checks, secure allocators and operating-system protections. Its announced coverage includes key attack surfaces on iPhone 17 and iPhone Air.
Apple's architecture announcementNVIDIA OpenShell reference architecture
NVIDIA describes policy enforcement outside agent workloads and connects runtime governance to CPU, memory and I/O design. Its reference architecture illustrates joint attention to protection, execution and resource efficiency.
NVIDIA's technical announcementVendor-described architectures; coverage and assumptions vary by platform.
One vision across the computing stack#
Application permissions need explicit resource mappings and trusted enforcement to become effective limits in runtimes and hardware. Our research connects these interfaces through reproducible evaluation.
A connected architecture
Illustrative document-analysis workflow
Applications and services
Express the task's allowed resources, operations and review requirements.
Approved records; permitted analysis; operator review.
Map resources and mediate requests
System software
Enforce runtime and service controls, isolate components and manage their lifecycle.
Scoped files and destinations; isolated tools; execution budgets.
Connect runtime controls to supported hardware
Hardware and devices
Provide memory and device protection mechanisms that software can configure and evaluate.
Memory bounds; device mappings; state ownership.
We combine two complementary design lenses. Passive defense isolates components and contains faults or compromise. Active defense prevents specified invalid operations through enforced rules. These roles overlap: runtime isolation, permission checks and hardware protection can contribute to both. The relevant question is which properties the combined system establishes under its stated assumptions.
Protection and efficiency belong in the same design#
Memory movement, scheduling and state reuse affect both isolation and resource cost. We study them together, measuring legitimate task completion, denied operations, failure handling, latency, memory use and energy where measurable. Security and execution-safety requirements set the constraints; efficiency and compatibility determine whether teams can adopt the result.
Our commercial hypothesis is that platform and security teams will pay for infrastructure they can integrate, evaluate and operate within clear limits. Early use-case discussions and partner evaluations will test that hypothesis.
A practical first step toward the wider vision#
Our current foundation is research and technical artifacts. The planned reference platform and SDK will begin on existing hardware or emulation, using a controlled document-analysis workflow to connect permissions, isolated tool execution and reproducible evaluation. Evidence from this work will guide deeper ISA, chip and device co-design.
We welcome prospective design partners with a concrete execution boundary to investigate, researchers connecting hardware and software, and seed investors interested in bringing this agenda into use.
Explore the research · Follow the roadmap · Discuss a deployment problem