Research at AGIACC
Trustworthiness and efficiencyacross the AI computing stack
We investigate task-scoped authorization, fault containment and resource efficiency—from instruction sets and devices to runtimes, frameworks and SDKs.
Read the research scopeoperationFrom an agent
Record the decision, outcome and resource use.
AI agents use tools to read data, execute code and call services. Systems, platform and security engineers need to limit what those tools can affect while preserving legitimate task performance.
AGIACC’s research premise is trustworthiness-native AI: access control, memory protection, confidentiality and fault containment should be designed across the computing stack and evaluated alongside computation, memory, communication and energy costs. The focus is system security and execution safety.
From task policy to execution#
The initial reference case is document analysis: an agent reads approved maintenance records and images, invokes an untrusted analysis tool, and returns a recommendation for operator review. Controlled tests will exercise unauthorized file or network requests and, on a supporting platform, invalid native-memory operations.
The research contribution we are pursuing is explicit policy refinement across interfaces. Record-level authorization, runtime handles, memory bounds and device mappings describe different objects. Their relationships require documented resource mappings, trusted mediation, expiry and revocation; a permission at one layer cannot be assumed to establish the corresponding property at another.
Research motivation and evidence#
The agenda responds to concrete execution risks and resource constraints. The following external work illustrates those problems and the mechanisms available to investigate them.
Agents expose operational interfaces
MCP defines tool discovery and invocation. CaMeL studies policy enforcement around models exposed to untrusted inputs, illustrating the need to examine delegated authority alongside task behavior.
Native code and shared state create distinct risks
Chromium's memory-safety analysis and LeftoverLocals' GPU-memory demonstration document different ways defects or residual state can expose data. Their findings are specific to the systems studied.
Existing mechanisms support comparative evaluation
Arm MTE and agent runtimes such as OpenShell provide distinct mechanisms for memory checks and controlled execution. Platform support and configuration determine their coverage.
Memory management affects resource cost
Research on Transformer workloads identifies memory-bandwidth and data-movement constraints. This motivates workload-specific evaluation of state placement, transfers and compute–memory tradeoffs.
We combine containment of faults or compromise with prevention of specified invalid operations, under a defined threat model.
Research scope and technical context
Research across the computing stack
Three connected themes organize the agenda. Each branch links AGIACC’s research questions to external reference technologies, their protection boundaries and their design tradeoffs.
Hardware: protection, memory and device interfaces#
We study architectural support for memory protection, compartment interfaces and device access, including how compilers establish and optimize the protected operations. Independent process or virtual-machine boundaries may contain a native-code compromise; finer-grained protection can address exposure within a component.
The efficiency questions concern buffer sharing, ownership transitions and data movement. Evaluation begins on existing hardware or emulation. ISA, chip, accelerator and new-device designs become relevant where those experiments identify a concrete limitation and a measurable opportunity.
Memory authority & control flowCHERI · CHERIoT · Arm MTE · Apple MIE · PAC / BTI
The AGIACC research questionWe investigate how architectural bounds and permissions interact with compiler optimization, allocation and compartment design, evaluating protection coverage and resource cost on representative workloads.
CHERI & CHERIoT
CHERI capabilities carry bounds and permissions enforced by hardware. CHERIoT combines capability enforcement, compartment interfaces and runtime memory management for embedded systems.
Design considerations. Object lifetimes and revocation need an integrated design; generic CHERI support alone does not establish temporal safety.
Arm Memory Tagging Extension
MTE compares four-bit pointer and allocation tags on 16-byte memory granules. Tag mismatches help detect classes of out-of-bounds and use-after-free accesses when the allocator and operating system configure tagged memory.
Design considerations. Tag comparison is deterministic, but collisions, allocation policy and granularity limit detection coverage. Synchronous fault handling can block a mismatching access; asynchronous reporting can occur after execution continues. Tagging does not establish full object bounds or lifetime safety.
Apple Memory Integrity Enforcement
Apple's MIE combines synchronous Enhanced MTE with typed memory allocators and tag-confidentiality protections. Its September 2025 announcement describes always-on deployment on key attack surfaces, including the kernel, in iPhone 17 models and iPhone Air.
Design considerations. The cited announcement addresses key attack surfaces, not every application. Protection depends on allocator, hardware and operating-system integration; MIE is broader than enabling generic MTE.
Pointer authentication & branch protection
Arm PAC authenticates selected pointers; BTI restricts indirect branches to designated landing points. These mechanisms help harden control flow in native software.
Design considerations. Compiler instrumentation and platform configuration determine coverage. Allocation bounds and object lifetime remain separate concerns.
Device trust & secure foundationsTrustZone for Cortex-M · TF-M · OpenTitan
The AGIACC research questionWe study how authenticated startup, firmware updates, peripheral access and recovery can support task-specific authorization throughout a device's lifecycle.
Arm TrustZone for Cortex-M
Secure and Non-secure execution states separate code, memory and peripherals on supported microcontrollers.
Design considerations. Security attribution, entry interfaces and firmware determine the boundary. Memory safety within either domain requires additional mechanisms.
Trusted Firmware-M & secure boot
TF-M provides secure services and partition management. Secure boot authenticates firmware against configured trust anchors before execution.
Design considerations. Update policy, rollback protection and key lifecycle matter alongside startup verification; runtime behavior still needs its own protection.
Hardware roots of trust
Hardware roots of trust such as OpenTitan support device identity, key protection and boot verification.
Design considerations. Provisioning, trusted firmware, updates and ownership management remain part of the system's security model.
Confidential computingArm CCA · Intel TDX · AMD SEV-SNP · GPU protection
The AGIACC research questionWe investigate how confidential execution composes with service authorization, tool permissions and accelerator access, with explicit assumptions about platform trust, attestation and resource cost.
Arm Confidential Compute Architecture
CCA supports isolated Realm execution and attestation on supporting platforms, under a security model that includes trusted firmware.
Design considerations. Platform configuration, trusted firmware and the attestation policy determine what a relying party can establish.
Intel TDX & AMD SEV-SNP
Confidential VMs protect private guest memory and execution state from specified external threats, including a privileged host, under their respective platform models.
Design considerations. Guest software, shared interfaces and deployment assumptions remain important. Confidentiality does not establish the correctness of a tool or model.
Confidential accelerator execution
NVIDIA confidential computing supports protected GPU execution and CPU–GPU communication on specified platforms, with attestation for the GPU environment.
Design considerations. The linked VM threat model assumes whole-GPU assignment to one confidential VM. CPU, GPU, driver and interconnect assumptions require joint evaluation.
Accelerators, memory & data movementIOMMU / SMMU · CXL · accelerator memory · co-design
The AGIACC research questionWe investigate processor, accelerator and memory interfaces that coordinate data placement, access permissions and ownership transitions. Evaluation considers workload performance, communication cost and energy under defined protection requirements.
IOMMU & Arm SMMU
Device address translation and permission checks restrict DMA access to configured memory mappings.
Design considerations. DMA mappings, device grouping and platform topology define the isolation available to software.
CXL & heterogeneous memory
CXL provides coherent interconnect and memory-expansion protocols for processors and attached devices, including memory pooling in supporting systems.
Design considerations. Pooling benefits depend on workload, topology, latency, bandwidth and infrastructure cost. Coherence alone does not establish authorization or confidentiality.
Accelerator memory residue
The 2024 LeftoverLocals disclosure demonstrated residual GPU local-memory leakage, enabling reconstruction of LLM responses in a tested llama.cpp deployment.
Design considerations. The attack required a co-resident adversary executing GPU code. Exposure depended on device, driver and workload; historical findings do not establish current vulnerability. Clearing and partitioning need separate evaluation.
System software: isolated execution and resource efficiency#
Our systems agenda connects runtime isolation with scheduling, memory management and recovery. For short-lived agent tools, we investigate environment preparation and reuse, including prediction errors, credential expiry and residual state.
Training and inference raise related questions about model-state placement, caching, offloading, sharding and recomputation. We evaluate performance alongside ownership and lifecycle constraints. We assess reuse within declared trust domains and test timing leakage, state clearing and recovery separately.
Safer software & explicit resource limitsRust · compiler CFI · namespaces · cgroups · seccomp · Landlock
The AGIACC research questionWe study how memory-safe implementation, compiler enforcement and resource limits affect the size, auditability and performance of trusted software, including legacy code and foreign-function interfaces.
Memory-safe software & compiler hardening
Rust's safe-language rules and compiler techniques such as LLVM control-flow integrity address different classes of invalid behavior before or during execution.
Design considerations. Unsafe code, foreign interfaces and instrumentation coverage define where additional reasoning and protection are required.
Namespaces & cgroups
Linux namespaces separate resource views; cgroups account for and constrain resources such as CPU and memory. Together they support bounded execution and more predictable sharing.
Design considerations. These mechanisms operate within a shared kernel. Resource controls and authority controls serve different purposes.
seccomp & Landlock
System-call filtering and application-applied access restrictions narrow what a process may request from the operating system. They are building blocks for a runtime's enforcement policy.
Design considerations. Kernel support, policy composition and mediated operations determine coverage; syscall filtering alone is one part of containment.
Execution compartmentsgVisor · Firecracker · Kata Containers · WebAssembly / WASI
The AGIACC research questionWe compare isolation mechanisms for native tools, services and portable modules in terms of their trusted interfaces, compatibility, fault containment and execution overhead.
gVisor
A userspace application kernel mediates application interactions with the host, reducing the host-kernel interface directly exposed to a workload.
Design considerations. System-call compatibility, I/O behavior and workload-specific overhead are central to choosing this boundary.
Firecracker & Kata Containers
MicroVMs and VM-backed container workflows use virtualization to create an execution boundary with a separate guest kernel.
Design considerations. The host, hypervisor and device interfaces remain relevant; the isolation unit may be a VM or pod. Snapshot clones require guest-appropriate entropy and application-state handling.
WebAssembly, WASI & Wasmtime
Portable modules combine bounds-checked linear memory with explicit host interfaces. Capability-oriented interfaces can expose only the resources a module is intended to use.
Design considerations. Linear-memory bounds are not object-level memory safety. Fuel and epochs limit guest execution; blocking host calls and memory consumption need separate controls.
Agent runtime & policy enforcementNVIDIA OpenShell · declarative policy · mediated access
The AGIACC research questionWe investigate how application policies can be translated into inspectable runtime controls, with defined denial behavior, exception handling and evidence of the operations each enforcement mechanism mediates.
NVIDIA OpenShell
OpenShell combines isolated agent execution with declarative filesystem, network, process and provider-credential controls.
Design considerations. The linked Dev documentation describes evolving behavior. Enforcement depends on configured policies, operating modes and the kernel facilities in use.
Policy analysis & protocol-aware mediation
OpenShell's SMT-based prover checks selected policy properties; separate protocol-aware network rules mediate application traffic.
Design considerations. The prover excludes MCP and GraphQL and does not prove live runtime enforcement. MCP tool-name rules do not validate arguments.
Inference, attention & reusable statePagedAttention · KV-cache reuse · batching · FlashAttention · cache isolation
The AGIACC research questionWe study scheduling, memory placement and state reuse within explicit trust domains, including cache ownership, lifetime, cancellation and the resource costs of maintaining isolation.
PagedAttention & prefix caching
PagedAttention manages KV-cache memory in blocks; prefix caching can reuse suitable prior computation. These approaches address fragmentation, duplication and repeated inference work.
Design considerations. Cache identity, ownership and eviction need explicit policy. Reuse across requests should respect the intended trust domain.
Batching, chunked prefill & serving schedules
Serving systems can schedule prompt processing and token generation differently to balance utilization, responsiveness and interference between requests.
Design considerations. Measure time to first token, inter-token latency and tail behavior together; workload and communication costs determine the tradeoff.
FlashAttention & I/O-aware kernels
Attention tiling reduces transfers between high-bandwidth memory and on-chip storage, making data movement explicit in kernel optimization.
Design considerations. Hardware, tensor shapes and numerical precision determine the benefit. Kernel efficiency is evaluated within the surrounding system's protection requirements.
Cache-sharing side channels
Research has demonstrated prompt leakage through timing differences caused by shared prefix and KV-cache reuse in evaluated multi-tenant serving systems.
Design considerations. vLLM's optional cache_salt limits prefix reuse to matching-salt requests. It does not isolate all KV memory or GPU execution; salt assignment and lost reuse require evaluation.
Training & distributed resource managementFSDP / ZeRO · offloading · activation recomputation
The AGIACC research questionWe investigate distributed resource management that coordinates model-state placement, computation and communication while preserving declared data-access policies, resource limits and recovery behavior.
Sharding & parallel execution
FSDP, ZeRO and distributed parallelism divide model state or computation across devices, trading local memory requirements against communication and scheduling costs.
Design considerations. Sharding is a resource-management mechanism; confidentiality and access control need their own enforcement.
CPU / GPU / storage offloading
Moving model state between memory tiers can make larger workloads feasible. Overlapping transfers with computation can help manage the resulting data-movement cost.
Design considerations. Transfer bandwidth, residency, temporary buffers and sensitive-state lifetime belong in both the performance model and the protection design.
Activation recomputation & recovery
Activation checkpointing trades extra computation for less saved intermediate state. Persistent training checkpoints serve a separate role in restarting interrupted work.
Design considerations. Evaluate memory savings and recomputation cost separately from recovery time, checkpoint integrity and the authority to restore state.
Application frameworks: policy, information flow and evidence#
Frameworks and SDKs should express permitted resources and operations, resource budgets and human-authorization requirements. Our research investigates which restrictions can be translated into service or runtime controls and which must remain at an application boundary.
Information-flow policies can constrain how untrusted data influences protected decisions while allowing the data-dependent operations a task needs. Execution records should connect authorization decisions, observed operations, failures and resource cost. Their evidential value depends on trusted capture, integrity, freshness and coverage.
Identity and delegated authorityMCP authorization · SPIFFE
The AGIACC research questionWe study how user authorization, workload identity and credential lifetime can remain aligned across services, while evaluating authentication overhead and the authority delegated to each operation.
MCP authorization
The HTTP MCP authorization specification uses OAuth and server-defined scopes to authorize client access to protected MCP servers.
Design considerations. Authorization is optional and transport-specific. Servers must still validate tokens and authorize individual operations; the protocol does not establish tool isolation.
SPIFFE workload identity
Defines portable workload identities and short-lived cryptographic identity documents, enabling services to authenticate each other across heterogeneous environments without treating a network address as identity.
Design considerations. Authentication identifies a workload; operators still define trust domains, issuance conditions, authorization policies and the actions that identity may perform.
Untrusted content & information flowCaMeL · agent information-flow control · AgentDojo
The AGIACC research questionWe investigate how information-flow policies, runtime enforcement and memory protection can constrain the influence of untrusted data on protected decisions, with evaluation of task completion and resource cost.
CaMeL
CaMeL separates planning over a trusted query from subsequently retrieved data. Its interpreter tracks provenance and capabilities and checks policies at tool calls.
Design considerations. Security claims depend on trusted inputs, interpreter behavior and configured policies. AgentDojo evaluation reports a task-utility tradeoff; results are specific to the models and configurations tested.
Information-flow control for agents · Fides
Fides tracks confidentiality and integrity labels and checks policies over tool actions. Selective hiding and inspection of values support controlled data-dependent planning.
Design considerations. Guarantees assume correct labels, trusted wrappers and policy enforcement; they do not establish tool correctness or memory safety.
AgentDojo
An extensible environment of realistic agent tasks, security test cases and attacks for measuring both attack success and the utility a defense preserves.
Design considerations. Supports adaptive-attack evaluation; findings remain bounded by the tested tasks, attacks, models and configurations.
Actions, policies and approvalsOPA · MCP tool controls
The AGIACC research questionWe investigate how application policies can govern tool operations, resources and human authorization, including the correctness and overhead of policy evaluation and enforcement.
Open Policy Agent
Evaluates declarative policies over structured inputs so an application can decide which tools, parameters or resources are permitted before dispatching an operation.
Design considerations. OPA supplies decisions; the calling application must enforce them and provide trustworthy context, including any resource and budget constraints.
Tool validation and human approval
MCP defines tool descriptions and invocation structures around which applications can expose actions, validate inputs, handle results and give people an opportunity to approve or deny operations.
Design considerations. Human review is recommended rather than universally mandated; tool metadata and returned content require trust checks and careful application handling.
Evidence from build to executionOpenTelemetry · RATS · SLSA · Sigstore
The AGIACC research questionWe study how software provenance, platform attestation and execution telemetry can support verifiable claims about protection and resource use, with explicit capture, integrity, freshness and coverage assumptions.
OpenTelemetry
OpenTelemetry provides instrumentation and export for traces, metrics and logs used to inspect execution, latency and resource use.
Design considerations. Telemetry depends on instrumentation and collection; it does not prove enforcement or platform integrity.
IETF RATS remote attestation
Describes how an attester presents platform evidence, a verifier appraises it and a relying party uses the resulting assessment when deciding whether to grant access.
Design considerations. Results depend on evidence, freshness, trust anchors and appraisal policy; attestation does not establish the correctness of every application action.
SLSA provenance
Defines supply-chain assurance concepts and provenance describing where, when and how software artifacts were produced, giving consumers information they can verify against expected sources and builders.
Design considerations. Assurance depends on trusted builders and verified expectations; provenance describes artifact production, rather than proving that the software is defect-free.
Sigstore artifact signing
Provides tools to sign and verify software artifacts, linking signatures to signer identity and recording signing events in transparency logs that support independent checking and investigation.
Design considerations. Verification depends on trusted identities, issuers and log evidence; an authentic signature does not establish software correctness or acceptable runtime behavior.
Research hypotheses#
The first platform prioritizes policy composition, isolated tool execution, native-fault behavior and execution records on existing platforms. The hypotheses below also cover longer-term sharing, device and actuation research; they are questions to test, not simultaneous product commitments.
Policy-preserving compositionApplications · System software · Hardware
- Research problem
- Task policies and enforcement layers use different resource representations.
- Working hypothesis
- Explicit mappings can preserve task restrictions across service, runtime and supported hardware controls.
- Evaluation criteria
- Measure excess permissions per domain; test allowed operations, denials, revocation and mapping failures.
Controlled influence of untrusted dataApplications · System software
- Research problem
- Untrusted input can steer delegated authority toward unauthorized effects.
- Working hypothesis
- Provenance and information-flow policies can constrain protected decisions while permitting legitimate data-dependent operations.
- Evaluation criteria
- Report attack success, task completion and false denials, including adaptive attacks and trusted-enforcement assumptions.
Predictive isolation and lifecycle controlSystem software
- Research problem
- Short-lived tools incur setup costs; reused environments retain state.
- Working hypothesis
- Preparing likely environments can overlap setup with inference, provided expiry and reset semantics are explicit.
- Evaluation criteria
- Compare latency, prediction errors, reserved resources and residual state against baselines meeting equivalent protection requirements.
Capability-based sharing and revocationHardware · System software
- Research problem
- Isolation incurs communication and scheduling costs; shared-memory alternatives already exist.
- Working hypothesis
- Capability compartments and lifetime mechanisms may reduce communication cost while preserving required isolation properties.
- Evaluation criteria
- Compare copying and shared-memory baselines; test bounds, permissions and revocation alongside latency and memory cost.
Bounded interfaces for physical systemsApplications · System software · Hardware
- Research problem
- Faults or unauthorized commands can propagate toward an actuation interface.
- Working hypothesis
- Separated components and narrow command interfaces may limit requested actions alongside independent actuation constraints.
- Evaluation criteria
- Test commands, faults and timing on a defined setup and hazard model; memory isolation alone does not establish physical safety.
Verifiable execution recordsApplications · System software · Hardware
- Research problem
- Execution records depend on what the platform can reliably observe.
- Working hypothesis
- Records can associate task grants, platform evidence, code identity, observed operations and resource cost.
- Evaluation criteria
- Test tampering, replay and missing records; document trusted capture, freshness and completeness assumptions.
Evaluation: protection, task performance and resource cost#
Each study will specify its task, policy, threat model, trusted components and tested configuration.
| Evaluation question | Evidence required |
|---|---|
| Do permissions match the policy? | Effective access to resources, operations, duration and budgets, assessed separately for each domain |
| Are faults and attacks contained? | Permitted/denied-operation tests, memory faults, malicious tool outputs, resource exhaustion and lifecycle tests; recovery behavior |
| Does legitimate work complete? | Task success, output quality and false denials on matched workloads; repeated trials for stochastic behavior |
| What are the resource costs? | Latency, startup time, memory, throughput and energy where measurable, against stated baselines |
| Can another engineer reproduce it? | Published versions, policies, instructions and measurements; reproduction and integration effort |
Engineering alternatives must meet equivalent protection requirements; less-protected ablations measure incremental cost separately. Passing adversarial tests provides evidence limited to the tested attacks and configurations; formal guarantees require explicit models and proofs.
Research progression#
Progression depends on reproducible protection results, deployment needs and suitable resources. The roadmap gives the proposed delivery horizons.
Reference experiments
Validate protection properties and resource cost
- Research focus
- Integrate one workflow on existing hardware or emulation.
- Evidence to advance
- Reproducible protection cases, fault outcomes and resource measurements.
- Potential translation
- Reference platform and SDK.
Integration studies
Evaluate portability and adoption
- Research focus
- Test interfaces, lifecycle control and selected user workflows.
- Evidence to advance
- External reproduction, integration effort, reliability and evidence of user value.
- Potential translation
- Selected integrations and supported platform capabilities.
Evidence-gated co-design
Investigate architectural limitations
- Research focus
- Study processor, accelerator and device changes identified by reference experiments.
- Evidence to advance
- Demonstrated need, measurable benefit, suitable partners and resources.
- Potential translation
- Hardware–software co-design.
From research to a reference platform#
The founder’s research background spans memory protection, compiler optimization, agent execution and efficient AI systems. Seed resources would support platform integration, reproducible evaluation and design-partner discovery. The initial commercial hypothesis is paid platform evaluation and integration; supported offerings would follow evidence of demand.
Research collaboration and platform evaluation
Contact AGIACC