Security | InferLayer AI — Applied AI Engineering

Architecture

InferLayer is an OpenAI-compatible LLM gateway that sits between your application and the language models that serve it. Requests enter over HTTPS and are authenticated per tenant. The gateway classifies each request by complexity, serves what it can from a semantic cache, routes simpler work to local models and complex work to frontier providers, enforces per-tenant spend limits, and records every request in a tamper-evident audit log.

Security is built around a few principles: authenticate every request, isolate every tenant, encrypt data in transit and at rest, and make the audit trail verifiable rather than merely trusted.

Key controls

API key hashing

API keys are random 256-bit values stored only as SHA-256 hashes. Plaintext keys are shown once at creation and never persisted — a database compromise does not expose usable keys.

Tamper-evident audit chain

Every request is recorded in an append-only audit log linked by a SHA-256 hash chain. Any retroactive edit or deletion breaks the chain and is detectable on verification, either through the verification API or offline with a standalone script your auditors run themselves. The chain head is anchored to write-once (S3 Object Lock) storage; third-party timestamping (RFC 3161, OpenTimestamps) is rolling out.

Multi-tenant isolation

Keys are bound to a tenant at creation. Stats, logs, audit, and budget endpoints are scoped to the requesting key's tenant on every request — cross-tenant access is denied by design.

Encryption

TLS 1.2 and above protect data in transit. Data at rest is encrypted with AES-256 using AWS-managed keys.

Secrets management

Production provider and infrastructure credentials can be held in AWS Secrets Manager and delivered to compute via scoped IAM roles rather than static files. Access is least-privilege.

Rate limiting & spend caps

Per-key rate limiting and per-tenant spend governance bound abuse and runaway cost before requests reach a model backend.

Data handling

Prompts and responses are processed in the request path and logged for audit, billing, and dispute resolution. Logs are encrypted at rest with restricted IAM access. We do not use customer prompt content to train models, and we do not sell data. For full detail on what is collected and retained, see our Privacy Policy.

Requests classified as complex may be routed to frontier providers (such as OpenAI, Anthropic, or Google), which then process that prompt under their own terms. On the default tier, every request, simple or complex, is served by a third-party API provider. Local-only serving for simple requests is available on the private-GPU tier and in BYOC deployments.

Deployment options

For customers with stricter data-residency requirements, InferLayer offers a self-hosted (BYOC) deployment in which all prompt and response content, API key material, tenant identity, and audit data remain entirely within the customer's own infrastructure.

Compliance posture

The audit trail is designed as evidence, not a log file. It supports the audit-control requirement in HIPAA (45 CFR §164.312(b)), SOC 2 evidence collection for processing integrity and change tracking, GDPR accountability (Art. 5(2)) and the records of processing it feeds (Art. 30), and the automatic logging and log-retention duties the EU AI Act places on high-risk AI systems (Arts. 12 and 19). Prompt content is kept out of operational logs by default, and in BYOC deployments the audit data never leaves your VPC.

Cryptographic audit-trail integrity available today. Every inference is signed into a tamper-evident hash chain that your auditors can independently recompute — no certification framework required to verify what was billed. The integrity claim is the product, not a future deliverable.

SOC 2 Type I: in preparation on the Drata compliance-automation platform, with a Type II observation window to follow.

Verifiability doesn't wait on the certification: it ships today, and you can check it yourself. We are happy to share our current roadmap and the certification timeline with customers under evaluation: contact@inferlayer.ai.

Vulnerability disclosure

We welcome reports from security researchers. If you believe you have found a vulnerability in the Service, please email security@inferlayer.ai with enough detail to reproduce the issue. Please give us a reasonable opportunity to investigate and remediate before any public disclosure, and avoid accessing or modifying data that is not yours during testing. We will acknowledge legitimate reports and keep you updated on remediation.

Contact

General security questions: security@inferlayer.ai. For everything else, reach us at contact@inferlayer.ai.