Applied AI Engineering

We build the AI systems.
Then we prove what they cost.

InferLayer is an applied AI engineering firm. We make AI infrastructure spend something you can verify, not estimate. We build agent workflows, retrieval systems, and production AI products for clients, and we operate our own inference gateway to prove the case ourselves.

The gateway is live. It caches repeat requests, routes each one to the model that actually fits, and records every result in a tamper-evident, hash-chained audit log. We're taking on design partners selectively before opening it up more broadly.

Estimate your savings →
Gateway, measured
~50%
Cost reduction, measured live against a frontier-model baseline. No GPU required. Faster, not slower.
OpenAI Anthropic Google Gemini Amazon Bedrock Self-hosted OSS

Live in production, measured on real calls.

100 users
Concurrent, load-tested. Zero failures.
~50%
Measured cost reduction. Faster, not slower. No GPU required.
2–4hrs
Typical hosted integration. One base_url change in any OpenAI-compatible client.
Measured separately, on public data, with a different baseline model and dataset than the numbers above. We don't blend the two. Full methodology →
74%
Cost reduction from routing alone. 432 real API calls on the public OASST2 conversation dataset.
70%+
Of head-to-head comparisons where the routed response tied or beat the frontier model. Blind third-party judge, 432 prompts.

Five things.
One engineering team.

Most of what we do is custom engineering work for clients. The gateway is the one piece we productized ourselves, because our own client work needed it.

01
AI Product Engineering
Full-stack builds for AI platforms, SaaS products, and enterprise MVPs. We write the code, not just the architecture diagram.
02
Agent Workflows & Integration
Multi-agent pipelines that hold state, recover from failure, and actually finish the task instead of looping forever.
03
Enterprise RAG & Data Systems
Hybrid search, chunking, and re-ranking tuned to your data, not a generic vector-database tutorial.
04
Cost & Compute Optimization
Model routing and token optimization. This is the InferLayer Gateway — available self-serve today.
↓ See how it works
05
Guardrails & Compliance
Data privacy controls and audit infrastructure built to support HIPAA, GDPR, and EU AI Act obligations, and to survive a procurement team's questions. The same evidence trail runs inside our own gateway.
↓ What gets logged, and how it's proven

The InferLayer Gateway.

The gateway sits between your application and whichever model provider you use. Every request passes through cache, classification, and cryptographic logging before it reaches a model. Integration is a one-line change to your base URL.

01
Semantic Cache
Recognizes when a new request means the same thing as one you've already answered, even worded differently, and returns the prior response instantly. Zero tokens for repeat queries.
Eliminates repeated cost
02
Intelligent Model Routing
Not every request needs a flagship model. Classification, extraction, and short Q&A go to a lightweight tier. Complex reasoning goes to a frontier model. You configure the split per tenant.
Right model for each task
03
Tamper-Evident Audit Chain
Every request is written into a SHA-256 hash chain. Change one row and every row after it breaks. You or your auditors can recompute the chain independently. You don't have to trust us.
Cryptographically verifiable
04
Per-Workflow Cost Attribution
Tag a request with a workflow ID and the gateway surfaces cost per resolution for that workflow specifically, the number a CFO actually wants, not an aggregate token count.
Maps tokens to outcomes
Request flow
01
Your App
Sends request to InferLayer endpoint
→
02
Authenticate
API key, tenant scope, rate limit
→
03
Cache Check
Return instantly if matched
→
04
Classify & Route
Pick the right model per request
→
05
Inference
OpenAI · Anthropic · Gemini · Bedrock · open-source
→
06
Audit & Attribute
Hash-chained log, workflow tag, cost dashboard

Every request
leaves evidence.

Most AI stacks can tell you what they spent. Few can prove it to someone outside the team. Every call through InferLayer produces a record your auditors can check on their own, without access to our systems and without taking our word for anything.

Built to support the audit-control requirement in HIPAA, SOC 2 evidence collection, GDPR accountability, and the EU AI Act's logging and log-retention duties for high-risk systems.

HIPAA §164.312(b) SOC 2 GDPR Art. 5(2) EU AI Act Art. 12 & 19
Full security posture & SOC 2 timeline →
Hash-chained log
Each record is SHA-256 chained to the one before it. Edit or delete a row and verification fails from that point forward.
Independent verification
Verify the chain through an API endpoint, or offline with a standalone script we hand to your auditors, so the check doesn't depend on our systems.
Write-once anchoring
The chain head is anchored to write-once (S3 Object Lock) storage. Third-party timestamping (RFC 3161, OpenTimestamps) is rolling out so the proof doesn't depend on us at all.
Data handling
Operational logs redact prompt content by default. The audit log keeps full prompts and responses for 12 months by default, configurable per contract. API keys are stored only as hashes. In BYOC, all of it stays in your VPC.
Spend governance
Server-enforced spend caps per API key, plus per-agent limits, checked before inference runs rather than reconciled after the invoice lands.
Attribution
Every record carries tenant, team, agent, model, and cost, so it answers who spent what, on which model. Workflow tags roll that up to business processes.

Teams spending real money
on tokens.

Enterprise AI teams
You're spending five figures a month across OpenAI, Anthropic, Gemini, or Bedrock, and your CFO wants to know what each workflow actually costs. That mapping, tokens to workflows to outcomes, doesn't exist until someone builds it. We build it, with an audit trail procurement can sign off on.
AI ops and FinOps
Usage exports from three different providers, reconciled by hand in a spreadsheet every month. The gateway gives you one dashboard for cost, latency, and cache performance across all of it, so that spreadsheet stops being someone's job.
SaaS companies with AI features
Usage is growing and your margins are moving the wrong direction. Semantic caching and routing cut measured cost by roughly half without touching your application code. Drop-in, OpenAI-compatible.
Regulated industries
Healthcare, financial services, legal: anywhere your LLM use has to survive an actual audit, not just a dashboard screenshot. Deploy in your own VPC so the hash-chained audit log, prompts included, never leaves your network.

Why we're
building this.

HP
Hiral Patel
CEO & Co-Founder
Running product, engineering, and the design-partner conversations that decide what gets built next.
Architecture
Layer 3
InferLayer Gateway — OpenAI-compatible API surface, semantic cache, intelligent model routing, per-tenant isolation and spend governance, tamper-evident audit chain, workflow attribution.
Layer 2
Substrate-agnostic backend. Runs in your own cloud with no GPU required on the default tier. Amazon Bedrock for AWS-native deployments. Built to scale to multi-node, Kubernetes-orchestrated deployment.
Models
Flagship: OpenAI, Anthropic, and Google Gemini frontier classes. Cloud-managed: the Amazon Bedrock model family. Self-hosted: a tunable open-source backend selected per deployment.

Token costs are one problem. The bigger one is that nobody knows where the money went. Enterprises run AI workflows across multiple providers with no real way to tie spend to a workflow, a business unit, or an outcome. The CFO sees a bill. The engineering team sees usage logs. Nobody sees both at once.

InferLayer sits between your applications and your inference providers as the layer that actually attributes spend. It routes each request to the right model, caches repeat queries, and hash-chains every log row, so the cost numbers hold up without anyone having to take our word for it.

The gateway is live in production, the audit chain is verified and running, and the benchmarks are real: roughly 50% measured cost reduction against a frontier-model baseline, with lower latency, on the default CPU tier. We're onboarding design partners now. What you're running shapes what we build next.

Always talking to
strong engineers.

No open roles listed right now. If you're an inference engineer, an AI infrastructure researcher, or a systems developer who cares about getting cost attribution right, send us what you've built. We read everything.

Questions teams ask
before integrating.

Does InferLayer only build the inference gateway, or do you take on custom AI engineering work? +
The gateway is one of five things we do, and the only one that's self-serve today. The other four are engagements: full-stack AI product builds, agent workflows and multi-step automation, enterprise RAG and data systems, and compliance and guardrail infrastructure. We built the gateway to prove out cost and compute optimization on our own traffic before selling it. The same engineering discipline carries into client work.
How does InferLayer reduce LLM inference cost? +
Three mechanisms, working together. Semantic caching returns a cached response when a new request matches a prior one above a tuned similarity threshold, so repeat queries cost zero tokens. Intelligent model routing sends simple tasks, classification, extraction, short Q&A, to lightweight tiers, and saves flagship models for complex reasoning. Per-workflow attribution shows which workflows actually account for spend, so engineering knows what to optimize first. Benchmarked at ~50% cost reduction against a frontier-model baseline on the default CPU/API tier, no GPU, plus 100% on warm-cache hits. That number blends routing and cache hits. Your real repeat rate gets measured during a diagnostic, not assumed.
You cite both a ~50% and a 74% cost-reduction number. Which one is real? +
Both, and they measure different things on purpose. The ~50% figure was measured on the live gateway over a 100-query mixed benchmark, routing plus cache hits together, against a frontier-model baseline. The 74% figure is an isolated, reproducible benchmark: routing alone, run four independent times against 432 public prompts from OpenAssistant/OASST2, a dataset skewed toward short, simple questions that route to cheap tiers more easily than typical production traffic does. We never merge the two into one claim. Full methodology is on the benchmarks page.
Is InferLayer OpenAI API compatible? +
Yes. InferLayer exposes the OpenAI Chat Completions API surface. Integration is a one-line change in your application: point your OpenAI base_url at https://api.inferlayer.ai/v1 and use an il- API key. Streaming, function calling, and tool use work without modification, same SDKs, same response shape.
How is InferLayer different from LiteLLM, Portkey, or Helicone? +
LiteLLM and Portkey route traffic between providers. Helicone shows you what happened after the fact. InferLayer answers a different question: can you prove what something cost to someone who has no reason to trust you by default? The tamper-evident audit chain and per-workflow attribution are built for that conversation with finance and procurement, not just an engineering dashboard.
Can InferLayer be deployed in our own AWS, Azure, or GCP environment? +
Yes. InferLayer ships as a Docker Compose deployment that runs inside your VPC, so your inference traffic and audit logs never leave your network. Kubernetes (EKS) manifests ship today for multi-node deployments, with llm-d integration on the roadmap. BYOC onboarding typically runs 5-14 days depending on your environment and security review.
What models does InferLayer support? +
Frontier models through OpenAI, Anthropic, and Google Gemini. Cloud-managed through the Amazon Bedrock model family. Self-hosted through a tunable open-source backend, chosen per deployment. Switch backends through configuration, not code changes. Bring your own fine-tuned model and the gateway routes to it like anything else.
What is the tamper-evident audit chain and why does it matter? +
Every inference request is written into a SHA-256 hash chain. Change or remove a row and every row after it breaks, visibly. You or your auditors can recompute the chain independently to verify the numbers without taking InferLayer's word for it. That's what makes savings-share pricing enforceable, satisfies a compliance team asking for a real audit trail, and lets finance treat AI spend with the same rigor as cloud or SaaS spend.
Is InferLayer SOC 2 compliant? +
Not yet. The cryptographic audit chain, the part that actually matters for verifying cost, is live today and independently recomputable by your own auditors, no certification required to check it. SOC 2 Type I is in preparation on Drata. Ask us for the current timeline if it matters for procurement.
How long does it take to integrate InferLayer? +
Hosted integration: usually 2 to 4 hours. Change the base_url in any OpenAI-compatible client, confirm the first request shows up in your dashboard, done. BYOC deployment: typically 5 to 14 days, depending on your cloud environment and security review. Covers Docker Compose setup, audit log verification, dashboard configuration, and workflow-attribution tagging.

Interested?
Say hello.

We're talking to design partners: teams running production LLM workloads who care about proving cost, not just cutting it. If that's you, send us a sample of your logs and we'll run a real diagnostic.

We reply personally. No automated sequences.

01
Shape the roadmap. The gateway is live and we're hardening it against real workloads. What you describe influences what we prioritize next.
02
Priority onboarding. Design partners get dedicated deployment support and first access to new capabilities.
03
Pricing input. Pricing is usage-based and scoped per engagement. Conversations with design partners directly shape what model makes sense for different usage patterns.
04
A direct line. You talk to the engineers who build it, not a sales rep.