Problem
Employees at community banks will use AI tools. The question is whether the bank controls what leaves the building and can prove it later. EchelonAI is a self-hosted gateway that puts a bank-governed assistant in that position. The audience is loan officers, compliance officers, examiners, and admins.
Request pipeline
A request to /v1/chat/completions first clears auth, CSRF, and rate limit middleware, then resolves the user from a session token. The chat endpoint runs thirteen stages:
- Stages 1-4: Presidio redaction, PII to tokens.
- Stage 5: risk scoring, 0.0 to 1.0.
- Stage 6: complexity classification, LOW/MID/HIGH.
- Stage 7: system prompt assembly from guardrails and user profile.
- Stages 8-9: circuit breaker, then the LLM call.
- Stage 10: response scan for guardrail violations.
- Stage 11: principal review check.
- Stage 12: WORM log insert.
- Stage 13: return the response, a 202 Queued, or a 403 Blocked.
Redaction and the token vault
Presidio with spaCy detects SSNs, account numbers, names, emails, dates, phone numbers, and IP addresses. Originals are replaced with tokens such as <SSN_x123> and <PERSON_y456>. An encrypted vault stores the mappings. Rehydration happens server-side only. The client never receives keys. Redaction is mandatory, and any failure rejects the request. A scheduled task purges PII tokens after one hour.
Fair lending guardrails
The Reg B / ECOA and Fair Housing guardrails are hard-coded and non-toggleable. The system refuses credit determinations and refuses to use race, color, religion, national origin, sex, marital status, age, or disability. A proxy detector flags ZIP codes, neighborhoods, and census tracts. Ambiguous requests trigger an engagement layer that asks clarifying questions. Loan decision support is advisory.
Requests scoring above 0.6 enter the principal review queue. A compliance officer approves or denies before delivery, and the decision carries an audit trail.
Audit log
Each chat request is recorded with user ID, prompt hash, risk score, guardrail violations, and decision outcome. HMDA correlation fields (hmda_application_id, hmda_loan_purpose, hmda_loan_amount) are captured as well. Immutability is enforced in the database. Triggers reject UPDATE, DELETE, and TRUNCATE on compliance_audit_log. Redacted prompts are encrypted with AES-256-GCM. A weekly job verifies the HMAC chain. A daily Celery task deletes records past six years. Examiners get a read-only forensic view with full conversation detail, and their queries are logged separately.
Model provider chain
| Tier | Provider | Models | Residency |
|---|---|---|---|
| Primary | OpenRouter | Qwen 2.5 7B (LOW), Claude Haiku (MID), Claude Sonnet (HIGH) | Cloud, US, ZDR enforced |
| Fallback 1 | Azure OpenAI | GPT-4o-mini (LOW/MID), GPT-4o (HIGH) | Bank’s Azure tenant |
| Fallback 2 | Ollama | Qwen 2.5 Coder 14B | Local only |
The circuit breaker opens after 3 consecutive failures or latency above 5 seconds, then moves to the next configured tier. State is held in Redis and exposed at /health/circuit-breaker. Every cloud call carries X-OpenRouter-ZDR: true. The header is mandatory.
Disparate impact monitoring
A daily Celery Beat job at 3 AM UTC snapshots approval rates by cohort. Variance above 10% raises an alert. The Compliance and Disparate Treatment dashboards show cohort rates and statistical significance flags.
Access control and secrets
Four roles exist: loan_officer, compliance_officer, examiner, admin. Production login uses Azure AD OIDC with MFA. Session tokens are Fernet-encrypted and carried in HttpOnly cookies. API keys are stored encrypted in app_settings, and settings changes are logged.
Stack and verification
FastAPI on Python 3.11+, async SQLAlchemy 2.0, PostgreSQL 16, Redis 7+, Celery 5.3+, and a React/TypeScript/Vite frontend with Tailwind and Recharts. The schema has 44 Alembic migrations. A 59-check smoke test covers health, auth, the chat pipeline, conversations, folders, admin endpoints, compliance, and WORM insertion. Unit and integration suites run under pytest.
Status
Version 1.0.0, Phase 3 complete. The README describes a possible split into separate chat and admin services sharing auth and database. That split is planned, not shipped.