# Killing Long-Lived Credentials with Workload Identity and Dynamic Secrets

Canonical: https://kaansystems.com/library/killing-long-lived-credentials
Topic: Platform · Published: 2026-08-08 · Updated: 2026-10-12
Summary: Static credentials are the breach vector that actually gets used: they work until someone revokes them, and nobody revokes them. A phased plan for replacing IAM user keys, CI secrets, and database passwords with workload identity, OIDC federation, and dynamic secrets, with the static-credential count as the program KPI.

## Key takeaways

- Static credentials are the breach vector attackers actually use: a stolen key keeps working until a human revokes it, and credential-based intrusions have averaged around 240 days to identify.
- Inventory comes first — `aws iam generate-credential-report` plus a gitleaks or trufflehog scan across full git history builds a register whose row count becomes the program KPI, published weekly.
- Give workloads identity instead of stored keys: IRSA or EKS Pod Identity issues hour-long credentials that live in process memory, with nothing on disk, in a Kubernetes Secret, or in etcd to steal.
- In CI/CD, replace GitHub repository secrets with OIDC federation and pin the trust policy's `sub` claim to a specific repo and ref; a `repo:your-org/*` wildcard lets any workflow in the org assume your deploy role.
- For human access, put IAM Identity Center in front of your existing IdP and then delete every IAM user, keeping only two break-glass users with offline hardware-MFA keys wired to a CloudTrail alarm.
- Migrate one credential class at a time — inventory, build the new path, run both in parallel, then revoke — and accept the asymptote: the durable end state is break-glass plus the handful of vendors that only support API keys.

**How do you kill long-lived credentials?** Stop storing secrets and issue short-lived ones on proof of identity: workload identity (IRSA or EKS Pod Identity) for pods, OIDC federation for CI, SSO sessions for humans, and RDS IAM auth or rotating secrets for databases. Around 90% of a typical AWS estate can drop static keys entirely.

The phased plan assumes a real account boundary. If you do not have one yet, start with a [multi-account landing zone](/library/multi-account-aws-landing-zone). The [7-day Assessment](/assessment) includes the IAM credential report review.

## Why do long-lived keys survive every rotation reminder?

An attacker who wants into your AWS account does not need a zero-day. They need one of the several hundred static credentials your organization has minted over the years, and they only need it once. GitHub is scraped for `AKIA` prefixes continuously; a canary access key pushed to a public repo receives its first unauthorized API call in under ten minutes. The CircleCI breach in January 2023 worked this way at one remove: an engineer's laptop was compromised, a session was hijacked, and every customer who had stored AWS keys, deploy tokens, or signing certificates in CircleCI got the worst email of the quarter. Rotate everything, now, and hope your rotation runbooks exist.

The property that makes static credentials the dominant real-world breach vector is simple: stolen does not mean expired. A short-lived credential lifted from a compromised pod is worth an hour to an attacker. A long-lived one is worth however many months pass before someone notices, and breach-cost studies have put mean identification time for credential-based intrusions at around 240 days for years running. The secret in your CI settings, the IAM user key sitting in a developer's `~/.aws/credentials` since 2021, the database password in a Helm values file: each one works until a human revokes it, and revocation requires the human to know it was taken.

For a regulated SMB the problem doubles. HIPAA and SOC 2 both push you toward credential rotation, so teams perform quarterly rotation theater: a spreadsheet, a painful afternoon, and a couple of production incidents per year caused by the rotation itself. The exercise mostly proves that the process is manual and feared. The better answer is structural. Stop storing credentials at all, wherever a workload, a pipeline, or a human can prove its identity and be issued a short-lived credential on demand. For roughly 90% of a typical AWS estate, they can.

## How do you replace them with workload identity?

The organizing principle: replace possession of a secret with proof of identity at every boundary. AWS already operates the exchange desk (STS). Your job is to register the identity providers and delete the static keys. The trust flow for a Kubernetes workload looks like this; the CI and human flows are the same shape with a different token issuer.

```mermaid
flowchart LR
    subgraph EKS["EKS cluster"]
        POD["payments-api pod<br/>projected SA token"]
    end
    ISSUER["Cluster OIDC issuer<br/>signs JWT with sub claim<br/>system:serviceaccount:prod:payments-api"]
    STS["AWS STS<br/>AssumeRoleWithWebIdentity"]
    TRUST["IAM role trust policy<br/>issuer and sub must match"]
    CREDS["Temporary credentials<br/>1h TTL, auto-refreshed<br/>never written to disk"]
    API["AWS APIs<br/>S3, SQS, KMS"]

    POD -->|"presents JWT"| STS
    ISSUER -.->|"publishes signing keys"| STS
    STS -->|"validates against"| TRUST
    TRUST -->|"issues"| CREDS
    CREDS -->|"signs requests to"| API

    classDef box fill:#0d1f1a,stroke:#3FA585,stroke-width:1.5px,color:#E8F5EE;
    classDef hot fill:#1F4A3E,stroke:#2E8B6F,stroke-width:1.5px,color:#E8F5EE;
    classDef grp fill:#0a1612,stroke:#2E8B6F,color:#E8F5EE;
    class POD,ISSUER,STS,TRUST,API box;
    class CREDS hot;
    class EKS grp;
```

### The inventory: where static credentials hide

Counting comes first. `aws iam generate-credential-report` lists every IAM user access key with its age and last-used date; on a typical 30-engineer estate that report shows 40 to 80 keys, a third of them unused in the last 90 days. Then the places the report cannot see: repository and organization secrets in GitHub Actions, database passwords in ECS task definitions and Helm values, kubeconfigs with embedded client certificates that never expire, service-account JSON files, and the vendor API key living in a founder's dotfiles because it predates the secrets manager. Run gitleaks or trufflehog across full git history, not just HEAD; the file someone deleted in 2023 is still in the object store. The output is a register: credential, system, owner, blast radius if leaked, last-used date. The row count is the KPI for the entire program, and every phase below ends by making it smaller.

### Workloads: identity instead of storage

IRSA (IAM Roles for Service Accounts) is the established path on EKS. Each cluster ships an OIDC issuer; you register it in IAM once, then write a role trust policy that matches the token's `sub` claim to a specific service account, such as `system:serviceaccount:prod:payments-api`. The pod mounts a projected, auto-rotated token, and the AWS SDK exchanges it via `AssumeRoleWithWebIdentity` with no application changes. Credentials live in process memory, last an hour, and refresh themselves. Nothing on disk, nothing in a Kubernetes Secret, nothing in etcd to exfiltrate.

EKS Pod Identity (GA since late 2023) reaches the same end with less ceremony: no per-cluster OIDC provider registration, an agent handling `AssumeRoleForPodIdentity`, and role associations managed through a plain EKS API call. That matters once you run three or four clusters and tire of trust policies referencing cluster-specific issuer URLs. The trade-offs are minor but real: Pod Identity does not cover Fargate pods, and cross-account access needs explicit role chaining where IRSA can trust directly. Pick one mechanism per cluster and stay consistent. On GKE the same design is called Workload Identity Federation; on AKS, Entra Workload ID. Same JWT, same exchange.

### CI/CD: federation, not repository secrets

The `AWS_ACCESS_KEY_ID` stored in GitHub repository secrets is usually the highest-blast-radius static credential an SMB owns: it carries deploy-level permissions and is exposed to every workflow, third-party action, and maintainer account. GitHub Actions issues OIDC tokens natively. Register `token.actions.githubusercontent.com` as an identity provider in IAM and the pipeline assumes a role directly, so the next CircleCI-class vendor breach is someone else's rotation emergency.

The trust policy conditions do the security work (full JSON in the template below): `aud` pinned to `sts.amazonaws.com`, and `sub` pinned to `repo:your-org/your-repo:ref:refs/heads/main`. The classic misconfiguration is a `StringLike` condition on `repo:your-org/*`, which lets any workflow in any repository in the org — including a pull request against some archived experiment — assume your deploy role. Security researchers have found `repo:*` in the wild, which is an open door for every GitHub account on the planet. Pin to repository and ref, and give pull-request builds a separate role that can run `terraform plan` and nothing else.

### Humans: SSO sessions, zero IAM users

Put IAM Identity Center in front of whatever IdP you already pay for (Google Workspace, Okta, Entra). Engineers get permission sets, sessions of 8 to 12 hours, and `aws sso login` for the CLI. Then the uncomfortable part: delete the IAM users. All of them. The end state is a credential report with zero rows, plus two break-glass IAM users with hardware MFA whose access keys exist only on paper in a safe, wired to a CloudTrail alarm that pages on any use. This is Phase 1 of [zero trust for SMBs](/library/zero-trust-for-smbs) applied to the AWS control plane: identity-based, short-lived, centrally revocable. When someone leaves, disabling one IdP account kills every session within hours. Compare that with the offboarding checklist line that reads "rotate any keys they may have had."

### Databases and third-party APIs

RDS IAM authentication swaps the password for a 15-minute connection token, on Postgres, MySQL, and MariaDB engines including Aurora. It fits services with modest connection churn and works cleanly behind RDS Proxy. It does not fit everything: IAM-auth connection establishment carries extra overhead on the database side — token verification and mandatory TLS — and AWS recommends it only below roughly 200 new connections per second; some ORMs fight the auth plugin, and per-user tokens map awkwardly onto shared connection pools without a proxy in the path.

Where a password must exist, make it self-rotating. Secrets Manager with the managed Lambda rotators handles RDS credentials on a schedule using the alternating-users strategy, and applications that fetch the secret at connect time never notice a rotation. Vault's dynamic database secrets go further: per-lease database users created on demand with a TTL, giving every workload its own expiring identity inside the database. Honest guidance at 10-80 engineers: Vault is a distributed system you have to operate (HA, unseal, upgrades, policy sprawl), and running it solely for database credentials is overkill when Secrets Manager rotation delivers most of the value with none of the pager load. Vault earns its keep when you are multi-cloud, need dynamic credentials across many engines, or already run it for something else. HCP Vault splits the difference if lease-based credentials are genuinely required without the ops burden.

### The migration: parallel run, then revoke

Four moves per credential class: inventory, build the new issuance path, run both in parallel, revoke the old one. Deactivation is the safety net. IAM access keys support an Inactive state, so flip a key inactive for a week before deleting it and watch CloudTrail for whatever screams. Last-used data sets the sequence: dead keys first (instant wins, zero risk), then CI, then workloads, then humans. Publish the static-credential count weekly. A graph that reads 74, 61, 40, 19 does more for program momentum than any policy document.

Then the residue: the three vendors that only support API keys. There are always three. Scope each key to the minimum the integration needs, store it in Secrets Manager, put a 90-day rotation cadence on the calendar, assign a named owner, and alert on usage anomalies. Those keys become line items in your [vendor risk scoring](/library/vendor-risk-scoring), because their rotation story is now part of the vendor's risk posture, not only yours.

## The Template

The migration checklist below is phased so that each phase ends with a revocation, not a deployment. A phase is not done while the old credential still works.

```markdown
# No-Long-Lived-Credentials Migration

## Phase 0: Inventory (week 1)
- [ ] Run `aws iam generate-credential-report`; export all access keys with age and last-used
- [ ] Enumerate CI secrets: GitHub org, repo, and environment secrets
- [ ] Grep deploy artifacts for embedded creds: task definitions, Helm values, compose files, .env
- [ ] gitleaks/trufflehog across full git history of all active repos
- [ ] List kubeconfigs with embedded client certs; note expiry (often: none)
- [ ] Build the register: credential | system | owner | blast radius | last used
- [ ] Record the baseline count; this number is the program KPI

## Phase 1: Workloads (weeks 2-4)
- [ ] Register cluster OIDC provider in IAM (IRSA) or enable the EKS Pod Identity agent
- [ ] One IAM role per service; trust pinned to namespace + service account
- [ ] Migrate services off node-role and env-var creds; verify AssumeRoleWithWebIdentity in CloudTrail
- [ ] Delete AWS credentials from Kubernetes Secrets and task-definition env vars

## Phase 2: CI/CD (weeks 3-5)
- [ ] Create the token.actions.githubusercontent.com OIDC provider in IAM
- [ ] Deploy role trust pinned to repo + branch (JSON below); separate plan-only role for PRs
- [ ] Cut workflows over to aws-actions/configure-aws-credentials with role-to-assume
- [ ] Deactivate, then delete, the CI IAM user and its stored repo secrets

## Phase 3: Humans (weeks 4-6)
- [ ] IAM Identity Center connected to the IdP; permission sets replace per-user policies
- [ ] CLI access via `aws sso login`; session duration 12h or less
- [ ] Create 2 break-glass IAM users: hardware MFA, keys stored offline, CloudTrail alarm on use
- [ ] Flip all remaining IAM user keys Inactive for 7 days, then delete the users

## Phase 4: Data stores (weeks 5-8)
- [ ] RDS IAM auth for low-churn services (Postgres/MySQL/Aurora)
- [ ] Secrets Manager rotation (managed Lambda, alternating users) for the rest
- [ ] Apps fetch DB credentials at connect time from Secrets Manager, never from env vars
- [ ] Rotate every DB password once through the new path to invalidate leaked history

## Phase 5: Third parties (ongoing)
- [ ] Per vendor key: minimum scope, Secrets Manager entry, named owner, 90-day rotation
- [ ] Usage alerting per key (API audit log or billing anomaly)
- [ ] Fold key-rotation posture into vendor risk reviews

## Steady state
- [ ] Static-credential count published weekly (target: break-glass plus vendor keys, nothing else)
- [ ] CI check blocking AKIA* and other known key formats in commits
- [ ] SCP denying iam:CreateUser and iam:CreateAccessKey outside the break-glass path (SCPs never bind the Organizations management account, so this holds only if workloads live in member accounts; single-account estates: explicit deny in a permissions boundary plus a CloudTrail/EventBridge alarm on CreateUser/CreateAccessKey)
```

The GitHub Actions trust policy, with the conditions that matter:

```json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::111122223333:oidc-provider/token.actions.githubusercontent.com"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "token.actions.githubusercontent.com:aud": "sts.amazonaws.com",
          "token.actions.githubusercontent.com:sub": "repo:your-org/your-repo:ref:refs/heads/main"
        }
      }
    }
  ]
}
```

Never widen the `sub` condition to `repo:your-org/*` on the deploy role. If several repos genuinely need deploy access, give each its own statement or its own role.

## Operating Notes

### Revocation is the real test

If revoking a particular credential feels frightening, you have located a system whose ownership and issuance path nobody understands. That fear is the finding; write it down before you fix it. The parallel-run pattern exists to convert fear into boredom: the new path serves traffic for a week while the old credential sits deactivated but recoverable. Apply the same discipline to break-glass. Twice a year, actually open the safe, actually use the offline key, and confirm the alarm pages someone. A break-glass procedure that has never been exercised is a decorative one.

### The audit trail arrives for free

Every `AssumeRole` and `AssumeRoleWithWebIdentity` call is a CloudTrail event binding an action to an identity, a session, and a timestamp. Once credentials are issued rather than stored, your rotation control is satisfied structurally: the auditor question "how do you rotate credentials?" gets the answer "they expire hourly, and issuance is logged." That is a materially stronger position for SOC 2 CC6.1 than a spreadsheet of quarterly rotations, and it feeds directly into [evidence pipelines](/library/evidence-pipelines-101), because the proof is a queryable log rather than a screenshot someone remembers to take.

### Don't chase the last three

The static-credential count follows an asymptote. It falls fast through the dead keys and CI secrets, slower through data stores, and then lands on a small stubborn number: break-glass plus the vendors that will not do OIDC. Accept the asymptote and guard the perimeter instead. An SCP denying new access-key creation means the count cannot silently climb back up, and a CI secret-format check means the next leaked key is caught at commit time. The durable win is cultural: engineers who joined after the migration have never configured a static AWS key and would not know where to put one. At that point the migration has stopped being a project and become the default, which is the only finish line that holds.

## Frequently asked questions

**Q: Why are long-lived credentials such a security risk?**

Because stolen does not mean expired: a static key keeps working until a human notices and revokes it, and credential-based intrusions have averaged around 240 days to identify. A short-lived credential lifted from a compromised pod is worth an hour to an attacker; a long-lived one is worth however many months pass before someone catches it.

**Q: How do I replace AWS access keys in my GitHub Actions pipeline?**

Register `token.actions.githubusercontent.com` as an IAM identity provider and have the pipeline assume a role directly via GitHub's native OIDC token, instead of storing `AWS_ACCESS_KEY_ID` in repository secrets. Pin the trust policy's `sub` to a specific repo and ref rather than a `repo:your-org/*` wildcard, and give pull-request builds a separate plan-only role.

**Q: What is the difference between IRSA and EKS Pod Identity?**

Both give pods short-lived credentials without stored keys, but IRSA registers a per-cluster OIDC provider in IAM and matches a role's trust policy to the token's `sub` claim, while EKS Pod Identity (GA since late 2023) skips that registration and manages role associations through a plain EKS API call — easier once you run several clusters. The trade-offs: Pod Identity does not cover Fargate pods and needs explicit role chaining for cross-account access, so pick one mechanism per cluster and stay consistent.

**Q: Do I need HashiCorp Vault to get rid of database passwords?**

Usually not. RDS IAM authentication swaps the password for a 15-minute connection token on Postgres, MySQL, and Aurora, and Secrets Manager with managed Lambda rotators handles the rest on a schedule. Vault's dynamic per-lease database users go further, but at 10-80 engineers running Vault solely for database credentials is overkill — it earns its keep when you are multi-cloud or already operate it for something else.

**Q: How do I migrate off static credentials without breaking production?**

Work one credential class at a time in four moves: inventory, build the new issuance path, run both in parallel, then revoke the old credential. Deactivation is the safety net — flip an IAM key Inactive for a week and watch CloudTrail before deleting it. Sequence by last-used data: dead keys first, then CI, then workloads, then humans.

**Q: Does killing long-lived credentials help with SOC 2 compliance?**

Yes. Once credentials are issued rather than stored, every AssumeRole call is a CloudTrail event binding an action to an identity, a session, and a timestamp, so your rotation control is satisfied structurally — they expire hourly and issuance is logged. That is a materially stronger position for SOC 2 CC6.1 than a spreadsheet of quarterly rotations, and it feeds evidence pipelines directly.

Template: "No-Long-Lived-Credentials Migration Checklist" — branded PDF at https://kaansystems.com/templates/killing-long-lived-credentials.pdf

Source: https://kaansystems.com/library/killing-long-lived-credentials · Work with Kaan Systems: https://kaansystems.com/contact