stateofsecurity.com
A Secure Pipeline Is Not the Same as a Trustworthy Release
The deployment dashboard is green.
The build completed. The scanners ran. The change ticket is closed. The application is in production.
Now ask a harder question:
Can the organization prove that the software running in production came from the reviewed source, was built in the approved environment, passed the required controls, retained every accepted exception, and arrived without substitution?
If the answer is only, “The pipeline succeeded,” the organization has not answered the question.
A pipeline is a workflow. A trustworthy release is a security claim supported by evidence.
That distinction matters because modern delivery systems do more than compile code. They pull dependencies, execute third-party actions, create infrastructure, retrieve secrets, generate artifacts, promote packages, and deploy to production. A green status can show that automation reached its final step. It does not automatically prove that every important input was authorized, every control ran as intended, or the exact artifact that passed review is the artifact now operating.
The National Institute of Standards and Technology reinforced this point in a September 24 update to its live Secure Software Development, Security, and Operations Practices project. The new material maps the Secure Software Development Framework to a DevSecOps model and adds functional scenarios across the software lifecycle. Those scenarios connect build isolation, integrity checks, provenance, software bills of materials, test records, signing, release policy, and deployment verification.
The practical lesson is straightforward:
> Do not trust a release because the pipeline is busy with security tools. Trust it when the evidence remains connected from approved input to verified production artifact.
## Green Is a Workflow State, Not a Trust Decision
Security teams commonly describe a pipeline by listing its tools.
There is source-code analysis. There is dependency scanning. There is infrastructure-as-code scanning. There is secret detection. There may be container scanning, dynamic testing, artifact signing, and a software bill of materials.
Each capability can reduce risk. Together, however, they still may not answer basic release questions:
* Which source commit was actually built?
* Which dependencies and build instructions were used?
* Which identity requested and executed the build?
* Did every required control run, or was one skipped, disabled, or misconfigured?
* Were findings resolved, suppressed, or accepted as risk?
* Which immutable artifact was approved?
* Was that exact artifact deployed?
* Can the evidence be reconstructed after the pipeline logs expire?
A scanner can report on the artifact it received without proving that the artifact was the one later deployed. A signature can protect integrity without proving that the signer was authorized to approve the release. An SBOM can describe components without proving that the build environment was trustworthy. A ticket can record approval without being bound to a specific artifact digest.
These are not arguments against scanners, signatures, SBOMs, or change records. They are reasons to connect them.
NIST Special Publication 800-204D describes CI/CD pipelines as the flow that takes software through build, test, package, and deployment activities, and it outlines strategies for integrating software-supply-chain security into that flow. The security objective is therefore not merely to add controls at individual stages. It is to maintain confidence as software and evidence move between them. (NIST SP 800-204D)
That requires a release evidence chain.
## Build the Release Evidence Chain
A useful release evidence chain answers four questions in sequence:
1. **What entered the build?**
2. **What happened during the build and test process?**
3. **What exactly was approved for release?**
4. **What exactly reached production?**
The value comes from continuity. If any link refers to a different commit, run, artifact, environment, or approval, the chain is incomplete.
### 1. Establish the Inputs
Start with the inputs that define the release.
At minimum, the evidence should identify:
* The source repository and exact commit.
* The approved change, review, or pull request.
* The build definition and versioned pipeline configuration.
* Direct dependencies and other retrieved build materials.
* Infrastructure-as-code and deployment configuration.
* The builder identity and requesting identity.
* The target environment and release criteria.
This is where apparently reproducible work often becomes ambiguous. A pipeline may reference a mutable branch, a floating container tag, an unpinned third-party action, or a package version range. The build can succeed twice while consuming different material.
The Supply-chain Levels for Software Artifacts provenance specification defines provenance as verifiable information about where, when, and how an artifact was produced. Its model includes the build process, top-level inputs, resolved dependencies, builder identity, and output artifacts. Not every organization needs to implement the highest SLSA level immediately. Every organization should be able to identify the inputs that materially determine a production artifact.
The governing question is:
> Can we reconstruct what the build trusted, not merely where the source repository lives?
### 2. Prove What Executed
The next link is the execution record.
NIST’s functional DevSecOps scenarios describe automated builds using ephemeral environments, integrated analysis, repeatable processes, and logged results. They also demonstrate integrity checks for source, commits, images, binaries, and libraries; provenance creation; signed SBOM generation; and captured scanning output. These are examples rather than universal prescriptions, but they show what trustworthy execution must make visible. (NIST functional demonstration scenarios)
For a consequential release, evidence should show:
* The build ran in an approved and appropriately isolated environment.
* The expected pipeline definition executed.
* Required tests and security checks ran against the intended input or artifact.
* Results were recorded, including tool errors and incomplete scans.
* Policy gates evaluated those results.
* Any manual intervention or override was attributable.
* The build environment did not quietly modify the release after testing.
This last point is easy to miss. If a package is rebuilt during promotion, if configuration is injected outside the governed process, or if a production image is assembled from components different from those tested, the organization has created a new artifact. Prior evidence may no longer describe it.
The proof should also distinguish **passed** from **did not run**. An unavailable scanner, an empty report, a timed-out test, and a successful control are different states. A secure release process fails closed where risk demands it and makes degraded controls explicit where business continuity requires an exception.
### 3. Bind Approval to the Artifact
Release approval should name the thing being approved.
That normally means an immutable identifier such as a cryptographic digest, accompanied by evidence that explains why the artifact is eligible for promotion. The release packet can be assembled automatically and may include:
* Artifact name, version, and digest.
* Build provenance or attestation.
* Signature and signing identity.
* SBOM and dependency analysis.
* Test and scan results.
* Security and functional acceptance criteria.
* Open findings and their disposition.
* Approved exceptions, owners, expiration dates, and compensating controls.
* Change and release approvals.
NIST’s new functional scenarios explicitly connect release documentation with records from tests, scans, and compliance checks. They also include signing and verification before deployment and policies intended to allow only signed, validated software from approved build and test processes to proceed. (NIST functional demonstration scenarios)
This is more than audit packaging.
If an approver accepts risk for artifact A, the approval should not be reusable for artifact B. If a critical scanner did not run, the record should not look identical to a fully tested release. If an emergency change bypasses a normal gate, the deviation should remain attached to the artifact until it is resolved or retired.
An exception hidden in a ticketing system is not part of the release decision unless the promotion process can find and enforce it.
### 4. Verify Promotion and Deployment
The final link is often the weakest.
Teams may secure the build, sign the artifact, and preserve provenance—then deploy by a path that does not verify any of it.
Provenance is useful only when a consumer checks it. The SLSA artifact-verification guidance calls for comparing the artifact with its provenance, validating the provenance signature, checking the builder identity, and confirming that build parameters match expectations.
At promotion and deployment, the organization should verify:
* The artifact digest matches the approved release record.
* The signature and attestation are authentic and valid.
* The builder and signer are trusted for that application and environment.
* Required evidence exists and satisfies policy.
* The target environment received the approved immutable artifact.
* Unauthorized or unsigned substitutions are rejected.
* Deployment and runtime inventory can identify the artifact actually operating.
NIST’s scenarios carry this principle into deployment by verifying SBOM and provenance data before deployment and applying policies so only signed and validated software is deployed. The objective is not to sign everything and declare victory. It is to make authenticity and policy verification part of the consuming action.
This closes the most important gap:
> The artifact that passed the controls must be the artifact that received the approval, and the artifact that received the approval must be the artifact that reached production.
## An Evidence Packet Is Not a Folder Full of Reports
Organizations sometimes respond to assurance demands by saving more output.
That creates volume, not necessarily proof.
A release evidence packet should be machine-associated with a specific build and artifact. It should preserve relationships, not just documents:
* Commit to build invocation.
* Build invocation to artifact digest.
* Artifact digest to test and scan results.
* Findings to disposition and exception.
* Approval to artifact digest.
* Artifact digest to deployment event.
* Deployment event to running workload or asset inventory.
The packet does not need to be a single file or a new platform. It can be a set of signed attestations, repository records, policy decisions, tickets, and deployment events. What matters is that the organization can follow the links without relying on memory, screenshots, or an engineer who remembers what happened.
The NIST Secure Software Development Framework is deliberately high-level so organizations can integrate its practices into different development lifecycles. That flexibility is useful. It also means leaders should define what evidence demonstrates each practice in their environment.
“We use the tool” is a capability statement.
“Here is the result for the exact artifact in production, and here is how policy consumed it” is an assurance statement.
## Measure Evidence Continuity, Not Security-Tool Count
Counting pipeline tools rewards installation. Counting findings rewards detection volume. Neither measure shows whether the release decision can be trusted.
More useful measures include:
* Percentage of production releases with verified provenance.
* Percentage of deployments performed by immutable digest rather than mutable tag or name.
* Percentage of production artifacts whose signature and approval were verified at deployment.
* Percentage of required controls that produced a conclusive result for each release.
* Percentage of exceptions linked to an artifact, named owner, compensating control, and expiration date.
* Time required to identify the source, builder, evidence, approval, and deployment history for a running artifact.
* Percentage of emergency releases whose evidence gaps were reconciled within the required time.
* Number of unauthorized, unsigned, or mismatched artifacts rejected during controlled testing.
One particularly revealing metric is **time to prove production** :
> Starting with a running workload, how long does it take to prove what source produced it, how it was built, what controls ran, who accepted the residual risk, and how the artifact was promoted?
If the answer requires several teams, multiple consoles, and manual correlation, the evidence chain is fragile even when every tool is technically present.
## Test One Release End to End
Do not begin with an enterprise transformation.
Choose one internet-facing or otherwise consequential service and one recent production release. Ask a developer, platform engineer, security analyst, and service owner to reconstruct the release together.
Start from production and work backward:
1. Identify the exact running artifact by immutable digest or equivalent identifier.
2. Find the deployment event and show that it referenced the same identifier.
3. Find the approval and confirm that it applied to that artifact.
4. Retrieve the test, scan, and policy results associated with the artifact.
5. Identify every exception and verify its owner, rationale, compensating control, and expiration.
6. Retrieve the provenance, signature, SBOM, and build logs.
7. Trace the build to the exact source commit, build definition, dependencies, builder, and requesting identity.
8. Confirm that the evidence is retained long enough for incident response, audit, and operational learning.
Then run a negative test in a safe environment.
Try to promote an unsigned artifact. Change the digest after approval. Skip a required control. Use an unapproved builder. Present provenance with an unexpected parameter. Confirm that the release process rejects the change and produces an event responders can use.
The negative test matters because a dashboard can show that the approved path works without proving that an unapproved path is blocked.
The exercise will probably reveal governance issues as well as technical ones. Teams may disagree about who owns the policy, which findings are release-blocking, how emergency changes are reconciled, how long evidence is retained, or whether a vendor-managed deployment exposes enough detail to verify the chain.
Those disagreements are part of the finding.
## Scale Assurance by Consequence
Not every internal script needs the same release process as a payment platform, clinical system, or internet-facing identity service.
Use consequence to set the assurance level.
For low-impact software, a minimum evidence packet may include the commit, build identity, artifact digest, required test results, and deployment record. Higher-impact systems may require isolated or hardened builders, signed provenance, verified SBOMs, independent approval, stronger separation of duties, policy enforcement at deployment, and longer evidence retention.
The important point is to make the differences explicit.
A lower-assurance release should be a risk decision, not an accidental byproduct of team maturity. The organization should know which systems lack provenance, which deployment paths accept mutable artifacts, which controls can be bypassed, and who owns the improvement plan.
This also keeps the program practical. The goal is not to create a paperwork tax that slows every change. Evidence should be generated and connected by the delivery process wherever possible. Human attention should focus on policy, exceptions, high-consequence decisions, and failed verification.
Automation moves software quickly. It should move proof with equal discipline.
## What Security Leaders Should Do This Week
Start with one production service and one release.
1. **Name the artifact.** Record the immutable identifier for what is actually running.
2. **Trace the chain backward.** Connect deployment, approval, test results, build record, and reviewed source.
3. **Find the silent states.** Identify controls that can time out, fail open, return empty results, or be skipped without a distinct release decision.
4. **Bind exceptions to releases.** Require an owner, rationale, compensating control, expiration date, and artifact reference.
5. **Verify at deployment.** Confirm that signatures, provenance, builder identity, and required evidence are checked when software is promoted—not merely created earlier.
6. **Run one negative test.** Attempt a safe substitution or skipped-control scenario and verify that the process blocks it.
7. **Measure time to prove production.** Set a target for reconstructing the complete evidence chain during an incident or audit.
A modern pipeline can run dozens of tools and still produce an untrustworthy release.
The difference is evidence continuity.
When source, build, controls, exceptions, approval, artifact, and deployment remain bound together, the organization can make a defensible claim about what reached production.
Without that chain, green means only that the workflow finished.
* * *
## More Information and Assistance
MicroSolved, Inc. can help organizations:
* Assess CI/CD, DevSecOps, and software-supply-chain controls.
* Map release evidence from source through production deployment.
* Review build isolation, signing, provenance, SBOM, and artifact-verification practices.
* Test release gates, exception handling, and unauthorized-artifact rejection.
* Design practical assurance tiers for high-consequence systems and vendors.
* Build release, incident-response, and audit evidence playbooks.
Contact MicroSolved at info@microsolved.com or +1.614.351.1237.
Relax. We’re on watch.
_AI tools were used as a research assistant for this content, but human moderation and writing are also included._
