Verify Results

After a scan completes, verify the results to make pass/fail decisions for your CI/CD pipeline. This page explains the verification logic and provides implementation examples.

Verification Logic

The verification workflow handles two scenarios:

Scenario 1: First Image

When no image was uploaded to the product before the one this build uploaded, the verification checks that image for findings matching the configured statuses. If any are found, the build fails.

Scenario 2: Comparison with the Previous Image

When the product has an image uploaded before this one, the verification:

Compares the image this build uploaded (IMAGE_ID) with the newest image uploaded before it
Reports statistics on resolved, unchanged, and newly introduced findings
Fails if any findings with configured statuses exist in the image this build uploaded

The script picks the comparison image by the upload time of IMAGE_ID. Images that another pipeline or a user uploads to the same product while the build runs don’t change the comparison.

Configuration

Finding Status Filter

Control which finding statuses cause the build to fail using the FAIL_ON_STATUS environment variable:

Status	Description
`new`	Newly discovered, not yet triaged
`inProgress`	Currently being investigated
`rejected`	Marked as false positive or won’t fix
`remediated`	Fixed but still detected (unusual)

Default: new,inProgress, which fails on findings that require attention. Examples:

# Fail only on new (untriaged) findings
export FAIL_ON_STATUS="new"

# Fail on new and in-progress (default)
export FAIL_ON_STATUS="new,inProgress"

# Strict: Fail on any non-remediated finding
export FAIL_ON_STATUS="new,inProgress,rejected"

Comparison Statistics

The comparison API categorizes findings into three groups:

Status	Meaning	API Filter
Resolved	Was in previous image, not in latest	`compareSide=left`
Unchanged	Present in both images	`compareSide=both`
New	Not in previous, appeared in latest	`compareSide=right`

Each query sets compareLeftImageId to the previous image and compareRightImageId to the latest image. The API ignores filter fields it doesn’t know instead of returning an error, so a misspelled field returns all findings.

API Endpoints Used

Endpoint	Purpose
`GET /api/v4/products/{productId}/images`	List images to find the previous image
`POST /api/v4/grids/findings:gridList`	Query findings with filters for status and comparison

Implementation

Bash

#!/bin/bash
# Binarly CI/CD Verification Script
# Requires: curl, jq
# Environment: BINARLY_CLIENT_ID, BINARLY_CLIENT_SECRET, BINARLY_AUTH_URL,
#   BINARLY_API_URL, BINARLY_PRODUCT_ID, IMAGE_ID (from the upload step)
# Optional: FAIL_ON_STATUS (default: "new,inProgress")

set -eo pipefail

# Access tokens expire after 30 minutes, so request a new one for this step
get_token() {
  curl -sS --fail-with-body \
    -H "Content-Type: application/x-www-form-urlencoded" \
    --data-urlencode "grant_type=client_credentials" \
    --data-urlencode "client_id=${BINARLY_CLIENT_ID}" \
    --data-urlencode "client_secret=${BINARLY_CLIENT_SECRET}" \
    "${BINARLY_AUTH_URL}" | jq -er '.access_token'
}

# Configuration: Which statuses should fail the build
FAIL_ON_STATUS="${FAIL_ON_STATUS:-new,inProgress}"

echo "Binarly Security Verification"
echo "Failing on statuses: ${FAIL_ON_STATUS}"
echo ""

TOKEN=$(get_token)

# Find the image uploaded before this one, to compare against
IMAGES_RES=$(curl -sS --fail-with-body -H "Authorization: Bearer ${TOKEN}" \
  "${BINARLY_API_URL}/api/v4/products/${BINARLY_PRODUCT_ID}/images")
PREVIOUS_ID=$(echo "$IMAGES_RES" | jq -r --arg id "$IMAGE_ID" '
  (.images[] | select(.id == $id) | .createTime) as $created
  | [.images[] | select(.createTime < $created)]
  | sort_by(.createTime) | last | .id // empty')

if [ -n "$PREVIOUS_ID" ]; then
  echo "Comparing images:"
  echo "   Previous: ${PREVIOUS_ID}"
  echo "   Latest:   ${IMAGE_ID}"
  echo ""

  # Count RESOLVED findings (only in previous image = left side)
  RESOLVED=$(curl -sS --fail-with-body -X POST \
    -H "Authorization: Bearer ${TOKEN}" \
    -H "Content-Type: application/json" \
    -d '{
      "filters": [
        {"field": "compareLeftImageId", "value": "'"${PREVIOUS_ID}"'", "comparator": "equals"},
        {"field": "compareRightImageId", "value": "'"${IMAGE_ID}"'", "comparator": "equals"},
        {"field": "compareSide", "value": "left", "comparator": "equals"}
      ]
    }' \
    "${BINARLY_API_URL}/api/v4/grids/findings:gridList" | jq -e '.total')

  # Count UNCHANGED findings (present in both images)
  UNCHANGED=$(curl -sS --fail-with-body -X POST \
    -H "Authorization: Bearer ${TOKEN}" \
    -H "Content-Type: application/json" \
    -d '{
      "filters": [
        {"field": "compareLeftImageId", "value": "'"${PREVIOUS_ID}"'", "comparator": "equals"},
        {"field": "compareRightImageId", "value": "'"${IMAGE_ID}"'", "comparator": "equals"},
        {"field": "compareSide", "value": "both", "comparator": "equals"}
      ]
    }' \
    "${BINARLY_API_URL}/api/v4/grids/findings:gridList" | jq -e '.total')

  # Count NEW findings (only in latest image = right side)
  NEW_ISSUES=$(curl -sS --fail-with-body -X POST \
    -H "Authorization: Bearer ${TOKEN}" \
    -H "Content-Type: application/json" \
    -d '{
      "filters": [
        {"field": "compareLeftImageId", "value": "'"${PREVIOUS_ID}"'", "comparator": "equals"},
        {"field": "compareRightImageId", "value": "'"${IMAGE_ID}"'", "comparator": "equals"},
        {"field": "compareSide", "value": "right", "comparator": "equals"}
      ]
    }' \
    "${BINARLY_API_URL}/api/v4/grids/findings:gridList" | jq -e '.total')

  echo "Comparison Statistics:"
  printf "  Resolved from previous:    %6d\n" "$RESOLVED"
  printf "  Unchanged (still present): %6d\n" "$UNCHANGED"
  printf "  Newly introduced:          %6d\n" "$NEW_ISSUES"
  echo ""
fi

# Count findings in this image with the configured statuses
# Convert comma-separated statuses to a JSON array
STATUSES_JSON=$(echo "$FAIL_ON_STATUS" | tr -d ' ' | tr ',' '\n' | jq -R . | jq -s .)

TOTAL_ACTIONABLE=$(curl -sS --fail-with-body -X POST \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{
    "filters": [
      {"field": "imageId", "value": "'"${IMAGE_ID}"'", "comparator": "equals"},
      {"field": "issueStatus", "value": '"${STATUSES_JSON}"', "comparator": "in"}
    ]
  }' \
  "${BINARLY_API_URL}/api/v4/grids/findings:gridList" | jq -e '.total')

echo "Statuses checked: ${FAIL_ON_STATUS}"
echo "Total actionable findings: ${TOTAL_ACTIONABLE}"
echo ""

# Pass/Fail decision
if [ "$TOTAL_ACTIONABLE" -gt 0 ]; then
  echo "BUILD FAILED: ${TOTAL_ACTIONABLE} actionable finding(s)"
  exit 1
fi
echo "BUILD PASSED: No actionable findings"

Output Example

Comparison with a previous image, build fails on actionable findings:

Binarly Security Verification
Failing on statuses: new,inProgress

Comparing images:
   Previous: 01JYJ0W0AACGC7QT1Q21SRWG94
   Latest:   01JYJX8K2BBDF9PT3R22TSWH05

Comparison Statistics:
  Resolved from previous:         0
  Unchanged (still present):     24
  Newly introduced:               0

Statuses checked: new,inProgress
Total actionable findings: 24

BUILD FAILED: 24 actionable finding(s)

Next Steps

Triage Findings: Review and triage findings in your Binarly dashboard at https://dashboard-{slug}.binarly.cloud
Adjust Thresholds: Combine with severity filters - see Compare Findings API
See Also: Compare Findings API for advanced filtering options

Overview

Use Cases

Reference