Skip to main content
After a scan completes, verify the results to make pass/fail decisions for your CI/CD pipeline. This page explains the verification logic and provides implementation examples.

Verification Logic

The verification workflow handles two scenarios:

Scenario 1: Single Image (First Scan)

When the product contains only one image (the latest scan), the verification checks for findings matching the configured statuses. If found, the build fails.

Scenario 2: Multiple Images (Comparison)

When multiple images exist, the verification:
  1. Compares the latest image with the previous image
  2. Reports statistics on resolved, unchanged, and newly introduced findings
  3. Fails if any findings with configured statuses exist in the latest image

Configuration

Finding Status Filter

Control which finding statuses cause the build to fail using the FAIL_ON_STATUS environment variable:
StatusDescription
newNewly discovered, not yet triaged
inProgressCurrently being investigated
rejectedMarked as false positive or won’t fix
remediatedFixed but still detected (unusual)
Default: new,inProgress – Fails on findings that require attention. Examples:
# Fail only on new (untriaged) findings
export FAIL_ON_STATUS="new"

# Fail on new and in-progress (default)
export FAIL_ON_STATUS="new,inProgress"

# Strict: Fail on any non-remediated finding
export FAIL_ON_STATUS="new,inProgress,rejected"

Comparison Statistics

The comparison API categorizes findings into three groups:
StatusMeaningAPI Filter
ResolvedWas in previous image, not in latestcomparison.status=notFound
UnchangedPresent in both imagescomparison.status=notChanged
NewNot in previous, appeared in latestcomparison.status=found

API Endpoints Used

EndpointPurpose
GET /api/v4/products/{productId}/imagesList images to determine count and get IDs
POST /api/v4/grids/findings:gridListQuery findings with filters for status and comparison

Implementation

#!/bin/bash
# Binarly CI/CD Verification Script
# Requires: curl, jq
# Environment: TOKEN, BINARLY_API_URL, BINARLY_PRODUCT_ID, IMAGE_ID (from upload step)
# Optional: FAIL_ON_STATUS (default: "new,inProgress")

set -e

# Configuration: Which statuses should fail the build
FAIL_ON_STATUS="${FAIL_ON_STATUS:-new,inProgress}"

echo "Binarly Security Verification"
echo "Failing on statuses: ${FAIL_ON_STATUS}"
echo ""

# 1. Get all images for the product
IMAGES_RES=$(curl -s -H "Authorization: Bearer ${TOKEN}" \
  "${BINARLY_API_URL}/api/v4/products/${BINARLY_PRODUCT_ID}/images")
IMAGE_COUNT=$(echo "$IMAGES_RES" | jq '.images | length')
echo "Product has ${IMAGE_COUNT} image(s)"

# Initialize statistics
RESOLVED=0
UNCHANGED=0
NEW_ISSUES=0

# 2. If multiple images exist, compare latest with previous
if [ "$IMAGE_COUNT" -gt 1 ]; then
  # Sort images by createTime descending and get IDs of two most recent
  LATEST_ID=$(echo "$IMAGES_RES" | jq -r '[.images | sort_by(.createTime) | reverse][0][0].id')
  PREVIOUS_ID=$(echo "$IMAGES_RES" | jq -r '[.images | sort_by(.createTime) | reverse][0][1].id')
  
  echo "Comparing images:"
  echo "   Previous: ${PREVIOUS_ID}"
  echo "   Latest:   ${LATEST_ID}"
  echo ""
  
  # Count RESOLVED findings (only in previous image = left side)
  RESOLVED=$(curl -s -X POST \
    -H "Authorization: Bearer ${TOKEN}" \
    -H "Content-Type: application/json" \
    -d '{
      "filters": [
        {"field": "compareLeftImageId", "value": "'"${PREVIOUS_ID}"'", "comparator": "equals"},
        {"field": "compareRightImageId", "value": "'"${LATEST_ID}"'", "comparator": "equals"},
        {"field": "compareSide", "value": "left", "comparator": "equals"}
      ]
    }' \
    "${BINARLY_API_URL}/api/v4/grids/findings:gridList" | jq '.total // 0')
  
  # Count UNCHANGED findings (present in both images)
  UNCHANGED=$(curl -s -X POST \
    -H "Authorization: Bearer ${TOKEN}" \
    -H "Content-Type: application/json" \
    -d '{
      "filters": [
        {"field": "compareLeftImageId", "value": "'"${PREVIOUS_ID}"'", "comparator": "equals"},
        {"field": "compareRightImageId", "value": "'"${LATEST_ID}"'", "comparator": "equals"},
        {"field": "compareSide", "value": "both", "comparator": "equals"}
      ]
    }' \
    "${BINARLY_API_URL}/api/v4/grids/findings:gridList" | jq '.total // 0')
  
  # Count NEW findings (only in latest image = right side)
  NEW_ISSUES=$(curl -s -X POST \
    -H "Authorization: Bearer ${TOKEN}" \
    -H "Content-Type: application/json" \
    -d '{
      "filters": [
        {"field": "compareLeftImageId", "value": "'"${PREVIOUS_ID}"'", "comparator": "equals"},
        {"field": "compareRightImageId", "value": "'"${LATEST_ID}"'", "comparator": "equals"},
        {"field": "compareSide", "value": "right", "comparator": "equals"}
      ]
    }' \
    "${BINARLY_API_URL}/api/v4/grids/findings:gridList" | jq '.total // 0')
  
  echo "Comparison Statistics:"
  printf "  Resolved from previous:    %6d\n" "$RESOLVED"
  printf "  Unchanged (still present): %6d\n" "$UNCHANGED"
  printf "  Newly introduced:          %6d\n" "$NEW_ISSUES"
  echo ""
fi

# 3. Check for findings matching configured statuses (using 'in' comparator with array)
# Convert comma-separated statuses to JSON array
STATUSES_JSON=$(echo "$FAIL_ON_STATUS" | tr ',' '\n' | jq -R . | jq -s .)

# Use LATEST_ID if available (from comparison), otherwise use IMAGE_ID from upload step
CHECK_IMAGE_ID="${LATEST_ID:-${IMAGE_ID}}"

TOTAL_ACTIONABLE=$(curl -s -X POST \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{
    "filters": [
      {"field": "imageId", "value": "'"${CHECK_IMAGE_ID}"'", "comparator": "equals"},
      {"field": "issueStatus", "value": '"${STATUSES_JSON}"', "comparator": "in"}
    ]
  }' \
  "${BINARLY_API_URL}/api/v4/grids/findings:gridList" | jq '.total // 0')

echo "   Statuses checked: ${FAIL_ON_STATUS}"

echo ""
echo "Total actionable findings: ${TOTAL_ACTIONABLE}"
echo ""

# 4. Pass/Fail decision
if [ "$TOTAL_ACTIONABLE" -gt 0 ]; then
  echo "BUILD FAILED: ${TOTAL_ACTIONABLE} actionable finding(s)"
  exit 1
else
  echo "BUILD PASSED: No actionable findings"
  exit 0
fi

Output Example

Multi-image scenario with issues:
Binarly Security Verification
Failing on statuses: new,inProgress
Product has 5 image(s)
Comparing images:
   Previous: 01JYJ0W0AACGC7QT1Q21SRWG94
   Latest:   01JYJX8K2BBDF9PT3R22TSWH05

Comparison Statistics:
  Resolved from previous:        12
  Unchanged (still present):     45
  Newly introduced:               3

   new: 2
   inProgress: 1

Total actionable findings: 3

BUILD FAILED: 3 actionable finding(s)

Next Steps