Verify Results

After a scan completes, verify the results to make pass/fail decisions for your CI/CD pipeline. This page explains the verification logic and provides implementation examples.

Verification Logic

The verification workflow handles two scenarios:

Scenario 1: Single Image (First Scan)

When the product contains only one image (the latest scan), the verification checks for findings matching the configured statuses. If found, the build fails.

Scenario 2: Multiple Images (Comparison)

When multiple images exist, the verification:

Compares the latest image with the previous image
Reports statistics on resolved, unchanged, and newly introduced findings
Fails if any findings with configured statuses exist in the latest image

Configuration

Finding Status Filter

Control which finding statuses cause the build to fail using the FAIL_ON_STATUS environment variable:

Status	Description
`new`	Newly discovered, not yet triaged
`inProgress`	Currently being investigated
`rejected`	Marked as false positive or won’t fix
`remediated`	Fixed but still detected (unusual)

Default: new,inProgress – Fails on findings that require attention. Examples:

# Fail only on new (untriaged) findings
export FAIL_ON_STATUS="new"

# Fail on new and in-progress (default)
export FAIL_ON_STATUS="new,inProgress"

# Strict: Fail on any non-remediated finding
export FAIL_ON_STATUS="new,inProgress,rejected"

Comparison Statistics

The comparison API categorizes findings into three groups:

Status	Meaning	API Filter
Resolved	Was in previous image, not in latest	`comparison.status=notFound`
Unchanged	Present in both images	`comparison.status=notChanged`
New	Not in previous, appeared in latest	`comparison.status=found`

API Endpoints Used

Endpoint	Purpose
`GET /api/v4/products/{productId}/images`	List images to determine count and get IDs
`POST /api/v4/grids/findings:gridList`	Query findings with filters for status and comparison

Implementation

Bash

#!/bin/bash
# Binarly CI/CD Verification Script
# Requires: curl, jq
# Environment: TOKEN, BINARLY_API_URL, BINARLY_PRODUCT_ID, IMAGE_ID (from upload step)
# Optional: FAIL_ON_STATUS (default: "new,inProgress")

set -e

# Configuration: Which statuses should fail the build
FAIL_ON_STATUS="${FAIL_ON_STATUS:-new,inProgress}"

echo "Binarly Security Verification"
echo "Failing on statuses: ${FAIL_ON_STATUS}"
echo ""

# 1. Get all images for the product
IMAGES_RES=$(curl -s -H "Authorization: Bearer ${TOKEN}" \
  "${BINARLY_API_URL}/api/v4/products/${BINARLY_PRODUCT_ID}/images")
IMAGE_COUNT=$(echo "$IMAGES_RES" | jq '.images | length')
echo "Product has ${IMAGE_COUNT} image(s)"

# Initialize statistics
RESOLVED=0
UNCHANGED=0
NEW_ISSUES=0

# 2. If multiple images exist, compare latest with previous
if [ "$IMAGE_COUNT" -gt 1 ]; then
  # Sort images by createTime descending and get IDs of two most recent
  LATEST_ID=$(echo "$IMAGES_RES" | jq -r '[.images | sort_by(.createTime) | reverse][0][0].id')
  PREVIOUS_ID=$(echo "$IMAGES_RES" | jq -r '[.images | sort_by(.createTime) | reverse][0][1].id')
  
  echo "Comparing images:"
  echo "   Previous: ${PREVIOUS_ID}"
  echo "   Latest:   ${LATEST_ID}"
  echo ""
  
  # Count RESOLVED findings (only in previous image = left side)
  RESOLVED=$(curl -s -X POST \
    -H "Authorization: Bearer ${TOKEN}" \
    -H "Content-Type: application/json" \
    -d '{
      "filters": [
        {"field": "compareLeftImageId", "value": "'"${PREVIOUS_ID}"'", "comparator": "equals"},
        {"field": "compareRightImageId", "value": "'"${LATEST_ID}"'", "comparator": "equals"},
        {"field": "compareSide", "value": "left", "comparator": "equals"}
      ]
    }' \
    "${BINARLY_API_URL}/api/v4/grids/findings:gridList" | jq '.total // 0')
  
  # Count UNCHANGED findings (present in both images)
  UNCHANGED=$(curl -s -X POST \
    -H "Authorization: Bearer ${TOKEN}" \
    -H "Content-Type: application/json" \
    -d '{
      "filters": [
        {"field": "compareLeftImageId", "value": "'"${PREVIOUS_ID}"'", "comparator": "equals"},
        {"field": "compareRightImageId", "value": "'"${LATEST_ID}"'", "comparator": "equals"},
        {"field": "compareSide", "value": "both", "comparator": "equals"}
      ]
    }' \
    "${BINARLY_API_URL}/api/v4/grids/findings:gridList" | jq '.total // 0')
  
  # Count NEW findings (only in latest image = right side)
  NEW_ISSUES=$(curl -s -X POST \
    -H "Authorization: Bearer ${TOKEN}" \
    -H "Content-Type: application/json" \
    -d '{
      "filters": [
        {"field": "compareLeftImageId", "value": "'"${PREVIOUS_ID}"'", "comparator": "equals"},
        {"field": "compareRightImageId", "value": "'"${LATEST_ID}"'", "comparator": "equals"},
        {"field": "compareSide", "value": "right", "comparator": "equals"}
      ]
    }' \
    "${BINARLY_API_URL}/api/v4/grids/findings:gridList" | jq '.total // 0')
  
  echo "Comparison Statistics:"
  printf "  Resolved from previous:    %6d\n" "$RESOLVED"
  printf "  Unchanged (still present): %6d\n" "$UNCHANGED"
  printf "  Newly introduced:          %6d\n" "$NEW_ISSUES"
  echo ""
fi

# 3. Check for findings matching configured statuses (using 'in' comparator with array)
# Convert comma-separated statuses to JSON array
STATUSES_JSON=$(echo "$FAIL_ON_STATUS" | tr ',' '\n' | jq -R . | jq -s .)

# Use LATEST_ID if available (from comparison), otherwise use IMAGE_ID from upload step
CHECK_IMAGE_ID="${LATEST_ID:-${IMAGE_ID}}"

TOTAL_ACTIONABLE=$(curl -s -X POST \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{
    "filters": [
      {"field": "imageId", "value": "'"${CHECK_IMAGE_ID}"'", "comparator": "equals"},
      {"field": "issueStatus", "value": '"${STATUSES_JSON}"', "comparator": "in"}
    ]
  }' \
  "${BINARLY_API_URL}/api/v4/grids/findings:gridList" | jq '.total // 0')

echo "   Statuses checked: ${FAIL_ON_STATUS}"

echo ""
echo "Total actionable findings: ${TOTAL_ACTIONABLE}"
echo ""

# 4. Pass/Fail decision
if [ "$TOTAL_ACTIONABLE" -gt 0 ]; then
  echo "BUILD FAILED: ${TOTAL_ACTIONABLE} actionable finding(s)"
  exit 1
else
  echo "BUILD PASSED: No actionable findings"
  exit 0
fi

Output Example

Multi-image scenario with issues:

Binarly Security Verification
Failing on statuses: new,inProgress
Product has 5 image(s)
Comparing images:
   Previous: 01JYJ0W0AACGC7QT1Q21SRWG94
   Latest:   01JYJX8K2BBDF9PT3R22TSWH05

Comparison Statistics:
  Resolved from previous:        12
  Unchanged (still present):     45
  Newly introduced:               3

   new: 2
   inProgress: 1

Total actionable findings: 3

BUILD FAILED: 3 actionable finding(s)

Next Steps

Triage Findings: Use the Binarly Dashboard to review and triage findings
Adjust Thresholds: Combine with severity filters - see Compare Findings API
See Also: Compare Findings API for advanced filtering options

Overview

Use Cases

Reference

Verification Logic

Scenario 1: Single Image (First Scan)

Scenario 2: Multiple Images (Comparison)

Configuration

Finding Status Filter

Comparison Statistics

API Endpoints Used

Implementation

Output Example

Next Steps

​Verification Logic

​Scenario 1: Single Image (First Scan)

​Scenario 2: Multiple Images (Comparison)

​Configuration

​Finding Status Filter

​Comparison Statistics

​API Endpoints Used

​Implementation

​Output Example

​Next Steps

Verification Logic

Scenario 1: Single Image (First Scan)

Scenario 2: Multiple Images (Comparison)

Configuration

Finding Status Filter

Comparison Statistics

API Endpoints Used

Implementation

Output Example

Next Steps