Documentation Index
Fetch the complete documentation index at: https://docs.binarly.io/llms.txt
Use this file to discover all available pages before exploring further.
After a scan completes, verify the results to make pass/fail decisions for your CI/CD pipeline. This page explains the verification logic and provides implementation examples.
Verification Logic
The verification workflow handles two scenarios:
Scenario 1: Single Image (First Scan)
When the product contains only one image (the latest scan), the verification checks for findings matching the configured statuses. If found, the build fails.
Scenario 2: Multiple Images (Comparison)
When multiple images exist, the verification:
- Compares the latest image with the previous image
- Reports statistics on resolved, unchanged, and newly introduced findings
- Fails if any findings with configured statuses exist in the latest image
Configuration
Finding Status Filter
Control which finding statuses cause the build to fail using the FAIL_ON_STATUS environment variable:
| Status | Description |
|---|
new | Newly discovered, not yet triaged |
inProgress | Currently being investigated |
rejected | Marked as false positive or won’t fix |
remediated | Fixed but still detected (unusual) |
Default: new,inProgress – Fails on findings that require attention.
Examples:
# Fail only on new (untriaged) findings
export FAIL_ON_STATUS="new"
# Fail on new and in-progress (default)
export FAIL_ON_STATUS="new,inProgress"
# Strict: Fail on any non-remediated finding
export FAIL_ON_STATUS="new,inProgress,rejected"
Comparison Statistics
The comparison API categorizes findings into three groups:
| Status | Meaning | API Filter |
|---|
| Resolved | Was in previous image, not in latest | comparison.status=notFound |
| Unchanged | Present in both images | comparison.status=notChanged |
| New | Not in previous, appeared in latest | comparison.status=found |
API Endpoints Used
| Endpoint | Purpose |
|---|
GET /api/v4/products/{productId}/images | List images to determine count and get IDs |
POST /api/v4/grids/findings:gridList | Query findings with filters for status and comparison |
Implementation
#!/bin/bash
# Binarly CI/CD Verification Script
# Requires: curl, jq
# Environment: TOKEN, BINARLY_API_URL, BINARLY_PRODUCT_ID, IMAGE_ID (from upload step)
# Optional: FAIL_ON_STATUS (default: "new,inProgress")
set -e
# Configuration: Which statuses should fail the build
FAIL_ON_STATUS="${FAIL_ON_STATUS:-new,inProgress}"
echo "Binarly Security Verification"
echo "Failing on statuses: ${FAIL_ON_STATUS}"
echo ""
# 1. Get all images for the product
IMAGES_RES=$(curl -s -H "Authorization: Bearer ${TOKEN}" \
"${BINARLY_API_URL}/api/v4/products/${BINARLY_PRODUCT_ID}/images")
IMAGE_COUNT=$(echo "$IMAGES_RES" | jq '.images | length')
echo "Product has ${IMAGE_COUNT} image(s)"
# Initialize statistics
RESOLVED=0
UNCHANGED=0
NEW_ISSUES=0
# 2. If multiple images exist, compare latest with previous
if [ "$IMAGE_COUNT" -gt 1 ]; then
# Sort images by createTime descending and get IDs of two most recent
LATEST_ID=$(echo "$IMAGES_RES" | jq -r '[.images | sort_by(.createTime) | reverse][0][0].id')
PREVIOUS_ID=$(echo "$IMAGES_RES" | jq -r '[.images | sort_by(.createTime) | reverse][0][1].id')
echo "Comparing images:"
echo " Previous: ${PREVIOUS_ID}"
echo " Latest: ${LATEST_ID}"
echo ""
# Count RESOLVED findings (only in previous image = left side)
RESOLVED=$(curl -s -X POST \
-H "Authorization: Bearer ${TOKEN}" \
-H "Content-Type: application/json" \
-d '{
"filters": [
{"field": "compareLeftImageId", "value": "'"${PREVIOUS_ID}"'", "comparator": "equals"},
{"field": "compareRightImageId", "value": "'"${LATEST_ID}"'", "comparator": "equals"},
{"field": "compareSide", "value": "left", "comparator": "equals"}
]
}' \
"${BINARLY_API_URL}/api/v4/grids/findings:gridList" | jq '.total // 0')
# Count UNCHANGED findings (present in both images)
UNCHANGED=$(curl -s -X POST \
-H "Authorization: Bearer ${TOKEN}" \
-H "Content-Type: application/json" \
-d '{
"filters": [
{"field": "compareLeftImageId", "value": "'"${PREVIOUS_ID}"'", "comparator": "equals"},
{"field": "compareRightImageId", "value": "'"${LATEST_ID}"'", "comparator": "equals"},
{"field": "compareSide", "value": "both", "comparator": "equals"}
]
}' \
"${BINARLY_API_URL}/api/v4/grids/findings:gridList" | jq '.total // 0')
# Count NEW findings (only in latest image = right side)
NEW_ISSUES=$(curl -s -X POST \
-H "Authorization: Bearer ${TOKEN}" \
-H "Content-Type: application/json" \
-d '{
"filters": [
{"field": "compareLeftImageId", "value": "'"${PREVIOUS_ID}"'", "comparator": "equals"},
{"field": "compareRightImageId", "value": "'"${LATEST_ID}"'", "comparator": "equals"},
{"field": "compareSide", "value": "right", "comparator": "equals"}
]
}' \
"${BINARLY_API_URL}/api/v4/grids/findings:gridList" | jq '.total // 0')
echo "Comparison Statistics:"
printf " Resolved from previous: %6d\n" "$RESOLVED"
printf " Unchanged (still present): %6d\n" "$UNCHANGED"
printf " Newly introduced: %6d\n" "$NEW_ISSUES"
echo ""
fi
# 3. Check for findings matching configured statuses (using 'in' comparator with array)
# Convert comma-separated statuses to JSON array
STATUSES_JSON=$(echo "$FAIL_ON_STATUS" | tr ',' '\n' | jq -R . | jq -s .)
# Use LATEST_ID if available (from comparison), otherwise use IMAGE_ID from upload step
CHECK_IMAGE_ID="${LATEST_ID:-${IMAGE_ID}}"
TOTAL_ACTIONABLE=$(curl -s -X POST \
-H "Authorization: Bearer ${TOKEN}" \
-H "Content-Type: application/json" \
-d '{
"filters": [
{"field": "imageId", "value": "'"${CHECK_IMAGE_ID}"'", "comparator": "equals"},
{"field": "issueStatus", "value": '"${STATUSES_JSON}"', "comparator": "in"}
]
}' \
"${BINARLY_API_URL}/api/v4/grids/findings:gridList" | jq '.total // 0')
echo " Statuses checked: ${FAIL_ON_STATUS}"
echo ""
echo "Total actionable findings: ${TOTAL_ACTIONABLE}"
echo ""
# 4. Pass/Fail decision
if [ "$TOTAL_ACTIONABLE" -gt 0 ]; then
echo "BUILD FAILED: ${TOTAL_ACTIONABLE} actionable finding(s)"
exit 1
else
echo "BUILD PASSED: No actionable findings"
exit 0
fi
Output Example
Multi-image scenario with issues:
Binarly Security Verification
Failing on statuses: new,inProgress
Product has 5 image(s)
Comparing images:
Previous: 01JYJ0W0AACGC7QT1Q21SRWG94
Latest: 01JYJX8K2BBDF9PT3R22TSWH05
Comparison Statistics:
Resolved from previous: 12
Unchanged (still present): 45
Newly introduced: 3
new: 2
inProgress: 1
Total actionable findings: 3
BUILD FAILED: 3 actionable finding(s)
Next Steps