Configure Binarly charts

This page describes how to configure the Binarly charts for your deployment. The configuration is done through a values file that is passed to the Helm chart during installation, and secrets set up prior to the installation.

Binarly Secrets

The following table details secrets that are required by the platform. It is recommended to use a more secure method of managing the manual secrets such as the External Secrets Operator or Sealed Secrets. The secrets and their uses are as follows (This list is not exhaustive):

Secret Name	Description	Usage	Deployment
keycloak	Keycloak admin user credentials	Used by Keycloak for authentication	Automatic
keycloak-database-connection	Keycloak database connection details	Used by Keycloak to connect to the database. If using the built-in database, these details are automatically managed.	Automatic
server-database-connection	Server database connection details	Used by the Binarly Server to connect to the database. If using the built-in database, these details are automatically managed.	Automatic
nvd-api-key	NVD API key for accessing the National Vulnerability Database	Used by the vulnerability database to fetch vulnerability data. Provided by customer support.	Manual
binarly-registry	Docker registry credentials for Binarly images	Used to pull Binarly images from the provided private registry.	Manual
docker-login-credentials	Docker registry credentials for pulling images	Used to pull images from the specified Docker registry.	Manual
server-integration-secret	Integration secret for the Binarly Server	Used by the Binarly Server.	Manual

Deploying the Secrets

Secrets should be deployed prior to installation. Any method to deploy the secrets will work and the following are provided as examples. Additional resources (Like ExternalSecrets) can be passed using the extraResources value, see here for configuration.

Manual Secret Deployment

It is recommended to use a more secure method of managing the secrets such as the External Secrets Operator or Sealed Secrets. To detail the structure of the secrets, here are examples of how to manually create the required secrets using kubernetes manifests.

apiVersion: v1
kind: Secret
type: Opaque
metadata:
  name: nvd-api-key
data:
  key: <base64 encoded nvd-api-key provided by us>

apiVersion: v1
kind: Secret
metadata:
  name: binarly-registry
type: kubernetes.io/dockerconfigjson
data:
  .dockerconfigjson: <base64-encoded-docker-config>

The Docker Config JSON can be constructed using the following template:

{
  "auths": {
    "<your-registry-url>": {
      "username": "_json_key",
      "password": "<A JSON key we will provide>",
      "email": "<An email we will provide>"
    }
  }
}

apiVersion: v1
kind: Secret
type: Opaque
metadata:
  name: docker-login-credentials
data:
  DOCKER_LOGIN_USERNAME: < _json_key, base64 encoded >
  DOCKER_LOGIN_PASSWORD: <A JSON key we will provide, base64 encoded>

apiVersion: v1
kind: Secret
type: Opaque
metadata:
  name: server-integration-secret
data:
  server-integration-secret: <A base64 encoded string of your choosing, please back it up>

With External Secrets Operator

The External Secrets Operator can be used to manage secrets more securely. Here is an example of how to define an external secret for the binarly registry:

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: binarly-registry
spec:
  dataFrom:
  - extract:
      conversionStrategy: Default
      decodingStrategy: None
      key: binarly-registry
      metadataPolicy: None
  refreshInterval: 1h
  secretStoreRef:
    kind: ClusterSecretStore # Or SecretStore
    name: my-secret-store # The name of the secret store
  target:
    creationPolicy: Owner
    deletionPolicy: Retain
    name: binarly-registry # The name of the secret on your secret store

Where the binarly-registry secret in your secret store contains the following JSON structure:

{
  "auths": {
    "<your-registry-url>": {
      "username": "_json_key",
      "password": "<A JSON key we will provide>",
      "email": "<An email we will provide>"
    }
  }
}

With Sealed Secrets

Sealed Secrets can be used to manage secrets more securely. Here is an example of how to define a sealed secret for the binarly registry:

apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
  name: binarly-registry
spec:
  encryptedData:
    .dockerconfigjson: <Encrypted Docker Config JSON>
  template:
    type: kubernetes.io/dockerconfigjson
    immutable: true

The encrypted docker config JSON would look like the following before encryption:

  {
    "auths": {
      "<your-registry-url>": {
        "username": "_json_key",
        "password": "<A JSON key we will provide>",
        "email": "<An email we will provide>"
      }
    }
  }

Values

A minimal deployment requires setting the following values in another file, e.g. specific-values.yaml: Please do not uses these values as-is, they must be adapted to your environment.

global:
  imagePullSecrets:
    - name: binarly-registry # The name of the secret containing the Binarly registry credentials
  storageClassName: standard # The storage class to use for persistent volumes for the application
  ingressClassName: tailscale # The ingress class to use for the application
  basedomain: binarly.io # The base domain for the application
  dashboard:
    hostname: "dashboard" # The hostname for the Binarly Dashboard
  keycloak:
    hostname: "keycloak" # The hostname for Keycloak
  bucketsConfig:
    publicEndpoint: https://minio-api.binarly.io # The public endpoint for MinIO, if using the built-in data storage option
keycloak:
  ingress:
    hostname: "keycloak.binarly.io" # Unfortunately this has to be set twice
scan-workflow: # Specific configuration for the scanner jobs
  workflow:
    storageClassName: "premium" # The storage class to use for the scanner jobs. Please ensure this storage class' reclaimPolicy is set to Delete.
    nodeSelector: # The node selector to use for the scanner jobs
      workload: tools
    tolerations:
      - effect: NoSchedule
        key: workload
        operator: Equal
        value: tools

Extra Resources

The Chart contains an extraResources section that allows you to deploy additional resources alongside the main chart. This can be used to deploy custom ConfigMaps, Secrets, or other Kubernetes resources that are not part of the main chart. This is useful for adding custom configurations or additional components that are not included in the chart by default, and are specific to your environment. For example:

extraResources:
  test:
    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: test-config
    data:
      key: value
  externalSecrets:
    apiVersion: external-secrets.io/v1beta1
    kind: ExternalSecret
    metadata:
      name: my-external-secret
    spec:
      backendType: secretsManager
      data:
        - key: my-secret-key
          name: my-secret-name

This will create a ConfigMap named test-config and an ExternalSecret named my-external-secret. This is an alternative to creating a wrapper chart around this chart.

Upgrading

The chart follows the semantic versioning convention so any breaking changes will be reflected in the major version number. Any other changes should be backwards compatible. To prevent constant major version updates some values are deprecated but not removed. When there is a breaking change released all deprecated values will be removed and a notice posted on this readme with upgrade instructions. Pre-upgrade, please ensure the Database is backed up. This depends on your deployment method, but if you are using the built-in PostgreSQL database, you can use the pg_dump command to create a backup of the database. If you are using an external database, please refer to your database provider’s documentation for backup instructions.

ArgoCD and FluxCD

Upgrades involving minor or patch versions should be carried out depending on your installation method. If you are using ArgoCD or FluxCD, the upgrade will be handled by updating the version on your Application or HelmRelease.

Helm

If you are using Helm, you can upgrade the chart using the following command:

helm upgrade binarly-transparency-platform oci://<Customer Registry>/charts/binarly-transparency-platform:<The required version> \
-f specific-values.yaml \
--namespace binarly-transparency-platform \
--skip-crds

To check for differences the helm diff plagin is useful (Requires separate installation):

helm plugin install https://github.com/databus23/helm-diff

helm diff upgrade binarly-transparency-platform oci://<Customer Registry>/charts/binarly-transparency-platform \
--version 1.2.3 \
-f specific-values.yaml \
--namespace binarly-transparency-platform

Rollback

There are two rollback scenarios to consider:

No Migrations

If the upgrade does not include any database migrations, you can simply rollback to the previous version using the following command:

helm rollback binarly-transparency-platform <previous-release-version> --namespace binarly-transparency-platform

With Migrations

Migrations are not backwards compatible so if the upgrade includes database migrations, you will need to restore the database from a backup. This can be done using the pg_restore command for PostgreSQL databases. If you are using an external database, please refer to your database provider’s documentation for rollback instructions. During the restore process you will need to disable the Binarly Server and VDB components to prevent them from trying to access the database while it is being restored. This can be done by setting the server.enabled and vulnerability-database.enabled values to false in the values.yaml file, or by using the --set flag when running the helm upgrade command. After this, restore the database and restart the components by setting the values back to true and running the helm upgrade command again.

High Availability

The chart contains partial support for high availability. The following components can be deployed in a highly available configuration:

Keycloak
Dashboard
VDB

To set these, use the following values:

keycloak:
  replicaCount: 3 # The number of Keycloak replicas to deploy
dashboard:
  replicaCount: 3 # The number of Dashboard replicas to deploy
  reportsService:
    replicaCount: 3 # The number of Reports Service replicas to deploy
vulnerability-database:
  hpa:
    enabled: true # Enable Horizontal Pod Autoscaler for the VDB service, defaults to three replicas

Ingress

Ingress is currently provisioned from the dashboard, minio-buckets, and keycloak chart. This will be simplified in the future and require fewer values. basedomain is base domain name for Binarly Transparency Platform:

global:
  basedomain: "binarly.domain.com"
  dashboard:
    hostname: "dashboard" # The hostname for the Binarly Dashboard
  keycloak:
    hostname: "keycloak" # The hostname for Keycloak
  bucketsConfig:
    publicEndpoint: https://minio-api.binarly.domain.com # The public endpoint for MinIO, if using the built-in data storage option
keycloak:
  ingress:
    enabled: true
    hostname: "keycloak.binarly.domain.com"
dashboard:
  ingress:
    enabled: true
minio-buckets:
  minioBuckets:
    ingress:
      enabled: true

The above config will create the following ingress resources:

dashboard.binarly.domain.com for the Binarly Dashboard
keycloak.binarly.domain.com for Keycloak
minio-api.binarly.domain.com for MinIO

Observability

The BTP Chart emits a variety of metrics, logs, and traces to help you monitor the health and performance of the applications.

Metrics

Metrics can be collected from prometheus metrics endpoints exposed by the various components. The following components expose metrics:

Keycloak
Server API
Server Scanner
Argo Workflows
Minio

And can be configured using the following values:

server:
  metrics:
    enabled: true
keycloak:
  metrics:
    enabled: true
argo-workflows:
  controller:
    metricsConfig:
      enabled: true

Logs

Logs are output to Stdout and Stderr in JSON format by default.

Traces

Traces are produced in OpenTelemetry format from the following components:

Server API
Server Scanner

And can be configured using the following values:

server:
  traces:
    enabled: true
    host: Collector Host
    port: Collector Port

Role-Based Access Control (RBAC)

By default all access is managed via Keycloak. To enable RBAC, you can set the following values:

auth:
  enableRBAC: true
  enableRBACMiddleware: true

dashboard:
  appConfigData:
    features:
      rbac: true

To read more about RBAC in the Binarly application, please see the dedicated page.

Custom Rules

To use the Custom Rules feature in an on-premise installtion, an installation of Harbor is required to manage the artefacts used by the system, and the user’s credentials must be available to the binarly application. Harbor can be deployed on kubernetes using the helm chart:

helm repo add harbor https://helm.goharbor.io
helm install harbor harbor/harbor -f values-specific-to-your-environment.yaml

It is strongly suggested to use the following values as part of the user’s specific values to prevent issues:

jobservice:
  jobLoggers: 
    - stdout

With a working harbor installation, the following values can be passed to the BTP Chart to enable custom rules with substitutions to match the user’s environment:

global:
  # -- The name of the project that will be used in harbor
  clusterName: harbor-project
  features:
    customRulesV2: true
    harbor: true
  harbor:
    # -- harbor host
    host: harbor.harbor-ha.svc.cluster.local
    # -- harbor insecure connection
    insecure: true
    # -- Name of the Kubernetes secret containing harbor username.
    usernameSecretName: "harbor"
    # -- Key in the secret for the harbor username.
    usernameSecretKey: "username"
    # -- Name of the Kubernetes secret containing harbor password.
    passwordSecretName: "harbor"
    # -- Key in the secret for the harbor password.
    passwordSecretKey: "password"

The secret name and keys can be customised. The default secret looks like this:

apiVersion: v1
kind: Secret
metadata:
  name: harbor
type: Opaque
data:
  password: "A base64 encoded password"
  username: "A base64 encoded username"

This secret will be mounted and user to interact with harbor. Please ensure:

A user has been provisioned on the harbor instance
The user has access to a project with a name defined in global.clusterName as above

Air Gapped Environment

The Binarly Application will work in an air-gapped environment with a few caveats:

There must be an internal registry capable of hosting images and charts.
Chart addresses must be updated to use the internal registry.
Image repository fields must be overwritten to use the internal registry.
One component of the Binarly Application (Vulnerability Database) requires internet access to fetch vulnerability data.

Internal Registry

The registry in use should be populated with the contents of the private Binarly registry provided by Customer Success. The exact contents will be communicated prior to the installation.

Images

In case you wish to distribute artifacts with your chosen artifact registry, we recommend using the helm images plugin. Further details on this plugin can be found in artifactHUB Installation of this plugin is simple as

helm plugin install https://github.com/nikhilsbhat/helm-images

To get a human friendly output with all the images that you need to distribute on your end, use the following:

helm images get . -o table -u -f <values-override>.yaml

This output can be used to pull images from the Binarly registry and push them to your internal registry.

Vulnerability Database

The Vulnerability Database component of the Binarly Application requires internet access to fetch vulnerability data. This can be achieved by setting up a proxy server that allows the Vulnerability Database to access the internet, and passing this config in k8s/apps/binarly/values.yaml.gotmpl:

vulnerability-database:
  env:
    http_proxy: "http://proxy:port"
    https_proxy: "http://proxy:port"

Configure Third-Party Charts

The Binarly Installation comes with a set of third-party charts, more information in considerations. These charts are configured in the charts/{chart name} directory.

Installation

​Configure Binarly charts

​Binarly Secrets

​Deploying the Secrets

​Values

​Extra Resources

​Upgrading

​ArgoCD and FluxCD

​Helm

​Rollback

​No Migrations

​With Migrations

​High Availability

​Ingress

​Observability

​Metrics

​Logs

​Traces

​Role-Based Access Control (RBAC)

​Custom Rules

​Air Gapped Environment

​Internal Registry

​Images

​Vulnerability Database

​Configure Third-Party Charts