The Binarly Transparency Platform can be deployed using a pre-configured virtual machine based on Talos Linux and Kubernetes.

Using this installation method is:

Less stable
Does not scale up and down
Less observable

This method is recommended for short proof-of-concept or very low traffic scenarios. Depending on the machine size, scan times can be greatly impacted.

Requirements

A machine with at least 32 cores and 128GB of RAM
A root disk of at least 100GB
A data disk of at least 300GB
A reverse proxy at front of this machine
- reverse proxy is to forward HTTP (80) traffic to HTTP (30080) and HTTPS (443) traffic to HTTPS (30443) on the VM

Setup

Setup will depend on your hypervisor of choice, this Virtual Machine has been tested using:

Kubevirt
Proxmox

Disks

The provided disk.img should be used as the root volume, and may need to be converted using qemu-img
The data disk should be attached to /dev/vdb and be blank. This disk should be detached between upgrades and reattached as it contains all application data

Ports

50000 for access to the talos API
6443 for access to the Kubernetes API
30080 and 30443 for application access

Example

During testing these VMs are deployed on Kubernetes via Kubevirt with the following configuration:

Kubevirt VM Deployment

apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  name: talos-node
  labels:
    kubevirt.io/domain: talos-node
spec:
  runStrategy: Always
  template:
    spec:
      networks:
        - name: defaultnetwork
          pod: {}
      tolerations:
        - key: dedicated
          operator: Equal
          value: kubevirt
          effect: NoSchedule
      domain:
        cpu:
          cores: 32
        resources:
          requests:
            memory: 128Gi
          limits:
            memory: 256Gi
        devices:
          interfaces:
            - name: defaultnetwork
              masquerade: {}
              ports:
                - port: 30080
                - port: 30443
                - port: 32000
                - port: 50000
                - port: 6443
          disks:
            - name: rootdisk
              disk: { bus: virtio }
            - name: secondarydisk
              disk: { bus: virtio }
      volumes:
        - name: rootdisk
          dataVolume:
            name: talos-boot-dv
        - name: secondarydisk
          persistentVolumeClaim:
            claimName: talos-data-pvc

Configuring BTP

As part of the installation package there are 4 more files that provide access to the VM:

talosconfig
kubeconfig
controlplane.yaml
worker.yaml
btp.yaml

These files should be treated as confidential. They contain the certificates that allow access to the machine and without them the machine cannot be changed. They are specific to the VM instance they were distributed with. Configuration of the application requires interacting with the machine using talosctl to update parameters. This can be done either from a machine with network access to the VM or from a Kubernetes pod running inside the VM.

Configuring the BTP App

The btp.yaml file contains the configuration for your deployment and will need to be updated. This can be done by adding values (more information in the configuring BTP section) to the yaml file and applying it to the cluster. As an example the domain will need to change to match your internal domain:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: binarly-transparency-platform
  namespace: argocd
spec:
  project: default
  source:
    helm:
      values: |
        # These values are provided by us and shouldn't be changed, except `basedomain`
        global:
          ...
          basedomain: dev.binarly.cloud # Change this value to your desired basedomain.
          ...

Domains

BTP comes with a set list of subdomains that will be accessible through the basedomain. These domains are the following:

dashboard.basedomain
kc.basedomain
minio-api.basedomain

Please make sure these domain names are set in your DNS server and the value set to the IP of your reverse proxy.

Machine with Network Access

Talos uses mutual transport layer security to ensure it can only be accessed on a strict list of domains. The domains are listed in the controlplane.yaml file, in these examples talos-node.local is used. Steps:

Add talos-node.local DNS entry for the VM (Or add an entry for talos-node.local in /etc/hosts)
Install:
1. talosctl
2. kubectl
Make changes to the btp.yaml file to suit your needs

Apply those changes with:

export TALOSCONFIG=/path/to/talosconfig
talosctl config endpoint talos-node.local
talosctl config node talos-node.local
talosctl kubeconfig
kubectl apply -f btp.yaml

Kubernetes Pod

The local address (127.0.0.1) is valid to reconfigure the VM and all Kubernetes calls originate from there. Steps:

Get access to the cluster

export TALOSCONFIG=/path/to/talosconfig
talosctl kubeconfig

Create a pod on the Kubernetes cluster:

kubectl run talos-config \
  --image=alpine \
  --restart=Never \
  -- sleep infinity

Wait for the pod to be ready:

kubectl wait --for=condition=Ready pod/talos-config

Make changes to the btp.yaml file to suit your needs

Copy the talosconfig and btp.yaml files to the pod:

kubectl cp /path/to/talosconfig default/talos-config:/talosconfig
kubectl cp /path/to/btp.yaml default/talos-config:/btp.yaml

Install talosctl and kubectl, then apply the changes:

kubectl --kubeconfig /path/to/kubeconfig exec -it talos-config -- sh -c \
  'apk add --no-cache jq curl && \
  curl -sL https://github.com/siderolabs/talos/releases/latest/download/talosctl-linux-amd64 \
    -o /usr/local/bin/talosctl && \
  chmod +x /usr/local/bin/talosctl
  apk add kubectl
  export TALOSCONFIG=/path/to/talosconfig
  talosctl config endpoint 127.0.0.1
  talosctl config node 127.0.0.1
  talosctl kubeconfig
  kubectl apply -f btp.yaml

Once complete, delete the pod:
kubectl delete pod talos-config

Routing to the Virtual Machine

The Virtual Machine can be routed to either with a load balancer that terminates TLS and forwards plaintext traffic, or TLS can be terminated by the virtual machine’s internal load balancer.

External Load Balancer

You must set up a load balancer with certificates for your domain, terminating TLS at the load balancer level and forwarding traffic to the application access ports on the load balancer. The load balancer should be configured with a DNS record for your domain, and the domain should be passed to the Binarly platform (more information in the configuring BTP section).

Using the Internal Load Balancer

The virtual machine employs a Kubernetes Gateway listening on the application ports and can be used as a traffic entrypoint. This requires some configuration of the Gateway in Kubernetes to set up the certificate and TLS termination.

Due to restricted ports when using this ingress method you will need to access the VM using the port as part of the domain. For example: https://binarly-dashboard.internal:30443 or http://binarly-dashboard.internal:30080.

Setting up Internal Certificates

Once there is a domain pointing to the virtual machine’s IP address, apply certificates to the gateway:

Get access to the kubernetes cluster following the instructions from the Machine with Network Access section

Create a secret containing your certificate in the default namespace by editing this template:

apiVersion: v1
kind: Secret
metadata:
  name: gateway-cert
  namespace: default
type: kubernetes.io/tls
data:
  tls.crt: <Your certificate, base64 encoded>
  tls.key: <Your certificate key, base64 encoded>

Deploy the secret to the cluster by the following command. kubectl apply -f <secret>.yaml
Run kubectl edit Gateway ingress-gateway -n default which will open the Gateway in a local editor

Replace the https listener entry (Whole file provided for an example):

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: ingress-gateway
spec:
  gatewayClassName: istio
  infrastructure:
    parametersRef:
      group: ""
      kind: ConfigMap
      name: ingress-gateway-config
  listeners:
    - name: http
      protocol: HTTP
      port: 80
      allowedRoutes:	
        namespaces:	
          from: All
    # This section below needs to be changed
    - name: https
      protocol: HTTPS
      port: 443
      allowedRoutes:	
        namespaces:	
          from: All
      hostname: "<The required hostname, supports wildcards.>" #must be quoted
      tls:
        certificateRefs:
        - group: ""
          kind: Secret
          name: gateway-cert
        mode: Terminate

Installation

Virtual Machine Deployment

Requirements

Setup

Disks

Ports

Example

Configuring BTP

Configuring the BTP App

Domains

Machine with Network Access

Kubernetes Pod

Routing to the Virtual Machine

External Load Balancer

Using the Internal Load Balancer

Setting up Internal Certificates

​Requirements

​Setup

​Disks

​Ports

​Example

​Configuring BTP

​Configuring the BTP App

​Domains

​Machine with Network Access

​Kubernetes Pod

​Routing to the Virtual Machine

​External Load Balancer

​Using the Internal Load Balancer

​Setting up Internal Certificates

Requirements

Setup

Disks

Ports

Example

Configuring BTP

Configuring the BTP App

Domains

Machine with Network Access

Kubernetes Pod

Routing to the Virtual Machine

External Load Balancer

Using the Internal Load Balancer

Setting up Internal Certificates