# DSYS Challenge Project

* Fabio Elvedi
* Timo Klingler

## Introduction

The idea of the challenge project was to combine the powerful DevOps capabilities of the [nix-project](https://nixos.org/) with a Ruby on Rails web application which allows the user to upload images.

The goals of the project were to simulate a distributed system, where 

1. A Ruby on Rails container can be built which is
   * Fully reproducible and (mostly) deterministic
   * Minimal in size
   * Is not required for local development
2. To structure the development environment in such a manner that the configurations are fully usable in production
3. Have a small-scale system that can be scaled and easily destroyed in case of misconfiguration

## Architecture

As a proof of concept, we run two instances of a Ruby on Rails with an attachd Sqlite database on a single-node k3s Kubernetes cluster, which uses [MetalLB](https://metallb.universe.tf/), a bare-metal load balancer, in conjunction with a [Traefik](https://doc.traefik.io/traefik/) reverse proxy.

## Configurations

### System

```nix
{ users.users.root.initialPassword = "";
}
```

Root user can log in without password

### K3s

```nix
{ config, lib, pkgs, ... }: {

  services = {
    k3s = {
      enable = true;
      role = "server";
      extraFlags = toString [ "--disable traefik" "--disable servicelb" ];
    };
  };

  environment.systemPackages = with pkgs; [ k3s iptables ];

  networking.firewall.enable = false;

  virtualisation.forwardPorts = [{ from = "host"; guest.port = 6443; host.port = 6443; }
  { from = "host"; guest.port = 80; guest.address = "10.0.2.240"; host.address = "127.0.0.1"; host.port = 8080; }];
}

```

* K3s is run as a single node cluster. Therefore we opt for the server configuration since we are not concerned with splitting it into a server or agent. We also follow the recommendations to disable the builtin load balancer (traefik, which we introduce later), and servicelb.
* We disable the firewall. This was done so that we do not have to deal with the restrictive settings for development and the proof of concept.
* Finally, for local development, we expose the ports needed for k3s and the load balancer to the host. This is not required for production, but does have a large impact because we do not rely on virtualization ourselves for production.

### Flake

```nix
{ description = "Instagram Clone Project Flake";

  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/6adf48f53d819a7b6e15672817fa1e78e5f4e84f";
    flake-utils.url = "github:numtide/flake-utils";
  };

  outputs = { self, nixpkgs, flake-utils }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        pkgs = import nixpkgs {
          inherit system;
          overlays = [ overlay ];
        };

        overlay = (final: prev: {
          containers = {
            database = final.callPackage ./mysql { };
            backend = final.callPackage ./rails { };
          };
        });

        base = { lib, modulesPath, ... }: {
          imports = [ "${modulesPath}/virtualisation/qemu-vm.nix" ];

          # https://github.com/utmapp/UTM/issues/2353
          networking.nameservers = lib.mkIf pkgs.stdenv.isDarwin [ "8.8.8.8" ];

          virtualisation = {
            graphics = false;
            host = { inherit pkgs; };
            memorySize = 4096;
            diskSize = 10240;
            sharedDirectories.deployment = {
              source = "$SRC";
              target = "/home/deployment";
            };
          };
        };

        machine = nixpkgs.lib.nixosSystem {
          system = builtins.replaceStrings [ "darwin" ] [ "linux" ] system;

          modules = [ base ./module.nix ./k3s/k3s.nix ];
        };

        program = pkgs.writeShellScript "run-vm.sh" ''
          export NIX_DISK_IMAGE=$(mktemp -u -t nixos.qcow2)

          trap "rm -f $NIX_DISK_IMAGE" EXIT

          ${machine.config.system.build.vm}/bin/run-nixos-vm
        '';

      in
        { packages = { 
            inherit machine;
            mysql = pkgs.containers.database.image;
            rails = pkgs.containers.backend.image;
          };

          apps.default = {
            type = "app";

            program = "${program}";
          };
        }
    );
}
```

* This is the configuration that ties all sub-configurations together, both for development and build stages.
* We run a QEMU-VM for nixos with 4GB of RAM and 10GB disk space.
* A mysql (not used) and a rails container can be built from here.
* What is missing: The development shell from rails. This has to be accessed in the flake inside of the `/rails` subdirectory (see setup).

### MetalLB

```yaml
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: first-pool
  namespace: metallb-system
spec:
  addresses:
  - 10.0.2.240-10.0.2.250
  
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: example
  namespace: metallb-system
spec:
  ipAddressPools:
  - first-pool
```

* We provide MetalLB with a hard-coded IP range

## Setup

For the setup to work, it is advised to perform the following steps using a Linux machine with a x86-64 architecture.

### Local Development with Ruby on Rails

```bash
cd ./rails
nix develop --ignore-environment
```

### Local Development with k3s

Make sure that port `8080` is free on `localhost` or change it in `k3s/k3s.nix`.

```bash
SRC="$PWD/deployment" nix run

# MetalLb
k3s kubectl apply -f /home/deployment/metallb-native.yaml
k3s kubectl apply -f /home/deployment/metallb-config.yaml

# Traefik
k3s kubectl apply -f /home/deployment/traefik-test.yaml

# Rails
k3s kubectl create secret docker-registry gitlab-registry --docker-server=registry.gitlab.ost.ch:45023 --docker-username=k3s_pull_token --docker-password=XoLgvQTuz8MnuZn3ypDz --docker-email=timo.klingler@ost.ch
k3s kubectl get secret gitlab-registry --output=yaml
k3s kubectl apply -f /home/deployment/rails-test-mult.yaml
```

## Limitations

* There is persistent storage with the rails instances, but no centralized storage
* The application only works with HTTP/1.1 and without TLS
* Security was left out

