# Image manager - Dsys 2023
This is the repository for the challenge task of the lecture Distributed Systems at OST Rapperswil in FS 2023.
It is a simple image store, whose main purpose is to show our understanding of the covered topics of the lecture.
Hence the focus was not on the available functions, but instead on loadbalancing, healthchecks, communication via reverse proxy and implementing http3.


![Basic layout](resources/image_manager.jpg)

# Setup
We are using the following setup:

![Setup](resources/setup.jpg)

Initially we wanted to have a direct access to our frontend, and relay the requests to the services from there through the loadbalancer.
However we struggeled a lot to get http3 running with that setup, so we decided to instead use the loadbalancer as the first point of contact, which routes to the frontend webserver and the backend services. This let us use your examples from the lectures as a base for the loadbalancer and using http3. At that point we also changed from Caddy to Traefik, as the setup was a bit clearer to us.
In the end we switched back to Caddy as we managed to consistenlty use http3 with Caddy, whereas we struggeled with using http3 with Traefik.
A single instance of a MongoDb is run as a persistent storage, to which all services build a connection.

# Frontend
We are using vitejs to build our frontend server, using Vuejs with typescript for the webpage. 
The entrypoint is the the index.html file, which calls src/main.ts which creates the page application.

We define our http request target in vite.config.ts, where we send /api calls to the loadbalancer. The following requests are sent from the frontend:

| Endpoint                  | HTTP Meth.  | Description                          |
|---------------------------|-------------|--------------------------------------|
| `/api/search/{searchTerm}` | GET        | Looks for images matching searchterm |
| `/api/files/{fileName}`    | GET        | Image on service built from db data  |
| `/api/upload`              | POST       | Uploads provided image               |


# Loadbalancer
We are using Caddy for loadbalancing and as a reverse proxy. The available ports are http `:80` and https `:443`. Requests to `:80` are redirected to `:443`.
The purpose behind that is that we advertise http3, for which using https is necessary. We found that we were able to enable http3 even without providing certificates, as such we removed them from out setup.


The following routes are defined for the reverse proxy:

| Endpoint  | Server                |
|-----------|-----------------------|
|`/api/`    |`http://api1:8080`     |
|`/api/`    |`http://api2:8080`     | 
|`/`        |`http://frontend:3000` |

A service can be stopped (e.g. stopping image in Docker), after a maximum of 10 seconds the loadbalancer will recognise unhealthy api services and not send traffic to that service anymore and use another running api service instead.
We set the timer to 10 seconds to not overload our log output. In deployment situations it would be decreased to 1-3 seconds.

# Http3
We are using http3 communication as seen here. ![http3](resources/http3.jpeg)

# Loadbalancer old
We used to use Traefik for loadbalancing and reverse proxy. It can be checked out it the [Traefik branch](https://gitlab.ost.ch/dsysfs23/image-manager/-/tree/Traefik) of this repo. For that http3 was not really visible, just in some packages when sniffing the traffic. However as we invested a lot of time into it, it deserves to stay in the repo.
For that we also used port `:8080` that we remapped to `:443`
In order to make https work properly, we provide our own certificates. They have to be created on each client and imported by putting them into the [certs directory](certs). A [readme](certs/README.md) is provided with instructions to do so.
Additionally Traefik was configured to send GET requests to `http://api1:8080/api/health` and `http://api2:8080/api/health` for healthchecks.
If one service were to go down, the traffic will be sent to the other service if it is still running. They are checked every 10s and are considered to be down after not responding for 1s. If both are runninc, requests will be distributed to either service.

# Http3 old
We wereadvertising http3 to the outside:

![Http3 advert on port 443](resources/http3_advert.jpg)


However, as this is only between the first contanct of the loadbalancer and the client, not between the frontend and the client, the files sent from the frontend use http2 instead of http3.

However one can check that http3 packets are actually sent by using [Http Indicator Chrome Extension](https://chrome.google.com/webstore/detail/http-indicator/hgcomhbcacfkpffiphlmnlhpppcjgmbl). We used that extension to check the sent packets and requests. We started logging, then openend `http://localhost`, uploaded an image and looked for an image with it's name. Then we stopped logging.

The result can be found in [this JSON file](resources/image_manager_http_indicator.json).

Even though it is hard to read, in the first part are the type mappings.
logEventTypes 475-499 correspond to different http3 event types.
An example would be `"HTTP3_DATA_FRAME_RECEIVED":485`. Looking for `"type":485` shows that http3 dataframes are actually sent.


# Backend Services
We are using node.js webservers for our backend services.
The base is [node js upload image mongodb by bezkoder](https://github.com/bezkoder/node-js-upload-image-mongodb), used under [ISC license](resources/ISC_license.md), with appropriate modifications and removals to fit our purpose.

We are using express to handle our requests. 
The above mentioned endpoints are defined. For file storage / access, Multer GridFs Storage is used to handle pictures as binary data in the MongoDB.
Upon search request, the database returns the filelink on the service. After a call from the frontend to `/api/files/{fileName}` the images gets built from the binary data in the database on the service itself. It gets then served to the frontend.

For healthchecks using Traefik the endpoint `/api/health` was configured on the services to always respond with `OK Code 200`. This was to prevent routing issues that we had in the middle of development. However with the use of caddy it is now obsolete.

# Backend MongoDB
We use MongoDB to store the binary data, title and path of each picture. Each services builds a connection to the MongoDB. We chose to store images as binary data in MongoDB as the task was only to make the service scalable, not the DB. As such saving binary images is not optimal but ok.






# Usage
For Traefik we used certificates, we managed without using caddy.
To create and provide certificates, read instructions in [readme](certs/README.md).

Run:
Windows: docker-compose -f .\docker-compose.yml up

Mac: docker-compose -f docker-compose.yml up

Cleanup:

docker-compose down -v

# Authors

Kailing Peng, Natalia Gerasimenko, Linus Flury
