openstapps/backend/backend/README.md

backend (a reference implementation of a StApps backend)

This project is a reference implementation for a StApps backend. It provides an HTTP API to index data into a database, perform full text search, sorts and filters. It also delivers the configuration needed by the app. The API is specified within the @openstapps/core.

If you want to perform requests, index data or search within JavaScript or TypeScript you should consider using our client @openstapps/api.

Or generate your own client using the openapi/swagger definitions you can get form the API documentation.

Usage

This backend is not a standalone software. It needs a database like Elasticsearch to work.

If you just want to use the backend you should consider using minimal-deployment. The minimal-deployment will provide you with everything you need to run this backend.

Local usage for development purposes

Requirements

• Elasticsearch (8.4)
  • ICU analysis plugin
  • OR Docker
• Node.js (~14) / NPM

Startup Behaviour

This might be important if you work on the Core

The backend is using Elasticsearch Mappings and Aggregations from its currently used core dependency.

Start Database (Elasticsearch)

Elasticsearch needs some configuration and plugins to be able to work with the backend. To save you some work we provide a docker image which only needs to be executed to work with the backend.

Run docker run -d -p 9200:9200 registry.gitlab.com/openstapps/database:latest

Elasticsearch should be running at port 9200 now. If you have problems with getting elasticsearch to work, have a look in the README of the image first.

Metrics collection

The backend contains an express middleware which can be optionally enabled by setting the environment variable PROMETHEUS_MIDDLEWARE to true. The middleware collects metrics and provides a Prometheus compatible endpoint from which the metrics may be scraped by Prometheus.

The middleware may be configured with JSON provided in config/prometheus.json, i.e.

{
  "metricsPath": "/metrics",
  "collectDefaultMetrics": true,
  "requestDurationBuckets": [0.1, 0.5, 1, 2, 5, 10, 20],
  "requestLengthBuckets": [512, 1024, 5120, 10240, 51200, 102400],
  "responseLengthBuckets": [512, 1024, 5120, 10240, 51200, 102400]
}

The available options are documented on npmjs or the homepage of the express-prometheus-middleware project. You may get a compatible grafana dashboard at grafana.com.

Start backend

Run npm install and npm run build, then start with npm start. The server should now be accepting connections at http://localhost:3000.

Environment Variables

To select a database implementation you have to set the NODE_CONFIG_ENV variable. At the time only NODE_CONFIG_ENV=elasticsearch is supported. Set NODE_ENV=production to run backend for production usages. In production the backend expects some kind of monitoring to be set via the environment. At the time only SMTP is being implemented. The backend wouldn't start if you don't provide SMTP authentification. Alternatively you can set ALLOW_NO_TRANSPORT=true. To set up an SMTP configuration have a look at @openstapps/logger.

The list of environment variables includes:

• NODE_ENV when set to production, there will be a reduced amount of output from the logger
• PORT when this is not set, the backend will default to port 3000
• ES_ADDR the Elasticsearch address, if not set it will default the Elasticsearch address to http://localhost:9200
• ALLOW_NO_TRANSPORT if set to true, the backend will allow starting without an Email configured that receives critical errors.
• ES_DEBUG setting this to true will result in Elasticsearch logging to be VERY extensive, in almost all situation this should no be enabled.
• PROMETHEUS_MIDDLEWARE if set to true will enable metrics collection with Express Prometheus Middleware

Config files

Each university can have it's specific config for the general backend and app and for all databases.

All config files can be found in ./config/. There is a default.ts which is used by default. You can create an university specific file with following naming scheme: default-<university license plate>.ts

A university specific file will only overwrite all properties of the default.ts that are set in the file itself. To start the backend using your configuration you have to provide the NODE_APP_INSTANCE environment variable with your university license plate.

To set a database you have to provide the NODE_CONFIG_ENV environment variable with the name of the database. At the time only Elasticsearch is implemented.

To create your university specific config file for the elasticsearch you have to create a file with following naming scheme: elasticsearch-<university license plate>.ts.

Debugging

Set ES_DEBUG=true to enable verbose Elasticsearch tracing information. This can be useful to debug some issues between backend and elasticsearch.

Contributing