refactor: build system

This commit is contained in:
2023-03-22 11:45:30 +01:00
parent 4df19e8c20
commit 8cb9285462
427 changed files with 3978 additions and 9810 deletions

View File

@@ -1,5 +1,7 @@
[![pipeline status](https://img.shields.io/gitlab/pipeline/openstapps/backend.svg?style=flat-square)](https://gitlab.com/openstapps/backend/commits/master)
# backend (a reference implementation of a StApps backend)
This project is a reference implementation for a StApps backend. It provides an HTTP API to index data into a database,
perform full text search, sorts and filters. It also delivers the configuration needed by the app. The API is specified
within the [@openstapps/core](https://gitlab.com/openstapps/core).
@@ -10,6 +12,7 @@ If you want to perform requests, index data or search within JavaScript or TypeS
Or generate your own client using the openapi/swagger definitions you can get form the [API documentation](https://openstapps.gitlab.io/backend).
# Usage
This backend is not a standalone software. It needs a database like Elasticsearch to work.
If you just want to use the backend you should consider using
@@ -17,19 +20,21 @@ If you just want to use the backend you should consider using
you with everything you need to run this backend.
# Local usage for development purposes
## Requirements
* Elasticsearch (8.4)
- [ICU analysis plugin](https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-icu.html)
- OR Docker
* Node.js (~14) / NPM
- Elasticsearch (8.4)
* [ICU analysis plugin](https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-icu.html)
* OR Docker
- Node.js (~14) / NPM
### Startup Behaviour
*This might be important if you work on the Core*
_This might be important if you work on the Core_
The backend is using Elasticsearch Mappings and Aggregations from its currently used `core` dependency.
## Start Database (Elasticsearch)
Elasticsearch needs some configuration and plugins to be able to work
with the backend. To save you some work we provide a
[docker image](https://gitlab.com/openstapps/database) which
@@ -43,6 +48,7 @@ getting elasticsearch to work, have a look in the
first.
## Metrics collection
The backend contains an express middleware which can be optionally enabled by setting the environment variable
`PROMETHEUS_MIDDLEWARE` to `true`. The middleware collects metrics and provides a [Prometheus](https://prometheus.io/)
compatible endpoint from which the metrics may be scraped by Prometheus.
@@ -58,12 +64,15 @@ The middleware may be configured with JSON provided in `config/prometheus.json`,
"responseLengthBuckets": [512, 1024, 5120, 10240, 51200, 102400]
}
```
The available options are documented on [npmjs](https://www.npmjs.com/package/express-prometheus-middleware) or the [homepage](https://github.com/joao-fontenele/express-prometheus-middleware#readme) of the express-prometheus-middleware project. You may get a compatible grafana dashboard at [grafana.com](https://grafana.com/grafana/dashboards/14565).
## Start backend
Run `npm install` and `npm run build`, then start with `npm start`. The server should now be accepting connections at `http://localhost:3000`.
# Environment Variables
To select a database implementation you have to set the `NODE_CONFIG_ENV` variable. At the time only `NODE_CONFIG_ENV=elasticsearch` is supported.
Set `NODE_ENV=production` to run backend for production usages. In production the backend expects some kind of monitoring to be set via the
environment. At the time only SMTP is being implemented. The backend wouldn't start if you don't provide SMTP authentification. Alternatively
@@ -71,14 +80,16 @@ you can set `ALLOW_NO_TRANSPORT=true`. To set up an SMTP configuration have a lo
[@openstapps/logger](https://gitlab.com/openstapps/logger).
The list of environment variables includes:
* `NODE_ENV` when set to `production`, there will be a reduced amount of output from the logger
* `PORT` when this is not set, the backend will default to port 3000
* `ES_ADDR` the Elasticsearch address, if not set it will default the Elasticsearch address to `http://localhost:9200`
* `ALLOW_NO_TRANSPORT` if set to true, the backend will allow starting without an Email configured that receives critical errors.
* `ES_DEBUG` setting this to `true` will result in Elasticsearch logging to be **VERY** extensive, in almost all situation this should no be enabled.
* `PROMETHEUS_MIDDLEWARE` if set to `true` will enable metrics collection with [Express Prometheus Middleware](https://www.npmjs.com/package/express-prometheus-middleware)
- `NODE_ENV` when set to `production`, there will be a reduced amount of output from the logger
- `PORT` when this is not set, the backend will default to port 3000
- `ES_ADDR` the Elasticsearch address, if not set it will default the Elasticsearch address to `http://localhost:9200`
- `ALLOW_NO_TRANSPORT` if set to true, the backend will allow starting without an Email configured that receives critical errors.
- `ES_DEBUG` setting this to `true` will result in Elasticsearch logging to be **VERY** extensive, in almost all situation this should no be enabled.
- `PROMETHEUS_MIDDLEWARE` if set to `true` will enable metrics collection with [Express Prometheus Middleware](https://www.npmjs.com/package/express-prometheus-middleware)
## Config files
Each university can have it's specific config for the general backend and app and for all databases.
All config files can be found in `./config/`. There is a `default.ts` which is used by default. You can create an
@@ -95,6 +106,7 @@ To create your university specific config file for the elasticsearch you have to
scheme: `elasticsearch-<university license plate>.ts`.
## Debugging
Set `ES_DEBUG=true` to enable verbose Elasticsearch tracing information.
This can be useful to debug some issues between backend and elasticsearch.