mirror of
https://gitlab.com/openstapps/openstapps.git
synced 2026-01-22 09:32:41 +00:00
feat: add minimal deployment
This commit is contained in:
1
.gitignore
vendored
Normal file
1
.gitignore
vendored
Normal file
@@ -0,0 +1 @@
|
|||||||
|
database/
|
||||||
122
README.md
Normal file
122
README.md
Normal file
@@ -0,0 +1,122 @@
|
|||||||
|
# Minimal deployment
|
||||||
|
This is an example for how a deployment for StApps can look like. The project also shows why we decided to use Docker
|
||||||
|
for the deployment on all universities. With help of Docker's virtualization you don't have to install and configure
|
||||||
|
anything beyond `docker` and `docker compose` itself.
|
||||||
|
|
||||||
|
A complete StApps deployment consists of following projects:
|
||||||
|
```
|
||||||
|
|*****| |*******| |*********| |**********|
|
||||||
|
| app | <----> | proxy | <----> | backend | <----> | database |
|
||||||
|
|*****| |*******| |*********| |**********|
|
||||||
|
^
|
||||||
|
|
|
||||||
|
|************|
|
||||||
|
| connectors |
|
||||||
|
|************|
|
||||||
|
```
|
||||||
|
|
||||||
|
This project shows a very fast way to deploy [backend](https://gitlab.com/openstapps/backend),
|
||||||
|
[database](https://gitlab.com/openstapps/database) and [api](https://gitlab.com/openstapps/api) (to copy some data from
|
||||||
|
an existing deployment of the TU Berlin).
|
||||||
|
|
||||||
|
|
||||||
|
With a running backend and database you should be able to run the [app](https://gitlab.tubit.tu-berlin.de/stapps/app)
|
||||||
|
with your deployment.
|
||||||
|
|
||||||
|
# Step by step to your own deployment
|
||||||
|
At first you need to install [docker](https://docs.docker.com/install/) and
|
||||||
|
[docker-compose](https://docs.docker.com/compose/install/). Then clone or download this repository. To download the
|
||||||
|
docker images from GitLab you have to be authenticated against GitLabs container registry. Please execute
|
||||||
|
`docker login gitlab-registry.tubit.tu-berlin.de`. The requested credentials are the same as for GitLab.
|
||||||
|
`docker login` stores the credentials the users home directory. If you plan to execute docker commands only from root
|
||||||
|
or via sudo you should run `docker login` with root or sudo as well.
|
||||||
|
|
||||||
|
|
||||||
|
On Linux machines you should be able to proceed on [Execution on Linux](#execution-on-linux) otherwise follow me step
|
||||||
|
by step.
|
||||||
|
|
||||||
|
|
||||||
|
Be sure to execute all the following commands in the repository and with root privileges or access to the Docker daemon.
|
||||||
|
|
||||||
|
## Start the database
|
||||||
|
Our current database is Elasticsearch and most configuration is contained in the Docker image. Since Docker is not a
|
||||||
|
full virtualization it uses the hosts kernel. That's why on some systems you may need to increase your virtual memory to
|
||||||
|
get Elasticsearch to work.
|
||||||
|
([documentation for increasing your virtual memory](https://www.elastic.co/guide/en/elasticsearch/reference/5.6/vm-max-map-count.html))
|
||||||
|
|
||||||
|
|
||||||
|
To start the database
|
||||||
|
```sh
|
||||||
|
# create a directory for all data
|
||||||
|
mkdir -p database
|
||||||
|
|
||||||
|
# let the database be the owner of the folder
|
||||||
|
chown 1000:1000 database
|
||||||
|
|
||||||
|
# start the database
|
||||||
|
docker-compose up -d database
|
||||||
|
|
||||||
|
# have a look at the logs of the database
|
||||||
|
docker-compose logs database
|
||||||
|
```
|
||||||
|
|
||||||
|
As stated in the `docker-compose.yml` the database will expose port `9200` to other services in the same deployment
|
||||||
|
(all services in the file).
|
||||||
|
|
||||||
|
## Start the backend
|
||||||
|
The backend is providing an interface to search and bulk index data into the database. Simply run:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
# start the backend
|
||||||
|
docker-compose up -d backend
|
||||||
|
|
||||||
|
# have a look at the logs of the backend
|
||||||
|
docker-compose logs backend
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
The backend will find the database on `http://database:9200` inside the deployment network. As stated in the
|
||||||
|
`docker-compose.yml` the backend will expose port 3000 on the host. You should be able to request the backend for it's
|
||||||
|
configuration on http://localhost:3000.
|
||||||
|
|
||||||
|
|
||||||
|
Now you have a running backend that can be queried by the app. You could already try to the install the
|
||||||
|
[app](https://gitlab.com/openstapps/app) and use it with your local deployment. That is pretty boring, because the
|
||||||
|
backend doesn't provide any data by now.
|
||||||
|
|
||||||
|
There is no public deployment for the app version 2.0.0 by now. But you could copy some data from there.
|
||||||
|
|
||||||
|
To import some `SCPlace`'s you could run:
|
||||||
|
```sh
|
||||||
|
docker-compose run --rm api copy --appVersion 2.0.0 place <url-of-public-deployment> http://backend:3000 20
|
||||||
|
```
|
||||||
|
This will copy data using the [api](https://gitlab.com/openstapps/api) from the deployment of the TU Berlin to your
|
||||||
|
deployment.
|
||||||
|
|
||||||
|
# Execution on Linux
|
||||||
|
On Linux you can execute a simple shell script that will run you through
|
||||||
|
steps above and copy some data in your deployment:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
sh start.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
# Where to go from here
|
||||||
|
## Writing your own connector
|
||||||
|
You now have your own deployment and can import some data from your university into the backend. To write your own
|
||||||
|
program to import data you should checkout the [minimal-connector](https://gitlab.com/openstapps/minimal-connector/)
|
||||||
|
This is a small program that shows the usage of the [api](https://gitlab.com/openstapps/api/) and imports some example
|
||||||
|
data from TypeScript code. Feel free to copy the [minimal-connector](https://gitlab.com/openstapps/minimal-connector/)
|
||||||
|
and write your own connector with it.
|
||||||
|
|
||||||
|
## Round off your deployment with the [proxy](https://gitlab.com/openstapps/proxy)
|
||||||
|
The backend is exposed on the port 3000 now. This means anyone can import data into your backend and you can only run
|
||||||
|
one version of the backend at a time. Have a look at the [proxy](https://gitlab.com/openstapps/proxy/) to secure and
|
||||||
|
round off your deployment. The proxy will allow you to run multiple deployments for different app version on one server
|
||||||
|
and provide some static data like images for your university.
|
||||||
|
|
||||||
|
## Explore docker capabilities
|
||||||
|
Docker is a great tool with many great features. Have a look at the
|
||||||
|
[docker cli documentation](https://docs.docker.com/engine/reference/commandline/cli/) or our
|
||||||
|
[docker cheat sheet](https://gitlab.tubit.tu-berlin.de/stapps/projectmanagement/blob/master/DOCKER.md) to learn
|
||||||
|
more about it.
|
||||||
30
docker-compose.yml
Normal file
30
docker-compose.yml
Normal file
@@ -0,0 +1,30 @@
|
|||||||
|
version: '2'
|
||||||
|
services:
|
||||||
|
database:
|
||||||
|
image: registry.gitlab.com/openstapps/database:master
|
||||||
|
volumes:
|
||||||
|
- ./database:/usr/share/elasticsearch/data
|
||||||
|
expose:
|
||||||
|
- "9200"
|
||||||
|
|
||||||
|
backend:
|
||||||
|
image: registry.gitlab.com/openstapps/backend/default:master
|
||||||
|
environment:
|
||||||
|
ES_PORT_9200_TCP_ADDR: database
|
||||||
|
ES_PORT_9200_TCP_PORT: 9200
|
||||||
|
NODE_CONFIG_ENV: "elasticsearch"
|
||||||
|
ALLOW_NO_TRANSPORT: "true"
|
||||||
|
expose:
|
||||||
|
- 3000
|
||||||
|
ports:
|
||||||
|
- 3000:3000
|
||||||
|
links:
|
||||||
|
- "database"
|
||||||
|
labels:
|
||||||
|
- stapps.version=1.0.0
|
||||||
|
|
||||||
|
api:
|
||||||
|
image: registry.gitlab.com/openstapps/api/copy:v0.0.3
|
||||||
|
links:
|
||||||
|
- "backend"
|
||||||
|
|
||||||
89
start.sh
Executable file
89
start.sh
Executable file
@@ -0,0 +1,89 @@
|
|||||||
|
SUDO=''
|
||||||
|
|
||||||
|
if [ $(id -u) != 0 ]; then
|
||||||
|
SUDO='sudo'
|
||||||
|
fi
|
||||||
|
|
||||||
|
if ! hash docker 2>/dev/null; then
|
||||||
|
echo "\"docker\" executable not found. Please install docker."
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
if ! hash docker-compose 2>/dev/null; then
|
||||||
|
echo "\"docker-compose\" executable not found. Please install docker-compose."
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# check kernel parameter
|
||||||
|
# check if vm.max_map_count is at least 262144 on host
|
||||||
|
# this is needed for elasticsearch >5.x
|
||||||
|
ACTUAL_MAX_MAP_COUNT=$(sysctl -n vm.max_map_count)
|
||||||
|
MINIMAL_MAX_MAP_COUNT=262144
|
||||||
|
|
||||||
|
if [ "$ACTUAL_MAX_MAP_COUNT" -lt "$MINIMAL_MAX_MAP_COUNT" ]; then
|
||||||
|
echo "vm.max_map_count is too low. This is a kernel parameter used by java which is used by elasticsearch. Elasticsearch won't run if vm.max_map_count is too low."
|
||||||
|
while true; do
|
||||||
|
read -p "Do you wish to increase vm.max_map_count to 262144? " yn
|
||||||
|
case $yn in
|
||||||
|
[Yy]* ) $SUDO sysctl -w "vm.max_map_count=262144"; echo "To make this change permanent after restart: Add \"vm.max_map_count=262144\" to \"/etc/sysctl.conf\""; break;;
|
||||||
|
[Nn]* ) exit 1;;
|
||||||
|
* ) echo "Please answer yes or no.";;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
fi
|
||||||
|
|
||||||
|
mkdir -p database
|
||||||
|
$SUDO chown 1000:1000 database
|
||||||
|
|
||||||
|
cat <<EOF
|
||||||
|
Please execute the following command in another shell, but in the same directory.
|
||||||
|
This will start backend (and database, which is its dependency) in interactive mode so that you can see the logging
|
||||||
|
output of the backend container in your shell.
|
||||||
|
|
||||||
|
$SUDO docker-compose up backend
|
||||||
|
|
||||||
|
After you've done this and no more log messages appear **PRESS ENTER** to import data from the b-tu backend.
|
||||||
|
|
||||||
|
Useful commands:
|
||||||
|
|
||||||
|
* If you want to see logs from database container in real time, you can use the following command:
|
||||||
|
|
||||||
|
$SUDO docker-compose logs -f database
|
||||||
|
|
||||||
|
* In case you don't need the containers to be running anymore, you can also stop them using docker-compose command (example for database container):
|
||||||
|
|
||||||
|
$SUDO docker-compose stop database
|
||||||
|
EOF
|
||||||
|
|
||||||
|
read _
|
||||||
|
|
||||||
|
while true; do
|
||||||
|
read -p "How much do you want to import? (A)ll, (M)inimal or (N)othing? " yn
|
||||||
|
case $yn in
|
||||||
|
[Aa]* ) CHOICE="a"; break ;;
|
||||||
|
[Mm]* ) CHOICE="m"; break ;;
|
||||||
|
[Nn]* ) CHOICE="n"; break ;;
|
||||||
|
* ) echo "Please choose A, M or N!";;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
if [ "$CHOICE" = "n" ]; then
|
||||||
|
exit;
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "This is currently not supported, because there is no public deployment for the app v2.0.0 by now."
|
||||||
|
exit 1
|
||||||
|
$SUDO docker-compose run --rm api copy --appVersion 2.0.0 place https://stappsbe01.innocampus.tu-berlin.de http://backend:3000 20
|
||||||
|
$SUDO docker-compose run --rm api copy --appVersion 2.0.0 dish https://stappsbe01.innocampus.tu-berlin.de http://backend:3000 20
|
||||||
|
|
||||||
|
if [ "$CHOICE" = "m" ]; then
|
||||||
|
exit;
|
||||||
|
fi
|
||||||
|
|
||||||
|
$SUDO docker-compose run --rm api copy --appVersion 2.0.0 article https://stappsbe01.innocampus.tu-berlin.de http://backend:3000 20
|
||||||
|
$SUDO docker-compose run --rm api copy --appVersion 2.0.0 floor https://stappsbe01.innocampus.tu-berlin.de http://backend:3000 20
|
||||||
|
$SUDO docker-compose run --rm api copy --appVersion 2.0.0 organization https://stappsbe01.innocampus.tu-berlin.de http://backend:3000 20
|
||||||
|
$SUDO docker-compose run --rm api copy --appVersion 2.0.0 person https://stappsbe01.innocampus.tu-berlin.de http://backend:3000 20
|
||||||
|
$SUDO docker-compose run --rm api copy --appVersion 2.0.0 catalog https://stappsbe01.innocampus.tu-berlin.de http://backend:3000 20
|
||||||
|
$SUDO docker-compose run --rm api copy --appVersion 2.0.0 event https://stappsbe01.innocampus.tu-berlin.de http://backend:3000 20
|
||||||
|
$SUDO docker-compose run --rm api copy --appVersion 2.0.0 date https://stappsbe01.innocampus.tu-berlin.de http://backend:3000 20
|
||||||
Reference in New Issue
Block a user