Theaninova/openstapps
1
0
Fork 0
mirror of https://gitlab.com/openstapps/openstapps.git synced 2026-01-09 11:12:52 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

feat: add minimal deployment

Browse Source
This commit is contained in:
Anselm Stordeur
2019-01-22 13:39:04 +01:00
commit d3a9f38539
4 changed files with 242 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
.gitignore vendored Normal file
View File

@@ -0,0 +1 @@
database/

122
README.md Normal file
View File

@@ -0,0 +1,122 @@
# Minimal deployment
This is an example for how a deployment for StApps can look like. The project also shows why we decided to use Docker
for the deployment on all universities. With help of Docker's virtualization you don't have to install and configure
anything beyond `docker` and `docker compose` itself.
A complete StApps deployment consists of following projects:
```
|*****| |*******| |*********| |**********|
| app | <----> | proxy | <----> | backend | <----> | database |
|*****| |*******| |*********| |**********|
^
|
|************|
| connectors |
|************|
```
This project shows a very fast way to deploy [backend](https://gitlab.com/openstapps/backend),
[database](https://gitlab.com/openstapps/database) and [api](https://gitlab.com/openstapps/api) (to copy some data from
an existing deployment of the TU Berlin).
With a running backend and database you should be able to run the [app](https://gitlab.tubit.tu-berlin.de/stapps/app)
with your deployment.
# Step by step to your own deployment
At first you need to install [docker](https://docs.docker.com/install/) and
[docker-compose](https://docs.docker.com/compose/install/). Then clone or download this repository. To download the
docker images from GitLab you have to be authenticated against GitLabs container registry. Please execute
`docker login gitlab-registry.tubit.tu-berlin.de`. The requested credentials are the same as for GitLab.
`docker login` stores the credentials the users home directory. If you plan to execute docker commands only from root
or via sudo you should run `docker login` with root or sudo as well.
On Linux machines you should be able to proceed on [Execution on Linux](#execution-on-linux) otherwise follow me step
by step.
Be sure to execute all the following commands in the repository and with root privileges or access to the Docker daemon.
## Start the database
Our current database is Elasticsearch and most configuration is contained in the Docker image. Since Docker is not a
full virtualization it uses the hosts kernel. That's why on some systems you may need to increase your virtual memory to
get Elasticsearch to work.
([documentation for increasing your virtual memory](https://www.elastic.co/guide/en/elasticsearch/reference/5.6/vm-max-map-count.html))
To start the database
```sh
# create a directory for all data
mkdir -p database
# let the database be the owner of the folder
chown 1000:1000 database
# start the database
docker-compose up -d database
# have a look at the logs of the database
docker-compose logs database
```
As stated in the `docker-compose.yml` the database will expose port `9200` to other services in the same deployment
(all services in the file).
## Start the backend
The backend is providing an interface to search and bulk index data into the database. Simply run:
```sh
# start the backend
docker-compose up -d backend
# have a look at the logs of the backend
docker-compose logs backend
```
The backend will find the database on `http://database:9200` inside the deployment network. As stated in the
`docker-compose.yml` the backend will expose port 3000 on the host. You should be able to request the backend for it's
configuration on http://localhost:3000.
Now you have a running backend that can be queried by the app. You could already try to the install the
[app](https://gitlab.com/openstapps/app) and use it with your local deployment. That is pretty boring, because the
backend doesn't provide any data by now.
There is no public deployment for the app version 2.0.0 by now. But you could copy some data from there.
To import some `SCPlace`'s you could run:
```sh
docker-compose run --rm api copy --appVersion 2.0.0 place <url-of-public-deployment> http://backend:3000 20
```
This will copy data using the [api](https://gitlab.com/openstapps/api) from the deployment of the TU Berlin to your
deployment.
# Execution on Linux
On Linux you can execute a simple shell script that will run you through
steps above and copy some data in your deployment:
```sh
sh start.sh
```
# Where to go from here
## Writing your own connector
You now have your own deployment and can import some data from your university into the backend. To write your own
program to import data you should checkout the [minimal-connector](https://gitlab.com/openstapps/minimal-connector/)
This is a small program that shows the usage of the [api](https://gitlab.com/openstapps/api/) and imports some example
data from TypeScript code. Feel free to copy the [minimal-connector](https://gitlab.com/openstapps/minimal-connector/)
and write your own connector with it.
## Round off your deployment with the [proxy](https://gitlab.com/openstapps/proxy)
The backend is exposed on the port 3000 now. This means anyone can import data into your backend and you can only run
one version of the backend at a time. Have a look at the [proxy](https://gitlab.com/openstapps/proxy/) to secure and
round off your deployment. The proxy will allow you to run multiple deployments for different app version on one server
and provide some static data like images for your university.
## Explore docker capabilities
Docker is a great tool with many great features. Have a look at the
[docker cli documentation](https://docs.docker.com/engine/reference/commandline/cli/) or our
[docker cheat sheet](https://gitlab.tubit.tu-berlin.de/stapps/projectmanagement/blob/master/DOCKER.md) to learn
more about it.

30
docker-compose.yml Normal file
View File

@@ -0,0 +1,30 @@
version: '2'
services:
database:
image: registry.gitlab.com/openstapps/database:master
volumes:
- ./database:/usr/share/elasticsearch/data
expose:
- "9200"
backend:
image: registry.gitlab.com/openstapps/backend/default:master
environment:
ES_PORT_9200_TCP_ADDR: database
ES_PORT_9200_TCP_PORT: 9200
NODE_CONFIG_ENV: "elasticsearch"
ALLOW_NO_TRANSPORT: "true"
expose:
- 3000
ports:
- 3000:3000
links:
- "database"
labels:
- stapps.version=1.0.0
api:
image: registry.gitlab.com/openstapps/api/copy:v0.0.3
links:
- "backend"

89
start.sh Executable file
View File

@@ -0,0 +1,89 @@
SUDO=''
if [ $(id -u) != 0 ]; then
SUDO='sudo'
fi
if ! hash docker 2>/dev/null; then
echo "\"docker\" executable not found. Please install docker."
exit 1
fi
if ! hash docker-compose 2>/dev/null; then
echo "\"docker-compose\" executable not found. Please install docker-compose."
exit 1
fi
# check kernel parameter
# check if vm.max_map_count is at least 262144 on host
# this is needed for elasticsearch >5.x
ACTUAL_MAX_MAP_COUNT=$(sysctl -n vm.max_map_count)
MINIMAL_MAX_MAP_COUNT=262144
if [ "$ACTUAL_MAX_MAP_COUNT" -lt "$MINIMAL_MAX_MAP_COUNT" ]; then
echo "vm.max_map_count is too low. This is a kernel parameter used by java which is used by elasticsearch. Elasticsearch won't run if vm.max_map_count is too low."
while true; do
read -p "Do you wish to increase vm.max_map_count to 262144? " yn
case $yn in
[Yy]* ) $SUDO sysctl -w "vm.max_map_count=262144"; echo "To make this change permanent after restart: Add \"vm.max_map_count=262144\" to \"/etc/sysctl.conf\""; break;;
[Nn]* ) exit 1;;
* ) echo "Please answer yes or no.";;
esac
done
fi
mkdir -p database
$SUDO chown 1000:1000 database
cat <<EOF
Please execute the following command in another shell, but in the same directory.
This will start backend (and database, which is its dependency) in interactive mode so that you can see the logging
output of the backend container in your shell.
$SUDO docker-compose up backend
After you've done this and no more log messages appear **PRESS ENTER** to import data from the b-tu backend.
Useful commands:
* If you want to see logs from database container in real time, you can use the following command:
$SUDO docker-compose logs -f database
* In case you don't need the containers to be running anymore, you can also stop them using docker-compose command (example for database container):
$SUDO docker-compose stop database
EOF
read _
while true; do
read -p "How much do you want to import? (A)ll, (M)inimal or (N)othing? " yn
case $yn in
[Aa]* ) CHOICE="a"; break ;;
[Mm]* ) CHOICE="m"; break ;;
[Nn]* ) CHOICE="n"; break ;;
* ) echo "Please choose A, M or N!";;
esac
done
if [ "$CHOICE" = "n" ]; then
exit;
fi
echo "This is currently not supported, because there is no public deployment for the app v2.0.0 by now."
exit 1
$SUDO docker-compose run --rm api copy --appVersion 2.0.0 place https://stappsbe01.innocampus.tu-berlin.de http://backend:3000 20
$SUDO docker-compose run --rm api copy --appVersion 2.0.0 dish https://stappsbe01.innocampus.tu-berlin.de http://backend:3000 20
if [ "$CHOICE" = "m" ]; then
exit;
fi
$SUDO docker-compose run --rm api copy --appVersion 2.0.0 article https://stappsbe01.innocampus.tu-berlin.de http://backend:3000 20
$SUDO docker-compose run --rm api copy --appVersion 2.0.0 floor https://stappsbe01.innocampus.tu-berlin.de http://backend:3000 20
$SUDO docker-compose run --rm api copy --appVersion 2.0.0 organization https://stappsbe01.innocampus.tu-berlin.de http://backend:3000 20
$SUDO docker-compose run --rm api copy --appVersion 2.0.0 person https://stappsbe01.innocampus.tu-berlin.de http://backend:3000 20
$SUDO docker-compose run --rm api copy --appVersion 2.0.0 catalog https://stappsbe01.innocampus.tu-berlin.de http://backend:3000 20
$SUDO docker-compose run --rm api copy --appVersion 2.0.0 event https://stappsbe01.innocampus.tu-berlin.de http://backend:3000 20
$SUDO docker-compose run --rm api copy --appVersion 2.0.0 date https://stappsbe01.innocampus.tu-berlin.de http://backend:3000 20