Compare commits

..

2 Commits

Author SHA1 Message Date
Rainer Killinger
b7f512b7bd refactor: update changelog
ci: publish release
2023-08-03 16:08:02 +02:00
Rainer Killinger
78d9690974 ci: polish publishing via ci pipelines 2023-08-03 16:00:32 +02:00
801 changed files with 27562 additions and 38794 deletions

View File

@@ -0,0 +1,5 @@
---
'@openstapps/app': major
---
Update to Angular 16.1

View File

@@ -0,0 +1,8 @@
---
'@openstapps/core-tools': major
---
Removed pack tool
Use a bundler like `tsup` in conjunction with an
`index.ts` that aggregates all exports instead.

View File

@@ -0,0 +1,11 @@
---
'@openstapps/api-plugin': major
'@openstapps/api-cli': major
'@openstapps/api': major
---
Split API into API, API-CLI and API-Plugin
Plugins are now required to use `api-plugin`.
Consumers of `api` can benefit from a slimmer package with
no NodeJS dependencies.

View File

@@ -0,0 +1,31 @@
---
'@openstapps/backend': major
---
Migrate config system to cosmiconfig
Configs are now written using type-checked JavaScript
```js
// @ts-check
// This file is now type-safe just like TypeScript, but
// without the compilation step
/** @type {import('@openstapps/package').Type} */
const foo = {};
```
You can write config files in JavaScript, JSON, or other
cosmiconfig-supported formats.
The config files now are
- `backendrc.{js,json,...}`
- `elasticsearchrc.{js,json,...}`
- `prometheusrc.{js,json,...}`
You can also split them into multiple files,
as well as using the `markdown.js` helper in
`config/default/tools` to include markdown files
as text (see `config/f-u/about-pages`)

View File

@@ -0,0 +1,51 @@
---
'@openstapps/projectmanagement': major
'@openstapps/prettier-config': major
'@openstapps/es-mapping-generator': major
'@openstapps/backend-config': major
'@openstapps/eslint-config': major
'@openstapps/minimal-connector': major
'@openstapps/collection-utils': major
'@openstapps/minimal-plugin': major
'@openstapps/tsconfig': major
'@openstapps/api-plugin': major
'@openstapps/core-tools': major
'@openstapps/gitlab-api': major
'@openstapps/easy-ast': major
'@openstapps/api-cli': major
'@openstapps/backend': major
'@openstapps/logger': major
'@openstapps/proxy': major
'@openstapps/core': major
'@openstapps/app': major
'@openstapps/api': major
---
Move project to a turbo monorepo & pnpm
Internal dependencies are now defined using `"@openstapps/package": "workspace:*"`
- Removed extraneous files from packages
- `.npmrc`
- `.npmignore`
- `.mailmap`
- `.gitignore`
- `CONTRIBUTING.md`
- `LICENSE` (Project license file is added upon publishing, see [pnpm.io](https://pnpm.io/cli/publish))
- `package-lock.json`
- `.editorconfig`
- `.eslintrc.json` (moved eslint config to `package.json`)
- `.eslintignore`
- `.gitlab-ci.yml` (Most workflows are workspace-level)
- `.gitlab/**` (issue templates etc. are now workspace-level)
- `.dockerignore` (Docker files are determined by which files are deployed with `pnpm deploy`, as per `package.json/files`)
- TSConfig has been moved to its own package (You can now use `"extends": "@openstapps/tsconfig"`)
- Removed ESLint and Prettier peer dependency hell by injecting them through the `.pnpmfile.cjs`
- Added syncpack for keeping dependency versions in sync (and consistent key ordering in `package.json`)
- Replaced conventional changelog with changesets
- Apps with binaries now use a top level `app.js`
```js
#!/usr/bin/env node
import './lib/app.js';
```

View File

@@ -0,0 +1,13 @@
---
'@openstapps/core': major
---
Migrate away from [ts-optchain](https://www.npmjs.com/package/ts-optchain)
The package has been deprecated with the last releast being 4 years ago.
TypeScript as well as ECMAScript have native support for optional
chaining with the `?.` operator now.
You will need to update any packages that rely on the translator
module.

View File

@@ -0,0 +1,25 @@
---
'@openstapps/projectmanagement': patch
'@openstapps/prettier-config': patch
'@openstapps/es-mapping-generator': patch
'@openstapps/eslint-config': patch
'@openstapps/minimal-connector': patch
'@openstapps/minimal-plugin': patch
'@openstapps/core-tools': patch
'@openstapps/gitlab-api': patch
'@openstapps/backend': patch
'@openstapps/logger': patch
'@openstapps/proxy': patch
'@openstapps/core': patch
'@openstapps/api': patch
---
Migrated changelogs to changeset format
```js
import fs from 'fs';
const path = 'packages/logger/CHANGELOG.md';
fs.writeFileSync(path, fs.readFileSync(path, 'utf8').replace(/^#+\s+\[/gm, '## ['));
```

View File

@@ -0,0 +1,8 @@
---
'@openstapps/backend': minor
'@openstapps/api-cli': major
---
Migrate integration tests from docker-compose solution to a shell script
`api-cli` no longer builds as a Docker container as a result.

View File

@@ -0,0 +1,34 @@
---
'@openstapps/projectmanagement': major
'@openstapps/prettier-config': major
'@openstapps/es-mapping-generator': major
'@openstapps/backend-config': major
'@openstapps/eslint-config': major
'@openstapps/minimal-connector': major
'@openstapps/collection-utils': major
'@openstapps/minimal-plugin': major
'@openstapps/tsconfig': major
'@openstapps/api-plugin': major
'@openstapps/core-tools': major
'@openstapps/gitlab-api': major
'@openstapps/easy-ast': major
'@openstapps/api-cli': major
'@openstapps/backend': major
'@openstapps/logger': major
'@openstapps/proxy': major
'@openstapps/core': major
'@openstapps/app': major
'@openstapps/api': major
---
Migrate to ESM
CommonJS is no longer supported in any capacity. To use the new
version, you will need to migrate your package to ESM.
We recommend using `tsup` and `Node 18`.
```json
{
"type": "module"
}
```

View File

@@ -0,0 +1,30 @@
---
'@openstapps/projectmanagement': major
'@openstapps/prettier-config': major
'@openstapps/es-mapping-generator': major
'@openstapps/backend-config': major
'@openstapps/eslint-config': major
'@openstapps/minimal-connector': major
'@openstapps/collection-utils': major
'@openstapps/minimal-plugin': major
'@openstapps/tsconfig': major
'@openstapps/api-plugin': major
'@openstapps/core-tools': major
'@openstapps/gitlab-api': major
'@openstapps/easy-ast': major
'@openstapps/api-cli': major
'@openstapps/backend': major
'@openstapps/logger': major
'@openstapps/proxy': major
'@openstapps/core': major
'@openstapps/app': major
'@openstapps/api': major
---
Migrate package to Node 18
- Consumers of this package will need to migrate to Node 18 or
higher.
- Packages have been migrated from promisified `readFile` or
`readFileSync` towards `fs/promises`
- Packages use native `flatMap` now

View File

@@ -0,0 +1,5 @@
---
'@openstapps/gitlab-api': minor
---
Migrate from request-promise-native to got

View File

@@ -0,0 +1,5 @@
---
'@openstapps/projectmanagement': patch
---
Moved project-docs to the [GitLab Wiki](https://gitlab.com/openstapps/openstapps/-/wikis/home) repo.

View File

@@ -0,0 +1,31 @@
---
'@openstapps/projectmanagement': minor
'@openstapps/prettier-config': minor
'@openstapps/es-mapping-generator': minor
'@openstapps/backend-config': minor
'@openstapps/eslint-config': minor
'@openstapps/minimal-connector': minor
'@openstapps/collection-utils': minor
'@openstapps/minimal-plugin': minor
'@openstapps/tsconfig': minor
'@openstapps/api-plugin': minor
'@openstapps/core-tools': minor
'@openstapps/gitlab-api': minor
'@openstapps/easy-ast': minor
'@openstapps/api-cli': minor
'@openstapps/backend': minor
'@openstapps/logger': minor
'@openstapps/proxy': minor
'@openstapps/core': minor
'@openstapps/app': minor
'@openstapps/api': minor
---
Migrate tests to C8/Chai/Mocha
- `@testdeck` OOP testing has been removed.
- Tests have been unified
- CommonJS module mocking has been replaced through
refactoring of tests, as ES Modules cannot be mocked
(do yourself a favor and don't try to mock them)
- C8 now replaces NYC as a native coverage tool

View File

@@ -0,0 +1,5 @@
---
'@openstapps/es-mapping-generator': patch
---
Remove @openstapps/logger dependency

68
.changeset/pre.json Normal file
View File

@@ -0,0 +1,68 @@
{
"mode": "pre",
"tag": "next",
"initialVersions": {
"@openstapps/backend": "2.0.0",
"@openstapps/database": "2.0.0",
"@openstapps/proxy": "2.0.0",
"@openstapps/backend-config": "2.0.0",
"@openstapps/eslint-config": "2.0.0",
"@openstapps/prettier-config": "2.0.0",
"@openstapps/projectmanagement": "2.0.0",
"@openstapps/tsconfig": "2.0.0",
"@openstapps/minimal-connector": "2.0.0",
"@openstapps/minimal-deployment": "2.0.0",
"@openstapps/minimal-plugin": "2.0.0",
"@openstapps/app": "2.0.0",
"@openstapps/api": "2.0.0",
"@openstapps/api-cli": "2.0.0",
"@openstapps/api-plugin": "2.0.0",
"@openstapps/collection-utils": "2.0.0",
"@openstapps/core": "2.0.0",
"@openstapps/core-tools": "2.0.0",
"@openstapps/easy-ast": "2.0.0",
"@openstapps/es-mapping-generator": "2.0.0",
"@openstapps/gitlab-api": "2.0.0",
"@openstapps/logger": "2.0.0",
"@openstapps/app-builder-image": "3.0.0-next.3",
"@openstapps/node-base": "3.0.0-next.3",
"@openstapps/node-builder": "3.0.0-next.3"
},
"changesets": [
"bright-dryers-act",
"cool-jars-kiss",
"cuddly-bobcats-roll",
"dull-news-appear",
"five-ears-wash",
"fuzzy-walls-greet",
"giant-crabs-cheer",
"gorgeous-flowers-reflect",
"healthy-steaks-shop",
"honest-sheep-train",
"late-zoos-breathe",
"moody-parrots-develop",
"neat-hats-trade",
"new-pianos-joke",
"pretty-timers-complain",
"proud-wolves-end",
"quick-houses-count",
"rare-squids-bake",
"serious-meals-sin",
"silent-maps-float",
"silly-news-punch",
"smart-ghosts-shout",
"soft-donuts-fail",
"sour-coins-visit",
"spicy-snails-sort",
"stale-garlics-share",
"strange-hounds-repair",
"tall-ducks-dream",
"tame-mayflies-hug",
"tame-rings-dream",
"tasty-islands-smell",
"thick-weeks-compete",
"thin-camels-give",
"tidy-buses-reflect",
"tough-hairs-provide"
]
}

View File

@@ -0,0 +1,8 @@
---
'@openstapps/projectmanagement': major
'@openstapps/node-base': major
---
Move images to separate packages
Removed builder image due to migration to Kaniko

View File

@@ -0,0 +1,9 @@
---
'@openstapps/eslint-config': minor
---
Update rules
- `unicorn/no-array-reduce` is now off
- `unicorn/no-non-null-assertion` is now off
- Removed prettier rules (use prettier formatting instead)

View File

@@ -0,0 +1,7 @@
---
'@openstapps/es-mapping-generator': major
---
Remove put-es-templates
The `put-es-templates` functionality has been removed.

View File

@@ -0,0 +1,24 @@
---
'@openstapps/projectmanagement': patch
'@openstapps/prettier-config': patch
'@openstapps/es-mapping-generator': patch
'@openstapps/backend-config': patch
'@openstapps/eslint-config': patch
'@openstapps/minimal-connector': patch
'@openstapps/collection-utils': patch
'@openstapps/minimal-plugin': patch
'@openstapps/tsconfig': patch
'@openstapps/api-plugin': patch
'@openstapps/core-tools': patch
'@openstapps/gitlab-api': patch
'@openstapps/easy-ast': patch
'@openstapps/api-cli': patch
'@openstapps/backend': patch
'@openstapps/logger': patch
'@openstapps/proxy': patch
'@openstapps/core': patch
'@openstapps/app': patch
'@openstapps/api': patch
---
Migrate away from @openstapps/configuration

View File

@@ -0,0 +1,5 @@
---
'@openstapps/projectmanagement': patch
---
Update release configs

View File

@@ -0,0 +1,29 @@
---
'@openstapps/projectmanagement': patch
'@openstapps/prettier-config': patch
'@openstapps/es-mapping-generator': patch
'@openstapps/backend-config': patch
'@openstapps/eslint-config': patch
'@openstapps/minimal-deployment': patch
'@openstapps/minimal-connector': patch
'@openstapps/collection-utils': patch
'@openstapps/minimal-plugin': patch
'@openstapps/tsconfig': patch
'@openstapps/node-builder': patch
'@openstapps/api-plugin': patch
'@openstapps/core-tools': patch
'@openstapps/gitlab-api': patch
'@openstapps/app-builder-image': patch
'@openstapps/easy-ast': patch
'@openstapps/database': patch
'@openstapps/node-base': patch
'@openstapps/api-cli': patch
'@openstapps/backend': patch
'@openstapps/logger': patch
'@openstapps/proxy': patch
'@openstapps/core': patch
'@openstapps/app': patch
'@openstapps/api': patch
---
Update to TypeScript 5.1.6

View File

@@ -0,0 +1,5 @@
---
'@openstapps/app': patch
---
Migrate collection helpers to use @openstapps/collection-utils

View File

@@ -0,0 +1,8 @@
---
'@openstapps/app': minor
---
Migrate to Ionic 7
- Migrate uses of `<ion-label>` with inputs to new syntax
- Fix infinite loop in schedule date picker (`datetime.confirm()` to `datetime.cancel()`)

View File

@@ -0,0 +1,7 @@
---
'@openstapps/api': minor
---
Migrate HttpClient to Node 18's native `fetch` API
HttpClient can now be used both in NodeJS and browsers.

View File

@@ -0,0 +1,7 @@
---
'@openstapps/es-mapping-generator': major
'@openstapps/backend': major
'@openstapps/core': major
---
Migrate es mapping types from es-mapping-generator to .d.ts next to generated mappings

View File

@@ -0,0 +1,5 @@
---
'@openstapps/eslint-config': patch
---
Add license header enforcement rule (unfinished)

View File

@@ -0,0 +1,5 @@
---
'@openstapps/core': minor
---
Include openapi.json in built package

View File

@@ -0,0 +1,26 @@
---
'@openstapps/projectmanagement': minor
'@openstapps/minimal-connector': minor
'@openstapps/minimal-plugin': minor
'@openstapps/collection-utils': minor
'@openstapps/core-tools': minor
'@openstapps/gitlab-api': minor
'@openstapps/backend': minor
'@openstapps/logger': minor
'@openstapps/core': minor
'@openstapps/api': minor
---
Migrate away from `@krlwlfrt/async-pool`
```ts
import {mapAsyncLimit} from '@openstapps/collection-utils';
await mapAsyncLimit(
[1, 2, 3],
async it => {
await someNetworkRequest(it);
},
5,
);
```

View File

@@ -0,0 +1,5 @@
---
'@openstapps/app': minor
---
Migrate away from JIT compilation

View File

@@ -0,0 +1,5 @@
---
'@openstapps/logger': patch
---
Fix an issue with chalk terminal colors in tests

View File

@@ -0,0 +1,9 @@
---
'@openstapps/core-tools': major
---
Migrate openapi generation to output single file
The OpenAPI generator now outputs a single file instead
of a directory with the `openapi.json` and copied schema
files.

View File

@@ -0,0 +1,6 @@
---
'@openstapps/app': patch
---
Migrate unit tests to karma-coverage and junit reports.
Fixes an issue where coverage reports would not be generated.

View File

@@ -0,0 +1,5 @@
---
'@openstapps/projectmanagement': minor
---
Migrate moment.js to date-fns

View File

@@ -0,0 +1,23 @@
---
'@openstapps/projectmanagement': major
'@openstapps/es-mapping-generator': major
'@openstapps/minimal-connector': major
'@openstapps/collection-utils': major
'@openstapps/minimal-plugin': major
'@openstapps/api-plugin': major
'@openstapps/core-tools': major
'@openstapps/gitlab-api': major
'@openstapps/easy-ast': major
'@openstapps/api-cli': major
'@openstapps/backend': major
'@openstapps/logger': major
'@openstapps/proxy': major
'@openstapps/core': major
'@openstapps/api': major
---
Migrate build system to `tsup`
All packages now use an `index.ts` file to expose contents.
You will need to migrate paths from `import foo from '@scope/package/lib/foo` to `import foo from '@scope/package'`

View File

@@ -0,0 +1,9 @@
---
'@openstapps/core-tools': major
'@openstapps/easy-ast': major
---
Migrate easy-ast to separate package
The `easy-ast` part of `core-tools` has been moved to its own package.
For migration, simply use the new package.

View File

@@ -0,0 +1,5 @@
---
'@openstapps/core-tools': patch
---
Migrate from away from lodash

1
.envrc
View File

@@ -1 +0,0 @@
use flake

4
.gitignore vendored
View File

@@ -24,7 +24,6 @@ report-junit.xml
# NixOS flake
result
hsperfdata_root
.direnv/
# Directory for instrumented libs generated by jscoverage/JSCover
lib-cov
@@ -98,8 +97,7 @@ typings/
# ignore ide files
.idea
.vscode?*
!.vscode/launch.json
.vscode
# ignore lib
lib

View File

@@ -21,16 +21,25 @@ include:
- local: /.gitlab/publishing.gitlab-ci.yml
variables:
TURBO_CACHE_BYPASS: "--force --no-cache"
TURBO_CACHE_BYPASS:
value: ''
options:
- '--force'
- '--no-cache'
- ''
description: Bypass turbo cache
default:
image: registry.gitlab.com/openstapps/openstapps/node-builder
tags:
- saas-linux-xlarge-amd64
- performance
interruptible: true
before_script:
- corepack enable
- corepack prepare pnpm@latest-10 --activate
- corepack prepare pnpm@latest-8 --activate
- echo TURBO_API=$TURBO_API >> .env.local
- echo TURBO_TOKEN=$TURBO_TOKEN >> .env.local
- echo TURBO_TEAM=$TURBO_TEAM >> .env.local
- pnpm config set store-dir .pnpm-store
cache: &pnpm_cache
key:
@@ -54,13 +63,13 @@ build:
# - pnpm i --prefer-offline
# - pnpm build:full:skip || pnpm i -r --prefer-offline
- pnpm install
# TODO: whats the benefit for CI? something doesn't work with git here...
# TODO: whats the benifit for CI? something doesn't work with git here...
# "🦋 error Error: Failed to find where HEAD diverged from master. Does master exist?"
# - pnpm changeset:status
- pnpm build:full $TURBO_CACHE_BYPASS
- .gitlab/ci/enableGitlabReviewToolbar.sh frontend/app/www/browser/index.html "$CI_PROJECT_ID" "$CI_OPEN_MERGE_REQUESTS"
- cp frontend/app/www/browser/index.html frontend/app/www/browser/200.html
- pnpm dlx surge -p frontend/app/www/browser -d https://$CI_PROJECT_NAME-$DEPLOY_ID.surge.sh/
- .gitlab/ci/enableGitlabReviewToolbar.sh frontend/app/www/index.html "$CI_PROJECT_ID" "$CI_OPEN_MERGE_REQUESTS"
- cp frontend/app/www/index.html frontend/app/www/200.html
- pnpm dlx surge -p frontend/app/www -d https://$CI_PROJECT_NAME-$DEPLOY_ID.surge.sh/
cache:
<<: *pnpm_cache
policy: pull-push
@@ -71,7 +80,7 @@ build:
rules: &deploy-rules
- if: $CI_PIPELINE_SOURCE == 'merge_request_event'
variables:
DEPLOY_ID: $CI_MERGE_REQUEST_IID
DEPLOY_ID: $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME
- if: $CI_COMMIT_BRANCH == 'main'
variables:
DEPLOY_ID: production
@@ -96,7 +105,13 @@ unit:
stage: test
needs: ['build']
script:
# - pnpm config set recursive-install false
# - pnpm i --prefer-offline
# - pnpm test:skip || pnpm i -r --prefer-offline
- pnpm install
- cd node_modules/.pnpm/re2*/node_modules/re2
- npm run install
- cd $CI_PROJECT_DIR
- pnpm test $TURBO_CACHE_BYPASS
coverage: '/Statements[^:]*\:[^:]*\s+([\d\.]+)%/'
artifacts:
@@ -118,7 +133,7 @@ audit:
allow_failure: true
needs: []
script:
- pnpm audit --prod --audit-level critical
- pnpm audit --prod
rules:
- if: $CI_COMMIT_BRANCH == 'main'
allow_failure: false

View File

@@ -1,11 +1,6 @@
.limit_publish_pipelines:
rules:
- if: '$CI_COMMIT_BRANCH == "develop" && $CI_COMMIT_MESSAGE =~ /ci: publish prerelease/ && $CI_PIPELINE_SOURCE != "schedule"'
variables:
PUBLISH_TAG: next
- if: '$CI_COMMIT_BRANCH == "main" && $CI_COMMIT_MESSAGE =~ /ci: publish release/ && $CI_PIPELINE_SOURCE != "schedule"'
variables:
PUBLISH_TAG: latest
- if: '($CI_COMMIT_BRANCH == "main" || $CI_COMMIT_BRANCH == "develop") && $CI_COMMIT_MESSAGE =~ /^ci: publish release/ && $CI_PIPELINE_SOURCE != "schedule"'
deploy:
stage: publish
@@ -27,8 +22,10 @@ publish image:
- tags
needs: ['deploy']
image:
name: gcr.io/kaniko-project/executor:v1.24.0-debug
name: gcr.io/kaniko-project/executor:v1.12.1-debug
entrypoint: [""]
variables:
PUBLISH_TAG: next
script:
- >
/kaniko/executor
@@ -53,6 +50,9 @@ publish image:
- IMAGE_NAME: app
DEPLOY_DIR: frontend/app
rules:
- if: $CI_COMMIT_BRANCH == 'main'
variables:
PUBLISH_TAG: latest
- !reference [.limit_publish_pipelines, rules]
publish packages:
@@ -61,12 +61,16 @@ publish packages:
variables:
GIT_STRATEGY: clone
GIT_DEPTH: 0
PUBLISH_TAG: next
script:
- pnpm install
- pnpm build
- pnpm config set '//registry.npmjs.org/:_authToken' "${NPM_AUTH_TOKEN}"
- pnpm publish -r --publish-branch ${CI_COMMIT_BRANCH} --tag ${PUBLISH_TAG} --no-git-checks # TODO: Git checks...
rules:
- if: $CI_COMMIT_BRANCH == 'main'
variables:
PUBLISH_TAG: latest
- !reference [.limit_publish_pipelines, rules]
publish docs:
@@ -80,4 +84,5 @@ publish docs:
paths:
- public
rules:
- if: $CI_COMMIT_BRANCH == 'main'
- !reference [.limit_publish_pipelines, rules]

View File

@@ -18,7 +18,7 @@ base image:
docker build
-t "${CI_REGISTRY_IMAGE}/${IMAGE_NAME}:$(grep -o '"version": "[^"]*' "${DEPLOY_DIR}/package.json" | cut -d'"' -f4)"
-t "${CI_REGISTRY_IMAGE}/${IMAGE_NAME}:latest" "${CI_PROJECT_DIR}/${DEPLOY_DIR}" &&
docker push "${CI_REGISTRY_IMAGE}/${IMAGE_NAME}" --all-tags
docker push "${CI_REGISTRY_IMAGE}/${IMAGE_NAME}"
cache: {} # disable irrelevant cache for this job
before_script: [] # do not run irrelevant before script for this job
parallel:
@@ -29,7 +29,5 @@ base image:
DEPLOY_DIR: images/node-builder
- IMAGE_NAME: app-builder
DEPLOY_DIR: images/app-builder
- IMAGE_NAME: app-cypress
DEPLOY_DIR: images/app-cypress
rules:
- !reference [.limit_scheduled_pipelines, rules]

View File

@@ -1 +0,0 @@
pnpm-lock.yaml

View File

@@ -2,13 +2,7 @@
/** @type {import('syncpack').RcFile} */
const config = {
semverGroups: [
{
range: '',
dependencies: ['**'],
packages: ['**'],
}
],
semverRange: '',
source: ['package.json', '**/package.json'],
indent: ' ',
sortFirst: [
@@ -27,7 +21,6 @@ const config = {
'bin',
'files',
'engines',
'packageManager',
'scripts',
'dependencies',
'devDependencies',
@@ -56,7 +49,7 @@ const config = {
{
label: 'Packages should use workspace version',
dependencies: ['@openstapps/**'],
dependencyTypes: ['prod', 'dev', 'peer'],
dependencyTypes: ['prod', 'dev'],
packages: ['**'],
pinVersion: 'workspace:*',
},

View File

@@ -1,3 +0,0 @@
nodejs 22.16.0
pnpm 10.12.3
python 3.11.5

16
.vscode/launch.json vendored
View File

@@ -1,16 +0,0 @@
{
// Use IntelliSense to learn about possible attributes.
// Hover to view descriptions of existing attributes.
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
{
"type":"node",
"request": "launch",
"name": "Run mocha for nodejs workspace",
"runtimeExecutable": "${fileDirname}/../node_modules/.bin/c8",
"cwd": "${fileDirname}/../",
"runtimeArgs": ["--config", "${workspaceFolder}/.c8rc.json", "${fileDirname}/../node_modules/.bin/mocha", "--exit", "--reporter", "spec"]
},
]
}

View File

@@ -37,13 +37,13 @@ Adding new types requires changes at multiple locations for it to work correctly
- Add your SCThing and SCThingWithoutReferences to `src/things/your-thing-name.ts` and make them extend `SCThingWithoutReferences` and `SCThing` respectively
- Add your SCThingMeta to `src/things/your-thing-name.ts` and make it extend `SCThingMeta`
- Add your SCThingMeta to `SCClasses` in `src/meta.ts`
- Add your SCThing to `SCIndexableThings ` in `src/meta.ts`
- Add your SCThing to `SCThingsWithoutDiff` in `src/meta.ts`
- Add your SCThingWithoutReferences to `SCAssociatedThingWithoutReferences` in `src/meta.ts`
- Add your SCThing to `SCAssociatedThing` in `src/meta.ts`
- Add your SCThing to the `SCThingType` enum in `src/things/abstract/thing.ts`
- Add an example file for your SCThing in `test/resources/YourThingName.json`
- Add the following lines for your SCThing in `test/type.spec.ts`:
- Make sure your SCThing (but not SCThingWithoutReferences!) includes the `@indexable` and `@validatable` JSDoc annotations, otherwise neither JSON Schemas nor Elasticsearch mappings will be generated
```typescript
/**
* Types of properties of SCYourThingName

View File

@@ -9,8 +9,8 @@ integration:
- name: registry.gitlab.com/openstapps/openstapps/database:latest
alias: elasticsearch
script:
- pnpm install
- pnpm test:integration:backend --force --no-cache
- pnpm --filter=@openstapps/backend install
- pnpm test:integration:backend
artifacts:
when: on_failure
paths:

View File

@@ -1,244 +1,5 @@
# @openstapps/backend
## 4.0.0
### Major Changes
- c8e29020: Update to Node 22
### Patch Changes
- Updated dependencies [c8e29020]
- @openstapps/core@4.0.0
- @openstapps/core-tools@4.0.0
- @openstapps/logger@4.0.0
## 3.3.2
### Patch Changes
- @openstapps/core@3.3.2
- @openstapps/core-tools@3.3.2
## 3.3.1
### Patch Changes
- 67ab1fd6: fix for geo.point mapping
## 3.3.0
### Minor Changes
- 688bc5f2: v3.3.0 changes
### Patch Changes
- Updated dependencies [688bc5f2]
- @openstapps/core@3.3.0
- @openstapps/core-tools@3.3.0
- @openstapps/logger@3.0.0
## 3.2.0
### Minor Changes
- 912ae422: Add the ability to filter by existence of a field
### Patch Changes
- 689ac68b: pin alpine version to 3.18 and add healthchecks
- e8d72683: Backend unit tests break every year
- Updated dependencies [912ae422]
- @openstapps/core@4.0.0
- @openstapps/core-tools@3.0.0
- @openstapps/logger@3.0.0
## 3.1.2
### Patch Changes
- Fix backend rejecting plugins
- Fix backend sliently falling back to default configs
## 3.1.1
### Patch Changes
- Fix version history offered by backend
- Updated dependencies
- @openstapps/core@3.1.1
## 3.1.0
### Minor Changes
- 06b8ca10: Add job portal feature
### Patch Changes
- Updated dependencies [06b8ca10]
- @openstapps/core@3.1.0
## 3.0.1
### Patch Changes
- Fix missing dependency
## 3.0.0
### Major Changes
- 64caebaf: Migrate config system to cosmiconfig
Configs are now written using type-checked JavaScript
```js
// @ts-check
// This file is now type-safe just like TypeScript, but
// without the compilation step
/** @type {import('@openstapps/package').Type} */
const foo = {};
```
You can write config files in JavaScript, JSON, or other
cosmiconfig-supported formats.
The config files now are
- `backendrc.{js,json,...}`
- `elasticsearchrc.{js,json,...}`
- `prometheusrc.{js,json,...}`
You can also split them into multiple files,
as well as using the `markdown.js` helper in
`config/default/tools` to include markdown files
as text (see `config/f-u/about-pages`)
- 64caebaf: Move project to a turbo monorepo & pnpm
Internal dependencies are now defined using `"@openstapps/package": "workspace:*"`
- Removed extraneous files from packages
- `.npmrc`
- `.npmignore`
- `.mailmap`
- `.gitignore`
- `CONTRIBUTING.md`
- `LICENSE` (Project license file is added upon publishing, see [pnpm.io](https://pnpm.io/cli/publish))
- `package-lock.json`
- `.editorconfig`
- `.eslintrc.json` (moved eslint config to `package.json`)
- `.eslintignore`
- `.gitlab-ci.yml` (Most workflows are workspace-level)
- `.gitlab/**` (issue templates etc. are now workspace-level)
- `.dockerignore` (Docker files are determined by which files are deployed with `pnpm deploy`, as per `package.json/files`)
- TSConfig has been moved to its own package (You can now use `"extends": "@openstapps/tsconfig"`)
- Removed ESLint and Prettier peer dependency hell by injecting them through the `.pnpmfile.cjs`
- Added syncpack for keeping dependency versions in sync (and consistent key ordering in `package.json`)
- Replaced conventional changelog with changesets
- Apps with binaries now use a top level `app.js`
```js
#!/usr/bin/env node
import './lib/app.js';
```
- 64caebaf: Migrate to ESM
CommonJS is no longer supported in any capacity. To use the new
version, you will need to migrate your package to ESM.
We recommend using `tsup` and `Node 18`.
```json
{
"type": "module"
}
```
- 64caebaf: Migrate package to Node 18
- Consumers of this package will need to migrate to Node 18 or
higher.
- Packages have been migrated from promisified `readFile` or
`readFileSync` towards `fs/promises`
- Packages use native `flatMap` now
- 0a7e6af1: Migrate es mapping types from es-mapping-generator to .d.ts next to generated mappings
- 64caebaf: Migrate build system to `tsup`
All packages now use an `index.ts` file to expose contents.
You will need to migrate paths from `import foo from '@scope/package/lib/foo` to `import foo from '@scope/package'`
### Minor Changes
- 64caebaf: Migrate integration tests from docker-compose solution to a shell script
`api-cli` no longer builds as a Docker container as a result.
- 64caebaf: Migrate tests to C8/Chai/Mocha
- `@testdeck` OOP testing has been removed.
- Tests have been unified
- CommonJS module mocking has been replaced through
refactoring of tests, as ES Modules cannot be mocked
(do yourself a favor and don't try to mock them)
- C8 now replaces NYC as a native coverage tool
- 64caebaf: Migrate away from `@krlwlfrt/async-pool`
```ts
import {mapAsyncLimit} from '@openstapps/collection-utils';
await mapAsyncLimit(
[1, 2, 3],
async it => {
await someNetworkRequest(it);
},
5,
);
```
### Patch Changes
- 64caebaf: Migrated changelogs to changeset format
```js
import fs from 'fs';
const path = 'packages/logger/CHANGELOG.md';
fs.writeFileSync(path, fs.readFileSync(path, 'utf8').replace(/^#+\s+\[/gm, '## ['));
```
- 98546a97: Migrate away from @openstapps/configuration
- 23481d0d: Update to TypeScript 5.1.6
- Updated dependencies [98546a97]
- Updated dependencies [64caebaf]
- Updated dependencies [98546a97]
- Updated dependencies [64caebaf]
- Updated dependencies [64caebaf]
- Updated dependencies [64caebaf]
- Updated dependencies [64caebaf]
- Updated dependencies [1f62b5c5]
- Updated dependencies [98546a97]
- Updated dependencies [23481d0d]
- Updated dependencies [0a7e6af1]
- Updated dependencies [64caebaf]
- Updated dependencies [64caebaf]
- Updated dependencies [98546a97]
- Updated dependencies [64caebaf]
- Updated dependencies [64caebaf]
- Updated dependencies [64caebaf]
- Updated dependencies [98546a97]
- @openstapps/core-tools@3.0.0
- @openstapps/logger@3.0.0
- @openstapps/core@3.0.0
## 3.0.0-next.4
### Patch Changes

View File

@@ -9,6 +9,4 @@ ENV NODE_ENV=production
WORKDIR /app
EXPOSE 3000
HEALTHCHECK --interval=10s --timeout=10s --start-period=10s --retries=12 CMD curl -s --fail --request POST --data '{}' --header 'Content-Type: application/json' http://localhost:3000/ >/dev/null || exit 1
ENTRYPOINT ["node", "app.js"]

View File

@@ -1,3 +1,5 @@
// @ts-check
/**
* This is the database configuration for the technical university of berlin
*

View File

@@ -22,7 +22,6 @@ const app = {
name: 'Goethe-Uni',
privacyPolicyUrl: 'https://mobile.server.uni-frankfurt.de/_static/privacy.md',
settings: [userGroupSetting, languageSetting],
versionHistory: [],
};
export default app;

View File

@@ -1,3 +1,4 @@
// @ts-check
import {SCSettingInputType, SCThingOriginType, SCThingType} from '@openstapps/core';
/** @type {import('@openstapps/core').SCLanguageSetting} */

View File

@@ -1,3 +1,4 @@
// @ts-check
/** @type {import('@openstapps/core').SCAppConfigurationMenuCategory[]} */
const menus = [
{
@@ -55,19 +56,6 @@ const menus = [
},
},
},
{
icon: 'work',
route: '/jobs',
title: 'job postings',
translations: {
de: {
title: 'Jobangebote',
},
en: {
title: 'job postings',
},
},
},
],
title: 'overview',
route: '/overview',

View File

@@ -1,3 +1,4 @@
// @ts-check
import {SCSettingInputType, SCThingOriginType, SCThingType} from '@openstapps/core';
/** @type {import('@openstapps/core').SCUserGroupSetting} */

View File

@@ -1,3 +1,4 @@
// @ts-check
import {SCThingType} from '@openstapps/core';
/** @type {import('@openstapps/core').SCBackendAggregationConfiguration[]} */

View File

@@ -1,3 +1,4 @@
// @ts-check
import {
month,
sommerRange,

View File

@@ -1,3 +1,4 @@
// @ts-check
import {SCThingType} from '@openstapps/core';
import aggregations from './aggregations.js';
import boostings from './boostings.js';
@@ -16,7 +17,7 @@ export const backend = {
hiddenTypes: [SCThingType.DateSeries, SCThingType.Diff, SCThingType.Floor],
mappingIgnoredTags: ['minlength', 'pattern', 'see', 'tjs-format'],
maxMultiSearchRouteQueries: 5,
maxRequestBodySize: 2e6,
maxRequestBodySize: 512 * 1024,
name: 'Goethe-Universität Frankfurt am Main',
namespace: '909a8cbc-8520-456c-b474-ef1525f14209',
sortableFields: [

View File

@@ -1,3 +1,4 @@
// @ts-check
import app from './app/index.js';
import {backend, internal} from './backend/index.js';

View File

@@ -1,3 +1,5 @@
// @ts-check
/**
* This is the default configuration for elasticsearch (a database)
*
@@ -9,13 +11,13 @@
*
* To get more information about the meaning of specific fields, please use your IDE to read the TSDoc documentation.
*
* @type {import('../../src/storage/elasticsearch/types/elasticsearch-config.js').ElasticsearchConfigFile}
* @type {import('../../src/storage/elasticsearch/types/elasticsearch-config.js')}
*/
const config = {
internal: {
database: {
name: 'elasticsearch',
version: '8.4.2',
version: '5.6',
query: {
minMatch: '75%',
queryType: 'dis_max',

View File

@@ -1,3 +1,4 @@
// @ts-check
import {readFile} from 'fs/promises';
import {SCAboutPageContentType} from '@openstapps/core';

View File

@@ -1,3 +1,4 @@
// @ts-check
/**
* Generates a range of numbers that represent consecutive calendar months
*

View File

@@ -1,37 +0,0 @@
import {readFile, readdir} from 'fs/promises';
import url from 'url';
import path from 'path';
/**
* @example version(1, import.meta.url)
* @param options {Omit<import('@openstapps/core').SCAppVersionInfo, 'releaseNotes' | 'translations'>}
* @param base {string}
* @returns {Promise<import('@openstapps/core').SCAppVersionInfo>}
*/
export async function version(options, base) {
const de = await readFile(new URL(`${options.version}.de.md`, base), 'utf8');
const en = await readFile(new URL(`${options.version}.en.md`, base), 'utf8');
return {
...options,
releaseNotes: de,
translations: {
en: {
releaseNotes: en,
},
},
};
}
/**
* @param base {string} Base path of the file as `import.meta.url`
* @returns {Promise<import('@openstapps/core').SCAppVersionInfo[]>}
*/
export async function versions(base) {
const directory = await readdir(path.dirname(url.fileURLToPath(base)));
const versions = [
...new Set(directory.filter(it => it.endsWith('.md')).map(it => it.replace(/\.\w+\.md$/, ''))),
].sort((a, b) => -a.localeCompare(b, undefined, {numeric: true}));
return Promise.all(versions.map(versionName => version({version: versionName}, base)));
}

View File

@@ -1,3 +1,4 @@
// @ts-check
import {SCAboutPageContentType} from '@openstapps/core';
import {markdown} from '../../default/tools/markdown.js';

View File

@@ -1,3 +1,4 @@
// @ts-check
import {SCAboutPageContentType} from '@openstapps/core';
/** @type {import('@openstapps/core').SCAboutPage} */

View File

@@ -1,3 +1,4 @@
// @ts-check
import about from './about.js';
import imprint from './imprint.js';
import privacy from './privacy.js';

View File

@@ -1,3 +1,4 @@
// @ts-check
import {markdown} from '../../default/tools/markdown.js';
/** @type {import('@openstapps/core').SCAboutPage} */

View File

@@ -1,7 +1,7 @@
// @ts-check
import aboutPages from './about-pages/index.js';
import defaultApp from '../default/app/index.js';
import {backend as defaultBackend, internal as defaultInternal} from '../default/backend/index.js';
import versionHistory from './version-history/index.js';
/**
* This is the default configuration for the Goethe university of Frankfurt
@@ -70,13 +70,8 @@ const config = {
authProvider: 'paia',
url: 'https://hds.hebis.de/paia/core',
},
/** TODO: idCards: {
authProvider: 'default',
url: 'TODO',
} */
},
},
versionHistory,
aboutPages,
},
backend: defaultBackend,

View File

@@ -1,52 +0,0 @@
# Goethe-Uni App 2.4
Wir freuen uns euch mehr in der Goethe-Uni App
bieten zu können.
## Navigation zu Gebäuden und Orten
Als eines der Ergebnisse des Ideenwettbewerbs wurde jetzt
ein Navigationsfeature in die App integriert.
Orte auf der Karte, Mensen, sowie sogar Termine (wenn hinterlegt)
bieten jetzt direkt die Option eine Verbindung zu finden, gestützt
durch die Karten App auf deinem Gerät.
## Integration der Jobbörse
Jobs findest du ab sofort auch in der Goethe-Uni App.
Auch das ist ein Ergebnis des Ideenwettbewerbs,
und wir freuen uns es euch hier präsentieren zu können!
## Der Umweltscore
Der Umweltscore für Gerichte wird nun auch in der App angezeigt.
> Nachhaltigkeit, Umweltschutz, Gesundheit und Klimawandel sind
> zentrale Begriffe im gesellschaftlichen Miteinander.
> Unsere Ernährung spielt hierbei eine wichtige Rolle.
> Das Studierendenwerk Frankfurt am Main zeichnet seine Speisenpläne
> ab sofort mit einem Umweltscore aus.
> Anhand dieser Bewertung können Sie direkt ersehen,
> welchen Einfluss Ihre Essenauswahl auf das Klima hat.
## Weitere Verbesserungen
### Performance
Die Performance der App beim Navigieren wurde stark verbessert und ist datensparender.
### Kalender
Die Kalenderabschnitte haben jetzt neue Namen bekommen:
- Der _Kalender_ zeigt Termine für spezifische Tage
- Die _Wochenübersicht_ ist ein Stundenplan mit allen Termine, die sich wiederholen (z. B. Vorlesungen)
- Die _Einzeltermine_ zeigen alle Termine, die sich nicht wiederholen
(z. B. Klausuren)
### Meine App
Der "Meine Kurse" Abschnitt wurde überarbeitet, und zeigt jetzt Termine
für die nächsten Tage und mit mehr Details an.

View File

@@ -1,49 +0,0 @@
# Goethe-Uni App 2.4
The Goethe-Uni App got even better!
## Navigation to buildings and places
As part of the "Ideenwettbewerb," the idea competition,
we have now integrated a navigation feature into the app.
Orte auf der Karte, Mensen, sowie sogar Termine (wenn hinterlegt)
bieten jetzt direkt die Option eine Verbindung zu finden, gestützt
durch die Karten App auf deinem Gerät.
## Integration of the job market
Jobs are now also available in the Goethe-Uni App.
This feature is also a result of the idea competition,
and we're happy to be able to present it to you here!
## The environment score
The environment score for dishes is now displayed inside the app.
> Sustainability, environment protection, health, and climate change are
> central topics in how we live today in our society.
> Our eating habits play an important role in it.
> The "Studierendenwerk Frankfurt am Main" is marking up its menus
> from now on with the so-called "Umweltscore," the environment score.
> Based on this rating, you can see the impact your meal choice would have on our climate.
## Further improvements
### Performance
The performance while navigating around the app has been heavily improved and requires less data to work.
### Calendar
The calendar sections have new names:
- The _calendar_ shows appointments on specific days
- The _week overview_ is a schedule with all events that repeat (e.g. lectures)
- The _single events_ show all appointments that don't repeat (e.g. exams)
### My App
The "my courses" section has been revamped,
and now shows events for the next days and with more detail.

View File

@@ -1,15 +0,0 @@
# Goethe-Uni App 2.5
Die Goethe-Uni App ist noch besser geworden!
## Komplett neue Kartenansicht
Wir haben die Karte überarbeitet, um eine klarere und schnellere Übersicht zu bieten.
## Deutschlandticket mit an Bord
Wenn du das Upgrade des Semesterticket zum Deutschlandticket gemacht hast und eingeloggt bist, findet es sich jetzt auch in der App.
## Bibliotheksdienste sind wieder voll funktionsfähig
Aufgrund einiger Adhoc-Änderungen im Bibliothekssystem haben wir die App so angepasst, dass sie damit umgehen kann.

View File

@@ -1,15 +0,0 @@
# Goethe-Uni App 2.5
The Goethe-Uni App got even better!
## Completely new map view
We overhauled the map to offer you a clearer and faster and overview.
## Deutschlandticket included
If you upgraded your Semesterticket to a Deutschlandticket it will now reside in the App if you are logged in.
## Library services are fully functional again
Due to some adhoc changes in the library system we adjusted the app to handle them properly.

View File

@@ -1,6 +0,0 @@
import {versions} from '../../default/tools/version.js';
/** @type {import('@openstapps/core').SCAppVersionInfo[]} */
const versionHistory = await versions(import.meta.url);
export default versionHistory;

View File

@@ -1,7 +1,7 @@
{
"name": "@openstapps/backend",
"description": "A reference implementation for a StApps backend",
"version": "4.0.0",
"version": "3.0.0-next.4",
"private": true,
"type": "module",
"license": "AGPL-3.0-only",
@@ -51,7 +51,7 @@
"@types/cors": "2.8.13",
"@types/express": "4.17.17",
"@types/geojson": "1.0.6",
"@types/node": "22.15.31",
"@types/node": "18.15.3",
"@types/node-cron": "3.0.7",
"@types/nodemailer": "6.4.7",
"@types/promise-queue": "2.2.0",
@@ -59,12 +59,11 @@
"body-parser": "1.20.2",
"cors": "2.8.5",
"cosmiconfig": "8.1.3",
"deepmerge": "4.3.1",
"express": "4.18.2",
"express-prom-bundle": "6.6.0",
"express-promise-router": "4.1.1",
"got": "12.6.0",
"moment": "2.30.1",
"moment": "2.29.4",
"morgan": "1.10.0",
"nock": "13.3.1",
"node-cache": "5.1.2",
@@ -80,27 +79,27 @@
"@openstapps/prettier-config": "workspace:*",
"@openstapps/tsconfig": "workspace:*",
"@testdeck/mocha": "0.3.3",
"@types/chai": "4.3.20",
"@types/chai-as-promised": "7.1.8",
"@types/mocha": "10.0.10",
"@types/chai": "4.3.5",
"@types/chai-as-promised": "7.1.5",
"@types/mocha": "10.0.1",
"@types/morgan": "1.9.4",
"@types/sinon": "10.0.14",
"@types/sinon-express-mock": "1.3.9",
"@types/supertest": "2.0.12",
"c8": "10.1.3",
"chai": "4.5.0",
"chai-as-promised": "7.1.2",
"c8": "7.14.0",
"chai": "4.3.7",
"chai-as-promised": "7.1.1",
"cross-env": "7.0.3",
"get-port": "5.1.1",
"mocha": "10.8.2",
"mocha": "10.2.0",
"mocha-junit-reporter": "2.2.0",
"mocked-env": "1.3.5",
"sinon": "15.0.4",
"sinon-express-mock": "2.2.1",
"supertest": "6.3.3",
"ts-node": "10.9.2",
"tsup": "8.5.0",
"typescript": "5.4.2"
"ts-node": "10.9.1",
"tsup": "6.7.0",
"typescript": "5.1.6"
},
"tsup": {
"entry": [

View File

@@ -2,7 +2,6 @@ import {cosmiconfig, PublicExplorer} from 'cosmiconfig';
import {SCConfigFile} from '@openstapps/core';
import path from 'path';
import deepmerge from 'deepmerge';
import {Logger} from '@openstapps/logger';
const fallbackNamespace = 'default';
const configPath = 'config';
@@ -24,25 +23,31 @@ function configLoader(moduleName: string): PublicExplorer {
* Find and load a config file
*/
async function findConfig<T>(moduleName: string, namespace = fallbackNamespace): Promise<T> {
const config = await configLoader(moduleName).search(path.posix.join('.', configPath, namespace));
if (config) {
Logger.info(`Using ${namespace} config for ${moduleName}`);
return config.config;
} else {
Logger.info(`Using ${fallbackNamespace} config for ${moduleName}`);
return configLoader(moduleName)
.search(path.posix.join('.', configPath, fallbackNamespace))
.then(it => it!.config);
}
return configLoader(moduleName)
.search(path.posix.join('.', configPath, namespace))
.then(it => it!.config as T)
.catch(() =>
configLoader(moduleName)
.search(path.posix.join('.', configPath, fallbackNamespace))
.then(it => it!.config),
);
}
const namespace = process.env.NODE_APP_INSTANCE;
const database = process.env.NODE_CONFIG_ENV;
/**
* Loads a config file
*/
async function loadConfig<T>(moduleName: string): Promise<T> {
const namespace = process.env.NODE_APP_INSTANCE;
const database = process.env.NODE_CONFIG_ENV;
export const prometheusConfig = await findConfig<unknown>('prometheus', namespace);
const config = await findConfig<T>(moduleName, namespace);
if (database) {
const databaseConfig = await findConfig<T>(database, namespace);
return deepmerge(config, databaseConfig);
}
const backendConfigWithoutDatabase = await findConfig<SCConfigFile>('backend', namespace);
export const backendConfig = database
? deepmerge(backendConfigWithoutDatabase, await findConfig<never>(database, namespace))
: backendConfigWithoutDatabase;
return config;
}
export const backendConfig = await loadConfig<SCConfigFile>('backend');
export const prometheusConfig = await loadConfig<unknown>('prometheus');

View File

@@ -13,34 +13,15 @@
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import {
SCConfigFile,
SCPlace,
SCPlaceWithoutReferences,
SCSearchQuery,
SCSearchResponse,
SCThingWithCategoriesWithoutReferences,
SCThings,
SCUuid,
} from '@openstapps/core';
import {SCConfigFile, SCSearchQuery, SCSearchResponse, SCThings, SCUuid} from '@openstapps/core';
import {MailQueue} from '../notification/mail-queue.js';
import {Bulk} from './bulk-storage.js';
import {FeatureCollection, Point, Polygon} from 'geojson';
/**
* Creates an instance of a database
*/
export type DatabaseConstructor = new (config: SCConfigFile, mailQueue?: MailQueue) => Database;
export type SupplementaryGeoJSON = FeatureCollection<Point | Polygon, SupplementaryGeoJSONThing>;
export type SupplementaryGeoJSONThing = Pick<
Extract<SCThings, SCPlace>,
Exclude<
keyof SCPlaceWithoutReferences | keyof SCThingWithCategoriesWithoutReferences<never, never>,
'geo' | 'origin' | 'translations'
>
>;
/**
* Defines what one database class needs to have defined
*/
@@ -101,9 +82,4 @@ export interface Database {
* @param params Parameters which form a search query to search the backend data
*/
search(parameters: SCSearchQuery): Promise<SCSearchResponse>;
/**
* Get geo info for display on a map
*/
geo(): Promise<SupplementaryGeoJSON>;
}

View File

@@ -26,7 +26,7 @@ import {Logger} from '@openstapps/logger';
import moment from 'moment';
import {MailQueue} from '../../notification/mail-queue.js';
import {Bulk} from '../bulk-storage.js';
import {Database, SupplementaryGeoJSON, SupplementaryGeoJSONThing} from '../database.js';
import {Database} from '../database.js';
import {parseAggregations} from './aggregations.js';
import * as Monitoring from './monitoring.js';
import {buildQuery} from './query/query.js';
@@ -46,7 +46,6 @@ import {
} from './util/index.js';
import {noUndefined} from './util/no-undefined.js';
import {retryCatch, RetryOptions} from './util/retry.js';
import {Feature, Point, Polygon} from 'geojson';
/**
* A database interface for elasticsearch
@@ -85,10 +84,7 @@ export class Elasticsearch implements Database {
* @param config an assembled config file
* @param mailQueue a mail queue for monitoring
*/
constructor(
private readonly config: SCConfigFile,
mailQueue?: MailQueue,
) {
constructor(private readonly config: SCConfigFile, mailQueue?: MailQueue) {
if (config.internal.database === undefined || typeof config.internal.database.version !== 'string') {
throw new TypeError('Database version is undefined. Check your config file');
}
@@ -406,49 +402,4 @@ export class Elasticsearch implements Database {
},
};
}
async geo(): Promise<SupplementaryGeoJSON> {
const searchResponse = await this.client.search<Extract<SCThings, {geo: unknown}>>({
body: {
query: {
exists: {
field: 'geo',
},
},
},
from: 0,
allow_no_indices: true,
index: ACTIVE_INDICES_ALIAS,
size: 1,
});
return {
type: 'FeatureCollection',
features: searchResponse.hits.hits
.map(thing => {
return thing._source?.geo
? ({
id: Number(thing._source.identifiers?.['OSM']) || undefined,
type: 'Feature',
geometry: thing._source.geo.polygon ?? thing._source.geo.point,
properties: {
name: thing._source.name,
sameAs: thing._source.sameAs,
image: thing._source.image,
alternateNames: thing._source.alternateNames,
description: thing._source.description,
identifiers: thing._source.identifiers,
categories: thing._source.categories,
categorySpecificValues: thing._source.categorySpecificValues,
openingHours: thing._source.openingHours,
address: thing._source.address,
uid: thing._source.uid,
type: thing._source.type,
},
} satisfies Feature<Polygon | Point, SupplementaryGeoJSONThing>)
: undefined;
})
.filter(noUndefined),
};
}
}

View File

@@ -25,7 +25,7 @@ export function buildDistanceFilter(
): QueryDslSpecificQueryContainer<'geo_distance'> {
const geoObject: QueryDslGeoDistanceQuery = {
distance: `${filter.arguments.distance}m`,
[`${filter.arguments.field}.point`]: {
[`${filter.arguments.field}.point.coordinates`]: {
lat: filter.arguments.position[1],
lon: filter.arguments.position[0],
},

View File

@@ -19,29 +19,14 @@ import {QueryDslSpecificQueryContainer} from '../../types/util.js';
* Converts a geo filter to elasticsearch syntax
* @param filter A search filter for the retrieval of the data
*/
export function buildGeoFilter(filter: SCGeoFilter): QueryDslSpecificQueryContainer<'bool'> {
export function buildGeoFilter(filter: SCGeoFilter): QueryDslSpecificQueryContainer<'geo_shape'> {
return {
bool: {
should: [
{
geo_shape: {
ignore_unmapped: true,
[`${filter.arguments.field}.polygon`]: {
shape: filter.arguments.shape,
relation: filter.arguments.spatialRelation,
},
},
} satisfies QueryDslSpecificQueryContainer<'geo_shape'>,
{
geo_shape: {
ignore_unmapped: true,
[`${filter.arguments.field}.point`]: {
shape: filter.arguments.shape,
relation: filter.arguments.spatialRelation,
},
},
} satisfies QueryDslSpecificQueryContainer<'geo_shape'>,
],
geo_shape: {
ignore_unmapped: true,
[`${filter.arguments.field}.polygon`]: {
shape: filter.arguments.shape,
relation: filter.arguments.spatialRelation,
},
},
};
}

View File

@@ -21,31 +21,16 @@ import {QueryDslSpecificQueryContainer} from '../../types/util.js';
*/
export function buildValueFilter(
filter: SCSearchValueFilter,
):
| QueryDslSpecificQueryContainer<'exists'>
| QueryDslSpecificQueryContainer<'term'>
| QueryDslSpecificQueryContainer<'terms'> {
switch (typeof filter.arguments.value) {
case 'undefined': {
return {
exists: {
field: filter.arguments.field,
): QueryDslSpecificQueryContainer<'term'> | QueryDslSpecificQueryContainer<'terms'> {
return Array.isArray(filter.arguments.value)
? {
terms: {
[`${filter.arguments.field}.raw`]: filter.arguments.value,
},
};
}
case 'string': {
return {
}
: {
term: {
[`${filter.arguments.field}.raw`]: filter.arguments.value,
},
};
}
case 'object': {
return {
terms: {
[`${filter.arguments.field}.raw`]: filter.arguments.value,
},
};
}
}
}

View File

@@ -25,7 +25,7 @@ export function buildDistanceSort(sort: SCDistanceSort): SortOptions {
mode: 'avg',
order: sort.order,
unit: 'm',
[`${sort.arguments.field}.point`]: {
[`${sort.arguments.field}.point.coordinates`]: {
lat: sort.arguments.position[1],
lon: sort.arguments.position[0],
},

View File

@@ -22,7 +22,7 @@ import http from 'http';
import {MailQueue} from '../src/notification/mail-queue.js';
import {Bulk, BulkStorage} from '../src/storage/bulk-storage.js';
import getPort from 'get-port';
import {Database, SupplementaryGeoJSON} from '../src/storage/database.js';
import {Database} from '../src/storage/database.js';
import {v4} from 'uuid';
import {backendConfig} from '../src/config.js';
import {getIndexUID} from '../src/storage/elasticsearch/util/index.js';
@@ -58,6 +58,7 @@ export async function startApp(): Promise<Express> {
* An elasticsearch mock
*/
export class ElasticsearchMock implements Database {
// @ts-expect-error never read
private bulk: Bulk | undefined;
private storageMock = new Map<string, SCThings>();
@@ -66,10 +67,6 @@ export class ElasticsearchMock implements Database {
// Nothing to do here
}
geo(): Promise<SupplementaryGeoJSON> {
throw new Error('Method not implemented.');
}
bulkCreated(bulk: Bulk): Promise<void> {
this.bulk = bulk;
return Promise.resolve(undefined);

View File

@@ -14,7 +14,6 @@
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import {
SCBook,
SCBulkAddRoute,
SCBulkDoneRoute,
SCBulkRequest,
@@ -24,30 +23,29 @@ import {
import {expect} from 'chai';
import {bulk, DEFAULT_TEST_TIMEOUT} from '../common.js';
import {testApp} from '../tests-setup.js';
import {readFile} from 'fs/promises';
import {v4} from 'uuid';
import bookFile from '@openstapps/core/test/resources/indexable/Book.2.json' with {type: 'json'};
const book = bookFile.instance as SCBook;
const book = JSON.parse(
await readFile('node_modules/@openstapps/core/test/resources/indexable/Book.2.json', 'utf8'),
).instance;
describe('Bulk routes', async function () {
// increase timeout for the suite
this.timeout(DEFAULT_TEST_TIMEOUT);
let request: SCBulkRequest;
let bulkRoute: SCBulkRoute;
let bulkAddRoute: SCBulkAddRoute;
let bulkDoneRoute: SCBulkDoneRoute;
const request: SCBulkRequest = {
expiration: bulk.expiration,
source: bulk.source,
type: bulk.type,
};
const bulkRoute = new SCBulkRoute();
const bulkAddRoute = new SCBulkAddRoute();
const bulkDoneRoute = new SCBulkDoneRoute();
before(function () {
request = {
expiration: bulk.expiration,
source: bulk.source,
type: bulk.type,
};
bulkRoute = new SCBulkRoute();
bulkAddRoute = new SCBulkAddRoute();
bulkDoneRoute = new SCBulkDoneRoute();
});
// afterEach(async function() {
// TODO: Delete saved bulks
// });
it('should create bulk', async function () {
const {status, body, error} = await testApp

View File

@@ -21,12 +21,7 @@ import {expect} from 'chai';
describe('Index route', async function () {
// increase timeout for the suite
this.timeout(DEFAULT_TEST_TIMEOUT);
let indexRoute: SCIndexRoute;
before(function () {
indexRoute = new SCIndexRoute();
});
const indexRoute = new SCIndexRoute();
it('should respond with both app and backend configuration', async function () {
const request: SCIndexRequest = {};

View File

@@ -30,11 +30,15 @@ import chaiAsPromised from 'chai-as-promised';
import {DEFAULT_TEST_TIMEOUT} from '../common.js';
import {testApp} from '../tests-setup.js';
import {backendConfig} from '../../src/config.js';
import registerRequest from '@openstapps/core/test/resources/PluginRegisterRequest.1.json' with {type: 'json'};
import {readFile} from 'fs/promises';
// for using promises in expectations (to.eventually.be...)
use(chaiAsPromised);
const registerRequest = JSON.parse(
await readFile('node_modules/@openstapps/core/test/resources/PluginRegisterRequest.1.json', 'utf8'),
);
// cast it because of "TS2322: Type 'string' is not assignable to type '"add"'"
export const registerAddRequest: SCPluginAdd = registerRequest.instance as SCPluginAdd;

View File

@@ -47,8 +47,8 @@ describe('Create route', async function () {
const statusCodeSuccess = 222;
const bodySuccess = {foo: true};
const sandbox = sinon.createSandbox();
let validationError: SCValidationErrorResponse;
let internalServerError: SCInternalServerErrorResponse;
const validationError = new SCValidationErrorResponse([]);
const internalServerError = new SCInternalServerErrorResponse();
beforeEach(function () {
app = express();
@@ -64,9 +64,6 @@ describe('Create route', async function () {
handler = (_request, _app) => {
return Promise.resolve(bodySuccess);
};
validationError = new SCValidationErrorResponse([]);
internalServerError = new SCInternalServerErrorResponse();
});
afterEach(function () {

View File

@@ -29,21 +29,14 @@ import {backendConfig} from '../../src/config.js';
describe('Search route', async function () {
// increase timeout for the suite
this.timeout(DEFAULT_TEST_TIMEOUT);
let searchRoute: SCSearchRoute;
let multiSearchRoute: SCMultiSearchRoute;
let syntaxError: SCSyntaxErrorResponse;
let methodNotAllowedError: SCMethodNotAllowedErrorResponse;
let tooManyRequestsError: SCTooManyRequestsErrorResponse;
before(function () {
searchRoute = new SCSearchRoute();
multiSearchRoute = new SCMultiSearchRoute();
syntaxError = new SCSyntaxErrorResponse('Foo Message');
methodNotAllowedError = new SCMethodNotAllowedErrorResponse();
tooManyRequestsError = new SCTooManyRequestsErrorResponse();
});
const searchRoute = new SCSearchRoute();
const multiSearchRoute = new SCMultiSearchRoute();
const syntaxError = new SCSyntaxErrorResponse('Foo Message');
const methodNotAllowedError = new SCMethodNotAllowedErrorResponse();
const tooManyRequestsError = new SCTooManyRequestsErrorResponse();
it('should reject GET, PUT with a valid search query', async function () {
// const expectedParams = JSON.parse(JSON.stringify(defaultParams));
const {status} = await testApp.get('/search').set('Accept', 'application/json').send({
query: 'Some search terms',
});

View File

@@ -13,25 +13,23 @@
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import {SCBook, SCThingUpdateRoute} from '@openstapps/core';
import {SCThingUpdateRoute} from '@openstapps/core';
import chaiAsPromised from 'chai-as-promised';
import {bulkStorageMock, DEFAULT_TEST_TIMEOUT} from '../common.js';
import {expect, use} from 'chai';
import {testApp} from '../tests-setup.js';
import bookFile from '@openstapps/core/test/resources/indexable/Book.1.json' with {type: 'json'};
import {readFile} from 'fs/promises';
use(chaiAsPromised);
const book = bookFile.instance as SCBook;
const book = JSON.parse(
await readFile('node_modules/@openstapps/core/test/resources/indexable/Book.1.json', 'utf8'),
).instance;
describe('Thing update route', async function () {
// increase timeout for the suite
this.timeout(DEFAULT_TEST_TIMEOUT);
let thingUpdateRoute: SCThingUpdateRoute;
before(function () {
thingUpdateRoute = new SCThingUpdateRoute();
});
const thingUpdateRoute = new SCThingUpdateRoute();
it('should update a thing', async function () {
const thingUpdateRouteurlPath = thingUpdateRoute.urlPath

View File

@@ -39,6 +39,7 @@ import {Elasticsearch} from '../../../src/storage/elasticsearch/elasticsearch.js
import {bulk, DEFAULT_TEST_TIMEOUT, getTransport, getIndex} from '../../common.js';
import fs from 'fs';
import {backendConfig} from '../../../src/config.js';
import {readFile} from 'fs/promises';
import {
ACTIVE_INDICES_ALIAS,
getIndexUID,
@@ -49,11 +50,6 @@ import {
} from '../../../src/storage/elasticsearch/util/index.js';
import cron from 'node-cron';
import {query} from './query.js';
import messageFile from '@openstapps/core/test/resources/indexable/Message.1.json' with {type: 'json'};
import bookFile from '@openstapps/core/test/resources/indexable/Book.1.json' with {type: 'json'};
const message = messageFile.instance as SCMessage;
const book = bookFile.instance as SCBook;
use(chaiAsPromised);
@@ -64,6 +60,13 @@ function searchResponse<T>(...hits: SearchHit<T>[]): SearchResponse<T> {
return {hits: {hits}, took: 0, timed_out: false, _shards: {total: 1, failed: 0, successful: 1}};
}
const message = JSON.parse(
await readFile('node_modules/@openstapps/core/test/resources/indexable/Message.1.json', 'utf8'),
);
const book = JSON.parse(
await readFile('node_modules/@openstapps/core/test/resources/indexable/Book.1.json', 'utf8'),
);
describe('Elasticsearch', function () {
// increase timeout for the suite
this.timeout(DEFAULT_TEST_TIMEOUT);
@@ -71,15 +74,8 @@ describe('Elasticsearch', function () {
before(function () {
// eslint-disable-next-line no-console
console.log('before');
sandbox.stub(fs, 'readFileSync').returns('{}');
sandbox.stub(backendConfig.internal.boostings.default[0], 'fields').value({
'academicTerms.acronym': {
'SS 2023': 1.05,
'WS 2023/24': 1.1,
'SoSe 2023': 1.05,
'WiSe 2023/24': 1.1,
},
});
});
after(function () {
sandbox.restore();
@@ -449,7 +445,7 @@ describe('Elasticsearch', function () {
_id: '',
_index: '',
_score: 0,
_source: message,
_source: message as SCMessage,
};
sandbox.stub(es.client, 'search').resolves(searchResponse(foundObject));
@@ -479,7 +475,7 @@ describe('Elasticsearch', function () {
const object: SearchHit<SCMessage> = {
_id: '',
_index: oldIndex,
_source: message,
_source: message as SCMessage,
};
sandbox.stub(es.client, 'search').resolves(searchResponse<SCMessage>(object));
sandbox.stub(es, 'prepareBulkWrite').resolves(index);
@@ -493,7 +489,7 @@ describe('Elasticsearch', function () {
sandbox.stub(es.client, 'create').resolves({result: 'not_found'} as CreateResponse);
await es.init();
return expect(es.post(message, bulk)).to.rejectedWith('creation');
return expect(es.post(message as SCMessage, bulk)).to.rejectedWith('creation');
});
it('should create a new object', async function () {
@@ -506,7 +502,7 @@ describe('Elasticsearch', function () {
});
await es.init();
await es.post(message, bulk);
await es.post(message as SCMessage, bulk);
expect(createStub.called).to.be.true;
expect(caughtParameter.document).to.be.eql({
@@ -531,7 +527,7 @@ describe('Elasticsearch', function () {
_id: '',
_index: getIndex(),
_score: 0,
_source: message,
_source: message as SCMessage,
};
sandbox.stub(es.client, 'search').resolves(searchResponse());
@@ -545,7 +541,7 @@ describe('Elasticsearch', function () {
_id: '',
_index: getIndex(),
_score: 0,
_source: message,
_source: message as SCMessage,
};
sandbox.stub(es.client, 'search').resolves(searchResponse(object));
// @ts-expect-error unused
@@ -568,13 +564,13 @@ describe('Elasticsearch', function () {
_id: '123',
_index: getIndex(),
_score: 0,
_source: message,
_source: message as SCMessage,
};
const objectBook: SearchHit<SCBook> = {
_id: '321',
_index: getIndex(),
_score: 0,
_source: book,
_source: book as SCBook,
};
const fakeEsAggregations = {
'@all': {

View File

@@ -466,7 +466,7 @@ describe('Query', function () {
const expectedFilter: QueryDslSpecificQueryContainer<'geo_distance'> = {
geo_distance: {
'distance': '1000m',
'geo.point': {
'geo.point.coordinates': {
lat: 8.123,
lon: 50.123,
},
@@ -479,39 +479,18 @@ describe('Query', function () {
it('should build geo filter for shapes and points', function () {
const filter = buildFilter(searchFilters.geoPoint);
const expectedFilter = {
bool: {
should: [
{
geo_shape: {
'geo.polygon': {
relation: undefined,
shape: {
coordinates: [
[50.123, 8.123],
[50.123, 8.123],
],
type: 'envelope',
},
},
'ignore_unmapped': true,
},
geo_shape: {
'geo.polygon': {
relation: undefined,
shape: {
type: 'envelope',
coordinates: [
[50.123, 8.123],
[50.123, 8.123],
],
},
{
geo_shape: {
'geo.point': {
relation: undefined,
shape: {
coordinates: [
[50.123, 8.123],
[50.123, 8.123],
],
type: 'envelope',
},
},
'ignore_unmapped': true,
},
},
],
},
'ignore_unmapped': true,
},
};
@@ -521,39 +500,18 @@ describe('Query', function () {
it('should build geo filter for shapes only', function () {
const filter = buildFilter(searchFilters.geoShape);
const expectedFilter = {
bool: {
should: [
{
geo_shape: {
'geo.polygon': {
relation: 'contains',
shape: {
coordinates: [
[50.123, 8.123],
[50.123, 8.123],
],
type: 'envelope',
},
},
'ignore_unmapped': true,
},
geo_shape: {
'geo.polygon': {
relation: 'contains',
shape: {
type: 'envelope',
coordinates: [
[50.123, 8.123],
[50.123, 8.123],
],
},
{
geo_shape: {
'geo.point': {
relation: 'contains',
shape: {
coordinates: [
[50.123, 8.123],
[50.123, 8.123],
],
type: 'envelope',
},
},
'ignore_unmapped': true,
},
},
],
},
'ignore_unmapped': true,
},
};
@@ -636,7 +594,7 @@ describe('Query', function () {
'mode': 'avg',
'order': 'desc',
'unit': 'm',
'geo.point': {
'geo.point.coordinates': {
lat: 50.123,
lon: 8.123,
},

View File

@@ -29,7 +29,7 @@ export const query: QueryDslQueryContainer = {
should: [],
},
},
weight: 1.05,
weight: 1.1,
},
{
filter: {
@@ -42,14 +42,14 @@ export const query: QueryDslQueryContainer = {
},
{
term: {
'academicTerms.acronym.raw': 'WS 2023/24',
'academicTerms.acronym.raw': 'WS 2022/23',
},
},
],
should: [],
},
},
weight: 1.1,
weight: 1.05,
},
{
filter: {
@@ -69,7 +69,7 @@ export const query: QueryDslQueryContainer = {
should: [],
},
},
weight: 1.05,
weight: 1.1,
},
{
filter: {
@@ -82,14 +82,14 @@ export const query: QueryDslQueryContainer = {
},
{
term: {
'academicTerms.acronym.raw': 'WiSe 2023/24',
'academicTerms.acronym.raw': 'WiSe 2022/23',
},
},
],
should: [],
},
},
weight: 1.1,
weight: 1.05,
},
{
filter: {

View File

@@ -2,7 +2,7 @@
"extends": "@openstapps/tsconfig",
"compilerOptions": {
"resolveJsonModule": true,
"useUnknownInCatchVariables": false
},
"exclude": ["lib", "app.js"]
"useUnknownInCatchVariables": false,
"allowJs": true
}
}

View File

@@ -1,23 +1,5 @@
# @openstapps/database
## 4.0.0
### Major Changes
- c8e29020: Update to Node 22
## 3.2.0
### Patch Changes
- 689ac68b: pin alpine version to 3.18 and add healthchecks
## 3.0.0
### Patch Changes
- 23481d0d: Update to TypeScript 5.1.6
## 3.0.0-next.4
### Patch Changes

View File

@@ -14,6 +14,4 @@ RUN chown elasticsearch:elasticsearch config/elasticsearch.yml
USER elasticsearch
HEALTHCHECK --interval=10s --timeout=10s --start-period=60s --retries=12 CMD curl --fail -s http://localhost:9200/ >/dev/null || exit 1
CMD ["/usr/share/elasticsearch/bin/elasticsearch"]

View File

@@ -3,4 +3,3 @@ discovery.type: "single-node"
cluster.routing.allocation.disk.threshold_enabled: false
network.bind_host: 0.0.0.0
xpack.security.enabled: false
ingest.geoip.downloader.enabled: false

View File

@@ -1,6 +1,6 @@
{
"name": "@openstapps/database",
"version": "4.0.0",
"version": "3.0.0-next.4",
"private": true,
"files": [
"config",

View File

@@ -1,112 +1,5 @@
# @openstapps/proxy
## 4.0.0
### Major Changes
- c8e29020: Update to Node 22
### Patch Changes
- Updated dependencies [c8e29020]
- @openstapps/logger@4.0.0
## 3.0.0
### Major Changes
- 64caebaf: Move project to a turbo monorepo & pnpm
Internal dependencies are now defined using `"@openstapps/package": "workspace:*"`
- Removed extraneous files from packages
- `.npmrc`
- `.npmignore`
- `.mailmap`
- `.gitignore`
- `CONTRIBUTING.md`
- `LICENSE` (Project license file is added upon publishing, see [pnpm.io](https://pnpm.io/cli/publish))
- `package-lock.json`
- `.editorconfig`
- `.eslintrc.json` (moved eslint config to `package.json`)
- `.eslintignore`
- `.gitlab-ci.yml` (Most workflows are workspace-level)
- `.gitlab/**` (issue templates etc. are now workspace-level)
- `.dockerignore` (Docker files are determined by which files are deployed with `pnpm deploy`, as per `package.json/files`)
- TSConfig has been moved to its own package (You can now use `"extends": "@openstapps/tsconfig"`)
- Removed ESLint and Prettier peer dependency hell by injecting them through the `.pnpmfile.cjs`
- Added syncpack for keeping dependency versions in sync (and consistent key ordering in `package.json`)
- Replaced conventional changelog with changesets
- Apps with binaries now use a top level `app.js`
```js
#!/usr/bin/env node
import './lib/app.js';
```
- 64caebaf: Migrate to ESM
CommonJS is no longer supported in any capacity. To use the new
version, you will need to migrate your package to ESM.
We recommend using `tsup` and `Node 18`.
```json
{
"type": "module"
}
```
- 64caebaf: Migrate package to Node 18
- Consumers of this package will need to migrate to Node 18 or
higher.
- Packages have been migrated from promisified `readFile` or
`readFileSync` towards `fs/promises`
- Packages use native `flatMap` now
- 64caebaf: Migrate build system to `tsup`
All packages now use an `index.ts` file to expose contents.
You will need to migrate paths from `import foo from '@scope/package/lib/foo` to `import foo from '@scope/package'`
### Minor Changes
- 64caebaf: Migrate tests to C8/Chai/Mocha
- `@testdeck` OOP testing has been removed.
- Tests have been unified
- CommonJS module mocking has been replaced through
refactoring of tests, as ES Modules cannot be mocked
(do yourself a favor and don't try to mock them)
- C8 now replaces NYC as a native coverage tool
### Patch Changes
- 64caebaf: Migrated changelogs to changeset format
```js
import fs from 'fs';
const path = 'packages/logger/CHANGELOG.md';
fs.writeFileSync(path, fs.readFileSync(path, 'utf8').replace(/^#+\s+\[/gm, '## ['));
```
- 98546a97: Migrate away from @openstapps/configuration
- 23481d0d: Update to TypeScript 5.1.6
- Updated dependencies [64caebaf]
- Updated dependencies [64caebaf]
- Updated dependencies [64caebaf]
- Updated dependencies [64caebaf]
- Updated dependencies [64caebaf]
- Updated dependencies [98546a97]
- Updated dependencies [23481d0d]
- Updated dependencies [64caebaf]
- Updated dependencies [98546a97]
- Updated dependencies [64caebaf]
- @openstapps/logger@3.0.0
## 3.0.0-next.4
### Patch Changes

View File

@@ -16,7 +16,7 @@
// ESM is not supported, and cts is not detected, so we use type-checked cjs instead.
/** @type {import('../src/common').ConfigFile} */
module.exports = {
const configFile = {
activeVersions: ['1\\.0\\.\\d+', '2\\.0\\.\\d+'],
hiddenRoutes: ['/bulk'],
logFormat: 'default',
@@ -31,3 +31,5 @@ module.exports = {
dhparam: '/etc/nginx/certs/dhparam.pem',
},
};
export default configFile;

View File

@@ -1,7 +1,7 @@
{
"name": "@openstapps/proxy",
"description": "NGINX proxy that is dynamically configured by a Node.js script",
"version": "4.0.0",
"version": "3.0.0-next.4",
"private": true,
"type": "module",
"license": "AGPL-3.0-only",
@@ -44,39 +44,39 @@
"@openstapps/logger": "workspace:*",
"@types/config": "3.3.0",
"@types/dockerode": "3.3.17",
"@types/node": "22.15.31",
"@types/node": "18.15.3",
"@types/sha1": "1.1.3",
"config": "3.3.9",
"dockerode": "3.3.5",
"is-cidr": "4.0.2",
"mustache": "4.2.0",
"semver": "7.6.0",
"typescript": "5.4.2"
"semver": "7.3.8",
"typescript": "5.1.6"
},
"devDependencies": {
"@openstapps/api-cli": "workspace:*",
"@openstapps/eslint-config": "workspace:*",
"@openstapps/prettier-config": "workspace:*",
"@openstapps/tsconfig": "workspace:*",
"@types/chai": "4.3.20",
"@types/chai": "4.3.5",
"@types/config": "3.3.0",
"@types/dockerode": "3.3.17",
"@types/mocha": "10.0.10",
"@types/mocha": "10.0.1",
"@types/mustache": "4.2.2",
"@types/node": "22.15.31",
"@types/node": "18.15.3",
"@types/proxyquire": "1.3.28",
"@types/semver": "7.5.8",
"@types/semver": "7.3.13",
"@types/sha1": "1.1.3",
"@types/sinon": "10.0.14",
"@types/sinon-chai": "3.2.9",
"c8": "10.1.3",
"chai": "4.5.0",
"mocha": "10.8.2",
"c8": "7.14.0",
"chai": "4.3.7",
"mocha": "10.2.0",
"mocha-junit-reporter": "2.2.0",
"sinon": "15.0.4",
"sinon-chai": "3.7.0",
"ts-node": "10.9.2",
"tsup": "8.5.0"
"ts-node": "10.9.1",
"tsup": "6.7.0"
},
"tsup": {
"entry": [

View File

@@ -1,4 +1,4 @@
{
"extends": "@openstapps/tsconfig",
"exclude": ["config", "lib", "app.js"]
"exclude": ["./config/", "./lib/"]
}

View File

@@ -1,80 +1,5 @@
# @openstapps/backend-config
## 4.0.0
### Major Changes
- c8e29020: Update to Node 22
## 3.0.0
### Major Changes
- 64caebaf: Move project to a turbo monorepo & pnpm
Internal dependencies are now defined using `"@openstapps/package": "workspace:*"`
- Removed extraneous files from packages
- `.npmrc`
- `.npmignore`
- `.mailmap`
- `.gitignore`
- `CONTRIBUTING.md`
- `LICENSE` (Project license file is added upon publishing, see [pnpm.io](https://pnpm.io/cli/publish))
- `package-lock.json`
- `.editorconfig`
- `.eslintrc.json` (moved eslint config to `package.json`)
- `.eslintignore`
- `.gitlab-ci.yml` (Most workflows are workspace-level)
- `.gitlab/**` (issue templates etc. are now workspace-level)
- `.dockerignore` (Docker files are determined by which files are deployed with `pnpm deploy`, as per `package.json/files`)
- TSConfig has been moved to its own package (You can now use `"extends": "@openstapps/tsconfig"`)
- Removed ESLint and Prettier peer dependency hell by injecting them through the `.pnpmfile.cjs`
- Added syncpack for keeping dependency versions in sync (and consistent key ordering in `package.json`)
- Replaced conventional changelog with changesets
- Apps with binaries now use a top level `app.js`
```js
#!/usr/bin/env node
import "./lib/app.js";
```
- 64caebaf: Migrate to ESM
CommonJS is no longer supported in any capacity. To use the new
version, you will need to migrate your package to ESM.
We recommend using `tsup` and `Node 18`.
```json
{
"type": "module"
}
```
- 64caebaf: Migrate package to Node 18
- Consumers of this package will need to migrate to Node 18 or
higher.
- Packages have been migrated from promisified `readFile` or
`readFileSync` towards `fs/promises`
- Packages use native `flatMap` now
### Minor Changes
- 64caebaf: Migrate tests to C8/Chai/Mocha
- `@testdeck` OOP testing has been removed.
- Tests have been unified
- CommonJS module mocking has been replaced through
refactoring of tests, as ES Modules cannot be mocked
(do yourself a favor and don't try to mock them)
- C8 now replaces NYC as a native coverage tool
### Patch Changes
- 98546a97: Migrate away from @openstapps/configuration
- 23481d0d: Update to TypeScript 5.1.6
## 3.0.0-next.4
### Patch Changes
@@ -111,7 +36,7 @@
```js
#!/usr/bin/env node
import "./lib/app.js";
import './lib/app.js';
```
- 64caebaf: Migrate to ESM

Some files were not shown because too many files have changed in this diff Show More