Theaninova/openstapps
1
0
Fork 0
mirror of https://gitlab.com/openstapps/openstapps.git synced 2026-01-03 20:12:51 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

refactor: move es-mapping-generator to monorepo

Browse Source
This commit is contained in:
Rainer Killinger
2023-01-12 14:16:03 +01:00
committed by Thea Schöbl
parent f0a1f665ff
commit ceed1ed706
62 changed files with 4 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

64
packages/es-mapping-generator/README.md Normal file
View File

@@ -0,0 +1,64 @@
# @openstapps/es-mapping-generator
[![pipeline status](https://img.shields.io/gitlab/pipeline/openstapps/es-mapping-generator.svg?style=flat-square)](https://gitlab.com/openstapps/es-mapping-gen/commits/master)
[![npm](https://img.shields.io/npm/v/@openstapps/es-mapping-generator.svg?style=flat-square)](https://npmjs.com/package/@openstapps/es-mapping-gen)
[![license)](https://img.shields.io/npm/l/@openstapps/es-mapping-generator.svg?style=flat-square)](https://www.gnu.org/licenses/gpl-3.0.en.html)
[![documentation](https://img.shields.io/badge/documentation-online-blue.svg?style=flat-square)](https://openstapps.gitlab.io/es-mapping-gen)
Tools to convert and validate StAppsCore
## What is the ES Mapping Generator for?
The ES Mapping Generator is for converting TypeScript interfaces to Elasticsearch mappings,
with support for Integer and Float types, Dates, etc.
## Why is this a separate package?
The ES Mapping Generator relies on an outdated version of TypeDoc, that can't be updated.
Because of that, it then also relies on an outdated version of TypeScript, resulting
in a cascade effect things that can't be updated. Because of that, we decided to isolate
it to the best of our abilities.
## How to use the Elasticsearch Mapping generator
The mapping generator is intended to be used by the backend directly, but it can also be used to generate these files
manually.
### Generating the mapping files by hand
To generate the mapping files by hand, you need a local copy of the core-tools and the core, both need to be built first.
After that you can run
```
node lib/cli.js mapping path/to/core path/to/destination ignoredTag1,ignoredTag2,ignoredTag3
```
If you don't pass in any ignored tags, you will likely be prompted with errors, despite the core being absolutely correct.
This is because there are some tags that are not relevant to Elasticsearch, but the program has no direct way to tell
which ones simply lack an implementation and which ones can be ignored. Currently the ignored tags include
`minlength`, `pattern` and `see`.
### Generating the mapping directly from another TypeScript program
This is the more easy way, and it gives you direct access to the generated mapping as a (mostly typesafe) object. To
use it, call `generateTemplate`, However you will first need to generate a ProjectReflection of the core you are working
with. If you use the core as a dependency, you can for example use
```typescript
const map = generateTemplate(getProjectReflection(resolve('node_modules', '@openstapps', 'core', 'src')),
ignoredTags, false);
```
to generate the mappings. Note that the result object contains both a list of errors in `map.errors` and the actual mapping
in `map.template`. You can also specify whether you want the generator to show any errors while generating the mappings
in the console in the last parameter, `true` (default) being show errors and `false` to suppress them. That said it is very
easy to replace all `type: "MISSING_PREMAP"`, `type: "PARSE_ERROR"`, `type: "TYPE_CONFLICT"` with `dynamic: true` (you
can take these exactly like written here and run a replace over the file).
### Fixing a generated mapping by hand
If you get errors when generating the mappings, the mappings might not work, however they will still be generated to the
programs best efforts. Most small issues can be fixed after the mapping was generated as a temporary solution and without
changing anything in the mapper's code. This however requires some understanding of how mappings work.
The output of the program can easily reach 25.000 lines, but you can find errors quickly by searching for `MISSING_PREMAP`,
`PARSE_ERROR` and `TYPE_CONFLICT`. When you reach them you can then manually replace them with your code.
As a last resort you can also replace all errors with dynamic types, this should get the mapping working, but it is NOT
RECOMMENDED as a fix other than using it locally.