OpenSearch Index UM Entries

Goal: Install local OpenSearch via Docker and start indexing UM Entries.

You’ll need a working Docker setup on your local machine to follow all the steps in this tutorial.

You can download the whole Kickstarter-based sample application from our GitLab repository: um-public/tutorial-opensearch-index-um-entries

Start a new Kickstarter project

First, choose a Java-style package name for your new plugin – e.g. de.acme.tutorial. (See also ACME Corporation)
Please note that com..., net... and org... denote reserved namespaces and can not be used as top-level package names.

Second, use UM-Kickstarter to begin a new UM project:

umkickstarter -n de.acme.tutorial
cd umkickstarter

Add OpenSearch UM plugin

Edit build.gradle and add the following dependency:

dependencies {
    // ...

    runtime('de.pinuts.cmsbs:OpenSearch:1.0.0')
}

Setup and run local UM:

gradle setup run

As always, you should now be able to log in to your newly installed UM: http://localhost:8080/cmsbs (credentials: admin / admin)

Install OpenSearch via Docker

Pull the latest OpenSearch 2 Docker image:

docker pull opensearchproject/opensearch:2

In this tutorial we use mVTMDJWvmps57MShu9qy as the admin password. Change this to a strong random password in your environment!

The following command starts a Docker container named opensearch-node that will be deleted after termination:

docker run --rm -p 9200:9200 -p 9600:9600 \
    -e "OPENSEARCH_INITIAL_ADMIN_PASSWORD=mVTMDJWvmps57MShu9qy" \
    -e "discovery.type=single-node" \
    --name opensearch-node \
    opensearchproject/opensearch:2

Alternatively you can add an appropriate volume to keep the data directory and the container after termination:

docker run -p 9200:9200 -p 9600:9600 \
    -e "OPENSEARCH_INITIAL_ADMIN_PASSWORD=mVTMDJWvmps57MShu9qy" \
    -e "discovery.type=single-node" \
    -v opensearch-data:/usr/share/opensearch/data \
    --name opensearch-node \
    opensearchproject/opensearch:2

Once the container is up and running, you should be able to retreive some information about the running instance:

curl -k https://admin:mVTMDJWvmps57MShu9qy@localhost:9200/

See https://opensearch.org/docs/latest/install-and-configure/install-opensearch/docker/ for more information on how to run OpenSearch in a Docker environment.

Configure access to OpenSearch

Create a new config file env/devel/cmsbs-conf/conf.d/opensearch.properties:

cmsbs.opensearch.url                  = https://localhost:9200/
cmsbs.opensearch.username             = admin
cmsbs.opensearch.password             = mVTMDJWvmps57MShu9qy
cmsbs.opensearch.registerDefaultIndex = true

This tells the UM OpenSearch plugin to register and handle default index named cmsbs.

Customize UM data model

We will add a custom attribute of type TEXT to the our UM’s data model to see how we can affect the way full text fields can be indexed.

Create a new config file cmsbs-conf/conf.d/opensearch.attributes:

additional.attributes[] = notes
grp.std_personal.members[] = notes

notes.title = Notes
notes.type = TEXT

Now, stop your UM, delete the database and start it up again:

rm UM/cmsbs-work/db.h2.*
gradle setup run

When you login again, you should see the new Notes attribute when creating new Customer entries: http://localhost:8080/cmsbs/EditUser2.jsp?entrytype=customer&createnew=true

Create some Customer entries with Firstname, Lastname, Email and Notes.

Define which UM attributes to index

Edit cmsbs-conf/conf.d/opensearch.attributes again and add the following lines to define which attributes should be indexed in OpenSearch:

# Full text field with german and english stemming:
notes.config.opensearch = {"fields": {"german": {"type": "text", "analyzer": "german"}, "english": {"type": "text", "analyzer": "english"}}}

# Index some fields with default settings:
firstname.config.opensearch = {}
lastname.config.opensearch = {}
email.config.opensearch = {}

# Index salut attribute including the values' localized labels (= "nice" values):
salut.config.opensearch = {"storeNice": true}

# Override field name for certain attributes:
cmsbs.creation.config.opensearch = {"name": "creation"}
cmsbs.lastchanged.config.opensearch = {"name": "lastchanged"}

# Index these TABLE attributes as well:
consent_history.config.opensearch = {}
newsletter_channel_activity.config.opensearch = {}

Restart your UM.

Re-Index all Entries

Open the CSE Console and re-index all Entries:

de.pinuts.cmsbs.opensearch.reindexAll()

Run a simple query to get the first 100 UM entries from the OpenSearch index:

curl -k https://admin:mVTMDJWvmps57MShu9qy@localhost:9200/cmsbs/_search -X POST -H Content-Type:application/json -d '{"size": 100, "query": { "match_all": {} }}'

UM Developer Documentation