8be2516321
* Use Normalizer as a lib To normalize messages on the consumer side, Normalizer is moved to the internal pkgs. Writers being message consumers are modified to do message normalization instead of subscribing to normalized messages subject. Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix logging middleware for readers and writers Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove normalizer interface Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Use Normalizer in writers As we agreed on #919, we'll use normalizer as an interface and provide the default SenML implementation. Because of that, Normalizer is removed from `internal` and we'll use the project structure proposed in the aforementioned issue. Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix tests Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove unused batch settings from influxDB reader Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update docs Move Normalizer service to `addons`. Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Rename channels input topic Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update Noramlizer docs Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove commented code Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update readers logging Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update addons docker-compose files Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update topcis explanations Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>
Message normalizer
Normalizer service consumes events published by adapters, normalizes SenML-formatted ones, and publishes them to the post-processing stream. Normalizer can also be imported as a package and used independently for message normalization. This reduces internal traffic because messages are not published back to the broker, but transformed on the message consumer side. Mainflux (writers) [https://github.com/mainflux/mainflux/tree/master/writers] are using Normalizer to preprocess messages before storing them.
Configuration
The service is configured using the environment variables presented in the following table. Note that any unset variables will be replaced with their default values.
| Variable | Description | Default |
|---|---|---|
| MF_NATS_URL | NATS instance URL | nats://localhost:4222 |
| MF_NORMALIZER_LOG_LEVEL | Log level for the Normalizer | error |
| MF_NORMALIZER_PORT | Normalizer service HTTP port | 8180 |
Deployment
The service itself is distributed as Docker container. The following snippet provides a compose file template that can be used to deploy the service container locally:
version: "3.7"
services:
normalizer:
image: mainflux/normalizer:[version]
container_name: [instance name]
environment:
MF_NATS_URL: [NATS instance URL]
MF_NORMALIZER_LOG_LEVEL: [Normalizer log level]
MF_NORMALIZER_PORT: [Service HTTP port]
To start the service outside of the container, execute the following shell script:
# download the latest version of the service
go get github.com/mainflux/mainflux
cd $GOPATH/src/github.com/mainflux/mainflux
# compile the service; make sure to set the proper GOOS value
make normalizer
# copy binary to bin
make install
# set the environment variables and run the service
MF_NATS_URL=[NATS instance URL] MF_NORMALIZER_LOG_LEVEL=[Normalizer log level] MF_NORMALIZER_PORT=[Service HTTP port] $GOBIN/mainflux-normalizer