mirror of
https://github.com/mainflux/mainflux.git
synced 2025-04-29 13:49:28 +08:00

* MF-1254 - Create universal JSON writer (#1260) Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Add JSON support to Readers Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix Influx Reader tests Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix messages format query Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix Postgres reader Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix Cassandra Readers and writers Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix Mongo reader Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Extract utility method to the JSON transformer Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix Influx and Postgres count Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Update JSON transformer Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix Influxdb Reader total count Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Refactor init.go for Cassandra writer Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Create a Payload type Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Add comments for defaults Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix variable declarations Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Replace interface{} with a new type Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Don't set channel just to overwrite it later Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix range search Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Rename Messages field Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> Co-authored-by: Manuel Imperiale <manuel.imperiale@gmail.com>
Postgres reader
Postgres reader provides message repository implementation for Postgres.
Configuration
The service is configured using the environment variables presented in the following table. Note that any unset variables will be replaced with their default values.
Variable | Description | Default |
---|---|---|
MF_POSTGRES_READER_LOG_LEVEL | Service log level | debug |
MF_POSTGRES_READER_PORT | Service HTTP port | 8180 |
MF_POSTGRES_READER_CLIENT_TLS | TLS mode flag | false |
MF_POSTGRES_READER_CA_CERTS | Path to trusted CAs in PEM format | |
MF_POSTGRES_READER_DB_HOST | Postgres DB host | postgres |
MF_POSTGRES_READER_DB_PORT | Postgres DB port | 5432 |
MF_POSTGRES_READER_DB_USER | Postgres user | mainflux |
MF_POSTGRES_READER_DB_PASS | Postgres password | mainflux |
MF_POSTGRES_READER_DB | Postgres database name | messages |
MF_POSTGRES_READER_DB_SSL_MODE | Postgres SSL mode | disabled |
MF_POSTGRES_READER_DB_SSL_CERT | Postgres SSL certificate path | "" |
MF_POSTGRES_READER_DB_SSL_KEY | Postgres SSL key | "" |
MF_POSTGRES_READER_DB_SSL_ROOT_CERT | Postgres SSL root certificate path | "" |
MF_JAEGER_URL | Jaeger server URL | localhost:6831 |
MF_THINGS_AUTH_GRPC_URL | Things service Auth gRPC URL | localhost:8181 |
MF_THINGS_AUTH_GRPC_TIMEOUT | Things service Auth gRPC timeout in seconds | 1s |
Deployment
version: "3.7"
postgres-writer:
image: mainflux/postgres-writer:[version]
container_name: [instance name]
depends_on:
- postgres
- nats
restart: on-failure
environment:
MF_NATS_URL: [NATS instance URL]
MF_POSTGRES_READER_LOG_LEVEL: [Service log level]
MF_POSTGRES_READER_PORT: [Service HTTP port]
MF_POSTGRES_READER_DB_HOST: [Postgres host]
MF_POSTGRES_READER_DB_PORT: [Postgres port]
MF_POSTGRES_READER_DB_USER: [Postgres user]
MF_POSTGRES_READER_DB_PASS: [Postgres password]
MF_POSTGRES_READER_DB: [Postgres database name]
MF_POSTGRES_READER_DB_SSL_MODE: [Postgres SSL mode]
MF_POSTGRES_READER_DB_SSL_CERT: [Postgres SSL cert]
MF_POSTGRES_READER_DB_SSL_KEY: [Postgres SSL key]
MF_POSTGRES_READER_DB_SSL_ROOT_CERT: [Postgres SSL Root cert]
MF_JAEGER_URL: [Jaeger server URL]
MF_THINGS_AUTH_GRPC_URL: [Things service Auth gRPC URL]
MF_THINGS_AUTH_GRPC_TIMEOUT: [Things service Auth gRPC request timeout in seconds]
ports:
- 8180:8180
networks:
- docker_mainflux-base-net
To start the service, execute the following shell script:
# download the latest version of the service
git clone https://github.com/mainflux/mainflux
cd mainflux
# compile the postgres writer
make postgres-writer
# copy binary to bin
make install
# Set the environment variables and run the service
MF_POSTGRES_READER_LOG_LEVEL=[Service log level] \
MF_POSTGRES_READER_PORT=[Service HTTP port] \
MF_POSTGRES_READER_CLIENT_TLS =[TLS mode flag] \
MF_POSTGRES_READER_CA_CERTS=[Path to trusted CAs in PEM format] \
MF_POSTGRES_READER_DB_HOST=[Postgres host] \
MF_POSTGRES_READER_DB_PORT=[Postgres port] \
MF_POSTGRES_READER_DB_USER=[Postgres user] \
MF_POSTGRES_READER_DB_PASS=[Postgres password] \
MF_POSTGRES_READER_DB=[Postgres database name] \
MF_POSTGRES_READER_DB_SSL_MODE=[Postgres SSL mode] \
MF_POSTGRES_READER_DB_SSL_CERT=[Postgres SSL cert] \
MF_POSTGRES_READER_DB_SSL_KEY=[Postgres SSL key] \
MF_POSTGRES_READER_DB_SSL_ROOT_CERT=[Postgres SSL Root cert] \
MF_JAEGER_URL=[Jaeger server URL] \
MF_THINGS_AUTH_GRPC_URL=[Things service Auth GRPC URL] \
MF_THINGS_AUTH_GRPC_TIMEOUT=[Things service Auth gRPC request timeout in seconds] \
$GOBIN/mainflux-postgres-reader
Usage
Starting service will start consuming normalized messages in SenML format.