1
0
mirror of https://github.com/mainflux/mainflux.git synced 2025-04-26 13:48:53 +08:00

69 lines
3.6 KiB
Markdown
Raw Normal View History

# Postgres writer
Postgres writer provides message repository implementation for Postgres.
## Configuration
The service is configured using the environment variables presented in the
following table. Note that any unset variables will be replaced with their
default values.
MF-1264 - Add support for JSON readers (#1295) * MF-1254 - Create universal JSON writer (#1260) Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Add JSON support to Readers Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix Influx Reader tests Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix messages format query Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix Postgres reader Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix Cassandra Readers and writers Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix Mongo reader Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Extract utility method to the JSON transformer Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix Influx and Postgres count Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Update JSON transformer Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix Influxdb Reader total count Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Refactor init.go for Cassandra writer Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Create a Payload type Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Add comments for defaults Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix variable declarations Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Replace interface{} with a new type Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Don't set channel just to overwrite it later Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix range search Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Rename Messages field Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> Co-authored-by: Manuel Imperiale <manuel.imperiale@gmail.com>
2020-12-30 15:43:04 +01:00
| Variable | Description | Default |
| ----------------------------------- | ----------------------------------------------- | ---------------------- |
| MF_NATS_URL | NATS instance URL | nats://localhost:4222 |
| MF_POSTGRES_WRITER_LOG_LEVEL | Service log level | error |
| MF_POSTGRES_WRITER_PORT | Service HTTP port | 9104 |
| MF_POSTGRES_WRITER_DB_HOST | Postgres DB host | postgres |
| MF_POSTGRES_WRITER_DB_PORT | Postgres DB port | 5432 |
| MF_POSTGRES_WRITER_DB_USER | Postgres user | mainflux |
| MF_POSTGRES_WRITER_DB_PASS | Postgres password | mainflux |
| MF_POSTGRES_WRITER_DB | Postgres database name | messages |
| MF_POSTGRES_WRITER_DB_SSL_MODE | Postgres SSL mode | disabled |
| MF_POSTGRES_WRITER_DB_SSL_CERT | Postgres SSL certificate path | "" |
| MF_POSTGRES_WRITER_DB_SSL_KEY | Postgres SSL key | "" |
| MF_POSTGRES_WRITER_DB_SSL_ROOT_CERT | Postgres SSL root certificate path | "" |
| MF_POSTGRES_WRITER_CONFIG_PATH | Configuration file path with NATS subjects list | /config.toml |
| MF_POSTGRES_WRITER_CONTENT_TYPE | Message payload Content Type | application/senml+json |
| MF_POSTGRES_WRITER_TRANSFORMER | Message transformer type | senml |
## Deployment
The service itself is distributed as Docker container. Check the [`postgres-writer`](https://github.com/mainflux/mainflux/blob/master/docker/addons/postgres-writer/docker-compose.yml#L34-L59) service section in
docker-compose to see how service is deployed.
To start the service, execute the following shell script:
```bash
# download the latest version of the service
git clone https://github.com/mainflux/mainflux
cd mainflux
# compile the postgres writer
make postgres-writer
# copy binary to bin
make install
# Set the environment variables and run the service
MF-1096 - Fix AuthN and Things Auth ENVARS (#1066) * NOISSUE - Fix AuthN and Things Auth ENVARS Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Add DB envars to env.go Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Fix envars Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Add DefLogLLevelError Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Fix DB names Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Fix DB names Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Fix DB names and HTTP ports Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Fix .env Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Revert writers DB names Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Rm unused Twins envars Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Fix reviews Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Fix reviews Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Fix typo Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Remove definitions from env.go Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Revert HTTP gRPC ports Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Use default NATS URL as string Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Fix reviews Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Revert default ports Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Fix typo Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Fix cassandra ENVARS Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Fix cassandra reader ENVARS Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Fix readers and writers envars Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Fix readers and writers .env Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Fix reviews Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>
2020-04-07 12:02:17 +02:00
MF_NATS_URL=[NATS instance URL] \
MF_POSTGRES_WRITER_LOG_LEVEL=[Service log level] \
MF_POSTGRES_WRITER_PORT=[Service HTTP port] \
MF_POSTGRES_WRITER_DB_HOST=[Postgres host] \
MF_POSTGRES_WRITER_DB_PORT=[Postgres port] \
MF_POSTGRES_WRITER_DB_USER=[Postgres user] \
MF_POSTGRES_WRITER_DB_PASS=[Postgres password] \
MF_POSTGRES_WRITER_DB=[Postgres database name] \
MF_POSTGRES_WRITER_DB_SSL_MODE=[Postgres SSL mode] \
MF_POSTGRES_WRITER_DB_SSL_CERT=[Postgres SSL cert] \
MF_POSTGRES_WRITER_DB_SSL_KEY=[Postgres SSL key] \
MF_POSTGRES_WRITER_DB_SSL_ROOT_CERT=[Postgres SSL Root cert] \
MF-1264 - Add support for JSON readers (#1295) * MF-1254 - Create universal JSON writer (#1260) Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Add JSON support to Readers Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix Influx Reader tests Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix messages format query Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix Postgres reader Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix Cassandra Readers and writers Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix Mongo reader Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Extract utility method to the JSON transformer Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix Influx and Postgres count Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Update JSON transformer Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix Influxdb Reader total count Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Refactor init.go for Cassandra writer Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Create a Payload type Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Add comments for defaults Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix variable declarations Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Replace interface{} with a new type Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Don't set channel just to overwrite it later Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix range search Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Rename Messages field Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> Co-authored-by: Manuel Imperiale <manuel.imperiale@gmail.com>
2020-12-30 15:43:04 +01:00
MF_POSTGRES_WRITER_CONFIG_PATH=[Configuration file path with NATS subjects list] \
MF_POSTGRES_WRITER_TRANSFORMER=[Message transformer type] \
MF-1096 - Fix AuthN and Things Auth ENVARS (#1066) * NOISSUE - Fix AuthN and Things Auth ENVARS Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Add DB envars to env.go Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Fix envars Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Add DefLogLLevelError Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Fix DB names Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Fix DB names Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Fix DB names and HTTP ports Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Fix .env Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Revert writers DB names Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Rm unused Twins envars Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Fix reviews Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Fix reviews Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Fix typo Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Remove definitions from env.go Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Revert HTTP gRPC ports Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Use default NATS URL as string Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Fix reviews Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Revert default ports Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Fix typo Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Fix cassandra ENVARS Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Fix cassandra reader ENVARS Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Fix readers and writers envars Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Fix readers and writers .env Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com> * Fix reviews Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>
2020-04-07 12:02:17 +02:00
$GOBIN/mainflux-postgres-writer
```
## Usage
Starting service will start consuming normalized messages in SenML format.