1
0
mirror of https://github.com/mainflux/mainflux.git synced 2025-04-27 13:48:49 +08:00
Aryan Godara 986edacfc7
MF-1584 - Upgrade InfluxDB from 1.x to 2.x (#1709)
* Upgrade InfluxDB from 1.x to 2.x

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Influx DB configuration updated

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Connection to InfluxDBv2

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Token cannot be created

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Connected to InfluxDB2

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Connected to InfluxDB2

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* InfluxDB v2 Consumer Implementation

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* quickfix

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Influxdb-Writer Unit Tests Update

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Consumer Update

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* InfluxDB Writer Tests Implemented

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* fix

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* InfluxDB Connection Check

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Resolving Remarks

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Resolved consumer-test remark

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* consumer-test slow working version

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* reader changes

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Consumer tests time issue fixed

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Eof warning fixed

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Resolved Reviews

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Makefile Fixed

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Resolved Reviews

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Resolved Reviews

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Resolved Reviews

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Resolved Reviews

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Resolved Reviews

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Readers Initial Setup

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* consumer json fix

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* readers simple version

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* readers simple version fix

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* readers simple version fix

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* readers simple version fix

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* readers simple version fix

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* readers simple version fix

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* readers simple version fix

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Influxdb Reader Parsers

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Influxdb Reader Parsers

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Influxdb Reader Parsers

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Bugfix and resolves comments.

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* one test fails

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* solved last page read

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* writers future time problem fixed

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* weird

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* weird

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Tests Passes

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Resolve Semaphore Issues

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* UUID comment on consumer tests resolved

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Inclusive from and Exclusive to

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Inclusive from and Exclusive to

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* 1 second limits

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* 1 second limits

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* resolved review

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* resolved review

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* resolved review

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* resolved review

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* fixed json time

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* fixed CI error

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* resolved request

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* resolved request

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* resolved requests

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* removed blank line

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* resolved comment

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* deleted unnecessary string builder

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* initial commit

Signed-off-by: aryan <aryangodara03@gmail.com>

* change influxdb docker image version.

Signed-off-by: aryan <aryangodara03@gmail.com>

* go mod and vendor fixing

Signed-off-by: aryan <aryangodara03@gmail.com>

* Upgrade InfluxDB from 1.x to 2.x

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Influx DB configuration updated

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Connection to InfluxDBv2

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Token cannot be created

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Connected to InfluxDB2

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Connected to InfluxDB2

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* InfluxDB v2 Consumer Implementation

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* quickfix

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Influxdb-Writer Unit Tests Update

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Consumer Update

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* InfluxDB Writer Tests Implemented

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* fix

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* InfluxDB Connection Check

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Resolving Remarks

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Resolved consumer-test remark

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* consumer-test slow working version

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* reader changes

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Consumer tests time issue fixed

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Resolved Reviews

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Resolved Reviews

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Resolved Reviews

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Resolved Reviews

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Resolved Reviews

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Resolved Reviews

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Readers Initial Setup

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* consumer json fix

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* readers simple version

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* readers simple version fix

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* readers simple version fix

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* readers simple version fix

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* readers simple version fix

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* readers simple version fix

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* readers simple version fix

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Influxdb Reader Parsers

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Influxdb Reader Parsers

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Influxdb Reader Parsers

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Bugfix and resolves comments.

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* one test fails

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* solved last page read

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* writers future time problem fixed

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* weird

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* weird

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Tests Passes

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Resolve Semaphore Issues

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* UUID comment on consumer tests resolved

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Inclusive from and Exclusive to

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* Inclusive from and Exclusive to

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* 1 second limits

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* 1 second limits

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* resolved review

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* resolved review

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* resolved review

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* resolved review

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* fixed json time

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* fixed CI error

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* resolved request

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* resolved request

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* resolved requests

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* removed blank line

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* resolved comment

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* deleted unnecessary string builder

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>

* initial commit

Signed-off-by: aryan <aryangodara03@gmail.com>

* change influxdb docker image version.

Signed-off-by: aryan <aryangodara03@gmail.com>

* go mod and vendor fixing

Signed-off-by: aryan <aryangodara03@gmail.com>

* go mod fixing

Signed-off-by: aryan <aryangodara03@gmail.com>

* make tests pass locally, fix errors

Signed-off-by: aryan <aryangodara03@gmail.com>

* rem unsused variables/consts

Signed-off-by: aryan <aryangodara03@gmail.com>

* go mod tidy vendor

Signed-off-by: aryan <aryangodara03@gmail.com>

* add env var and data types

Signed-off-by: aryan <aryangodara03@gmail.com>

* update influxdb version to latest (2.3 to 2.12)

Signed-off-by: aryan <aryangodara03@gmail.com>

* change time precision for message tests

Signed-off-by: aryan <aryangodara03@gmail.com>

* renamed influxdb2 to influxdata

Signed-off-by: aryan <aryangodara03@gmail.com>

* address remarks on PR

Signed-off-by: aryan <aryangodara03@gmail.com>

* update influxbd according to latest mf commit

Signed-off-by: aryan <aryangodara03@gmail.com>

* temp commit, rem before pushing

Signed-off-by: aryan <aryangodara03@gmail.com>

* update main files and cassandra-reader messages_tests

Signed-off-by: aryan <aryangodara03@gmail.com>

* fix name of logger while importing

Signed-off-by: aryan <aryangodara03@gmail.com>

* remove unnecessary print lines

Signed-off-by: aryan <aryangodara03@gmail.com>

* correct env var name

Signed-off-by: aryan <aryangodara03@gmail.com>

* change to async consume

Signed-off-by: aryan <aryangodara03@gmail.com>

* add option to switch bw sync and async

Signed-off-by: aryan <aryangodara03@gmail.com>

* test for both async and sync

Signed-off-by: aryan <aryangodara03@gmail.com>

* update consumer and add writeAPIs to config

Signed-off-by: aryan <aryangodara03@gmail.com>

* revert back to sync consuming

Signed-off-by: aryan <aryangodara03@gmail.com>

* temp fix for default timeout value

Signed-off-by: aryan <aryangodara03@gmail.com>

* set default timeout in config.

Signed-off-by: aryan <aryangodara03@gmail.com>

* remove unwanted env vars, add required ones.

Signed-off-by: aryan <aryangodara03@gmail.com>

* rem unused username password from config

Signed-off-by: aryan <aryangodara03@gmail.com>

* update readme, env vars, and remove grafana

Signed-off-by: aryan <aryangodara03@gmail.com>

* update readme

Signed-off-by: aryan <aryangodara03@gmail.com>

* fix typo

Signed-off-by: aryan <aryangodara03@gmail.com>

* update readme description.

Signed-off-by: aryan <aryangodara03@gmail.com>

* fix more typos.

Signed-off-by: aryan <aryangodara03@gmail.com>

* add link to official docs to readme.

Signed-off-by: aryan <aryangodara03@gmail.com>

---------

Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu>
Signed-off-by: aryan <aryangodara03@gmail.com>
Co-authored-by: fatih <fatihdurmaz@sabanciuniv.edu>
Co-authored-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>
2023-03-16 02:36:14 +01:00
..

HTTP adapter

HTTP adapter provides an HTTP API for sending messages through the platform.

Configuration

The service is configured using the environment variables presented in the following table. Note that any unset variables will be replaced with their default values.

Variable Description Default
MF_HTTP_ADAPTER_LOG_LEVEL Log level for the HTTP Adapter info
MF_HTTP_ADAPTER_PORT Service HTTP port 8180
MF_BROKER_URL Message broker instance URL nats://localhost:4222
MF_HTTP_ADAPTER_CLIENT_TLS Flag that indicates if TLS should be turned on false
MF_HTTP_ADAPTER_CA_CERTS Path to trusted CAs in PEM format
MF_JAEGER_URL Jaeger server URL localhost:6831
MF_THINGS_AUTH_GRPC_URL Things service Auth gRPC URL localhost:8181
MF_THINGS_AUTH_GRPC_TIMEOUT Things service Auth gRPC request timeout in seconds 1s

Deployment

The service itself is distributed as Docker container. Check the http-adapter service section in docker-compose to see how service is deployed.

To start the service outside of the container, execute the following shell script:

# download the latest version of the service
git clone https://github.com/mainflux/mainflux

cd mainflux

# compile the http
make http

# copy binary to bin
make install

# set the environment variables and run the service
MF_BROKER_URL=[Message broker instance URL] \
MF_HTTP_ADAPTER_LOG_LEVEL=[HTTP Adapter Log Level] \
MF_HTTP_ADAPTER_PORT=[Service HTTP port] \
MF_HTTP_ADAPTER_CA_CERTS=[Path to trusted CAs in PEM format] \
MF_JAEGER_URL=[Jaeger server URL] \
MF_THINGS_AUTH_GRPC_URL=[Things service Auth gRPC URL] \
MF_THINGS_AUTH_GRPC_TIMEOUT=[Things service Auth gRPC request timeout in seconds] \
$GOBIN/mainflux-http

Setting MF_HTTP_ADAPTER_CA_CERTS expects a file in PEM format of trusted CAs. This will enable TLS against the Things gRPC endpoint trusting only those CAs that are provided.

Usage

HTTP Authorization request header contains the credentials to authenticate a Thing. The authorization header can be a plain Thing key or a Thing key encoded as a password for Basic Authentication. In case the Basic Authentication schema is used, the username is ignored. For more information about service capabilities and its usage, please check out the API documentation.