1
0
mirror of https://github.com/mainflux/mainflux.git synced 2025-04-27 13:48:49 +08:00

83 lines
2.1 KiB
Go
Raw Normal View History

// Copyright (c) Mainflux
// SPDX-License-Identifier: Apache-2.0
package writers
import (
"fmt"
NOISSUE - Adding subtopics filtering in writer services (#1072) * Add feature of filtering by subtopics in writer Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Fix mistake Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Refactoring writer sevices Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Rename variables related to filter (channels & subtopics) Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Set default value of filtering when configuration file doesn't exist Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Add a blank line at the end of the file Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Refactor loading filters configuration (moving into writer package, merge both loading methods & returning error) Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Remove useless log Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Change type of variables (channels & subtopics) and simplify loading method Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Add logging error when loading filters Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Simplify return configuration in loading method Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Merge both filter files into one file Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Move loading subjects into writer package Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Add subscribe to selected subjects Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Edit README of writer services Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Keep only subscribe loop Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Use full NATS subjects Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Edit comment in subjects files Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> Co-authored-by: Drasko DRASKOVIC <drasko.draskovic@gmail.com>
2020-03-30 18:24:27 +02:00
"io/ioutil"
NOISSUE - Adding subtopics filtering in writer services (#1072) * Add feature of filtering by subtopics in writer Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Fix mistake Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Refactoring writer sevices Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Rename variables related to filter (channels & subtopics) Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Set default value of filtering when configuration file doesn't exist Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Add a blank line at the end of the file Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Refactor loading filters configuration (moving into writer package, merge both loading methods & returning error) Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Remove useless log Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Change type of variables (channels & subtopics) and simplify loading method Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Add logging error when loading filters Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Simplify return configuration in loading method Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Merge both filter files into one file Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Move loading subjects into writer package Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Add subscribe to selected subjects Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Edit README of writer services Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Keep only subscribe loop Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Use full NATS subjects Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Edit comment in subjects files Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> Co-authored-by: Drasko DRASKOVIC <drasko.draskovic@gmail.com>
2020-03-30 18:24:27 +02:00
"github.com/BurntSushi/toml"
"github.com/mainflux/mainflux/logger"
"github.com/mainflux/mainflux/pkg/errors"
"github.com/mainflux/mainflux/pkg/messaging"
pubsub "github.com/mainflux/mainflux/pkg/messaging/nats"
"github.com/mainflux/mainflux/pkg/transformers"
)
NOISSUE - Adding subtopics filtering in writer services (#1072) * Add feature of filtering by subtopics in writer Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Fix mistake Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Refactoring writer sevices Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Rename variables related to filter (channels & subtopics) Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Set default value of filtering when configuration file doesn't exist Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Add a blank line at the end of the file Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Refactor loading filters configuration (moving into writer package, merge both loading methods & returning error) Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Remove useless log Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Change type of variables (channels & subtopics) and simplify loading method Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Add logging error when loading filters Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Simplify return configuration in loading method Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Merge both filter files into one file Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Move loading subjects into writer package Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Add subscribe to selected subjects Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Edit README of writer services Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Keep only subscribe loop Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Use full NATS subjects Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Edit comment in subjects files Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> Co-authored-by: Drasko DRASKOVIC <drasko.draskovic@gmail.com>
2020-03-30 18:24:27 +02:00
var (
NOISSUE - Refactor messaging (#1141) * Refactor messaging Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Rename SubscribeHandler to MessageHandler Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove `Auth` event logs Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update message pubsub APi Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix topics handling Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update CoAP adapter Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update Twins service Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update LoRa adapter Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update OPC UA adapter Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove broker package Package `broker` is conceptually renamed to package `nats`. Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update makefile Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Add comment explanation Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix MQTT adapter Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix typo Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Move NATS pub/sub implementation to pubsub pkg Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove an empty line in main methods Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Move messaging-related code to messaging package Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix Twins mocks Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Change Occurred back to Created Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix tranformer test Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix message proto commands Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Replace string literal with constant Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove alias from main method Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Change messaging pubsub alias Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Rename occured to created Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Handle NATS connection in the NATS PubSub Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Rename n to pub/pubSub Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix typos Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>
2020-04-28 11:02:35 +02:00
errOpenConfFile = errors.New("unable to open configuration file")
errParseConfFile = errors.New("unable to parse configuration file")
errMessageConversion = errors.New("error conversing transformed messages")
NOISSUE - Adding subtopics filtering in writer services (#1072) * Add feature of filtering by subtopics in writer Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Fix mistake Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Refactoring writer sevices Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Rename variables related to filter (channels & subtopics) Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Set default value of filtering when configuration file doesn't exist Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Add a blank line at the end of the file Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Refactor loading filters configuration (moving into writer package, merge both loading methods & returning error) Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Remove useless log Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Change type of variables (channels & subtopics) and simplify loading method Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Add logging error when loading filters Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Simplify return configuration in loading method Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Merge both filter files into one file Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Move loading subjects into writer package Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Add subscribe to selected subjects Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Edit README of writer services Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Keep only subscribe loop Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Use full NATS subjects Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Edit comment in subjects files Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> Co-authored-by: Drasko DRASKOVIC <drasko.draskovic@gmail.com>
2020-03-30 18:24:27 +02:00
)
type consumer struct {
MF-919 - Mainflux message updates (#924) * Remove RawMessage Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove Normalizer Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update tests Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Replace normalizer with senml-transformer Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Rename Transformer interface and package Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update docs Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove SenML transformer service Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove SenML Protobuf support Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix readers Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix writers tests Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Refactor tests and remove normalizer Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update docs Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Reanme Service interface to Transformer Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Use msg instead of rawmsg Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove rawMsg from Aedes code Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix VerneMQ files Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove RawMessage code Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix missing subtopic return Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove remaining RawMessage reference Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix formatting Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix readers and writers tests Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Rename SenML transformer variables Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix readers and writers tests constants Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>
2019-11-05 11:57:16 +01:00
repo MessageRepository
transformer transformers.Transformer
NOISSUE - Adding subtopics filtering in writer services (#1072) * Add feature of filtering by subtopics in writer Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Fix mistake Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Refactoring writer sevices Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Rename variables related to filter (channels & subtopics) Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Set default value of filtering when configuration file doesn't exist Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Add a blank line at the end of the file Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Refactor loading filters configuration (moving into writer package, merge both loading methods & returning error) Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Remove useless log Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Change type of variables (channels & subtopics) and simplify loading method Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Add logging error when loading filters Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Simplify return configuration in loading method Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Merge both filter files into one file Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Move loading subjects into writer package Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Add subscribe to selected subjects Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Edit README of writer services Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Keep only subscribe loop Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Use full NATS subjects Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Edit comment in subjects files Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> Co-authored-by: Drasko DRASKOVIC <drasko.draskovic@gmail.com>
2020-03-30 18:24:27 +02:00
logger logger.Logger
}
MF-919 - Mainflux message updates (#924) * Remove RawMessage Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove Normalizer Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update tests Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Replace normalizer with senml-transformer Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Rename Transformer interface and package Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update docs Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove SenML transformer service Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove SenML Protobuf support Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix readers Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix writers tests Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Refactor tests and remove normalizer Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update docs Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Reanme Service interface to Transformer Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Use msg instead of rawmsg Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove rawMsg from Aedes code Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix VerneMQ files Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove RawMessage code Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix missing subtopic return Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove remaining RawMessage reference Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix formatting Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix readers and writers tests Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Rename SenML transformer variables Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix readers and writers tests constants Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>
2019-11-05 11:57:16 +01:00
// Start method starts consuming messages received from NATS.
// This method transforms messages to SenML format before
// using MessageRepository to store them.
MF-1264 - Add support for JSON readers (#1295) * MF-1254 - Create universal JSON writer (#1260) Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Add JSON support to Readers Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix Influx Reader tests Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix messages format query Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix Postgres reader Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix Cassandra Readers and writers Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix Mongo reader Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Extract utility method to the JSON transformer Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix Influx and Postgres count Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Update JSON transformer Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix Influxdb Reader total count Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Refactor init.go for Cassandra writer Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Create a Payload type Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Add comments for defaults Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix variable declarations Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Replace interface{} with a new type Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Don't set channel just to overwrite it later Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix range search Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Rename Messages field Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> Co-authored-by: Manuel Imperiale <manuel.imperiale@gmail.com>
2020-12-30 15:43:04 +01:00
func Start(sub messaging.Subscriber, repo MessageRepository, transformer transformers.Transformer, subjectsCfgPath string, logger logger.Logger) error {
c := consumer{
MF-919 - Mainflux message updates (#924) * Remove RawMessage Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove Normalizer Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update tests Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Replace normalizer with senml-transformer Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Rename Transformer interface and package Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update docs Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove SenML transformer service Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove SenML Protobuf support Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix readers Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix writers tests Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Refactor tests and remove normalizer Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update docs Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Reanme Service interface to Transformer Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Use msg instead of rawmsg Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove rawMsg from Aedes code Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix VerneMQ files Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove RawMessage code Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix missing subtopic return Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove remaining RawMessage reference Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix formatting Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix readers and writers tests Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Rename SenML transformer variables Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix readers and writers tests constants Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>
2019-11-05 11:57:16 +01:00
repo: repo,
transformer: transformer,
logger: logger,
}
subjects, err := loadSubjectsConfig(subjectsCfgPath)
NOISSUE - Adding subtopics filtering in writer services (#1072) * Add feature of filtering by subtopics in writer Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Fix mistake Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Refactoring writer sevices Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Rename variables related to filter (channels & subtopics) Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Set default value of filtering when configuration file doesn't exist Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Add a blank line at the end of the file Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Refactor loading filters configuration (moving into writer package, merge both loading methods & returning error) Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Remove useless log Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Change type of variables (channels & subtopics) and simplify loading method Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Add logging error when loading filters Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Simplify return configuration in loading method Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Merge both filter files into one file Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Move loading subjects into writer package Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Add subscribe to selected subjects Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Edit README of writer services Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Keep only subscribe loop Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Use full NATS subjects Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Edit comment in subjects files Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> Co-authored-by: Drasko DRASKOVIC <drasko.draskovic@gmail.com>
2020-03-30 18:24:27 +02:00
if err != nil {
logger.Warn(fmt.Sprintf("Failed to load subjects: %s", err))
}
for _, subject := range subjects {
NOISSUE - Refactor messaging (#1141) * Refactor messaging Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Rename SubscribeHandler to MessageHandler Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove `Auth` event logs Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update message pubsub APi Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix topics handling Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update CoAP adapter Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update Twins service Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update LoRa adapter Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update OPC UA adapter Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove broker package Package `broker` is conceptually renamed to package `nats`. Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update makefile Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Add comment explanation Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix MQTT adapter Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix typo Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Move NATS pub/sub implementation to pubsub pkg Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove an empty line in main methods Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Move messaging-related code to messaging package Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix Twins mocks Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Change Occurred back to Created Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix tranformer test Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix message proto commands Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Replace string literal with constant Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove alias from main method Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Change messaging pubsub alias Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Rename occured to created Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Handle NATS connection in the NATS PubSub Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Rename n to pub/pubSub Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix typos Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>
2020-04-28 11:02:35 +02:00
if err := sub.Subscribe(subject, c.handler); err != nil {
NOISSUE - Adding subtopics filtering in writer services (#1072) * Add feature of filtering by subtopics in writer Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Fix mistake Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Refactoring writer sevices Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Rename variables related to filter (channels & subtopics) Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Set default value of filtering when configuration file doesn't exist Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Add a blank line at the end of the file Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Refactor loading filters configuration (moving into writer package, merge both loading methods & returning error) Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Remove useless log Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Change type of variables (channels & subtopics) and simplify loading method Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Add logging error when loading filters Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Simplify return configuration in loading method Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Merge both filter files into one file Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Move loading subjects into writer package Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Add subscribe to selected subjects Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Edit README of writer services Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Keep only subscribe loop Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Use full NATS subjects Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Edit comment in subjects files Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> Co-authored-by: Drasko DRASKOVIC <drasko.draskovic@gmail.com>
2020-03-30 18:24:27 +02:00
return err
}
}
return nil
}
NOISSUE - Refactor messaging (#1141) * Refactor messaging Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Rename SubscribeHandler to MessageHandler Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove `Auth` event logs Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update message pubsub APi Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix topics handling Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update CoAP adapter Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update Twins service Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update LoRa adapter Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update OPC UA adapter Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove broker package Package `broker` is conceptually renamed to package `nats`. Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update makefile Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Add comment explanation Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix MQTT adapter Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix typo Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Move NATS pub/sub implementation to pubsub pkg Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove an empty line in main methods Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Move messaging-related code to messaging package Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix Twins mocks Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Change Occurred back to Created Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix tranformer test Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix message proto commands Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Replace string literal with constant Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove alias from main method Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Change messaging pubsub alias Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Rename occured to created Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Handle NATS connection in the NATS PubSub Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Rename n to pub/pubSub Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix typos Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>
2020-04-28 11:02:35 +02:00
func (c *consumer) handler(msg messaging.Message) error {
MF-919 - Mainflux message updates (#924) * Remove RawMessage Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove Normalizer Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update tests Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Replace normalizer with senml-transformer Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Rename Transformer interface and package Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update docs Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove SenML transformer service Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove SenML Protobuf support Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix readers Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix writers tests Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Refactor tests and remove normalizer Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update docs Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Reanme Service interface to Transformer Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Use msg instead of rawmsg Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove rawMsg from Aedes code Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix VerneMQ files Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove RawMessage code Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix missing subtopic return Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove remaining RawMessage reference Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix formatting Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix readers and writers tests Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Rename SenML transformer variables Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix readers and writers tests constants Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>
2019-11-05 11:57:16 +01:00
t, err := c.transformer.Transform(msg)
MF-780 - Use Normalizer as a lib (#915) * Use Normalizer as a lib To normalize messages on the consumer side, Normalizer is moved to the internal pkgs. Writers being message consumers are modified to do message normalization instead of subscribing to normalized messages subject. Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix logging middleware for readers and writers Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove normalizer interface Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Use Normalizer in writers As we agreed on #919, we'll use normalizer as an interface and provide the default SenML implementation. Because of that, Normalizer is removed from `internal` and we'll use the project structure proposed in the aforementioned issue. Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix tests Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove unused batch settings from influxDB reader Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update docs Move Normalizer service to `addons`. Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Rename channels input topic Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update Noramlizer docs Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove commented code Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update readers logging Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update addons docker-compose files Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update topcis explanations Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>
2019-10-31 14:04:47 +01:00
if err != nil {
NOISSUE - Refactor messaging (#1141) * Refactor messaging Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Rename SubscribeHandler to MessageHandler Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove `Auth` event logs Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update message pubsub APi Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix topics handling Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update CoAP adapter Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update Twins service Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update LoRa adapter Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update OPC UA adapter Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove broker package Package `broker` is conceptually renamed to package `nats`. Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update makefile Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Add comment explanation Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix MQTT adapter Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix typo Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Move NATS pub/sub implementation to pubsub pkg Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove an empty line in main methods Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Move messaging-related code to messaging package Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix Twins mocks Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Change Occurred back to Created Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix tranformer test Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix message proto commands Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Replace string literal with constant Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove alias from main method Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Change messaging pubsub alias Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Rename occured to created Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Handle NATS connection in the NATS PubSub Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Rename n to pub/pubSub Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix typos Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>
2020-04-28 11:02:35 +02:00
return err
}
MF-1264 - Add support for JSON readers (#1295) * MF-1254 - Create universal JSON writer (#1260) Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Add JSON support to Readers Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix Influx Reader tests Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix messages format query Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix Postgres reader Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix Cassandra Readers and writers Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix Mongo reader Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Extract utility method to the JSON transformer Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix Influx and Postgres count Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Update JSON transformer Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix Influxdb Reader total count Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Refactor init.go for Cassandra writer Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Create a Payload type Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Add comments for defaults Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix variable declarations Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Replace interface{} with a new type Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Don't set channel just to overwrite it later Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix range search Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Rename Messages field Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> Co-authored-by: Manuel Imperiale <manuel.imperiale@gmail.com>
2020-12-30 15:43:04 +01:00
return c.repo.Save(t)
}
NOISSUE - Adding subtopics filtering in writer services (#1072) * Add feature of filtering by subtopics in writer Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Fix mistake Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Refactoring writer sevices Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Rename variables related to filter (channels & subtopics) Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Set default value of filtering when configuration file doesn't exist Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Add a blank line at the end of the file Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Refactor loading filters configuration (moving into writer package, merge both loading methods & returning error) Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Remove useless log Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Change type of variables (channels & subtopics) and simplify loading method Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Add logging error when loading filters Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Simplify return configuration in loading method Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Merge both filter files into one file Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Move loading subjects into writer package Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Add subscribe to selected subjects Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Edit README of writer services Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Keep only subscribe loop Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Use full NATS subjects Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Edit comment in subjects files Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> Co-authored-by: Drasko DRASKOVIC <drasko.draskovic@gmail.com>
2020-03-30 18:24:27 +02:00
type filterConfig struct {
MF-1264 - Add support for JSON readers (#1295) * MF-1254 - Create universal JSON writer (#1260) Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Add JSON support to Readers Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix Influx Reader tests Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix messages format query Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix Postgres reader Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix Cassandra Readers and writers Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix Mongo reader Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Extract utility method to the JSON transformer Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix Influx and Postgres count Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Update JSON transformer Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix Influxdb Reader total count Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Refactor init.go for Cassandra writer Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Create a Payload type Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Add comments for defaults Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix variable declarations Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Replace interface{} with a new type Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Don't set channel just to overwrite it later Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix range search Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Rename Messages field Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> Co-authored-by: Manuel Imperiale <manuel.imperiale@gmail.com>
2020-12-30 15:43:04 +01:00
Filter []string `toml:"filter"`
NOISSUE - Adding subtopics filtering in writer services (#1072) * Add feature of filtering by subtopics in writer Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Fix mistake Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Refactoring writer sevices Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Rename variables related to filter (channels & subtopics) Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Set default value of filtering when configuration file doesn't exist Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Add a blank line at the end of the file Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Refactor loading filters configuration (moving into writer package, merge both loading methods & returning error) Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Remove useless log Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Change type of variables (channels & subtopics) and simplify loading method Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Add logging error when loading filters Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Simplify return configuration in loading method Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Merge both filter files into one file Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Move loading subjects into writer package Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Add subscribe to selected subjects Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Edit README of writer services Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Keep only subscribe loop Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Use full NATS subjects Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Edit comment in subjects files Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> Co-authored-by: Drasko DRASKOVIC <drasko.draskovic@gmail.com>
2020-03-30 18:24:27 +02:00
}
type subjectsConfig struct {
Subjects filterConfig `toml:"subjects"`
}
func loadSubjectsConfig(subjectsConfigPath string) ([]string, error) {
NOISSUE - Adding subtopics filtering in writer services (#1072) * Add feature of filtering by subtopics in writer Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Fix mistake Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Refactoring writer sevices Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Rename variables related to filter (channels & subtopics) Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Set default value of filtering when configuration file doesn't exist Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Add a blank line at the end of the file Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Refactor loading filters configuration (moving into writer package, merge both loading methods & returning error) Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Remove useless log Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Change type of variables (channels & subtopics) and simplify loading method Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Add logging error when loading filters Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Simplify return configuration in loading method Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Merge both filter files into one file Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Move loading subjects into writer package Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Add subscribe to selected subjects Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Edit README of writer services Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Keep only subscribe loop Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Use full NATS subjects Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Edit comment in subjects files Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> Co-authored-by: Drasko DRASKOVIC <drasko.draskovic@gmail.com>
2020-03-30 18:24:27 +02:00
data, err := ioutil.ReadFile(subjectsConfigPath)
if err != nil {
NOISSUE - Refactor messaging (#1141) * Refactor messaging Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Rename SubscribeHandler to MessageHandler Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove `Auth` event logs Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update message pubsub APi Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix topics handling Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update CoAP adapter Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update Twins service Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update LoRa adapter Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update OPC UA adapter Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove broker package Package `broker` is conceptually renamed to package `nats`. Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update makefile Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Add comment explanation Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix MQTT adapter Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix typo Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Move NATS pub/sub implementation to pubsub pkg Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove an empty line in main methods Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Move messaging-related code to messaging package Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix Twins mocks Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Change Occurred back to Created Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix tranformer test Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix message proto commands Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Replace string literal with constant Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove alias from main method Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Change messaging pubsub alias Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Rename occured to created Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Handle NATS connection in the NATS PubSub Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Rename n to pub/pubSub Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix typos Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>
2020-04-28 11:02:35 +02:00
return []string{pubsub.SubjectAllChannels}, errors.Wrap(errOpenConfFile, err)
NOISSUE - Adding subtopics filtering in writer services (#1072) * Add feature of filtering by subtopics in writer Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Fix mistake Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Refactoring writer sevices Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Rename variables related to filter (channels & subtopics) Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Set default value of filtering when configuration file doesn't exist Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Add a blank line at the end of the file Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Refactor loading filters configuration (moving into writer package, merge both loading methods & returning error) Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Remove useless log Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Change type of variables (channels & subtopics) and simplify loading method Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Add logging error when loading filters Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Simplify return configuration in loading method Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Merge both filter files into one file Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Move loading subjects into writer package Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Add subscribe to selected subjects Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Edit README of writer services Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Keep only subscribe loop Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Use full NATS subjects Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> * Edit comment in subjects files Signed-off-by: Jonathan Dreyer <jonathan.dreyer@cleanenergie.ch> Co-authored-by: Drasko DRASKOVIC <drasko.draskovic@gmail.com>
2020-03-30 18:24:27 +02:00
}
var subjectsCfg subjectsConfig
if err := toml.Unmarshal(data, &subjectsCfg); err != nil {
NOISSUE - Refactor messaging (#1141) * Refactor messaging Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Rename SubscribeHandler to MessageHandler Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove `Auth` event logs Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update message pubsub APi Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix topics handling Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update CoAP adapter Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update Twins service Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update LoRa adapter Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update OPC UA adapter Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove broker package Package `broker` is conceptually renamed to package `nats`. Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Update makefile Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Add comment explanation Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix MQTT adapter Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix typo Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Move NATS pub/sub implementation to pubsub pkg Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove an empty line in main methods Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Move messaging-related code to messaging package Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix Twins mocks Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Change Occurred back to Created Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix tranformer test Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix message proto commands Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Replace string literal with constant Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Remove alias from main method Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Change messaging pubsub alias Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Rename occured to created Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Handle NATS connection in the NATS PubSub Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Rename n to pub/pubSub Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com> * Fix typos Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>
2020-04-28 11:02:35 +02:00
return []string{pubsub.SubjectAllChannels}, errors.Wrap(errParseConfFile, err)
}
MF-1264 - Add support for JSON readers (#1295) * MF-1254 - Create universal JSON writer (#1260) Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Add JSON support to Readers Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix Influx Reader tests Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix messages format query Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix Postgres reader Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix Cassandra Readers and writers Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix Mongo reader Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Extract utility method to the JSON transformer Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix Influx and Postgres count Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Update JSON transformer Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix Influxdb Reader total count Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Refactor init.go for Cassandra writer Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Create a Payload type Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Add comments for defaults Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix variable declarations Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Replace interface{} with a new type Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Don't set channel just to overwrite it later Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Fix range search Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> * Rename Messages field Signed-off-by: dusanb94 <dusan.borovcanin@mainflux.com> Co-authored-by: Manuel Imperiale <manuel.imperiale@gmail.com>
2020-12-30 15:43:04 +01:00
return subjectsCfg.Subjects.Filter, nil
}