mirror of
https://github.com/mainflux/mainflux.git
synced 2025-05-06 19:29:15 +08:00

* Upgrade InfluxDB from 1.x to 2.x Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Influx DB configuration updated Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Connection to InfluxDBv2 Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Token cannot be created Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Connected to InfluxDB2 Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Connected to InfluxDB2 Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * InfluxDB v2 Consumer Implementation Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * quickfix Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Influxdb-Writer Unit Tests Update Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Consumer Update Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * InfluxDB Writer Tests Implemented Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * fix Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * InfluxDB Connection Check Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Resolving Remarks Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Resolved consumer-test remark Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * consumer-test slow working version Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * reader changes Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Consumer tests time issue fixed Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Eof warning fixed Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Resolved Reviews Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Makefile Fixed Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Resolved Reviews Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Resolved Reviews Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Resolved Reviews Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Resolved Reviews Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Resolved Reviews Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Readers Initial Setup Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * consumer json fix Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * readers simple version Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * readers simple version fix Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * readers simple version fix Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * readers simple version fix Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * readers simple version fix Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * readers simple version fix Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * readers simple version fix Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Influxdb Reader Parsers Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Influxdb Reader Parsers Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Influxdb Reader Parsers Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Bugfix and resolves comments. Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * one test fails Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * solved last page read Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * writers future time problem fixed Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * weird Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * weird Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Tests Passes Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Resolve Semaphore Issues Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * UUID comment on consumer tests resolved Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Inclusive from and Exclusive to Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Inclusive from and Exclusive to Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * 1 second limits Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * 1 second limits Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * resolved review Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * resolved review Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * resolved review Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * resolved review Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * fixed json time Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * fixed CI error Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * resolved request Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * resolved request Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * resolved requests Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * removed blank line Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * resolved comment Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * deleted unnecessary string builder Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * initial commit Signed-off-by: aryan <aryangodara03@gmail.com> * change influxdb docker image version. Signed-off-by: aryan <aryangodara03@gmail.com> * go mod and vendor fixing Signed-off-by: aryan <aryangodara03@gmail.com> * Upgrade InfluxDB from 1.x to 2.x Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Influx DB configuration updated Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Connection to InfluxDBv2 Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Token cannot be created Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Connected to InfluxDB2 Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Connected to InfluxDB2 Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * InfluxDB v2 Consumer Implementation Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * quickfix Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Influxdb-Writer Unit Tests Update Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Consumer Update Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * InfluxDB Writer Tests Implemented Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * fix Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * InfluxDB Connection Check Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Resolving Remarks Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Resolved consumer-test remark Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * consumer-test slow working version Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * reader changes Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Consumer tests time issue fixed Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Resolved Reviews Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Resolved Reviews Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Resolved Reviews Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Resolved Reviews Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Resolved Reviews Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Resolved Reviews Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Readers Initial Setup Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * consumer json fix Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * readers simple version Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * readers simple version fix Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * readers simple version fix Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * readers simple version fix Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * readers simple version fix Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * readers simple version fix Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * readers simple version fix Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Influxdb Reader Parsers Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Influxdb Reader Parsers Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Influxdb Reader Parsers Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Bugfix and resolves comments. Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * one test fails Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * solved last page read Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * writers future time problem fixed Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * weird Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * weird Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Tests Passes Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Resolve Semaphore Issues Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * UUID comment on consumer tests resolved Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Inclusive from and Exclusive to Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * Inclusive from and Exclusive to Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * 1 second limits Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * 1 second limits Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * resolved review Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * resolved review Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * resolved review Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * resolved review Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * fixed json time Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * fixed CI error Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * resolved request Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * resolved request Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * resolved requests Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * removed blank line Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * resolved comment Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * deleted unnecessary string builder Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> * initial commit Signed-off-by: aryan <aryangodara03@gmail.com> * change influxdb docker image version. Signed-off-by: aryan <aryangodara03@gmail.com> * go mod and vendor fixing Signed-off-by: aryan <aryangodara03@gmail.com> * go mod fixing Signed-off-by: aryan <aryangodara03@gmail.com> * make tests pass locally, fix errors Signed-off-by: aryan <aryangodara03@gmail.com> * rem unsused variables/consts Signed-off-by: aryan <aryangodara03@gmail.com> * go mod tidy vendor Signed-off-by: aryan <aryangodara03@gmail.com> * add env var and data types Signed-off-by: aryan <aryangodara03@gmail.com> * update influxdb version to latest (2.3 to 2.12) Signed-off-by: aryan <aryangodara03@gmail.com> * change time precision for message tests Signed-off-by: aryan <aryangodara03@gmail.com> * renamed influxdb2 to influxdata Signed-off-by: aryan <aryangodara03@gmail.com> * address remarks on PR Signed-off-by: aryan <aryangodara03@gmail.com> * update influxbd according to latest mf commit Signed-off-by: aryan <aryangodara03@gmail.com> * temp commit, rem before pushing Signed-off-by: aryan <aryangodara03@gmail.com> * update main files and cassandra-reader messages_tests Signed-off-by: aryan <aryangodara03@gmail.com> * fix name of logger while importing Signed-off-by: aryan <aryangodara03@gmail.com> * remove unnecessary print lines Signed-off-by: aryan <aryangodara03@gmail.com> * correct env var name Signed-off-by: aryan <aryangodara03@gmail.com> * change to async consume Signed-off-by: aryan <aryangodara03@gmail.com> * add option to switch bw sync and async Signed-off-by: aryan <aryangodara03@gmail.com> * test for both async and sync Signed-off-by: aryan <aryangodara03@gmail.com> * update consumer and add writeAPIs to config Signed-off-by: aryan <aryangodara03@gmail.com> * revert back to sync consuming Signed-off-by: aryan <aryangodara03@gmail.com> * temp fix for default timeout value Signed-off-by: aryan <aryangodara03@gmail.com> * set default timeout in config. Signed-off-by: aryan <aryangodara03@gmail.com> * remove unwanted env vars, add required ones. Signed-off-by: aryan <aryangodara03@gmail.com> * rem unused username password from config Signed-off-by: aryan <aryangodara03@gmail.com> * update readme, env vars, and remove grafana Signed-off-by: aryan <aryangodara03@gmail.com> * update readme Signed-off-by: aryan <aryangodara03@gmail.com> * fix typo Signed-off-by: aryan <aryangodara03@gmail.com> * update readme description. Signed-off-by: aryan <aryangodara03@gmail.com> * fix more typos. Signed-off-by: aryan <aryangodara03@gmail.com> * add link to official docs to readme. Signed-off-by: aryan <aryangodara03@gmail.com> --------- Signed-off-by: fatih <fatihdurmaz@sabanciuniv.edu> Signed-off-by: aryan <aryangodara03@gmail.com> Co-authored-by: fatih <fatihdurmaz@sabanciuniv.edu> Co-authored-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>
53 lines
1.4 KiB
Go
53 lines
1.4 KiB
Go
// Copyright 2020-2021 InfluxData, Inc.. All rights reserved.
|
|
// Use of this source code is governed by MIT
|
|
// license that can be found in the LICENSE file.
|
|
|
|
// Package gzip provides GZip related functionality
|
|
package gzip
|
|
|
|
import (
|
|
"compress/gzip"
|
|
"io"
|
|
"sync"
|
|
)
|
|
|
|
// ReadWaitCloser is ReadCloser that waits for finishing underlying reader
|
|
type ReadWaitCloser struct {
|
|
pipeReader *io.PipeReader
|
|
wg sync.WaitGroup
|
|
}
|
|
|
|
// Close closes underlying reader and waits for finishing operations
|
|
func (r *ReadWaitCloser) Close() error {
|
|
err := r.pipeReader.Close()
|
|
r.wg.Wait() // wait for the gzip goroutine finish
|
|
return err
|
|
}
|
|
|
|
// CompressWithGzip takes an io.Reader as input and pipes
|
|
// it through a gzip.Writer returning an io.Reader containing
|
|
// the gzipped data.
|
|
// An error is returned if passing data to the gzip.Writer fails
|
|
// this is shamelessly stolen from https://github.com/influxdata/telegraf
|
|
func CompressWithGzip(data io.Reader) (io.ReadCloser, error) {
|
|
pipeReader, pipeWriter := io.Pipe()
|
|
gzipWriter := gzip.NewWriter(pipeWriter)
|
|
|
|
rc := &ReadWaitCloser{
|
|
pipeReader: pipeReader,
|
|
}
|
|
|
|
rc.wg.Add(1)
|
|
var err error
|
|
go func() {
|
|
_, err = io.Copy(gzipWriter, data)
|
|
gzipWriter.Close()
|
|
// subsequent reads from the read half of the pipe will
|
|
// return no bytes and the error err, or EOF if err is nil.
|
|
pipeWriter.CloseWithError(err)
|
|
rc.wg.Done()
|
|
}()
|
|
|
|
return pipeReader, err
|
|
}
|