1
0
mirror of https://github.com/mainflux/mainflux.git synced 2025-04-24 13:48:49 +08:00
b1ackd0t 43a263d7da
NOISSUE - Introduce NATS Jetstream as Default ES (#1907)
* Refactor message broker implementation

This commit refactors the Nats message broker implementation to include pubsub options. These changes include:

- Adding `Option` func that takes in the URL and prefix
- Implement `WithStream` option which can create a different stream for nats stream
- Implement `WithExchange` option which can create a different exchaange for rabbitmq channel
- Implement `WithPrefix` option which allows to you change the publisher prefix

These changes improve the organization and readability of the codebase.

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
Signed-off-by: Rodney Osodo <28790446+rodneyosodo@users.noreply.github.com>
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>

* Use redis URL to configure username, password and db

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
Signed-off-by: Rodney Osodo <28790446+rodneyosodo@users.noreply.github.com>
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>

* Make event store configurable on dev deployment

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
Signed-off-by: Rodney Osodo <28790446+rodneyosodo@users.noreply.github.com>
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>

* Fix adds options to messaging `PubSub` interface

Adding options to PubSub interface allows the use of messaging
package to do es.

The changes in this commit ensure that the code handles errors
properly and provides more informative error messages when
encountering unexpected types.

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
Signed-off-by: Rodney Osodo <28790446+rodneyosodo@users.noreply.github.com>
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>

* Add NATS event publisher implementation

This commit adds the implementation of the NATS event publisher.

The NATS event publisher is responsible for publishing events
to a NATS messaging system. It uses the `messaging`
package to interact with the messaging system.

The implementation includes the following features:

- Publishing events to NATS using the `Publish` method.
- Marshaling events to JSON before publishing.
- Setting the message subject and headers based on the event.
- Handling errors during publishing.

This implementation is built with the `!rabbitmq` build tag,
which means it will only be compiled if the `rabbitmq` build tag
is not present.

The NATS event publisher is part of the Mainflux events package
and provides support for the Mainflux NATS events source service functionality.

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
Signed-off-by: Rodney Osodo <28790446+rodneyosodo@users.noreply.github.com>
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>

* Add RabbitMQ event publisher implementation

This commit adds the implementation of the RabbitMQ event publisher.

The RabbitMQ event publisher is responsible for publishing events
to a RabbitMQ messaging system. It uses the `messaging`
package to interact with the messaging system.

The implementation includes the following features:

- Publishing events to RabbitMQ using the `Publish` method.
- Marshaling events to JSON before publishing.
- Setting the message subject and headers based on the event.
- Handling errors during publishing.

This implementation is built with the `rabbitmq` build tag,
which means it will only be compiled if the `rabbitmq` build tag
is present.

The RabbitMQ event publisher is part of the Mainflux events package
and provides support for the Mainflux RabbitMQ events source service functionality.

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
Signed-off-by: Rodney Osodo <28790446+rodneyosodo@users.noreply.github.com>
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>

* Add configurable implementation for events store

This commit adds a new file `brokers_*.go` which contains the implementation for the different event store. The file includes functions for creating a new publisher and subscriber using different es store.

This commit also includes an `init` function that logs a message indicating that the binary was built using the respective package as the events store.

The purpose of this commit is to add support for alternative events store.

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
Signed-off-by: Rodney Osodo <28790446+rodneyosodo@users.noreply.github.com>
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>

* Fix build flags

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
Signed-off-by: Rodney Osodo <28790446+rodneyosodo@users.noreply.github.com>
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>

* Refactor Makefile and Semaphore configuration

The Makefile has been refactored to include the `MF_ES_STORE_TYPE` tag in the `go build` command. Additionally, the Semaphore configuration has been updated to include a new task for compiling with Redis as the broker type.

This commit addresses the need to compile the codebase with Redis as the event store type and includes the necessary changes in the Makefile and Semaphore configuration.

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
Signed-off-by: Rodney Osodo <28790446+rodneyosodo@users.noreply.github.com>
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>

* Reduced due to memory on testing

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
Signed-off-by: Rodney Osodo <28790446+rodneyosodo@users.noreply.github.com>
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>

* Fix tests for es

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
Signed-off-by: Rodney Osodo <28790446+rodneyosodo@users.noreply.github.com>
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>

* Fix grammar

Co-authored-by: Sammy Kerata Oina <44265300+SammyOina@users.noreply.github.com>
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
Signed-off-by: Rodney Osodo <28790446+rodneyosodo@users.noreply.github.com>
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>

* Fix linting

Signed-off-by: Rodney Osodo <28790446+rodneyosodo@users.noreply.github.com>
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>

* feat(docker): update environment variables for message broker

The commit updates the environment variable `MF_ES_STORE_TYPE` in the `docker/.env` file. The variable is changed from `${MF_MQTT_BROKER_TYPE}` to `${MF_MESSAGE_BROKER_TYPE}` to accurately reflect the type of message broker being used. This change ensures that the correct message broker is configured for the Event Store.

Signed-off-by: Rodney Osodo <28790446+rodneyosodo@users.noreply.github.com>
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>

* feat: Update docker environment variables

- Removed the unused MF_ES_URL variable in the .env file
- Updated the MF_ES_STORE_TYPE and MF_ES_STORE_URL variables in the .env file to match the MF_MESSAGE_BROKER_TYPE and MF_NATS_URL variables respectively

Signed-off-by: Rodney Osodo <28790446+rodneyosodo@users.noreply.github.com>
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>

* Fix after rebase

Signed-off-by: rodneyosodo <blackd0t@protonmail.com>

* Add godocs for option parameters for brokers

Signed-off-by: rodneyosodo <blackd0t@protonmail.com>

* pass by value exchange and prefix names

Signed-off-by: rodneyosodo <blackd0t@protonmail.com>

* Rename option functions

Signed-off-by: rodneyosodo <blackd0t@protonmail.com>

* move variables to constants

Signed-off-by: rodneyosodo <blackd0t@protonmail.com>

* fix: option example comment

Signed-off-by: rodneyosodo <blackd0t@protonmail.com>

---------

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
Signed-off-by: Rodney Osodo <28790446+rodneyosodo@users.noreply.github.com>
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
Co-authored-by: Sammy Kerata Oina <44265300+SammyOina@users.noreply.github.com>
2023-10-24 17:22:53 +02:00

81 lines
2.4 KiB
Go

// Copyright (c) Mainflux
// SPDX-License-Identifier: Apache-2.0
package messaging
import "context"
type DeliveryPolicy uint8
const (
// DeliverNewPolicy will only deliver new messages that are sent after the consumer is created.
// This is the default policy.
DeliverNewPolicy DeliveryPolicy = iota
// DeliverAllPolicy starts delivering messages from the very beginning of a stream.
DeliverAllPolicy
)
// Publisher specifies message publishing API.
type Publisher interface {
// Publishes message to the stream.
Publish(ctx context.Context, topic string, msg *Message) error
// Close gracefully closes message publisher's connection.
Close() error
}
// MessageHandler represents Message handler for Subscriber.
type MessageHandler interface {
// Handle handles messages passed by underlying implementation.
Handle(msg *Message) error
// Cancel is used for cleanup during unsubscribing and it's optional.
Cancel() error
}
type SubscriberConfig struct {
ID string
Topic string
Handler MessageHandler
DeliveryPolicy DeliveryPolicy
}
// Subscriber specifies message subscription API.
type Subscriber interface {
// Subscribe subscribes to the message stream and consumes messages.
Subscribe(ctx context.Context, cfg SubscriberConfig) error
// Unsubscribe unsubscribes from the message stream and
// stops consuming messages.
Unsubscribe(ctx context.Context, id, topic string) error
// Close gracefully closes message subscriber's connection.
Close() error
}
// PubSub represents aggregation interface for publisher and subscriber.
type PubSub interface {
Publisher
Subscriber
}
// Option represents optional configuration for message broker.
//
// This is used to provide optional configuration parameters to the
// underlying publisher and pubsub implementation so that it can be
// configured to meet the specific needs.
//
// For example, it can be used to set the message prefix so that
// brokers can be used for event sourcing as well as internal message broker.
// Using value of type interface is not recommended but is the most suitable
// for this use case as options should be compiled with respect to the
// underlying broker which can either be RabbitMQ or NATS.
//
// The example below shows how to set the prefix and jetstream stream for NATS.
//
// Example:
//
// broker.NewPublisher(ctx, url, broker.Prefix(eventsPrefix), broker.JSStream(js))
type Option func(vals interface{}) error