1
0
mirror of https://github.com/mainflux/mainflux.git synced 2025-04-28 13:48:49 +08:00
Dušan Borovčanin 22fc26b375 MF-513 - Add Bootstrapping service (#524)
* Introduce Config response for bootstrap procedure

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Add inital service implementation

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Enable status change

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Fix logger import

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Update BSS to send config in valid format

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Use ConfigReader to create valid format response

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Update config retrieval error handle

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Enable Thing deletion API

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Add API support for fetching Thing by ID

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Add list Things endpoint

Update database schema

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Use MF API to update status

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Use Channels list

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Fix reading Thing from the database

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Create Mainflux Thing when adding new Thing to BS

Create MF Thing as soon as Bootstrap service thing is added. There are 2
main reasons to create Thing when adding a new BS Thing over creating
Thing on bootstrapping:
1) On bootstrapping time, user JWT will not be sent as a part of
request, so there is no mechanism to send a valid API call to Mainflux.
2) This way, Bootstrap service will be in sync with Mainlux: each Thing
existing in BS will also be in Mainflux.

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Add Thing update

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Remove API key from BS service

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Improve channels update algorithm

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Refactor code

Remove unused fields, comment code and simplfy some method signatures.

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Remove Identity Provider and use gRPC

Update dependencies

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Add external auth key

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Update BS config reader

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Update docker-compose

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Update env variable read

Add MQTT password to bootstrap response.

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Update response fields and tags

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Remove status check

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Enable BS of active Things

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Add NewThing state

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Rename Status to State

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Update README.md

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Add filterng

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Update List endpoint

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Fix Database query

Remove copyright headers.

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Add filter type

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Gateway provisioning (1.d)

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Update self-bootstrapping feature

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Add mocks

Update dependencies to the newest Mainflux version.

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Add thing service tests

Mocks fix.
Some of the service code intentionally left untested due to possible
changes in future.
Fix copyright headers and update Mainflux and other dependencies.

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Use name "Config" instead of "Thing"

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Refactor code

Remove commented code.
Fix typo.
Remove unused exported error.

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Simplify service tests

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Remove Assign method

Raise test coverage.

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Update database schema

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Store unknown bootstrap attempts

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Update unknown bootstrap handling

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Update naming

Fix uses of `Thing` in DB and `api` package.

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Add endpoint tests

Currently, only test for adding a new Config are implemented.

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Add initialization of DB tests

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Add DB tests

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Update readme file

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Add API docs

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Remove Mainflux from vendor

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Add licence headers

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Fix service and endpoint tests

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Raise test coverage

Remove unused repsonse type.

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Update build and deployment

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Update API docs

Fix typo.

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Update imports formatting

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Make state response empty

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Raise test coverage

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Update API docs

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Update readme file

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Use uuid as a primary key

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Use Mainflux ID

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Remove `Created` state.

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Move State to separate file

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Add Things prefix

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Update API and API docs

Be consistent in API naming and add some useful comments.

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Refactor repository implementation

Cleanup code, make it more readable. Fix missing drop in migrations.

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Use "cfg" insted of "thing"

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Fix tests

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Update tables names

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>
2019-01-09 15:42:23 +01:00
..

BOOTSTRAP SERVICE

New devices need to be configured properly and connected to the Mainflux. Bootstrap service is used in order to accomplish that. This service provides the following features: 1) Creating new Mainflux Things 2) Providing basic configuration for the newly created Things 3) Enabling/disabling Things

Pre-provisioning a new Thing is as simple as sending Thing data to the Bootstrap service. Once the Thing is active, it sends a request for initial config to Bootstrap service. Once the Thing is bootstrapped, its possible to add it to the whitelist, so that it can exchange messages using Mainflux. Bootstrapping does not implicitly enable Things, it has to be done manually.

In order to bootstrap successfully, the Thing needs to send bootstrapping request to the specific URL, as well as a secret key. This key and URL are pre-provisioned during manufacturing process. If the Thing is provisioned on the Bootstrap service side, corresponding configuration will be sent as a response. Otherwise, the Thing will be saved so that it can be provisioned later.

Thing Configuration

Thing Configuration consists of two logical parts: custom configuration (that can be interpreted by the Thing itself) and Mainflux-related configuration. Mainflux config contains: 1) corresponding Mainflux Thing ID 2) corresponding Mainflux Thing key 3) list of the Mainflux channels the Thing is connected to

Note: list of channels contains IDs of the Mainflux channels. These channels are pre-provisioned on the Mainflux side and, unlike corresponding Mainflux Thing, Bootstrap service does not create Mainflux Channels.

Enabling and disabling Thing (adding Thing to/from whitelist) is as simple as connecting corresponding Mainflux Thing to the given list of Channels. Configuration keeps state of the Thing:

State What it means
Inactive Thing is created, but isn't enabled
Active Thing is able to communicate using Mainflux

Switching between states Active and Inactive enables and disables Thing, respectively.

Configuration

The service is configured using the environment variables presented in the following table. Note that any unset variables will be replaced with their default values.

Variable Description Default
MF_BOOTSTRAP_LOG_LEVEL Log level for Bootstrap (debug, info, warn, error) error
MF_BOOTSTRAP_DB_HOST Database host address localhost
MF_BOOTSTRAP_DB_PORT Database host port 5432
MF_BOOTSTRAP_DB_USER Database user mainflux
MF_BOOTSTRAP_DB_PASS Database password mainflux
MF_BOOTSTRAP_DB Name of the database used by the service bootstrap
MF_BOOTSTRAP_DB_SSL_MODE Database connection SSL mode (disable, require, verify-ca, verify-full) disable
MF_BOOTSTRAP_DB_SSL_CERT Path to the PEM encoded certificate file
MF_BOOTSTRAP_DB_SSL_KEY Path to the PEM encoded key file
MF_BOOTSTRAP_DB_SSL_ROOT_CERT Path to the PEM encoded root certificate file
MF_BOOTSTRAP_CLIENT_TLS Flag that indicates if TLS should be turned on false
MF_BOOTSTRAP_CA_CERTS Path to trusted CAs in PEM format
MF_BOOTSTRAP_PORT Bootstrap service HTTP port 8181
MF_BOOTSTRAP_SERVER_CERT Path to server certificate in pem format 8181
MF_BOOTSTRAP_SERVER_KEY Path to server key in pem format 8181
MF_SDK_BASE_URL Base url for Mainflux SDK http://localhost
MF_SDK_THINGS_PREFIX SDK prefix for Things service
MF_USERS_URL Users service URL localhost:8181

Deployment

The service itself is distributed as Docker container. The following snippet provides a compose file template that can be used to deploy the service container locally:

version: "2"
  bootstrap:
    image: mainflux/bootstrap:latest
    container_name: mainflux-bootstrap
    depends_on:
      - bootstrap-db
    restart: on-failure
    ports:
      - 8900:8900
    environment:
      MF_BOOTSTRAP_LOG_LEVEL: [Bootstrap log level]
      MF_BOOTSTRAP_DB_HOST: [Database host address]
      MF_BOOTSTRAP_DB_PORT: [Database host port]
      MF_BOOTSTRAP_DB_USER: [Database user]
      MF_BOOTSTRAP_DB_PASS: [Database password]
      MF_BOOTSTRAP_DB: [Name of the database used by the service]
      MF_BOOTSTRAP_DB_SSL_MODE: [SSL mode to connect to the database with]
      MF_BOOTSTRAP_DB_SSL_CERT: [Path to the PEM encoded certificate file]
      MF_BOOTSTRAP_DB_SSL_KEY: [Path to the PEM encoded key file]
      MF_BOOTSTRAP_DB_SSL_ROOT_CERT: [Path to the PEM encoded root certificate file]
      MF_BOOTSTRAP_CLIENT_TLS: [Boolean value to enable/disable client TLS]
      MF_BOOTSTRAP_CA_CERTS: [Path to trusted CAs in PEM format]
      MF_BOOTSTRAP_PORT: 8900
      MF_BOOTSTRAP_SERVER_CERT: [String path to server cert in pem format]
      MF_BOOTSTRAP_SERVER_KEY: [String path to server key in pem format]
      MF_SDK_BASE_URL: [Base SDK URL for the Mainflux services]
      MF_SDK_THINGS_PREFIX: [SDK prefix for Things service]
      MF_USERS_URL: [Users service URL]

To start the service outside of the container, execute the following shell script:

# download the latest version of the service
go get github.com/mainflux/mainflux

cd $GOPATH/src/github.com/mainflux/mainflux

# compile the service
make bootstrap

# copy binary to bin
make install

# set the environment variables and run the service
MF_BOOTSTRAP_LOG_LEVEL=[Bootstrap log level] MF_BOOTSTRAP_DB_HOST=[Database host address] MF_BOOTSTRAP_DB_PORT=[Database host port] MF_BOOTSTRAP_DB_USER=[Database user] MF_BOOTSTRAP_DB_PASS=[Database password] MF_BOOTSTRAP_DB=[Name of the database used by the service] MF_BOOTSTRAP_DB_SSL_MODE=[SSL mode to connect to the database with] MF_BOOTSTRAP_DB_SSL_CERT=[Path to the PEM encoded certificate file] MF_BOOTSTRAP_DB_SSL_KEY=[Path to the PEM encoded key file] MF_BOOTSTRAP_DB_SSL_ROOT_CERT=[Path to the PEM encoded root certificate file] MF_BOOTSTRAP_CLIENT_TLS=[Boolean value to enable/disable client TLS] MF_BOOTSTRAP_CA_CERTS=[Path to trusted CAs in PEM format] MF_BOOTSTRAP_PORT=[Service HTTP port] MF_BOOTSTRAP_SERVER_CERT=[Path to server certificate] MF_BOOTSTRAP_SERVER_KEY=[Path to server key] MF_SDK_BASE_URL=[Base SDK URL for the Mainflux services] MF_SDK_THINGS_PREFIX=[SDK prefix for Things service] MF_USERS_URL=[Users service URL] $GOBIN/mainflux-bootstrap

Setting MF_BOOTSTRAP_CA_CERTS expects a file in PEM format of trusted CAs. This will enable TLS against the Users gRPC endpoint trusting only those CAs that are provided.

Usage

For more information about service capabilities and its usage, please check out the API documentation.