README
¶
JSON Message Transformer
JSON Transformer provides Message Transformer for JSON messages. To transform Mainflux Message successfully, the payload must be a JSON object.
For the messages that contain JSON array as the root element, JSON Transformer does normalization of the data: it creates a separate JSON message for each JSON object in the root. In order to be processed and stored properly, JSON messages need to contain message format information. For the sake of the simpler storing of the messages, nested JSON objects are flatten to a single JSON object, using composite keys with the default separator /. This implies that the separator character (/) is not allowed in the JSON object key. For example, the following JSON object:
{
"name": "name",
"id":8659456789564231564,
"in": 3.145,
"alarm": true,
"ts": 1571259850000,
"d": {
"tmp": 2.564,
"hmd": 87,
"loc": {
"x": 1,
"y": 2
}
}
}
will be transformed to:
{
"name": "name",
"id":8659456789564231564,
"in": 3.145,
"alarm": true,
"ts": 1571259850000,
"d/tmp": 2.564,
"d/hmd": 87,
"d/loc/x": 1,
"d/loc/y": 2
}
The message format is stored in the subtopic. It's the last part of the subtopic. In the example:
http://localhost:8185/channels/<channelID>/messages/home/temperature/myFormat
the message format is myFormat. It can be any valid subtopic name, JSON transformer is format-agnostic. The format is used by the JSON message consumers so that they can process the message properly. If the format is not present (i.e. message subtopic is empty), JSON Transformer will report an error. Since the Transformer is agnostic to the format, having format in the subtopic does not prevent the publisher to send the content of different formats to the same subtopic. It's up to the consumer to handle this kind of issue. Message writers, for example, will store the message(s) in the table/collection/measurement (depending on the underlying database) with the name of the format (which in the example is myFormat). Mainflux writers will try to save any format received (whether it will be successful depends on the writer implementation and the underlying database), but it's recommended that the publisher takes care not to send different formats to the same subtopic.
Having a message format in the subtopic means that the subscriber has an option to subscribe to only one message format. This is a nice feature because message subscribers know what's the expected format of the message so that they can process it. If the message format is not important, wildcard subtopic can always be used to subscribe to any message format:
http://localhost:8185/channels/<channelID>/messages/home/temperature/*
Documentation
¶
Index ¶
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var ( // ErrTransform represents an error during parsing message. ErrTransform = errors.New("unable to parse JSON object") // ErrFormatPayload represents an error during formatting message payload. ErrFormatPayload = errors.New("unable to format payload") // ErrInvalidKey represents the use of a reserved message field. ErrInvalidKey = errors.New("invalid object key") // ErrInvalidTimeField represents the use an invalid time field. ErrInvalidTimeField = errors.New("invalid time field") )
Functions ¶
func Flatten ¶
Flatten makes nested maps flat using composite keys created by concatenation of the nested keys.
Example ¶
package main
import (
"encoding/json"
"fmt"
mfjson "github.com/MainfluxLabs/mainflux/pkg/transformers/json"
)
func main() {
in := map[string]interface{}{
"key1": "value1",
"key2": "value2",
"key5": map[string]interface{}{
"nested1": map[string]interface{}{
"nested2": "value3",
"nested3": "value4",
},
"nested2": map[string]interface{}{
"nested4": "value5",
},
},
}
out, err := mfjson.Flatten(in)
if err != nil {
panic(err)
}
b, err := json.MarshalIndent(out, "", " ")
if err != nil {
panic(err)
}
fmt.Println(string(b))
}
Output: { "key1": "value1", "key2": "value2", "key5/nested1/nested2": "value3", "key5/nested1/nested3": "value4", "key5/nested2/nested4": "value5" }
func ParseFlat ¶
func ParseFlat(flat interface{}) interface{}
ParseFlat receives flat map that represents complex JSON objects and returns the corresponding complex JSON object with nested maps. It's the opposite of the Flatten function.
Example ¶
package main
import (
"encoding/json"
"fmt"
mfjson "github.com/MainfluxLabs/mainflux/pkg/transformers/json"
)
func main() {
in := map[string]interface{}{
"key1": "value1",
"key2": "value2",
"key5/nested1/nested2": "value3",
"key5/nested1/nested3": "value4",
"key5/nested2/nested4": "value5",
}
out := mfjson.ParseFlat(in)
b, err := json.MarshalIndent(out, "", " ")
if err != nil {
panic(err)
}
fmt.Println(string(b))
}
Output: { "key1": "value1", "key2": "value2", "key5": { "nested1": { "nested2": "value3", "nested3": "value4" }, "nested2": { "nested4": "value5" } } }
Types ¶
type Message ¶
type Message struct {
Created int64 `json:"created,omitempty" db:"created" bson:"created"`
Subtopic string `json:"subtopic,omitempty" db:"subtopic" bson:"subtopic,omitempty"`
Publisher string `json:"publisher,omitempty" db:"publisher" bson:"publisher"`
Protocol string `json:"protocol,omitempty" db:"protocol" bson:"protocol"`
Payload Payload `json:"payload,omitempty" db:"payload" bson:"payload,omitempty"`
Profile Profile `json:"profile,omitempty" db:"profile" bson:"profile,omitempty"`
}
Message represents a JSON messages.
Source Files