[Documentation] New features in engines and API
Major changes
Engines
New achitecture
The old architecture restricted us to use only two separate routines inside the engine, which made us not so flexible in developing some of the functionality. We've implemented the new engine architecture, which allows us to create and use an unlimited number of engine's routines such as rmq consumers, servers, periodical workers etc.
The old architecture is still supportable until we've migrated all our engines. The engines, which were migrated so far is:
- engine-action
- engine-axe
- engine-watcher
- engine-pbehavior
- engine-webhook
Communication between the engines
The engines work as the pipeline, which is normal, since the work of a certain engine requires the work result from previous engines. On the other hand it makes it difficult to implement a functionality, which requires to call engines in different order. It leads to various 'system' or 'empty' events inside the pipeline, which should be handled differently compared to normal events, that leads to unnecessary increase in project complexity. Also they may interfere with the main event flow and decrease the overall performance, just by increasing a number of events in the pipeline.
We've implemented an internal communication between the engines via RabbitMQ-RPC. Thanks to the new engine architecture it's now possible for an engine to call another engine to do a specific job and it doesn't interfere with the main event flow anymore.
To make the new feature working, we've added some rpc servers and clients to the engines. Since RabbitMQ-RPC requires additional rabbitmq queues, the configuration should be updated for:
- engine-axe (rpc server + pbehavior and watcher rpc clients added)
[[RabbitMQ.queues]]
name = "Engine_axe_rpc_server"
durable = true
autoDelete = false
exclusive = false
noWait = false
# args =
[[RabbitMQ.queues]]
name = "Engine_axe_watcher_rpc_client"
durable = true
autoDelete = false
exclusive = false
noWait = false
# args =
[[RabbitMQ.queues]]
name = "Engine_axe_pbehavior_rpc_client"
durable = true
autoDelete = false
exclusive = false
noWait = false
# args =
- engine-pbehavior (rpc server added)
[[RabbitMQ.queues]]
name = "Engine_pbehavior_rpc_server"
durable = true
autoDelete = false
exclusive = false
noWait = false
# args =
- engine-watcher (rpc server added)
[[RabbitMQ.queues]]
name = "Engine_watcher_rpc_server"
durable = true
autoDelete = false
exclusive = false
noWait = false
# args =
- engine-action (axe and webhook rpc client added)
[[RabbitMQ.queues]]
name = "Engine_action_axe_rpc_client"
durable = true
autoDelete = false
exclusive = false
noWait = false
# args =
[[RabbitMQ.queues]]
name = "Engine_action_webhook_rpc_client"
durable = true
autoDelete = false
exclusive = false
noWait = false
# args =
- engine-webhook (rpc server added)
[[RabbitMQ.queues]]
name = "Engine_webhook_rpc_server"
durable = true
autoDelete = false
exclusive = false
noWait = false
# args =
New engine-action
With the moving to a new architecture, we've also completely reworked the engine-action. Now it is possible to trigger action sequences, while with the old version only the single action is supported. This feature gives full control on triggers to the user's hands, so we hope it will cover some use cases, which weren't possible with the previous version.
Additionally, the trigger feature were revamped and now it's working well. The previous engine has a lot of problems with that.
The new engine is designed with statelessness and fault-tolerant design, so it's possible to run in several instances.
Let's take a look at new config:
{
"name": "test-scenario-name",
"author": "test-scenario-author",
"enabled": true,
"priority": 10,
"triggers": [
"create"
],
"disabled_during_periods": ["maintenance"],
"delay": {
"seconds": 3,
"unit": "s"
},
"actions": [
{
"alarm_patterns": [
{
"test_field": "test"
}
],
"entity_patterns": [
{
"test_field": "test"
}
],
"type": "snooze",
"parameters": {
"output": "test",
"duration": {
"seconds": 3,
"unit": "s"
},
"author": "engine-action"
},
"drop_scenario_if_not_matched": false,
"emit_trigger": true
},
{
"alarm_patterns": [
{
"test_field": "test"
}
],
"entity_patterns": [
{
"test_field": "test"
}
],
"type": "ack",
"parameters": {
"output": "test",
"author": "engine-action"
},
"drop_scenario_if_not_matched": false,
"emit_trigger": true
}
]
}
actions
array contains action configs. As you can see hook
is not supported anymore. Now the whole scenario can be triggered by triggers
, but each action has its own set of alarm/entity patterns. There are 2 new config fields:
-
drop_scenario_if_not_matched
allows a user to control the behavior when some of the actions are not matched. -
emit_trigger
allows a user to turn on/off emitting triggers. In the example snooze and ack actions emit a trigger, when executed. It's possible to turn it off if it's needed.
There are updated triggers list :
create
stateinc
statedec
changestate
changestatus
ack
ackremove
cancel
uncancel
comment
done
declareticket
declareticketwebhook
assocticket
snooze
unsnooze
resolve
activate
pbhenter
pbhleave
Engine-webhook
Since we've created a new action scenario functionality, we've turned a webhook to another kind of action with type = webhook
. With that change the engine-webhook no longer needs an event to send a webhook, because engine-webhook receives an rpc call to do webhook from engine-action directly. So we've decided to exclude the engine-webhook from the pipeline. The publishQueue
options should be updated in those engines, which have engine-webhook as the next engine.
Engine-action
must be run with argument -withWebhook=true
to bind to engine-webhook
rpc queues. Default value of withWebhook=false
since webhooks is cat feature.
Backward compatibility
Unfortunately we've decided to not support old action functionality due to a new engine-action architecture. But we've implemented a migrate tool, so it's possible to transform old action configs to a new scenario configs.
Api
Let's take as an example a simple snooze
action
With old api we had a payload:
{
"_id": "action_463a9921-48dd-4ff2-8454-b4fb9a67d80a",
"type": "snooze",
"enabled": true,
"priority": 10,
"disabled_during_periods": ["maintenance"],
"delay": "1s",
"hook": {
"alarm_patterns": [
{
"test_field": "test"
}
],
"entity_patterns": [
{
"test_field": "test"
}
],
"triggers": [
"create"
]
},
"parameters": {
"message": "test",
"duration": 3,
"author": "engine-action"
}
}
The payload for the same action with the new api will look like:
{
"name": "test-scenario-name",
"author": "test-scenario-author",
"enabled": true,
"priority": 10,
"triggers": [
"create"
],
"disabled_during_periods": ["maintenance"],
"delay": {
"seconds": 3,
"unit": "s"
},
"actions": [
{
"alarm_patterns": [
{
"test_field": "test"
}
],
"entity_patterns": [
{
"test_field": "test"
}
],
"type": "snooze",
"parameters": {
"output": "test",
"duration": {
"seconds": 3,
"unit": "s"
},
"author": "engine-action"
},
"drop_scenario_if_not_matched": false,
"emit_trigger": false
}
]
}
Each scenario contains:
-
name
- scenario name -
author
- scenario author -
enabled
- if scenario is enabled(the same as in old action api) -
priority
- scenario prioriry(the same as in old action api) -
triggers
- scenario triggers(previously it was inhook
object) -
disabled_during_periods
- ignore scenario if alarm in pbehavior canonical type -
delay
- scenario execution delay -
actions
- array of actions.
Each action in actions
contains:
-
alarm_patterns
- standard alarm patterns -
entity_patterns
- standard entity patterns -
type
- action type -
parameters
- action parameters -
drop_scenario_if_not_matched
- allow a user to control a scenario flow -
emit_trigger
- allows a user to control a trigger emitting
hook
is not supported anymore.
To create a scenario, you should send a payload to POST /api/v4/scenarios
WARNING: The Webhook API is deprecated!!!
The webhook is considered as an action with type = webhook
. More details here
{
"name": "test-scenario-action-webhook-1-name",
"author": "test-scenario-action-webhook-1-author",
"enabled": true,
"priority": 30,
"triggers": ["create"],
"disabled_during_periods": ["maintenance"],
"actions": [
{
"alarm_patterns": [
{
"v": {
"component": "test-component-action-webhook-1"
}
}
],
"entity_patterns": [
{
"_id": "test-resource-action-webhook-1/test-component-action-webhook-1"
}
],
"type": "webhook",
"parameters": {
"request": {
"method": "POST",
"url": "{{ .apiURL }}/api/v4/scenarios",
"auth": {
"username": "root",
"password": "test"
},
"headers": {"Content-Type": "application/json"},
"payload": "{\"name\":\"test-scenario-action-webhook-1\",\"author\":\"test-scenario-action-webhook-1\",\"enabled\":true,\"priority\":41,\"triggers\":[\"create\"],\"actions\":[{\"alarm_patterns\":[{\"_id\":\"test-scenario-action-webhook-1-alarm\"}],\"type\":\"ack\",\"drop_scenario_if_not_matched\":false,\"emit_trigger\":false}]}"
},
"declare_ticket": {
"empty_response": false,
"is_regexp": false,
"ticket_id": "_id",
"scenario_name": "name"
},
"retry_count": 3,
"retry_delay": {
"seconds": 10,
"unit": "s"
}
},
"drop_scenario_if_not_matched": false,
"emit_trigger": false
}
]
}