Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Do not merge] New config groups API endpoint #1637

Closed
wants to merge 35 commits into from
Closed
Show file tree
Hide file tree
Changes from 32 commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
4aa64c5
Fix_for_nodelib_issue_752
May 22, 2023
1016def
Fix for issue 752
KeshavSoni2511 May 22, 2023
ab1d728
Document improvements for issue 752
KeshavSoni2511 May 23, 2023
0b0973d
Syncing with master
KeshavSoni2511 May 23, 2023
1ea983f
Rebase
KeshavSoni2511 May 25, 2023
5c686c0
Update deviceConfigGroup.json
KeshavSoni2511 May 25, 2023
5899a2b
Update deviceService-test.js
KeshavSoni2511 May 25, 2023
a4db57a
Update test/unit/ngsiv2/provisioning/singleGroupConfigurationMode-tes…
KeshavSoni2511 Jun 27, 2023
4fad7ea
Latest Changes
KeshavSoni2511 Jun 27, 2023
a07215c
Latest Changes
Gauravp-NEC Jul 21, 2023
207cbb5
Rebased
KeshavSoni2511 Aug 14, 2023
5174f01
ChangeAccToMaster
KeshavSoni2511 Aug 14, 2023
92340d4
Rebased
KeshavSoni2511 Sep 1, 2023
2abd610
Rebased and latest changes done
KeshavSoni2511 Sep 20, 2023
06bcc74
Rebased
KeshavSoni2511 Sep 20, 2023
7483358
Remove_Code_Duplicacy
KeshavSoni2511 Oct 17, 2023
c0100be
q
KeshavSoni2511 Oct 17, 2023
8f91d13
FixLintError
KeshavSoni2511 Oct 17, 2023
1178705
Update groupService.js
KeshavSoni2511 Oct 17, 2023
7320d9a
Rebased_with_latest_changes
KeshavSoni2511 Jan 30, 2024
407c156
Rebased and resolved conflicts
KeshavSoni2511 Feb 26, 2024
bc1a1d4
Merge branch 'prelanding/new-config-groups' into Added_Feature_Config…
mapedraza Aug 7, 2024
e53d004
Merge pull request #1375 from KeshavSoni2511/Added_Feature_ConfigGroups
mapedraza Aug 7, 2024
aa64573
magickey to constant
mapedraza Aug 8, 2024
d69f338
Merge branch 'master' into prelanding/new-config-groups
mapedraza Aug 8, 2024
9e25475
Fix lint
mapedraza Aug 8, 2024
0b38688
Add cross usage tests
mapedraza Aug 9, 2024
e2ea6c2
remove package-lock.json
mapedraza Aug 9, 2024
d48cc4b
Add package-lock to gitignore
mapedraza Aug 9, 2024
e1f4e47
Add deprecation message
mapedraza Aug 9, 2024
5a423a8
Merge branch 'master' into prelanding/new-config-groups
mapedraza Aug 9, 2024
611b45f
remove bidirectional leftover file
mapedraza Aug 9, 2024
ceafc78
replace configGroups to groups
mapedraza Aug 23, 2024
c9b95e5
Merge branch 'master' into prelanding/new-config-groups
mapedraza Aug 23, 2024
cdb791c
replace group API endpoint
mapedraza Aug 23, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -14,4 +14,4 @@
.metadata
npm-debug.log
.eslintcache

package-lock.json
2 changes: 2 additions & 0 deletions CHANGES_NEXT_RELEASE
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,6 @@
- Fix: service header to use uppercase in case of update and delete (#1528)
- Fix: Allow to send to CB batch update for multimeasures for NGSI-LD (#1623)
- Add: new JEXL transformations for including into an array keys that have a certain value: valuePicker and valuePickerMulti
- Add /iot/configGroups API endpoints (as equivalent to /iot/services) (#752)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Add /iot/configGroups API endpoints (as equivalent to /iot/services) (#752)
- Add /iot/configGroups API endpoints (as equivalent to /iot/services) (#752)
- Deprecated: /iot/services API routes

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added here:e1f4e47

This commit also includes an entry in deprecated.md file

- Add: attribute metadata and static attributes metadata added to jexl context (#1630)
- Deprecated: /iot/services API routes
12 changes: 7 additions & 5 deletions doc/admin.md
Original file line number Diff line number Diff line change
Expand Up @@ -125,9 +125,9 @@ allowing the computer to interpret the rest of the data with more clarity and de
```

Under mixed mode, **NGSI v2** payloads are used for context broker communications by default, but this payload may also
be switched to **NGSI LD** at group or device provisioning time using the `ngsiVersion` field in the
provisioning API. The `ngsiVersion` field switch may be added at either group or device level, with the device level
overriding the group setting.
be switched to **NGSI LD** at group or device provisioning time using the `ngsiVersion` field in the provisioning API.
The `ngsiVersion` field switch may be added at either group or device level, with the device level overriding the group
setting.

#### `server`

Expand Down Expand Up @@ -306,7 +306,8 @@ added `agentPath`:

#### `types`

This parameter includes additional groups configuration as described into the [Config group API](api.md#config-group-api) section.
This parameter includes additional groups configuration as described into the
[Config group API](api.md#config-group-api) section.

#### `service`

Expand Down Expand Up @@ -415,7 +416,8 @@ IotAgents, as all Express applications that use the body-parser middleware, have
size that the application will handle. This default limit for ioiotagnets are 1Mb. So, if your IotAgent receives a
request with a body that exceeds this limit, the application will throw a “Error: Request entity too large”.

The 1Mb default can be changed setting the `expressLimit` configuration parameter (or equivalente `IOTA_EXPRESS_LIMIT` environment variable).
The 1Mb default can be changed setting the `expressLimit` configuration parameter (or equivalente `IOTA_EXPRESS_LIMIT`
environment variable).

### Configuration using environment variables

Expand Down
63 changes: 33 additions & 30 deletions doc/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -136,9 +136,9 @@ For every config group, the pair (resource, apikey) _must_ be unique (as it is u
which device). Those operations of the API targeting specific resources will need the use of the `resource` and `apikey`
parameters to select the appropriate instance.

Config groups can be created with preconfigured sets of attributes, service and subservice information, security information and other
parameters. The specific parameters that can be configured for a given config group are described in the
[Config group datamodel](#config-group-datamodel) section.
Config groups can be created with preconfigured sets of attributes, service and subservice information, security
information and other parameters. The specific parameters that can be configured for a given config group are described
in the [Config group datamodel](#config-group-datamodel) section.

### Devices

Expand Down Expand Up @@ -218,7 +218,7 @@ device preprovisioning). Device measures can have four different behaviors:
an attribute in the device's entity, for which the IoT Agent will be regitered as Context Provider. The IoT Agent
will return an immediate response to the Context Broker, and will be held responsible of contacting the device to
perform the command itself using the device specific protocol. Special `status` and `info` attributes should be
update. For each command, its `name` and `type` must be provided. For further information, please refer to
update. For each command, its `name` and `type` must be provided. For further information, please refer to
[Command execution](#command-execution) section.

All of them have the same syntax, a list of objects with the following attributes:
Expand Down Expand Up @@ -565,17 +565,18 @@ expression. In all cases the following data is available to all expressions:
- `id`: device ID
- `entity_name`: NGSI entity Name (principal)
- `type`: NGSI entity type (principal)
- `service`: device service (`Fiware-Service`)
- `service`: device service (`Fiware-Service`)
- `subservice`: device subservice (`Fiware-ServicePath`)
- `staticAttributes`: static attributes defined in the device or config group

Additionally, for attribute expressions (`expression`, `entity_name`), `entityNameExp` and metadata expressions
(`expression`) the following is available in the **context** used to evalute:

- measures, as `<AttributeName>`
- metadata (both for attribute measurement in the case of NGSI-v2 measurements and static attribute) are available in the **context** under the following convention:
`metadata.<AttributeName>.<MetadataName>` or `metadata.<StaticAttributeName>.<MetadataName>` in a similar way of defined
for [Context Broker](https://github.com/telefonicaid/fiware-orion/blob/master/doc/manuals/orion-api.md#metadata-support)
- measures, as `<AttributeName>`
- metadata (both for attribute measurement in the case of NGSI-v2 measurements and static attribute) are available in
the **context** under the following convention: `metadata.<AttributeName>.<MetadataName>` or
`metadata.<StaticAttributeName>.<MetadataName>` in a similar way of defined for
[Context Broker](https://github.com/telefonicaid/fiware-orion/blob/master/doc/manuals/orion-api.md#metadata-support)

### Examples of JEXL expressions

Expand Down Expand Up @@ -736,7 +737,8 @@ e.g.:
}
```

Note that there is no order into metadata structure and there is no warranty about which metadata attribute expression will be evaluated first.
Note that there is no order into metadata structure and there is no warranty about which metadata attribute expression
will be evaluated first.

## Measurement transformation

Expand Down Expand Up @@ -1070,21 +1072,21 @@ As part of the device to entity mapping process, the IoT Agent creates and updat
attribute called `TimeInstant`. This timestamp is represented as two different properties of the mapped entity:

- An attribute `TimeInstant` is added to updated entities in the case of NGSI-v2, which captures as an ISO8601
timestamp when the associated measurement was observed. With NGSI-LD, the Standard
`observedAt` property is used instead
timestamp when the associated measurement was observed. With NGSI-LD, the Standard `observedAt` property is used
instead

- With NGSI-v2, an attribute metadata named `TimeInstant` per active or lazy attribute mapped, which captures as an ISO8601
timestamp when the associated measurement (represented as attribute value) was observed. With NGSI-LD, the Standard
`observedAt` property-of-a-property is used instead.
- With NGSI-v2, an attribute metadata named `TimeInstant` per active or lazy attribute mapped, which captures as an
ISO8601 timestamp when the associated measurement (represented as attribute value) was observed. With NGSI-LD, the
Standard `observedAt` property-of-a-property is used instead.

If timestamp is not explicily defined when sending the measures through the IoT Agent (for further details on how to
attach timestamp information to the measures, please refer to the particular IoT Agent implementation documentation),
the arrival time on the server when receiving the measurement will be used to generate a `TimeInstant` for both the
entity attribute and the attribute metadata.

This functionality can be turned on and off globaly through the use of the `timestamp` configuration flag or `IOTA_TIMESTAMP`
variable as well as `timestamp` flag in device or group provision (in this case, the device or group level flag takes
precedence over the global one). The default value is `true`.
This functionality can be turned on and off globaly through the use of the `timestamp` configuration flag or
`IOTA_TIMESTAMP` variable as well as `timestamp` flag in device or group provision (in this case, the device or group
level flag takes precedence over the global one). The default value is `true`.

The `timestamp` configuration value used, according to the priority:

Expand All @@ -1104,6 +1106,7 @@ the IoTA behaviour is described in the following table:
| false | No | TimeInstant and metadata updated with server timestamp |
| Not defined | Yes | TimeInstant and metadata updated with measure value |
| Not defined | No | TimeInstant and metadata updated with server timestamp |

Some additional considerations to take into account:

- If there is an attribute which maps a measure to `TimeInstant` attribute (after
Expand Down Expand Up @@ -1724,7 +1727,7 @@ Config group is represented by a JSON object with the following fields:

The following actions are available under the config group endpoint:

#### Retrieve config groups `GET /iot/services`
#### Retrieve config groups GET `/iot/configGroups` or `GET /iot/services`

List all the config groups for the given `fiware-service` and `fiware-servicepath`. The config groups that match the
`fiware-servicepath` are returned in any other case.
Expand All @@ -1748,14 +1751,14 @@ Successful operations return `Content-Type` header with `application/json` value

_**Response payload**_

A JSON object with a services field that contains an array of services that match the request. See the
A JSON object with a configGroups or services field that contains an array of services that match the request. See the
[config group datamodel](#service-group-datamodel) for more information.

Example:

```json
{
"services": [
"configGroups": [
{
"resource": "/deviceTest",
"apikey": "801230BJKL23Y9090DSFL123HJK09H324HV8732",
Expand Down Expand Up @@ -1792,7 +1795,7 @@ Example:
}
```

#### Create config group `POST /iot/services`
#### Create config group `POST /iot/configGroups` or `POST /iot/services`

Creates a set of config groups for the given service and service path. The service and subservice information will taken
from the headers, overwritting any preexisting values.
Expand All @@ -1806,14 +1809,14 @@ _**Request headers**_

_**Request payload**_

A JSON object with a `services` field. The value is an array of config groups objects to create. See the
[config group datamodel](#service-group-datamodel) for more information.
A JSON object with a `configGroups` or `services` field. The value is an array of config groups objects to create. See
the [config group datamodel](#service-group-datamodel) for more information.

Example:

```json
{
"services": [
"configGroups": [
{
"resource": "/deviceTest",
"apikey": "801230BJKL23Y9090DSFL123HJK09H324HV8732",
Expand Down Expand Up @@ -1847,11 +1850,11 @@ _**Response headers**_

Successful operations return `Content-Type` header with `application/json` value.

#### Modify config group `PUT /iot/services`
#### Modify config group `PUT /iot/configGroups` or `PUT /iot/services`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd suggest to remove references to /iot/services in any file, as it is deprecated feature and, as stated in doc

Documentation on deprecated features is removed from the repository documentation. Documentation is still available in the documentation set associated to older versions (in the repository release branches).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pending to choose the final term for configGroups


Modifies the information of a config group, identified by the `resource` and `apikey` query parameters. Takes a service
group body as the payload. The body does not have to be complete: for incomplete bodies, just the attributes included in
the JSON body will be updated. The rest of the attributes will remain unchanged.
Modifies the information of a config group, identified by the `resource` and `apikey` query parameters. Takes a
configuration/service group body as the payload. The body does not have to be complete: for incomplete bodies, just the
attributes included in the JSON body will be updated. The rest of the attributes will remain unchanged.

_**Request query parameters**_

Expand Down Expand Up @@ -1887,7 +1890,7 @@ _**Response code**_
- 400 MISSING_HEADERS if any of the mandatory headers is not present.
- 500 SERVER ERROR if there was any error not contemplated above.:

#### Remove config group `DELETE /iot/services`
#### Remove config group `DELETE /iot/configGroups` or `DELETE /iot/services`

Removes a config group, identified by the `resource` and `apikey` query parameters.

Expand Down
2 changes: 2 additions & 0 deletions doc/deprecated.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ A list of deprecated features and the version in which they were deprecated foll
- Bidirectinal pluging (finally removed in 3.4.0)
- appendMode configuration (`IOTA_APPEND_MODE` env var) (finally removed in 3.4.0)
- `config.stats` section, and push-mode statistics.
- Services API routes (`/iot/services`) in favor of the `/iot/configGroups`. Both are still supported, but the former
is deprecated.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Services API routes (`/iot/services`) in favor of the `/iot/configGroups`. Both are still supported, but the former
is deprecated.
- Services API routes (`/iot/services`) in favor of the `/iot/configGroups`.

Looking the above elements, it is assumed that every deprecated feature is still supported, except the ones with the "(finally removed in x)" part.


The use of Node.js v14 is highly recommended.

Expand Down
6 changes: 3 additions & 3 deletions doc/devel/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -1831,9 +1831,9 @@ iotAgentLib.startServer(config, iotAgent, function (error) {

### Configuration management

For some IoT Agents, it will be useful to know what devices or config groups were registered in the Agent, or to do
some actions whenever a new device is registered. All this configuration and provisioning actions can be performed using
two mechanisms: the provisioning handlers and the provisioning API.
For some IoT Agents, it will be useful to know what devices or config groups were registered in the Agent, or to do some
actions whenever a new device is registered. All this configuration and provisioning actions can be performed using two
mechanisms: the provisioning handlers and the provisioning API.

#### Provisioning handlers

Expand Down
26 changes: 13 additions & 13 deletions doc/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,22 +10,22 @@ custom settings may also be required dependent upon the actual IoT Agent used.

```javascript
config = {
logLevel: "DEBUG",
logLevel: 'DEBUG',
contextBroker: {
host: "orion",
port: "1026",
host: 'orion',
port: '1026'
},
server: {
port: 4041,
host: "0.0.0.0",
host: '0.0.0.0'
},
deviceRegistry: {
type: "memory",
type: 'memory'
},
service: "openiot",
subservice: "/",
providerUrl: "http://iot-agent:4041",
defaultType: "Thing",
service: 'openiot',
subservice: '/',
providerUrl: 'http://iot-agent:4041',
defaultType: 'Thing'
};
```

Expand All @@ -43,8 +43,8 @@ All configuration settings can also updated using Docker environment variables.
### Provisioning a Config Group

Settings which are common to a group of devices can be passed to the IoT Agent using the Config Group API. The
`fiware-service` and `fiware-service-path` to be used to access the API are defined within the `config.js`. Each config group
may override values previously defined in the global configuration if necessary.
`fiware-service` and `fiware-service-path` to be used to access the API are defined within the `config.js`. Each config
group may override values previously defined in the global configuration if necessary.

```bash
curl -iX POST \
Expand Down Expand Up @@ -138,8 +138,8 @@ curl -iX POST \
```

The IoT Agent South port is listen to the path defined in the config group, and the API key is recognized to match, so
the config group configuration will be used. No mappings will be made for the Entity `id` or the attribute names and
the following entity will be created:
the config group configuration will be used. No mappings will be made for the Entity `id` or the attribute names and the
following entity will be created:

```json
{
Expand Down
Loading
Loading