Merge pull request #1507 from telefonicaid/fix/static_attrs_when_expl…

…icitAttrs

Fix/static attrs when explicit attrs

CHANGES_NEXT_RELEASE

@@ -1,3 +1,4 @@
+- Fix: add static attributes when use explicitAttrs (#1506)
 - Remove: single configuration mode (along with IOTA_SINGLE_MODE env var) (#1469)
 - Remove: extractVariables from jexl plugin (no needed anymore since the removal of bidireational plugin)
 - Fix: ensure service and subservice in context of error handlers using req headers

doc/api.md

@@ -409,21 +409,22 @@ Case 1 (default):
 "explicitAttrs": false
 ```
 
-every measure will be propagated to NGSI interface.
+every measure will be propagated to NGSI interface, including all static attributes.
 
 Case 2:
 
 ```
 "explicitAttrs": true
 ```
 
-In this case, should only progress active and static attributes defined in the device or group provision (`TimeInstant` attribute 
-will be also included if enabled). In other words, having `"explicitAttrs":true` would prevent the IoTA creating attributes into the related 
-entity  within the context broker from measures that are not explicitly defined in the device or group provision. 
+In this case, should only progress active and static attributes defined in the device or group provision (`TimeInstant`
+attribute will be also included if enabled), including also all static attributes. In other words, having
+`"explicitAttrs":true` would prevent the IoTA creating attributes into the related entity within the context broker from
+measures that are not explicitly defined in the device or group provision.
 
-Note that attributes defined in the provision that are not receiving a measure (or having a expression defined that is resulting `null`) 
-will not progress (this means, the NGSI request to update the entity in the context broker is not going to include that attribute) 
-unless `skipValue` is defined to other value than `null`
+Note that attributes defined in the provision that are not receiving a measure (or having a expression defined that is
+resulting `null`) will not progress (this means, the NGSI request to update the entity in the context broker is not
+going to include that attribute) unless `skipValue` is defined to other value than `null`
 
 Case 3:
 
@@ -433,7 +434,8 @@ Case 3:
 
 just NGSI attributes defined in the array (identified by their attribute names, not by their object_id, plus
 conditionally TimeInstant) will be propagated to NGSI interface (note that in this case the value of `explicitAttrs` is
-not a JSON but a JEXL Array that looks likes a JSON).
+not a JSON but a JEXL Array that looks likes a JSON). Only static attributes included in that array will be propagated
+to NGSI interface.
 
 Case 4:
 
@@ -443,7 +445,8 @@ Case 4:
 
 just NGSI attributes defined in the array (identified by their attribute names and/or by their object_id) will be
 propagated to NGSI interface (note that in this case the value of `explicitAttrs` is not a JSON but a JEXL Array/Object
-that looks likes a JSON). This is necessary when same attribute names are used within multiple entities.
+that looks likes a JSON). This is necessary when same attribute names are used within multiple entities. Only static
+attributes included in that array will be propagated to NGSI interface.
 
 Case 5:
 
@@ -768,10 +771,10 @@ following to CB:
 
 ### Measurement transformation order
 
-The IoTA executes the transformaion looping over the `attributes` provision field. Every time a new expression is 
-evaluated, the JEXL context is updated with the expression result. The order defined in the `attributes` array is 
-taken for expression evaluation. This should be considered when using **nested expressions**, that uses values 
-calculated in other attributes. 
+The IoTA executes the transformaion looping over the `attributes` provision field. Every time a new expression is
+evaluated, the JEXL context is updated with the expression result. The order defined in the `attributes` array is taken
+for expression evaluation. This should be considered when using **nested expressions**, that uses values calculated in
+other attributes.
 
 For example, let's consider the following provision for a device which send a measure named `level`:
 
@@ -790,8 +793,8 @@ For example, let's consider the following provision for a device which send a me
 ]
 ```
 
-The expression for `correctedLevel` is evaluated first (using `level` measure as input). Next, the `normalizedLevel` 
-is evaluated (using `correctedLevel` calculated attribute, just calculated before).
+The expression for `correctedLevel` is evaluated first (using `level` measure as input). Next, the `normalizedLevel` is
+evaluated (using `correctedLevel` calculated attribute, just calculated before).
 
 Note that if we reserve the order, this way:
 
@@ -806,16 +809,17 @@ Note that if we reserve the order, this way:
         "name": "correctedLevel",
         "type": "Number",
         "expression": "level * 0.897"
-    },    
+    },
 ]
 ```
 
-It is not going to work. The first expression expects a `correctedLevel` which is neither a measure (remember the only 
-measure sent by the device is named `level`) nor a previously calculated attribute. Thus, `correctedLevel` will end 
-with a `null` value, so will not be part of the update request send to the Context Broker unless `skipValue` (check 
+It is not going to work. The first expression expects a `correctedLevel` which is neither a measure (remember the only
+measure sent by the device is named `level`) nor a previously calculated attribute. Thus, `correctedLevel` will end with
+a `null` value, so will not be part of the update request send to the Context Broker unless `skipValue` (check
 [Devices](#devices) section avobe) is defined with a different value thant the default one (`null`).
 
-In conclusion: **the order of attributes in the `attributes` arrays at provising time matters with regards to nested expression evaluation**.
+In conclusion: **the order of attributes in the `attributes` arrays at provising time matters with regards to nested
+expression evaluation**.
 
 Let's consider the following example. It is an anti-pattern but it's quite illustrative on how ordering works:
 
@@ -843,11 +847,10 @@ When receiving a measure with the following values:
 }
 ```
 
-Then, as they are executed sequentially, the first attribute expression to be evaluated will be `a`, taking the 
-value of the attribute `b` multiplied by 10, in this case, `200`. After that, the second attribute expression to be 
-evaluated is  the one holded by `b`. In this case, that attribute would take 10 times the value of `a`. In that case, 
-since the JEXL context was updated with the lastest execution, the value of `b` will be `2000`, being update at Context 
-Broker entity:
+Then, as they are executed sequentially, the first attribute expression to be evaluated will be `a`, taking the value of
+the attribute `b` multiplied by 10, in this case, `200`. After that, the second attribute expression to be evaluated is
+the one holded by `b`. In this case, that attribute would take 10 times the value of `a`. In that case, since the JEXL
+context was updated with the lastest execution, the value of `b` will be `2000`, being update at Context Broker entity:
 
 ```json
     "a": {"value": 200, "type": "Number"},

lib/services/ngsi/entities-NGSI-v2.js

@@ -324,30 +324,30 @@ function sendQueryValueNgsi2(entityName, attributes, typeInformation, token, cal
 }
 
 /**
- * Makes an update in the Device's entity in the context broker, with the values given in the 'attributes' array. This
+ * Makes an update in the Device's entity in the context broker, with the values given in the 'measures' array. This
  * array should comply to the NGSIv2's attribute format.
  *
  * @param {String} entityName       Name of the entity to register.
- * @param {Array} attributes        Attribute array containing the values to update.
+ * @param {Array} measures        Attribute array containing the values to update.
  * @param {Object} typeInformation  Configuration information for the device.
  * @param {String} token            User token to identify against the PEP Proxies (optional).
  */
-function sendUpdateValueNgsi2(entityName, attributes, typeInformation, token, callback) {
+function sendUpdateValueNgsi2(entityName, measures, typeInformation, token, callback) {
     logger.debug(
         context,
-        'sendUpdateValueNgsi2 called with: entityName=%s  attributes=%j  typeInformation=%j',
+        'sendUpdateValueNgsi2 called with: entityName=%s  measures=%j  typeInformation=%j',
         entityName,
-        attributes,
+        measures,
         typeInformation
     );
     // if any attribute name of a measure is 'id' or 'type' should be removed
-    var attributesWithoutIdType = [];
-    attributes.forEach(function (attribute) {
+    var measuresWithoutIdType = [];
+    measures.forEach(function (attribute) {
         if (attribute.name !== 'id' && attribute.name !== 'type') {
-            attributesWithoutIdType.push(attribute);
+            measuresWithoutIdType.push(attribute);
         }
     });
-    attributes = attributesWithoutIdType;
+    measures = measuresWithoutIdType;
 
     const payload = {
         entities: [
@@ -370,7 +370,7 @@ function sendUpdateValueNgsi2(entityName, attributes, typeInformation, token, ca
     let options = NGSIUtils.createRequestObject(url, typeInformation, token);
 
     if (typeInformation && typeInformation.staticAttributes) {
-        attributes = attributes.concat(typeInformation.staticAttributes);
+        measures = measures.concat(typeInformation.staticAttributes);
     }
 
     if (!typeInformation || !typeInformation.type) {
@@ -381,23 +381,30 @@ function sendUpdateValueNgsi2(entityName, attributes, typeInformation, token, ca
     let idTypeSSSList = pluginUtils.getIdTypeServSubServiceFromDevice(typeInformation);
     logger.debug(context, 'sendUpdateValueNgsi2 idTypeSSS are %j ', idTypeSSSList);
     let measureAttrsForCtxt = [];
+    // This ctxt should include all possible attrs
+    let attributesCtxt = [];
+    if (typeInformation && typeInformation.staticAttributes) {
+        typeInformation.staticAttributes.forEach(function (att) {
+            attributesCtxt.push(att);
+        });
+    }
 
     // Check explicitAttrs: adds all final needed attributes to payload
     if (
         typeInformation.explicitAttrs === undefined ||
         (typeof typeInformation.explicitAttrs === 'boolean' && !typeInformation.explicitAttrs)
         // explicitAttrs is not defined => default case: all attrs should be included
     ) {
-        // This loop adds all measure values (attributes) into payload entities (entity[0])
-        for (let i = 0; i < attributes.length; i++) {
-            if (attributes[i].name && attributes[i].type) {
-                payload.entities[0][attributes[i].name] = {
-                    value: attributes[i].value,
-                    type: attributes[i].type
+        // This loop adds all measure values (measures) into payload entities (entity[0])
+        for (let i = 0; i < measures.length; i++) {
+            if (measures[i].name && measures[i].type) {
+                payload.entities[0][measures[i].name] = {
+                    value: measures[i].value,
+                    type: measures[i].type
                 };
-                const metadata = NGSIUtils.getMetaData(typeInformation, attributes[i].name, attributes[i].metadata);
+                const metadata = NGSIUtils.getMetaData(typeInformation, measures[i].name, measures[i].metadata);
                 if (metadata) {
-                    payload.entities[0][attributes[i].name].metadata = metadata;
+                    payload.entities[0][measures[i].name].metadata = metadata;
                 }
             } else {
                 callback(new errors.BadRequest(null, entityName));
@@ -428,33 +435,32 @@ function sendUpdateValueNgsi2(entityName, attributes, typeInformation, token, ca
         }
     } else {
         let selectedAttrs = [];
+        if (typeInformation.staticAttributes) {
+            typeInformation.staticAttributes.forEach(function (att) {
+                selectedAttrs.push(att.name);
+            });
+        }
         if (typeof typeInformation.explicitAttrs === 'string') {
             // explicitAttrs is a jexlExpression
-            // This ctxt should include all possible attrs
-            const attributesCtxt = [];
-            if (typeInformation.static) {
-                typeInformation.static.forEach(function (att) {
-                    attributesCtxt.push(att);
-                });
-            }
+
             // Measures
-            for (let i = 0; i < attributes.length; i++) {
-                if (attributes[i].name && attributes[i].type) {
+            for (let i = 0; i < measures.length; i++) {
+                if (measures[i].name && measures[i].type) {
                     const measureAttr = {
-                        name: attributes[i].name,
-                        value: attributes[i].value,
-                        type: attributes[i].type
+                        name: measures[i].name,
+                        value: measures[i].value,
+                        type: measures[i].type
                     };
                     attributesCtxt.push(measureAttr);
                     // check measureAttr by object_id -> if in active
                     let j = 0;
                     let found = false;
                     while (j < typeInformation.active.length && !found) {
-                        if (attributes[i].name === typeInformation.active[j].object_id) {
+                        if (measures[i].name === typeInformation.active[j].object_id) {
                             let measureAttrByObjectId = {
                                 name: typeInformation.active[j].name,
-                                value: attributes[i].value,
-                                type: attributes[i].type
+                                value: measures[i].value,
+                                type: measures[i].type
                             };
                             attributesCtxt.push(measureAttrByObjectId);
                             found = true;
@@ -506,11 +512,11 @@ function sendUpdateValueNgsi2(entityName, attributes, typeInformation, token, ca
                         selectedAttrs.push(attr.name);
                         selectedAttrs.push(attr.object_id);
                     });
-                    for (let i = 0; i < attributes.length; i++) {
-                        selectedAttrs.push(attributes[i].name);
+                    for (let i = 0; i < measures.length; i++) {
+                        selectedAttrs.push(measures[i].name);
                     }
                 } else {
-                    selectedAttrs = res; // TBD: Check ensure is an array of strings
+                    selectedAttrs = res;
                 }
                 if (selectedAttrs.length === 0) {
                     // implies do nothing
@@ -539,7 +545,7 @@ function sendUpdateValueNgsi2(entityName, attributes, typeInformation, token, ca
             // explicitAtts is true => Add just measures which are defined in active attributes
             // and active attributes with expressions
             // and TimeInstant
-            selectedAttrs = ['TimeInstant'];
+            selectedAttrs.push('TimeInstant');
             typeInformation.active.forEach((attr) => {
                 // Measures
                 if (attr.expression !== undefined) {
@@ -549,9 +555,9 @@ function sendUpdateValueNgsi2(entityName, attributes, typeInformation, token, ca
                     // check if active attr is receiving a measure
                     let i = 0;
                     let found = false;
-                    while (i < attributes.length && !found) {
-                        if (attributes[i].name && attributes[i].type) {
-                            if (attributes[i].name === attr.object_id || attributes[i].name === attr.name) {
+                    while (i < measures.length && !found) {
+                        if (measures[i].name && measures[i].type) {
+                            if (measures[i].name === attr.object_id || measures[i].name === attr.name) {
                                 selectedAttrs.push(attr.name);
                                 selectedAttrs.push(attr.object_id);
                                 found = true;
@@ -562,29 +568,29 @@ function sendUpdateValueNgsi2(entityName, attributes, typeInformation, token, ca
                 }
             });
         }
-        // This loop adds selected measured values (attributes) into payload entities (entity[0])
-        for (let i = 0; i < attributes.length; i++) {
-            if (attributes[i].name && selectedAttrs.includes(attributes[i].name) && attributes[i].type) {
+        // This loop adds selected measured values (measures) into payload entities (entity[0])
+        for (let i = 0; i < measures.length; i++) {
+            if (measures[i].name && selectedAttrs.includes(measures[i].name) && measures[i].type) {
                 const attr = typeInformation.active.find((obj) => {
-                    return obj.name === attributes[i].name;
+                    return obj.name === measures[i].name;
                 });
-                payload.entities[0][attributes[i].name] = {
-                    value: attributes[i].value,
-                    type: attributes[i].type
+                payload.entities[0][measures[i].name] = {
+                    value: measures[i].value,
+                    type: measures[i].type
                 };
                 // ensure payload has attr with proper object_id
                 if (attr && attr.object_id) {
-                    payload.entities[0][attributes[i].name].object_id = attr.object_id;
+                    payload.entities[0][measures[i].name].object_id = attr.object_id;
                 }
-                const metadata = NGSIUtils.getMetaData(typeInformation, attributes[i].name, attributes[i].metadata);
+                const metadata = NGSIUtils.getMetaData(typeInformation, measures[i].name, measures[i].metadata);
                 if (metadata) {
-                    payload.entities[0][attributes[i].name].metadata = metadata;
+                    payload.entities[0][measures[i].name].metadata = metadata;
                 }
-            } else if (attributes[i].name && !selectedAttrs.includes(attributes[i].name) && attributes[i].type) {
+            } else if (measures[i].name && !selectedAttrs.includes(measures[i].name) && measures[i].type) {
                 const att = {
-                    name: attributes[i].name,
-                    type: attributes[i].type,
-                    value: attributes[i].value
+                    name: measures[i].name,
+                    type: measures[i].type,
+                    value: measures[i].value
                 };
                 measureAttrsForCtxt.push(att);
             }
@@ -649,12 +655,7 @@ function sendUpdateValueNgsi2(entityName, attributes, typeInformation, token, ca
             return ![constants.COMMAND_STATUS, constants.COMMAND_RESULT].includes(obj.type);
         });
     }
-    let attributesCtxt = [...attsArrayFiltered]; // just copy
-    if (typeInformation.static) {
-        typeInformation.static.forEach(function (att) {
-            attributesCtxt.push(att);
-        });
-    }
+    attributesCtxt = attributesCtxt.concat(attsArrayFiltered); // just copy
     if (measureAttrsForCtxt) {
         measureAttrsForCtxt.forEach(function (att) {
             attributesCtxt.push(att);
@@ -1026,9 +1027,9 @@ function sendUpdateValueNgsi2(entityName, attributes, typeInformation, token, ca
 }
 
 exports.sendQueryValue = sendQueryValueNgsi2;
-exports.sendUpdateValue = function (entityName, attributes, typeInformation, token, callback) {
-    NGSIUtils.applyMiddlewares(NGSIUtils.updateMiddleware, attributes, typeInformation, () => {
-        return sendUpdateValueNgsi2(entityName, attributes, typeInformation, token, callback);
+exports.sendUpdateValue = function (entityName, measures, typeInformation, token, callback) {
+    NGSIUtils.applyMiddlewares(NGSIUtils.updateMiddleware, measures, typeInformation, () => {
+        return sendUpdateValueNgsi2(entityName, measures, typeInformation, token, callback);
     });
 };
 exports.addTimestamp = addTimestampNgsi2;