Uploaded image for project: 'Help-Desk'
  1. Help-Desk
  2. HELP-5285

FIWARE.Request.Tech.Data.CEP.PROTON CEP

    Details

    • Type: extRequest
    • Status: Closed
    • Priority: Major
    • Resolution: Done
    • Fix Version/s: 2021
    • Component/s: FIWARE-TECH-HELP
    • Labels:
      None

      Description

      Dear Tech Support,

      We would like to set in PROTON CEP the following alert settings:

      For every row captured from the sensor we want to calculate the average measure of last hour measures and we want to compare with captured value.
      If the difference between the average is greater than 10% we would like to generate and alarm event.

      Would you help us how we set this alert in the PROTON CEP? We reviewed the document and try many settings combination but we haven't found the solution.

      Thanks,
      Zora Barta

      Since January 1st, old domains won't be supported and messages sent to any domain different to @lists.fiware.org will be lost.
      Please, send your messages using the new domain (Fiware-tech-help@lists.fiware.org) instead of the old one.
      _______________________________________________
      Fiware-tech-help mailing list
      Fiware-tech-help@lists.fiware.org
      https://lists.fiware.org/listinfo/fiware-tech-help
      [Created via e-mail received from: =?iso-8859-2?Q?Barta_Z=F3ra?= <barta.zora@seacon.hu>]

      1. image001.png
        79 kB
      2. image002.png
        82 kB
      3. image003.png
        76 kB
      4. image004.png
        83 kB
      5. image005.png
        76 kB
      6. image006.png
        87 kB
      7. image007.png
        89 kB
      8. image008.png
        90 kB
      9. image009.png
        133 kB

        Activity

        Hide
        mev Manuel Escriche added a comment -

        Dear Zora,

        This issue is assigned to the GE Owner for you support.

        Kind regards,
        Manuel

        Show
        mev Manuel Escriche added a comment - Dear Zora, This issue is assigned to the GE Owner for you support. Kind regards, Manuel
        Hide
        TALI Fabiana Fournier added a comment -

        Dear Zora,

        I'm sure there are several options to implement your scenario.
        Here is my suggestion.

        First I would implement an EPA that computes the average measure for the last hour. Let call it avgMeasureEPA.

        This EPA should use a temporal context of sliding time window of 1 hour duration and one hour sliding period. It should be initiated at startup and never end.

        This EPA should be of type aggregate and compute the average measure above its input sensor events. Its output event, let call it avgMeasureEvent should hold the average value in one of its attribute, let call it average.

        I would recommend you to implement this EPA first and test it (you can change the sliding window duration for a shorter reasonable testing value).

        When you are happy with this EPA, you can define another EPA, let call it alarmEPA, that will compare between the previous hour average and the current measurement.

        For this alarmEPA I would define a temporal context with initiator of avgMeasureEvent, and a relative terminator of one hour.
        This alarmEPA can be of sequence type, with two input event. The first one is avgMeasureEvent, and the second is your sensor input event. It should have a condition to detect the pattern if the sensor measure is greater than 10% from the avgMeasureEvent average. Since the first input event of this pattern can be used to detect several deviations, you should set its consumption parameter to be "reuse". Since the second input event can be used only once, you should set its consumption parameter to be "consume".

        Hope this will give you a good starting point.
        Best regards,
        Tali Yatzkar Haham

        Show
        TALI Fabiana Fournier added a comment - Dear Zora, I'm sure there are several options to implement your scenario. Here is my suggestion. First I would implement an EPA that computes the average measure for the last hour. Let call it avgMeasureEPA. This EPA should use a temporal context of sliding time window of 1 hour duration and one hour sliding period. It should be initiated at startup and never end. This EPA should be of type aggregate and compute the average measure above its input sensor events. Its output event, let call it avgMeasureEvent should hold the average value in one of its attribute, let call it average. I would recommend you to implement this EPA first and test it (you can change the sliding window duration for a shorter reasonable testing value). When you are happy with this EPA, you can define another EPA, let call it alarmEPA, that will compare between the previous hour average and the current measurement. For this alarmEPA I would define a temporal context with initiator of avgMeasureEvent, and a relative terminator of one hour. This alarmEPA can be of sequence type, with two input event. The first one is avgMeasureEvent, and the second is your sensor input event. It should have a condition to detect the pattern if the sensor measure is greater than 10% from the avgMeasureEvent average. Since the first input event of this pattern can be used to detect several deviations, you should set its consumption parameter to be "reuse". Since the second input event can be used only once, you should set its consumption parameter to be "consume". Hope this will give you a good starting point. Best regards, Tali Yatzkar Haham
        Hide
        fw.ext.user FW External User added a comment -

        Dear Tali,

        Thanks for quick answer. Your suggestion was very logical. We wanted to execute step by step.

        Unfortunately the first average EPA doesn’t work. It always gives the last received value. We found some other mysterious settings what we cannot decide. We tried some combinations but unsuccessfully. We used shorter period, 10 sec.

        I send you our settings maybe you find immediately the wrong one.

        Thanks for your help:

        Zora

        Setting Json:

        {"epn":{"events":[{"name":"SensorContextUpdate","createdDate":"Wed Aug 05 2015","attributes":[

        {"name":"entityId","type":"String","dimension":"0"}

        ,

        {"name":"entityType","type":"String","dimension":"0"}

        ,

        {"name":"Id","type":"String","dimension":"0"}

        ,

        {"name":"MeasureDate","type":"String","dimension":"0"}

        ,

        {"name":"Value","type":"String","dimension":"0"}

        ]},{"name":"AvgMeasureEvent","createdDate":"Tue Nov 17 2015","attributes":[

        {"name":"Certainty","type":"Double","defaultValue":"1","dimension":0,"description":"The certainty that this event happen (value between 0 to 1)"}

        ,

        {"name":"OccurrenceTime","type":"Date","dimension":0,"description":"No value means it equals the event detection time, other option is to use one of the defined distribution functions with parameters"}

        ,

        {"name":"ExpirationTime","type":"Date","dimension":0}

        ,

        {"name":"Cost","type":"Double","dimension":0,"description":"The cost of this event occurrence. Negative if this is an opportunity"}

        ,

        {"name":"Duration","type":"Double","defaultValue":"0","dimension":0,"description":"Used in case the this event occur within an interval"}

        ,

        {"name":"Average","type":"Double","dimension":"0"}

        ]}],"epas":[{"name":"AvgMeasureEPA","createdDate":"Tue Nov 17 2015","epaType":"Aggregate","context":"AvgMeasureContext","inputEvents":[

        {"name":"SensorContextUpdate","consumptionPolicy":"Reuse","instanceSelectionPolicy":"First"}

        ],"computedVariables":[

        {"name":"AvgMeasure","aggregationType":"Average","SensorContextUpdate":"StringToDouble(SensorContextUpdate.Value)"}

        ],"evaluationPolicy":"Immediate","cardinalityPolicy":"Single","internalSegmentation":[],"derivedEvents":[{"name":"AvgMeasureEvent","reportParticipants":false,"expressions":{"Duration":"0","Average":"AvgMeasure"}}]}],"contexts":{"temporal":[

        {"name":"AvgMeasureContext","createdDate":"Tue Nov 17 2015","type":"SlidingTimeWindow","atStartup":true,"neverEnding":true,"duration":10000,"slidingPeriod":10000,"initiators":[],"terminators":[]}

        ],"segmentation":[],"composite":[]},"consumers":[{"name":"AvgMeasureConsumer","createdDate":"Tue Nov 17 2015","type":"File","properties":[

        {"name":"filename","value":"AvgMeasure.csv"}

        ,

        {"name":"formatter","value":"tag"}

        ,

        {"name":"delimiter","value":";"}

        ,

        {"name":"tagDataSeparator","value":"="}

        ,

        {"name":"SendingDelay","value":"1"}

        ,

        {"name":"dateFormat","value":"dd/MM/yyyy-HH:mm:ss"}

        ],"events":[

        {"name":"AvgMeasureEvent"}

        ]}],"producers":[],"name":"AvgMeasure"}}

        User inferface:

        [cid:image002.png@01D1213A.6566A2E0]

        [cid:image003.png@01D1213A.6566A2E0]

        [cid:image004.png@01D1213A.6566A2E0]

        [cid:image005.png@01D1213A.6566A2E0]

        [cid:image006.png@01D1213A.6566A2E0]

        [cid:image007.png@01D1213A.6566A2E0]

        [cid:image008.png@01D1213A.6566A2E0]
        [cid:image009.png@01D1213A.6566A2E0]

        Message 1:
        <notifyContextRequest>
        <subscriptionId>5502d5ca984d98db424f19ae</subscriptionId>
        <originator>localhost</originator>
        <contextResponseList>
        <contextElementResponse>
        <contextElement>
        <entityId type="Sensor" isPattern="false">
        <id>101</id>
        </entityId>
        <contextAttributeList>
        <contextAttribute>
        <name>Id</name>
        <type>String</type>
        <contextValue>102</contextValue>
        </contextAttribute>
        <contextAttribute>
        <name>MeasureDate</name>
        <type>Date</type>
        <contextValue>2015.11.17 10:32:58</contextValue>
        </contextAttribute>
        <contextAttribute>
        <name>Value</name>
        <type>String</type>
        <contextValue>65.1</contextValue>
        </contextAttribute>
        </contextAttributeList>
        </contextElement>
        <statusCode>
        <code>200</code>
        <reasonPhrase>OK</reasonPhrase>
        </statusCode>
        </contextElementResponse>
        </contextResponseList>
        </notifyContextRequest>

        Message 2:

        <notifyContextRequest>
        <subscriptionId>5502d5ca984d98db424f19ae</subscriptionId>
        <originator>localhost</originator>
        <contextResponseList>
        <contextElementResponse>
        <contextElement>
        <entityId type="Sensor" isPattern="false">
        <id>101</id>
        </entityId>
        <contextAttributeList>
        <contextAttribute>
        <name>Id</name>
        <type>String</type>
        <contextValue>102</contextValue>
        </contextAttribute>
        <contextAttribute>
        <name>MeasureDate</name>
        <type>Date</type>
        <contextValue>2015.11.17 10:32:58</contextValue>
        </contextAttribute>
        <contextAttribute>
        <name>Value</name>
        <type>String</type>
        <contextValue>75.1</contextValue>
        </contextAttribute>
        </contextAttributeList>
        </contextElement>
        <statusCode>
        <code>200</code>
        <reasonPhrase>OK</reasonPhrase>
        </statusCode>
        </contextElementResponse>
        </contextResponseList>
        </notifyContextRequest>

        Result (server log):

        [cid:image001.png@01D1213A.019644A0]

        Show
        fw.ext.user FW External User added a comment - Dear Tali, Thanks for quick answer. Your suggestion was very logical. We wanted to execute step by step. Unfortunately the first average EPA doesn’t work. It always gives the last received value. We found some other mysterious settings what we cannot decide. We tried some combinations but unsuccessfully. We used shorter period, 10 sec. I send you our settings maybe you find immediately the wrong one. Thanks for your help: Zora Setting Json: {"epn":{"events":[{"name":"SensorContextUpdate","createdDate":"Wed Aug 05 2015","attributes":[ {"name":"entityId","type":"String","dimension":"0"} , {"name":"entityType","type":"String","dimension":"0"} , {"name":"Id","type":"String","dimension":"0"} , {"name":"MeasureDate","type":"String","dimension":"0"} , {"name":"Value","type":"String","dimension":"0"} ]},{"name":"AvgMeasureEvent","createdDate":"Tue Nov 17 2015","attributes":[ {"name":"Certainty","type":"Double","defaultValue":"1","dimension":0,"description":"The certainty that this event happen (value between 0 to 1)"} , {"name":"OccurrenceTime","type":"Date","dimension":0,"description":"No value means it equals the event detection time, other option is to use one of the defined distribution functions with parameters"} , {"name":"ExpirationTime","type":"Date","dimension":0} , {"name":"Cost","type":"Double","dimension":0,"description":"The cost of this event occurrence. Negative if this is an opportunity"} , {"name":"Duration","type":"Double","defaultValue":"0","dimension":0,"description":"Used in case the this event occur within an interval"} , {"name":"Average","type":"Double","dimension":"0"} ]}],"epas":[{"name":"AvgMeasureEPA","createdDate":"Tue Nov 17 2015","epaType":"Aggregate","context":"AvgMeasureContext","inputEvents":[ {"name":"SensorContextUpdate","consumptionPolicy":"Reuse","instanceSelectionPolicy":"First"} ],"computedVariables":[ {"name":"AvgMeasure","aggregationType":"Average","SensorContextUpdate":"StringToDouble(SensorContextUpdate.Value)"} ],"evaluationPolicy":"Immediate","cardinalityPolicy":"Single","internalSegmentation":[],"derivedEvents": [{"name":"AvgMeasureEvent","reportParticipants":false,"expressions":{"Duration":"0","Average":"AvgMeasure"}}] }],"contexts":{"temporal":[ {"name":"AvgMeasureContext","createdDate":"Tue Nov 17 2015","type":"SlidingTimeWindow","atStartup":true,"neverEnding":true,"duration":10000,"slidingPeriod":10000,"initiators":[],"terminators":[]} ],"segmentation":[],"composite":[]},"consumers":[{"name":"AvgMeasureConsumer","createdDate":"Tue Nov 17 2015","type":"File","properties":[ {"name":"filename","value":"AvgMeasure.csv"} , {"name":"formatter","value":"tag"} , {"name":"delimiter","value":";"} , {"name":"tagDataSeparator","value":"="} , {"name":"SendingDelay","value":"1"} , {"name":"dateFormat","value":"dd/MM/yyyy-HH:mm:ss"} ],"events":[ {"name":"AvgMeasureEvent"} ]}],"producers":[],"name":"AvgMeasure"}} User inferface: [cid:image002.png@01D1213A.6566A2E0] [cid:image003.png@01D1213A.6566A2E0] [cid:image004.png@01D1213A.6566A2E0] [cid:image005.png@01D1213A.6566A2E0] [cid:image006.png@01D1213A.6566A2E0] [cid:image007.png@01D1213A.6566A2E0] [cid:image008.png@01D1213A.6566A2E0] [cid:image009.png@01D1213A.6566A2E0] Message 1: <notifyContextRequest> <subscriptionId>5502d5ca984d98db424f19ae</subscriptionId> <originator>localhost</originator> <contextResponseList> <contextElementResponse> <contextElement> <entityId type="Sensor" isPattern="false"> <id>101</id> </entityId> <contextAttributeList> <contextAttribute> <name>Id</name> <type>String</type> <contextValue>102</contextValue> </contextAttribute> <contextAttribute> <name>MeasureDate</name> <type>Date</type> <contextValue>2015.11.17 10:32:58</contextValue> </contextAttribute> <contextAttribute> <name>Value</name> <type>String</type> <contextValue>65.1</contextValue> </contextAttribute> </contextAttributeList> </contextElement> <statusCode> <code>200</code> <reasonPhrase>OK</reasonPhrase> </statusCode> </contextElementResponse> </contextResponseList> </notifyContextRequest> Message 2: <notifyContextRequest> <subscriptionId>5502d5ca984d98db424f19ae</subscriptionId> <originator>localhost</originator> <contextResponseList> <contextElementResponse> <contextElement> <entityId type="Sensor" isPattern="false"> <id>101</id> </entityId> <contextAttributeList> <contextAttribute> <name>Id</name> <type>String</type> <contextValue>102</contextValue> </contextAttribute> <contextAttribute> <name>MeasureDate</name> <type>Date</type> <contextValue>2015.11.17 10:32:58</contextValue> </contextAttribute> <contextAttribute> <name>Value</name> <type>String</type> <contextValue>75.1</contextValue> </contextAttribute> </contextAttributeList> </contextElement> <statusCode> <code>200</code> <reasonPhrase>OK</reasonPhrase> </statusCode> </contextElementResponse> </contextResponseList> </notifyContextRequest> Result (server log): [cid:image001.png@01D1213A.019644A0]
        Hide
        TALI Fabiana Fournier added a comment -

        Dear Zora,

        The issue is with your EPA definition.

        Since the Evaluation policy is set to "immediate", it means that immediately after every input event the pattern is evaluated and if it holds the conditions, it will generate an output event. This will happen after the first input event arrives. Since the Cardinality policy is set to "Single" the pattern will be detected only once during the temporal context, with the value of the first event only.

        What you need to do is to change the Evaluation policy to "Deferred". This way the pattern will be evaluated at the end of the temporal context (after an hour), and will have average based on all the input events that arrived in this period.

        Hope it helps,
        Tali Yatzkar Haham

        Show
        TALI Fabiana Fournier added a comment - Dear Zora, The issue is with your EPA definition. Since the Evaluation policy is set to "immediate", it means that immediately after every input event the pattern is evaluated and if it holds the conditions, it will generate an output event. This will happen after the first input event arrives. Since the Cardinality policy is set to "Single" the pattern will be detected only once during the temporal context, with the value of the first event only. What you need to do is to change the Evaluation policy to "Deferred". This way the pattern will be evaluated at the end of the temporal context (after an hour), and will have average based on all the input events that arrived in this period. Hope it helps, Tali Yatzkar Haham
        Hide
        fw.ext.user FW External User added a comment -

        Dear Tali,

        Thanks for the correction. We tried this settings and also the other part of your suggestions.
        It works if the measures are received strict frequency. If the time difference is not periodical because of delay sometimes we experienced missing alarm event because the avg time window expired.

        Thanks for your help.

        Best regards,
        Zora

        Show
        fw.ext.user FW External User added a comment - Dear Tali, Thanks for the correction. We tried this settings and also the other part of your suggestions. It works if the measures are received strict frequency. If the time difference is not periodical because of delay sometimes we experienced missing alarm event because the avg time window expired. Thanks for your help. Best regards, Zora
        Hide
        TALI Fabiana Fournier added a comment -

        If the current CEP definitions doesn't fit your scenario you have several options to change them. Two options I can think of:

        1. Increase the period of the temporal context that is used to compute the average so there are enough samples in it.
        2. Change the second EPA (of type sequence) to have an "always" context (starts at startup and never ends) and have its first event instance selection policy to "override", and its consumption policy to "reuse", while having its second event selection policy to "override" and its consumption policy to "consume".
        This way the EPA will evaluate based on the last "average" and the last sensor event. Which means that even if in the last hour no average was reported, it will evaluate based on the last reported average.

        Hope it helps,
        Tali Yatzkar Haham

        Show
        TALI Fabiana Fournier added a comment - If the current CEP definitions doesn't fit your scenario you have several options to change them. Two options I can think of: 1. Increase the period of the temporal context that is used to compute the average so there are enough samples in it. 2. Change the second EPA (of type sequence) to have an "always" context (starts at startup and never ends) and have its first event instance selection policy to "override", and its consumption policy to "reuse", while having its second event selection policy to "override" and its consumption policy to "consume". This way the EPA will evaluate based on the last "average" and the last sensor event. Which means that even if in the last hour no average was reported, it will evaluate based on the last reported average. Hope it helps, Tali Yatzkar Haham
        Hide
        fw.ext.user FW External User added a comment -

        Dear Tali,

        Thanks for your suggestions specially as your weekend activity. We wil try them. I hope it will be succesful.

        Best regards,
        Zora

        Show
        fw.ext.user FW External User added a comment - Dear Tali, Thanks for your suggestions specially as your weekend activity. We wil try them. I hope it will be succesful. Best regards, Zora

          People

          • Assignee:
            TALI Fabiana Fournier
            Reporter:
            fw.ext.user FW External User
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: