Wattsight API updates

Wattsight API WAPI
Product
Oct. 13, 2020Barbara Ferreira

We have released updates to our API. You will find new functions to aggregate data with.

With these updates, we have introduced changes to our aggregation functions. We now have strict and non-strict aggregations:

  • AVERAGE: Calculates the average value of the values that exist in the curve. Missing values are ignored in the calculation.
  • SAVERAGE: Calculates the average value with strict mode calculation allowing no gaps in the curve values. It will not return a value if there are gaps.
  • SUM: Sum of values. Missing values are ignored in the calculation.
  • SSUM: Calculates sum of values with strict calculation mode allowing no gaps in the curve values. It will not return a value if there are gaps.

Using non-strict and strict aggregation

When aggregating data from curves with frequency in Weeks (‘W’), while using a target frequency of Months (‘M’), Quarters (‘Q’), Seasons (‘S’) or Years (‘Y), there are some edge cases, or pitfalls, to look out for. These occur when configuring both target frequency and date range such that:

  • Target frequency translates into more days per period than the curve frequency does
  • The periods of the source and target frequencies does not line up at either or both the start or the end of the date range.

For instance: when aggregating one month worth of data using a frequency of months, on a curve that has data for each week or a frequency of ‘W’. If the targeted month is such that the first week (Monday through Sunday) is split between it and the previous month, setting the start date after Monday will result in a number of days worth of data missing from the the start of the aggregated period. Strict aggregation (SAVERAGE, SSUM) will in this case not return any value for the targeted period. Non-strict aggregation (AVERAGE, SUM) will return a value for the period, however since the calculation in this case is based on curve values with gaps in the data, the result will not actually be the sum or average value of the period that was intended.

How to avoid pitfalls

Curve data, at the source frequency, for a period is represented by a timestamp at the start of that period. And when selecting data, “data_from” is the first timestamp that should be included at the start, while “data_to” is the first timestamp that should be excluded at the end.

So to avoid the pitfalls mentioned, the start (‘data_from’) should be set to the start of your source frequency, while the end (‘data_to’) should be at the start of the “first unwanted” day of your target frequency.

Examples

In the following examples, we want to aggregate data for the period March through July 2019. We are aggregating with a frequency of ‘M’ on a curve with frequency ‘W’.

Example 1 - Aggregating values for the whole period as intended

example1-week-month-correct-from-to-pri-fi-fwd-raw-weather45-mwh-cet-w-f-2019-02-25-2019-08-01_.png
  • Parameter from_date is set to Monday, the 25th of February. This is the start of the week that crosses into, and constitutes the beginning of March. So the first data point in the selected range will be as intended.
  • Parameter to_date is set to the 1st of August. This is the first day after the date range of interest. Since to_date is not included in the calculation, the data for the last month will be as intended.

Example 2 - Date range starting too late

example2-week-month-start-to-late-pri-fi-fwd-raw-weather45-mwh-cet-w-f-2019-02-26-2019-08-01_.png
  • Parameter from_date is set to the 26th of February. Since this is a Tuesday and the data for weeks are indexed on Mondays, the data for the first days in March, belonging to this week, will not be included. And so, the first month in the selection will have data gaps. Non-strict aggregation will still return a data point for March, however this result will not be the actual sum or average for March, since not all days are present in the values.
  • Parameter to_date is set to the 1st of August. This is the first day after the date range of interest. Since to_date is not included, the data for the last month will be as intended.

Example 3 - Date range ending too late

example3-week-month-end-to-late-pri-fi-fwd-raw-weather45-mwh-cet-w-f-2019-02-25-2019-08-02_.png
  • Parameter from_date is set to Monday, the 25th of February. This is the start of the week that crosses into, and constitutes the beginning of March. So the first data point in the selected range will be as intended.
  • Parameter to_date is set to the 2nd of August. Since this means that the 1st of August is included in the aggregated date range, non-strict aggregation will return a data point for August in addition to the range date we intended to aggregate for. However this data point will not be the actual sum or average for August, since not all days are present in the values.

For more on this, consult our API documentation or reach out to us at support@wattsight.com

RETURN TO BLOG