Aggregates
Vessel Insight apply both real time aggregation and pre-aggregation to provide performant responses to data queries used in the Aggregates, vesselsensorevents and OnGrid endpoints.
The aggregate interval options are:
1s, 5s, 10s, 1m, 5m, 10m, 1h, 6h, 1d, 1w, 1M, 3M, 1y
Real time aggregate freshness
Any aggregate levels below 5m utilize the raw data to compute the aggregates to the API consumer, thus these will always be fresh within few minutes if raw data is available.
Pre-aggregation freshness
Aggregates from 5m and up utilize the pre-aggregates which is depending on the pre-aggregation policy. Calculation of pre-aggregates are initiated on schedule every 10 minutes, and will calculate the previous 10 minute window with a 10 minute buffer offset. This means that aggregates are not generated until after 10-19 minutes for fresh data.
Lets look at a few examples:
- 5m aggregates for the period of 11:00 to 11:09. Policy starts after 10 minutes buffer time, which could be any time after 11:20 (but before 11:30), depending on system resources. Aggregates will start coming for the period 11:00 to 11:09 any time after 11:20.
- 5m aggregates for the period of 11:00 to 11:10. Policy for 11:00 to 11:09 starts as per above example. Policy for period 11:10 and onwards starts after 10 minutes buffer time, which could be any time after 11:30 (but before 11:40), depending on system resources. Aggregates will start coming for the period 11:10 any time after 11:30.
- The same concept for freshness will apply for all aggregates above 5m
- 6h and 1d aggregates are starting from 00:00
- 1w aggregates are starting from Monday at 00:00
- 1M, 3M and 1y aggregates are starting from the 1. day of the month at 00:00
Time Weighted Average aggregate
In cases where we are interested in accumulating values from e.g. a flow rate sensor or a power sensor and the sampling frequency is irregular, the standard "Average" aggregate is not sufficient. We are providing a Time weighted average as a new aggregate in the vesselsensorevents endpoint. This new aggregate will interpolate values to ensure a proper average for accumulates. The last observed value is applied as interpolated values which is appropriate to use in cases where sensor values updates at change.
As an example, we will use the time information from a fictitious Voyage to get the data from a path mapped to a mass flow sensor FO_Mass_Flow :
Curl --location 'https://api.kognif.ai/galore/v2/timeseries/vesselsensoreventsbypath'
\--header 'Ocp-Apim-Subscription-Key: [YOUR-OCP-APIM-SUBSCRIPTION-KEY]'
\--header 'Authorization: ••••••'
\--data '{<body>]}
This mass flow sensor gives us time irregular data, in this case kg/h, and we are using the preview API endpoint to get the data:
We are using a time_weighted_average aggregation to ensure we are getting values for each period within the time interval, this is how body looks like:
{
"from":"2024-11-01T01:01:01Z",
"to":"2024-11-04T10:00:30Z",
"interval":"1m",
"resamplingMethods":[
"time_weighted_average"
],
"paths":[
"/Fleet/imoxxxxxxx/Engines/Main/1/FO_Mass_Flow"
]
}
The response, shortened for readability:
{
"intervalInSeconds":60,
"resamplingMethods":[
"time_weighted_average"
],
"time":"2024-11-01T01:00:00Z",
"interval":"1m",
"columns":[
{
"path":"/Fleet/imoxxxxxxx/Engines/Main/1/FO_Mass_Flow",
"unit":"kg/h",
"samples":{
"time_Weighted_Average":[
741.6849276974416,
738.2358333333333,
"…"
],
"dataValidationErrors":{}
}
}
],
"errors":[]
}
We can now summarize all 1-hour time weighted averages and divide by 60 to get the accumulated kg.