A diffing transform computes the difference between consecutive data points.
Quantile properties
| Property | Type | Description | 
|---|---|---|
| diffing | string[] | Fields to compute diffings. Because this transformation reshape the data, provide all the fields to compute diffing together. | 
| carryOver | boolean | (Optional, default: true, only for streaming specs) Whether to get the last value from the lastest instance in history. | 
| keepFirstAsZero | boolean | (Optional, default: true) If add a zero value before the first datum. For streaming specs, ifcarryOveris set totrueand there are historical data points, then this is ignored. | 
| as | string[] | (Optional) New field names. The length must match that of diffing. | 
| groupby | Array[String] | Nominal fields to group the diffed values by. | 
Behavior
Suppose we have a dataset with three fields: category, speed, and distance, and we compute diffing for speed and distance.
| category | speed | distance | 
|---|---|---|
| car | 10 | 300 | 
| car | 11 | 290 | 
| bird | 1 | 12 | 
| bird | 1.1 | 13 | 
If groupby is not set, then it will result in:
| category | speed_diff | distance_diff | 
|---|---|---|
| car | 10 | 300 | 
| car | 1 | -10 | 
| bird | -10 | -278 | 
| bird | 0.1 | 1 | 
If groupby is set to ['category], then it will result in:
| category | speed_diff | distance_diff | 
|---|---|---|
| car | 10 | 300 | 
| car | 1 | -10 | 
| bird | 1 | 12 | 
| bird | 0.1 | 1 | 
As shown above, ‘next’ values are preserved for the non-diffed fields.
If keepFirstAsZero is set to false (with groupby), it will reslut in:
| category | speed_diff | distance_diff | 
|---|---|---|
| car | 1 | -10 | 
| bird | 0.1 | 1 | 
Usage pattern
JSON
{
  ...
  "transform" : [
    {
      "diffing": ["speed", "distance"],
      "groupby" ["category"],
      "carryOver": true
    },
  ]
  ...
}JavaScript
let stream = new Erie.Stream();
...
let diffing = new Erie.Diffing(["speed", "distance"], true, true, ["speed_diff", "distance_diff"]) // fields, carryOver, keepFirstAsZero, new field names
diffing.groupby(['category']);
stream.transform.add(diffing);
... Erie Documentation (Future)
            Erie Documentation (Future)