Technical documentation
This section documents the mechanisms around Insights: how structure and data get in, how you model your own domain on top of ours, how derived data comes out, and the syntaxes shared across the platform.
It is the counterpart of the API documentation. The API section describes the endpoints; this one describes the file formats, the templates, the grammars and the background processes those endpoints drive.
The map
Read it across: structure exists first, then data flows into it, then everything you build on top.
Getting structure in
Master Data is the bulk mechanism for creating and updating sites, sources, variables, events and users — the alternative to calling the API entity by entity.
Master Data introduction — the template file plus the data file, and how the two combine.
Whatif mode — run the whole thing and see what would change, without changing it.
Golden rule
Run a master data load in whatif mode before running it for real.
A master data file touches every entity it matches, and a business key that resolves one row too broadly is indistinguishable from a correct one until after the load. Whatif turns that into a diff you can read.
Getting data in
The JSON payload of the push API, and the webhook that reports what actually happened.
Invoice data pushed in bulk.
Files arriving by FTP, storage or HTTP are matched by a trigger — see Files and triggers.
Modelling your own domain
The native model — site, source, variable — is deliberately simple. Two mechanisms let you express a richer domain on top of it without leaving Insights.
Open Data Model over SQL — Insights entities kept in sync in your own SQL tables, queried with SQL.
Open Data Model over GraphQL — a custom schema, queries and mutations.
Geozones — territorial grouping of sites, computed from their coordinates.
Getting data out
Second level data query — querying the result of a first query.
Statistics on variables — aggregates computed and kept per variable.
Signal loss detection — spotting variables that stopped reporting.
Webhooks — events pushed to your endpoint instead of you polling ours.
Data forwarding — streaming incoming data points on to your own queue.
Webhooks and forwarding both replace polling. Reach for them before writing a loop that asks "anything new?".
Shared syntaxes
These are used from several places in the platform, and are worth reading once.
Custom filter query syntax — the grammar behind CustomFilter on /sites and /sources, and behind the portal search box. Operators, form paths, and a BNF grammar.
Cron syntax — the scheduling expressions used by reports and jobs.
Reports template syntax — building a custom report layout.
Security
Authentication flow — how a token is issued and what it carries.
Impersonate — acting on behalf of another user, and the limits that apply.
Connect to the API — credentials, scopes and the practical steps.
Related sections
- API documentation — the endpoints themselves, and how to query them.
- User Manual — the same features, seen from the interface.
- Documentation changelog — how this documentation is versioned.