Elastic aggregation query (WIP)
Runs an aggregation of your own design against your own indices and maps the result onto assets. It is the escape hatch of the catalogue: when the data you need is already in Elasticsearch but no typed integration exposes it, this is how you get it into the inventory.
| Type name | elastic_aggregation_query_integration |
| Creates | Nothing by default — enriches existing endpoints |
| Runs on | A sensor with network reach to Elasticsearch |
| Fields written | Whichever you map |
How it works
Unlike every other integration, this one runs per asset rather than over a global list:
The query string supports {{field_name}} placeholders, replaced with that
asset's value for field_name. An asset missing any referenced field is skipped
rather than queried with a blank.
The aggregation is built from the fields named in field mapping aggregation — nested terms aggregations, each ordered by descending document count. Leaf buckets are flattened into rows carrying each level's key plus the document count.
Requirements
The integration is outbound-only: it runs an operator-authored aggregation on a schedule, so nothing is installed in the Elastic stack and nothing has to reach into the deployment.
Network access
| From | To | Protocol | Port |
|---|---|---|---|
| The sensor running the integration | Elasticsearch, at the URL configured on the instance | HTTPS over TCP | Whatever the URL names — 9200 on a default self-managed install, 443 on Elastic Cloud |
One host, and it is Elasticsearch itself rather than Kibana: the run issues a
_search against the configured index pattern. This is usually an internal
destination rather than an internet one, so the rule belongs in the internal
firewall.
Access in Elastic
| Need | Why |
|---|---|
| An account that can create Elasticsearch users or roles | The credential is a user with read and view_index_metadata on the index pattern being queried — see Permissions in Elastic |
| A role scoped to exactly that pattern | The query is operator-authored, so a wide role here is a standing capability to read anything in the cluster |
If Elasticsearch presents a certificate the sensor host does not trust, install the CA on the host in preference to enabling Ignore SSL — the credential is sent on every request.
In this deployment
| Need | Why |
|---|---|
| A registered, healthy sensor with reach to Elasticsearch | The integration runs on a sensor, on the platform's schedule — see Sensors |
| Assets already in the inventory | This integration enriches rather than discovers — each aggregation row is matched back to an existing asset, and rows matching none are skipped |
| A field on those assets that the aggregation can match on | The Field mapping aggregation pair names it; without a populated match field on both sides nothing is written |
Configuration
| Field | Required | Notes |
|---|---|---|
| Name | Yes | Label for this instance |
| URL | Yes | Elasticsearch base URL |
| Username | Yes | Elasticsearch user |
| Password | Yes | Password for that user |
| Cookies | No | Extra cookies added to requests |
| Sensor | Yes | Sensor that will run the integration |
| Asset type | No | Asset type to analyse |
| Index pattern | No | Index pattern to query, e.g. logs-* |
| Query | No | Query string, may contain {{field}} placeholders |
| Look back time (seconds) | No | Size of the time window |
| Look back field | No | Asset field supplying the window's end; current time if empty |
| Field mapping aggregation | No | Fields to aggregate on, as elastic_field → lumenedge_field pairs |
| Field mapping result | No | Bucket values to write, as elastic_field → lumenedge_field pairs |
| Processing mode | No | Process single hit only or Process popular hit for all |
Field mapping aggregation decides what the aggregation groups by and how a row is matched back to an asset. Field mapping result decides which of those values get written to which platform field. They frequently reference different fields.
Mapped values fill an empty field but never replace a populated one, regardless of the template's action settings. That makes this integration safe to point at an existing inventory, but it also means correcting an already-wrong value is not something this integration can do.
Permissions in Elastic
A user with read and view_index_metadata on the index pattern being queried.
Grant nothing wider — the query is operator-authored, so a broad role here is a
standing capability to read anything in the cluster.
The identification stage takes the first row for a matched asset and processes assets that resolve to exactly one match. Aggregations resolving to several assets are logged and skipped. Design the aggregation so its top-level key is unique per asset.
This type is currently wired for endpoints regardless of the Asset type setting. Document the supported asset types once the other types are handled, and add a worked end-to-end example with a real index, query and both mappings.
See also
- Elastic agent — for Fleet-enrolled hosts, no query required
- Elastic asset owner — a purpose-built version of this pattern for ownership
- Integrations overview — the shared configuration model