Running Cassiopeia
cassiopeia map runs the pipeline. It reads a source, maps each record to an NGSI-LD entity, validates the result, and writes the entities to files or a context broker. This guide walks through that command and its sibling commands by task. It explains what each capability does, when to use it, and which flags control it. It is not an exhaustive flag reference. cassiopeia map --help lists every flag with its current default, and the linked topic pages cover the details.
Two ways to run
A run is described either by inline flags or by a manifest. The two forms are mutually exclusive. Inline flags spell out the whole run on the command line:
cassiopeia map --input stations.csv --mapping station.json5 --output out
A manifest packages the same description, including inputs, output, validation, and schedule, into a reusable file. Run it with --manifest (-M):
cassiopeia map --manifest run.json5
Because a manifest describes the entire run, it conflicts with every inline run-shaping flag. Mixing --manifest with an inline flag such as --output or --validation-mode is a parse error, not a silent override. Engine knobs and the global --config are exceptions. They tune how the pipeline executes rather than what the run does, so they remain valid alongside a manifest. See Control how the run behaves.
Many run settings have two equivalent forms: a command-line flag and a manifest field. Examples include --validation-mode and output.validation, and --on-failure and the top-level onFailure. This guide names both forms. The manifest field reference is in the manifest guide.
Choose the source
Every run needs an input and a mapping that shapes it:
--input(-i): the data source, a local file path or anhttp/httpsURL.--input-header: an extra request header sent when fetching a remote input, written"Name: Value". Repeat the flag for several headers. This is where a source's API key goes, for example--input-header "X-Api-Key: <key>". A local input refuses it. The manifest form is the input'sheaders(see the manifest guide).--mapping(-m): the JSON5 mapping file that defines the transformation rules.--type(-t): the source format. It defaults toauto, which detects the format from the content, so you rarely need to set it. Give it explicitly when detection cannot tell, such as with a headerless CSV, or when you want to force a specific reader. The accepted values areauto,csv,json,geojson,kml,kmz,grib,shapefile,geopackage,netcdf, andxml.
The source-format guide describes the records each format produces and how detection works.
Choose the destination
--writer (-w) selects where entities go: file (the default) or context-broker.
Write to files
A file run needs an output directory and can optionally set a framing:
--output(-o): the output directory. Required when the writer isfile. Cassiopeia writes one file per entity type.--framing:array(the default) writes one JSON array per type;line-delimitedwrites one entity per line, which suits streaming and append-oriented tools.
cassiopeia map -i stations.csv -m station.json5 -o out --framing line-delimited
The output guide covers the file-name and extension rules in full.
Send to a broker
A broker run needs the broker URL and can optionally set options for each HTTP request:
-
--broker-url(-u): the broker base URL. Required when the writer iscontext-broker. -
--broker-operation: the NGSI-LD operation each request performs. One of:Value Use it for upsertOrdinary bulk delivery. The default. createInserting entities that must not already exist. updateUpdating attributes of existing entities. mergeMerging incoming attributes into existing entities. temporalPosting to the temporal API, one entity per request. -
--upsert-mode: forupsert, whether an existing entity is replaced wholesale (replace, the default) or updated in place (update). -
--attribute-overwrite: forupdate, whether existing attributes are overwritten (overwrite, the default) or preserved (no-overwrite). -
--atomic: spool every entity locally and push it to the broker only after a clean finish, so a failed run delivers nothing. Without it, entities stream to the broker as they are produced. -
--tenant: theNGSILD-Tenantheader for a multi-tenant broker. -
--user-agent: override the defaultUser-Agentthe writer sends. -
--output-header: an extra request header, written"Name: Value". Repeat the flag for several headers. This is where broker credentials go, for example--output-header "Authorization: Bearer <token>". -
--link-header(-L): send@contextin aLinkheader instead of embedding it in each entity body.
cassiopeia map -i readings.json -m reading.json5 \
--writer context-broker \
--broker-url http://localhost:1026/ \
--broker-operation upsert \
--output-header "Authorization: Bearer replace-me"
The output guide documents the operations, atomic delivery, and context delivery in full.
Shape the entities
Two flags shape the entities themselves, independently of the destination:
--writer-representation: the NGSI-LD representation for written entities:normalized(the default),concise, orsimplified. Choose it for whatever consumes the output.--temporal-representation: the temporal output shape.seriesfolds each base ID's observations into a single temporal entity with time-ordered instance arrays (an EntityTemporal, ETSI GS CIM 009 v1.9.1 clause 5.2.20); absent writes current-state, with one entity per ID and each attribute keeping its latest observation. This applies to file and broker writers. It is not the same as--broker-operation temporal: this flag shapes the output, while the broker operation only selects the temporal endpoint. Aseriesrepresentation against a broker requires--broker-operation temporal.
The representation guide explains how the written representation stays independent of the one validation checks; the output guide explains the temporal shapes.
Attach @context
--context (-C) selects how Cassiopeia attaches the JSON-LD @context:
| Mode | Meaning |
|---|---|
none | Attach no @context. |
default | Resolve the context for the mapped data model from the local Smart Data Models catalog. The default. |
url | Use an explicit remote URL, given with --context-url. |
local | Inline a local .jsonld file, given with --context-file. |
--context-url is required when the mode is url, and --context-file is required when the mode is local. This mode and argument form is specific to the command line. A manifest instead carries context as a single string that is read by content (see the manifest guide). The output guide explains how the resolved context reaches a file body or a broker request, and what a local context file must hold. A local file Cassiopeia cannot attach fails the run before anything is written.
Validate
Validation checks each entity against its data model's JSON Schema before writing. Use these command-line controls:
--validation-mode: how strict a failure is:warn,fail-when-schema(the default), orfail.--validation-representation: the representation used to check entities. It defaults to the key-values (simplified) form described by the schemas.--validation-report(-r): write a JSON validation report to the given path.
The validation guide covers the modes, the outcome matrix, and where schemas come from.
Handle failures
A cycle fails when a stage reports a fatal error, such as a validation abort under --validation-mode fail or a write that cannot be completed. --on-failure decides what the run does next and which exit status it returns. The same policy governs one-shot runs and repeating schedules. Its manifest equivalent is the top-level onFailure field. Like --validation-mode, the flag shapes the run and is mutually exclusive with --manifest.
--on-failure | Scheduled run: after a failed cycle | One-shot run | Process exit |
|---|---|---|---|
abort (the default) | stop the schedule immediately | the single cycle fails | non-zero |
continue | log it, run the remaining cycles | finish | non-zero if any cycle failed |
ignore | log it, run the remaining cycles | finish | zero always |
Warnings never fail a run. --validation-mode warn and the missing-schema warning under the default fail-when-schema mode both leave the process at exit 0 under every --on-failure mode.
Exit status
Cassiopeia reports a run's outcome to the OS, so it can act as a CI gate. Exit 0 means success, with warnings allowed. A non-zero exit means a fatal abort under abort or continue. ignore always exits 0. The run summary's errors and warnings counts reflect the failures used to determine the exit status.
Read the run summary
When a run counted any errors or warnings, the summary ends with a Reasons block naming what they were:
Reasons
reason count example
broker-entity-rejected 142 attribute 'dateObserved' is not a valid DateTime
schema-nonconformant 18 /temperature: required property missing
Each row is one reason code, how many occurrences it accounted for, and one example of it. The codes are stable, so they can be grepped for and quoted in a bug report. The rows are ordered worst first: errors before warnings, and within each, the loudest reason first.
The block is drawn only when the run actually counted something to explain. A clean run ends at the counters.
Mapping problems
Cassiopeia checks every template in a mapping when it loads the mapping. A template it cannot compile fails the cycle before any record is read, and --on-failure decides the exit status as for any other failed cycle. The error names the template's place in the mapping and the mapping file, and its second line says what to change:
× The attribute `station` template in the mapping document at 'station.json5' cannot be compiled
╰─ `station-id` in `{{ station-id }}` is ambiguous: write `this['station-id']` to read the field, or `station - id` to subtract
A geometry declaration that could never run is rejected at load time the same way, naming the attribute: a GeoProperty whose transformation produces no geometry, such as string, a geometry block on an attribute whose value is never converted to a geometry, and a geometry conversion that cannot produce the declared type. See GeoProperty.
Problems that depend on one record's data, on the rows and header of one CSV file, or on one table of a GeoPackage, do not stop the run. Each is a warning, and the Reasons block counts it under one of these codes:
| Code | What happened |
|---|---|
expander-urn-ungeneratable | The record's identity resolved to nothing, or its identity.entityName or a scope template failed to render, so Cassiopeia skipped the record. An identity resolves to nothing when a field it reads is missing, null, or empty, or when its text has no letter or digit, such as -. A synthetic entity's scope that fails to render or is not a valid scope skips the record too. A scope or relationship target that resolves to nothing, rather than failing, is left out without a warning. |
expander-expires-at-unreadable | An entity expiresAt resolved to text Cassiopeia cannot read as a date and time, so the entity is written without it. One warning per entity type names the first entity that lost it, the text, and how many records did. |
expander-expires-at-unresolvable | An entity expiresAt template failed to render, so Cassiopeia skipped the record. |
expander-relationship-unresolvable | A relationship's source, or the identity of the synthetic entity it declares, failed to render, so Cassiopeia left out that link and that synthetic entity and kept the entity. This covers a Relationship, a ListRelationship, each of their instances, and a relationship nested under another attribute. One warning per relationship names it by its path, such as refRoad or hasLeadActor.playsCharacter, with the first entity that lost it. Run with -v to see Tera's reason. |
extractor-boolean-unreadable | A value read with the boolean transformation spells no boolean, such as maybe, an array, or an object, so Cassiopeia dropped that attribute, sub-attribute, or instance and kept the entity. A missing, null, or blank value produces no warning. The warning names the attribute by its path and the first value it could not read. See Convert values. |
extractor-geometry-unreadable | A value read as a geometry, by a GeoProperty or under a geometry transformation such as point, is there but spells no GeoJSON geometry, such as text that is not GeoJSON, an object that is not a valid geometry, or a number, so Cassiopeia dropped that attribute, sub-attribute, or instance and kept the entity. A missing, null, or blank value produces no warning. The warning names the attribute by its path and the first value it could not read as a geometry. See GeoProperty. |
extractor-iri-unreadable | A value read as an IRI, by a VocabProperty or the iri transformation, is not an absolute IRI, such as not an iri or the relative reference /a/b, so Cassiopeia dropped that attribute, sub-attribute, or instance and kept the entity. The warning names the attribute by its path, such as temperature.level for a sub-attribute, and the first value it could not use. A missing, null, or blank value produces no warning. See VocabProperty and Convert values. |
extractor-json-payload-misshapen | A JsonProperty's source is not a JSON object or an array of objects, such as text, a number, or an array of numbers, so Cassiopeia dropped that attribute and kept the entity. The warning names the attribute and what the source was. See ListProperty and JsonProperty. |
extractor-json-text-unreadable | A value read with the json transformation is text that is not JSON, so Cassiopeia dropped that attribute, sub-attribute, or instance and kept the entity. The warning names the attribute by its path and the first text it could not read. See Convert values. |
extractor-language-map-misshapen | A LanguageProperty's source is not an object of text keyed by language tag, such as plain text or a number, so Cassiopeia dropped that attribute, sub-attribute, or instance and kept the entity. The warning names the attribute by its path, such as temperature.label for a sub-attribute, and the first value it could not use. See LanguageProperty. |
extractor-null-dropped | An attribute's source value held a null inside an array or object. NGSI-LD does not allow null in an entity, so Cassiopeia wrote the attribute without it. One warning per attribute counts the records that lost a null. See Convert values. |
extractor-number-unreadable | A value read with the integer or float transformation spells no number of that kind, such as abc, an array, a number outside the 64-bit range under integer, or NaN under float, so Cassiopeia dropped that attribute, sub-attribute, or instance and kept the entity. A missing, null, or blank value produces no warning. The warning names the attribute by its path and the first value it could not read. See Convert values. |
extractor-object-misshapen | A value read with the object transformation is no JSON object, such as an array, a number, or text that does not read as an object, so Cassiopeia dropped that attribute, sub-attribute, or instance and kept the entity. The warning names the attribute by its path and the first value it could not use. See Convert values. |
extractor-template-unresolvable | An attribute's template failed to render for one entity. Cassiopeia dropped that attribute and kept the entity. The warning names the template, the mapping file, an entity that lost the attribute, and a hint. When the failing template is the observedAt that ranks the records of one entity, a record it failed on ranks as carrying no observation time; if another record of the entity wins the ranking, the failing record is discarded and the warning says so, naming an entity and counting the records it cost. |
extractor-timestamp-unreadable | A value read with a datetime, date, or time transformation is text Cassiopeia cannot read as a date and time. The attribute is dropped, or, for an observedAt, the attribute is written without it. |
extractor-urn-unreadable | A value read with the urn transformation is not a URN, such as B-12, so Cassiopeia dropped that attribute, sub-attribute, or instance and kept the entity. The warning names the attribute by its path and the first value it could not use. A missing, null, or blank value produces no warning. See Convert values. |
extractor-value-only-null | An attribute's source value was an object or an array holding nothing but null, such as {"maybe":null} or [null]. NGSI-LD does not allow null in a Property's value, and nothing would be left without it, so Cassiopeia dropped that attribute, sub-attribute, or instance and kept the entity. The warning names the attribute by its path and the first value. An empty object or array produces no warning, and a JsonProperty keeps the nulls. See Convert values. |
geometry-coordinates-out-of-range | A geometry carried a position whose longitude is outside [-180, 180] or whose latitude is outside [-90, 90], so it is no WGS 84 position (RFC 7946 section 4). Cassiopeia dropped that attribute, sub-attribute, or instance and kept the entity. One warning per attribute names it by its path and the first position refused, with where it sits in the geometry, such as coordinates[0][2], and counts every geometry refused. The usual causes are a source in an undeclared coordinate reference system, swapped longitude and latitude, or a wrong conversion or scale. See GeoProperty. |
geometry-* | A source geometry could not become the geometry its GeoProperty declares, such as a GeometryCollection (geometry-collection-inadmissible) or a MultiPolygon of several surfaces asked for a Polygon (geometry-ambiguous-multi-geometry). Cassiopeia dropped that attribute, sub-attribute, or instance and kept the entity. One warning per attribute and reason names the attribute by its path, such as location or temperature.footprint for a sub-attribute, and the reason, so two attributes that each declare a footprint sub-attribute are reported separately. |
ingest-columns-renamed | A CSV file's header gave two columns the same field name, so Cassiopeia kept both and renamed each repeat. One warning per file names up to three renamed columns and the field name to read each by, and counts the rest. With --verbose, it lists every renamed column. See CSV. |
ingest-dates-nonexistent | A NetCDF time variable in the 360_day or all_leap model calendar holds dates the Gregorian calendar does not have, such as February 30, which name no instant. The records keep them as labelled, and a datetime transformation cannot read them. One warning per variable names the variable, the file, the calendar, and the first such date, and counts them. See NetCDF. |
ingest-rows-misshapen | Rows of a CSV file had another number of fields than its table, so Cassiopeia skipped them and kept the rows around them. One warning per file names up to three skipped rows and counts every one. See CSV. |
ingest-table-unreadable | A GeoPackage table failed before Cassiopeia read its first row, such as a view, which has no row id to read its rows by, so Cassiopeia skipped it and read the other tables. One warning per table names the table and the file, and gives the reason. See GeoPackage. |
transform-expires-at-unreadable | An attribute's expiresAt could not be read as a date and time, so the attribute is written without it. |
transform-observed-at-unreadable | An observedAt declared without a transformation could not be read as a date and time, so the attribute is written without it. |
The run's warning count adds one for each renamed column of an ingest-columns-renamed warning, one for each date of an ingest-dates-nonexistent warning, and one for each skipped row of an ingest-rows-misshapen warning.
With --verbose, each of these also lists its causes, including Tera's own message when Tera reported one. For a template rejected at load time, that message gives the column where Tera stopped. The templates reference lists what fails at load time and how to guard a template against a missing field.
Control how the run behaves
These settings tune execution rather than describe the run, so they remain valid alongside --manifest. Several also have a manifest equivalent.
Memory profile. --low-memory (equivalent to memoryProfile: low-memory in a manifest) selects the low-memory pipeline profile: smaller batches, sequential extraction, and disk-backed (redb) resolver stores. It trades throughput for a smaller peak footprint, allowing a large feed to run on a constrained host. It is the coarse switch. The store and mode options below tune the same trade-off more finely.
Engine knobs. Two more settings tune the pipeline and override configuration, with no manifest field of their own:
--mode:batch(the default) buffers and processes items in parallel;singleprocesses each item immediately as it arrives. Usesinglefor the lowest latency or the simplest ordering, andbatchfor throughput.--entity-store(-e) and--relationship-store: the resolver's backing stores,dashmap(in-memory, the default) orredb(disk-backed). Useredbwhen the set of resolved entities or relationships is too large to hold in memory.--low-memoryselects it for you.
The manifest guide is the field reference for memoryProfile and onFailure.
Schedule a repeating run
The schedule flags (--every, --cron, --at, and the bounds --repeat, --schedule-duration, --jitter, --retry, --retry-backoff) make a run repeat instead of running once. The scheduling guide covers them, along with cron and timezone details. --on-failure governs failed cycles in a schedule, as described under Handle failures.
Global options
--config (-c) points to a configuration file. It is global, so it applies to any subcommand and remains valid alongside a manifest. Note the case: lowercase -c is the config file, while uppercase -C is the @context mode of a map run.
--verbose (-v) shows the full detail of every problem the run reports. Without it, a problem is one symbol-prefixed headline plus at most two indented lines, enough to know what went wrong and act on it. With it, the same problem lists its whole cause chain and every fact behind it: the status a broker answered with, the endpoint the request went to, the problem type it named, the entities the request carried, and the explanation it gave. It is global, so it applies to any subcommand, and it changes only how much is shown, never what the run does or what it exits with.
Repeated problems are collapsed whether or not the flag is given: an identical failure is drawn once, however many times it happens, and the run summary's Reasons block says how often each one fired. See Read the run summary.
Other commands
map is the workhorse, but the CLI carries several sibling commands for the jobs around a run:
cassiopeia sdm downloadfetches the published Smart Data Models catalog.cassiopeia sdm listandcassiopeia sdm search <query>inspect what it holds. The catalog supplies the schemas used by validation. See the validation guide.cassiopeia profile <file>detects and reports a data file's source format without running a pipeline. It is the quickest way to see whatautodetection will decide. See the source-format guide.cassiopeia exploreropens a terminal browser for a Smart Data Model schema, andcassiopeia wizardguides you through authoring a mapping against one. Both take--lightfor a light theme. See the terminal-interface guide.cassiopeia mapping format <file>pretty-prints a JSON5 mapping, with--replaceto rewrite it in place.cassiopeia mapping convert <file> --output <file>converts a mapping to plain JSON.cassiopeia schema dereference <file>inlines a JSON Schema's external references into a single self-contained schema.cassiopeia manifest generatewrites a manifest file from the same run-shaping flags accepted bymap. You can build one interactively and then run it with--manifest. See the manifest guide.cassiopeia config generatecreates a fully populated default configuration file at the location a later run discovers automatically.
Next steps
- Manifests: package a run's inputs, output, and schedule into a reusable file.
- Scheduling: make a run repeat.
- Command-line reference: see every flag and subcommand with its default.
- Examples: follow complete runs you can reproduce.