Writing a connector for an HTTP/REST system

Most tedge-dot connectors speak a fieldbus. Many industrial systems sit behind an HTTP API instead: historians, MES and SCADA servers, building management systems. This guide shows how to bring one into tedge-dot, using the AVEVA PI Web API connector (aveva-pi) as the worked example. Read it with the Connector SDK and the connector spec template.

The connector stays a driver: it reads and writes values. Units, names, alarms, measurements and device parameters are handled by the SDK runtime and the flows, as for every protocol.

Split the module into three layers

Layer	aveva-pi	Contains
HTTP client	client.rs, pi_http.c	TLS, authentication, timeouts, request size limits, the server’s error format, classified for the link status
Value decoder	value.rs, pi_value.c	A pure function from the server’s JSON value to a sample: value, quality, error, flags
Connector glue	lib.rs, connector_aveva_pi.c	Configuration, connect, read_points, subscribe, execute

The decoder has no I/O, so both builds check it against one file of golden vectors (connectors/aveva-pi/conformance/decode-vectors.json, generated by an independent Python reference), and the Rust build fuzzes it. Write the vectors before the decoder.

Look things up once

REST APIs address data by path or name, and most also have a stable internal id (PI’s WebId). Resolve every point on connect, keep the ids, and never look a point up per poll. A point the server does not have gets a bad sample not found: <path> and is looked up again after a delay (resolve_retry); it does not take the device’s link down.

Read many points in one request

Use the API’s batch endpoint (streamsets/value?webId=...&webId=...) and read all of a device’s due points in one call. Split long lists:

• by count (max_web_ids_per_request);
• by URL length (max_url_bytes): ids in the query string are limited by the web server, not the API (HTTP.sys allows 16 KB for the request line and headers, and answers 400 above it);
• on 414, or a 400 without the API’s error body, halve the chunk size and log once.

Run the chunks of one call concurrently, with a limit. Every sample of one read_points call shares the call’s read time (contract §5.4).

Push through a WebSocket

If the API can stream changes (PI Web API Channels), deliver points through subscribe:

• Open the socket inside subscribe and return. Do nothing long there: the Rust runtime bounds the call with operation_timeout.
• Run the reading loop in a task (Rust) or a worker thread (C) that hands samples to the runtime: the Rust sink is a bounded channel, the C side a ring drained by drain_subscriptions.
• Carry the server’s timestamp as the sample’s source time.
• Detect a silent server. PI sends empty heartbeat messages; three missed heartbeats fail check_subscription, and the runtime reconnects.
• If the upgrade is refused (a proxy that blocks WebSockets), fall back to polling the history endpoint for that device until the next reconnect.

Fill gaps from history

A historian keeps what happened while the link was down. Keep a per-point cursor (the time and values of the last event handed to the runtime) and persist it to a small file (<state_dir>/<service>/<device>.json, written by atomic rename at most every few seconds). On every subscribe:

1. Open the live stream and buffer it.
2. Read history from the cursor in pages, oldest first. Without a continuation link, restart each page at the last timestamp and drop what the cursor already has. If a whole page shares one timestamp, ask for a bigger page, or paging stalls.
3. Mark those samples replayed (Sample::replayed, tdot_sample_t.replayed), so the reporting policy measures min_interval and debounce on source time (contract §5.3 rule 7). Without the flag, a burst of history collapses to its first and last value.
4. Then deliver the buffered live events, dropping any at or before the cursor.
5. Bound the catch-up by age and count, and log the part you skip.

Let the runtime pace the catch-up: send to the bounded sink and wait (Rust), or wait while the C ring is above half its bound. The C ring must never drop a replayed sample.

Keep secrets out of the configuration

Read passwords, tokens and keys from files the configuration names (password_file, token_file, cert_file, key_file), and refuse an inline password or token. Declare those keys, tls.* and the state directory as local-only settings (local_only_settings), so a management command from the cloud cannot point them elsewhere or turn TLS verification off. Re-read a token file after a 401 before reporting the credentials as rejected, so an external refresher can rotate it.

Verify TLS by default

Verify against the system trust store plus an optional CA bundle (tls.ca_file). Make skipping verification an explicit setting that logs a warning on every connect. Put the TLS library’s reason into the link status, so “certificate not valid for name” reaches the operator instead of “connection failed”.

Rust: build one rustls::ClientConfig (with the ring provider, which cross-compiles without a C toolchain step) and give it to both the HTTP client and the WebSocket client. C: libcurl on the in-tree mbedTLS (impl/c/cmake/curl.cmake), with no CA path compiled in.

Map errors to link status

Answer	Link
Success	connected
401 (403 on a read)	disconnected, reason unauthorized
5xx, timeout, TLS, DNS, closed socket, silent stream	degraded, the runtime reconnects with backoff
Other 4xx on a read	a bad sample for that point
Any failure of a write	the command fails with the status and the API’s error text; the link follows the reads

Never retry a write.

Test against a simulator you own

Write a small simulator of the API (connectors/aveva-pi/sim, Python and aiohttp) that implements the documented behavior you rely on and a control API for faults: pause (accept, never answer), fail with a status, refuse the WebSocket, go silent, take the API down while history keeps growing, and insert events at past times. Unit-test the simulator, then run the same Robot suite against both builds. Record which API behavior you verified against the vendor’s reference and which you inferred (spec §9 of the AVEVA PI connector).