Changelog

All notable changes to restgdf are documented here. This project follows Semantic Versioning.

Unreleased

[3.2.0] - 2026-07-24

Added

• FeatureLayer.from_config / Directory.from_config — opt-in classmethods that build a prepared instance from an explicit restgdf.Config, applying config.auth.token as the request token before delegating to from_url (W5-14 / CONFIG-02). They are deliberately explicit: config is a required argument, so nothing implicitly reads the process-global get_config() at construction, and no global Config is threaded into the request path (CONFIG-03). The forwarded token rides the standard token= path (POST-forced by AUTH-01, never URL-serialized); for header-transport secured services prefer building an ArcGISTokenSession.from_config(...) and passing it as session=.

Changed

• Multi-page offset/count pagination now sends a deterministic orderByFields. When a layer is traversed with explicit resultOffset/resultRecordCount pages (the get_gdf / get_df / stream_* common path), each batch now defaults orderByFields to the layer’s resolved OID field unless the caller already supplied one — Esri’s documented remedy for reliable resultOffset paging, which otherwise leaves row order server-dependent and can silently duplicate or drop features across page boundaries (W4-2 / PAGINATION-02). A caller-supplied orderByFields (any case) is never overridden, and a layer whose OID cannot be resolved degrades gracefully to today’s un-sorted batches. Wire-payload change (semver-relevant): explicit-pagination request bodies now carry an orderByFields member. The OID-chunked WHERE-fallback and on_truncation='split' paths are unchanged.

• The GeoDataFrame path now raises on truncated responses instead of silently dropping rows. get_gdf and stream_gdf_chunks (via chunk_generator → get_sub_gdf) now inspect each page’s parsed JSON for exceededTransferLimit=true and raise restgdf.errors.PaginationError (mirroring the raw-feature engine), rather than returning a GeoDataFrame that is silently missing rows when ArcGIS hits its byte/geometry transfer cap (W4-1 / PAGINATION-01, the audit’s flagship silent-data-loss fix). The flag is read from the response JSON directly because pyogrio/read_file discards the top-level member. Behavior change: code that previously received a short-but-successful GeoDataFrame on a capped layer now gets a PaginationError; page the layer (smaller resultRecordCount, a tighter where, or the iter_pages engine with on_truncation='split') to read it completely.

• The default User-Agent on ArcGIS REST data requests is now sourced from get_config().transport.user_agent (default restgdf/<version>, e.g. restgdf/3.1.0) instead of the hardcoded "Mozilla/5.0" (restgdf.utils._http.default_headers, W2-10 / CONFIG-01 / AUTH-03). Wire-visible behavior change: some Esri deployments sniff the User-Agent, so a WAF or usage policy keyed on "Mozilla/5.0" may now see restgdf/<version>. Override it explicitly per request (headers={"User-Agent": ...}, still wins) or globally via RESTGDF_TRANSPORT_USER_AGENT / TransportConfig.user_agent. The exported DEFAULT_METADATA_HEADERS constant keeps its historical value for back-compat but no longer determines the wire User-Agent.

• Setting a currently-inert RESTGDF_* knob now warns. Config.from_env (hence get_config()) emits one restgdf._config.InertConfigWarning (a UserWarning) naming any set-but-unwired RESTGDF_RETRY_* / RESTGDF_LIMITER_* / RESTGDF_AUTH_REFRESH_THRESHOLD_S variable — those validate into Config but no live path consumes them (the resilience executor hardcodes its retry/limiter policy; token sessions never read AuthConfig.refresh_threshold_s). The knobs and their env aliases stay wired for back-compat; real executor/session wiring is deferred (warn-now, wire-later; W2-13 / TRANSPORT-01 / AUTH-04). Silence with warnings.filterwarnings("ignore", category=restgdf._config.InertConfigWarning).

• nested_count / FeatureLayer.get_nested_count now require exactly two fields. Passing any other arity raises a clear ValueError instead of an IndexError deep in post-processing (one field) or leaving a redundant *_count column with an incomplete sort (three or more) (W5-3 / API-04).

• Schema-drift log records now carry the originating service context (in the message and a drift_context extra field) so the first, non-deduped occurrence is attributable to which service drifted. The dedupe key stays the context-free (model, field, kind, type) 4-tuple, preserving the anti-spam guarantee for many-layer servers (W5-13 / TELEMETRY-02).

Added

• ArcGISTokenSession.from_config(session, credentials, config=...) and TokenSessionConfig.from_auth_config(auth_config, credentials) — opt-in factories that thread an AuthConfig namespace (its transport, header_name, referer, token_url, and refresh_leeway_s/clock_skew_s refresh knobs) onto a validated token session (W3-3 / W2-11, CONFIG-02 / AUTH-04). This is the only sanctioned way AuthConfig reaches a session: it is strictly opt-in and read only at call time. ArcGISTokenSession construction remains unchanged — __post_init__ never reads the process-global get_config(), so a plain ArcGISTokenSession(session, credentials) keeps its dataclass defaults (e.g. token_refresh_threshold=60 rather than the AuthConfig-derived 150). When config is omitted, from_config reads get_config().auth.

Fixed

• A 4xx from /generateToken no longer escapes as a raw aiohttp.ClientResponseError. ArcGISTokenSession.update_token now maps a true-HTTP 400/401/403 credential rejection to restgdf.errors.InvalidCredentialsError and any other non-2xx to restgdf.errors.RestgdfResponseError, both under the RestgdfError umbrella and chaining the originating aiohttp error as __cause__ (W2-2 / AUTH-02 / ERRTAX-01). The error is raised deterministically (not retried). The HTTP-200 {"error": {...}} bad-credentials envelope path is unchanged (still RestgdfResponseError via the strict TokenResponse tier). The InvalidCredentialsError docstring now describes the 4xx contract, and TokenRequiredError is documented as reserved/not-raised (a 499 surfaces as AuthNotAttachedError, the single live 499 raise site).

• The token refresh retry filter no longer swallows deterministic auth errors. restgdf’s own exceptions co-inherit OSError via PermissionError (AuthenticationError → PermissionError → OSError), so a None-credentials AuthenticationError (and the new W2-2 InvalidCredentialsError) were being caught by the OSError retry bucket, retried three times, and re-raised as the wrong class (TokenRefreshFailedError). An except RestgdfError: raise guard now sits before the retryable-error handler, so deterministic restgdf errors propagate immediately with zero backoff, distinguished by real exception instance (MRO) rather than by class name. Genuine transient OSError/ConnectionError/asyncio.TimeoutError still hit the retry ladder (W2-3 / ERRTAX-02).

• ArcGISTokenSession reactive token refresh is now single-flight under concurrent 498 Invalid Token responses: it snapshots the token before the request and, inside the refresh lock, only re-mints when the token is still unchanged. Previously N concurrent 498s issued N /generateToken calls; now they collapse onto one, and the later requests retry with the freshly minted token (W2-4 / ASYNC-01).

• A referer-bound ArcGISTokenSession (built from AGOLUserPass(referer=...) / TokenSessionConfig.referer) now attaches a matching Referer HTTP header to its data requests, not only to the /generateToken mint — so a client="referer" token is honoured end-to-end instead of being rejected (498/499) on the query. A client="requestip" (non-referer) session attaches no Referer header (no referer leak). Closes the “planned follow-up” limitation noted in MIGRATION.md for the 3.1 referer feature (#175 review NOTE-1).

• ArcGISTokenSession now forwards its own verify_ssl flag to token-attached data requests (not just the /generateToken POST), via setdefault("ssl", self.verify_ssl) in _call_with_auth_retry — so a session built with verify_ssl=False (self-signed ArcGIS Enterprise) no longer fails TLS verification on the actual query/metadata requests. A caller-supplied ssl= still wins (W2-10 / CONFIG-01 / AUTH-03).

• The library-owned session that get_gdf builds when called with session=None is now constructed with a TCPConnector whose ssl policy comes from get_config().transport.verify_ssl (default True), so RESTGDF_TRANSPORT_VERIFY_SSL=false / TransportConfig(verify_ssl=False) is finally honored on the flagship GeoDataFrame data path — previously the bare ClientSession() used the default connector and silently kept TLS verification on. A caller-supplied session is passed through untouched (its own connector owns its TLS policy). This completes the three-seam verify_ssl wiring (config source W3-1, token/_http W2-10, getgdf connector W4-5 / CONFIG-01 / AUTH-03).

• FeatureLayer.get_gdf/get_unique_values/get_value_counts/ get_nested_count now return an independent copy of the cached frame/list on every call (W5-1, ASYNC-02) instead of the shared cached object by reference. Previously, mutating a returned GeoDataFrame or DataFrame in place (e.g. df.rename(..., inplace=True)) silently corrupted what every later call on the same instance returned. Cache population is unaffected and remains non-atomic under concurrent asyncio.gather awaiters — this fix protects reads only, not writes.

• get_fields(types=True), _field_rows, and get_fields_frame no longer raise KeyError/AttributeError on a permissive-tier field entry missing name and/or type (W5-4, ADAPTERS-01) — a real ArcGIS server can emit such entries and FieldSpec already declares both as optional. A field with no resolvable name is now silently dropped (documented in the affected docstrings); a missing/None type defaults to "" instead of crashing on None.replace(...). get_fields(types=False) gained the same nameless-field guard.

• The _SpanContextFilter log-correlation filter now always stamps record.trace_id/record.span_id (defaulting to "" outside a valid OpenTelemetry span) instead of leaving them unset (W5-12, TELEMETRY-01). Previously, any restgdf.* log record emitted outside a feature_layer.stream span — e.g. the auth.refresh.start DEBUG record or the pagination exceededTransferLimit warning — had no trace_id/span_id attributes at all, so the documented %(trace_id)s log-correlation recipe raised ValueError: Formatting field not found in record: 'trace_id' (surfaced by stdlib logging as a swallowed stderr “— Logging error —” dump). span_context_fields() is unaffected and still returns {} outside a span.

• restgdf.utils.token._auth_logger (the restgdf.auth logger backing token-refresh debug logging) is now created through the library’s get_logger("auth") factory instead of a raw logging.getLogger(...) call, so it carries the documented NullHandler like every other restgdf.* logger (it previously silently lacked one).

• restgdf.resilience._errors._parse_retry_after now rejects non-finite Retry-After header values ("nan", "inf", "-inf", "Infinity", etc.) instead of returning them as a poisoned float. Previously a NaN/+Inf value passed the existing negative-value guard unmolested and could reach the 429 cooldown deadline computation and the public RateLimitError.retry_after attribute.

• The optional-dependency gate (require_pandas/require_geopandas/ require_pyogrio and friends) now catches ImportError instead of only ModuleNotFoundError, so a present-but-broken geo dependency (e.g. a native GDAL/shapely load failure) surfaces as OptionalDependencyError naming the restgdf[geo] hint instead of escaping as a raw, unwrapped ImportError.

• Removed the false .env file-loading claims from docs/configuration.rst’s precedence list and docs/authentication.rst’s credentials recipe. restgdf has never read .env files or depended on python-dotenv/pydantic-settings — Config.from_env() resolves only from the process environment (os.environ). The docs now show how to opt in explicitly with python-dotenv yourself (W3-5, CONFIG-04).

• FeatureLayer.get_value_counts / get_nested_count now send a conservative statistics body. The instance request data (carrying returnGeometry=True / outFields="*" / returnCountOnly=False) previously clobbered the stats-only flags, so the server received a geometry+all-fields query instead of the grouped statistics one. The bodies now forward only where+token from the caller data (matching get_unique_values), so the stats flags win while the instance where filter is preserved (W5-2 / API-01).

• resolve_domains (used by FeatureLayer.get_df(resolve_domains=True)) is now robust to malformed-but-real domain metadata: a non-dict domain is skipped instead of raising AttributeError, and coded values are mapped only when they carry both code and name. A name-less code passes through unchanged rather than being silently replaced with NaN (W5-6 / ADAPTERS-03).

• iter_pages/stream_*’s on_truncation='split' path no longer re-fetches the OID list at every recursion node (each bisected half now reuses the parent’s already-materialized slice) and no longer emits an unbounded IN (...) literal for a single oversized half — a half exceeding a new 1000-element cap (the common ArcGIS backing-store IN-predicate limit) is bisected further before being fetched, instead of being sent as one arbitrarily large literal list (W4-3, PAGINATION-03). No caller-visible contract change (same pages yielded, same on_truncation semantics).

3.1.0 - 2026-07-24

Added

• restgdf.utils.getinfo.build_spatial_filter_payload(geometry, *, in_sr=None, spatial_rel="esriSpatialRelIntersects") — pure helper that converts an ArcGIS-JSON geometry, a GeoJSON-style mapping, or any object exposing __geo_interface__ (e.g. shapely geometries) into the ArcGIS REST geometry / geometryType / spatialRel (+ optional inSR) query-payload fragment. Handles points, multipoints, polylines, polygons, envelopes, curve paths / curve rings, and 3D/ZM coordinates, and stamps hasZ / hasM on array-based geometries when the coordinates carry Z/M ordinates. Re-exported through getinfo.__all__ following the build_pagination_plan pattern (not a top-level restgdf export).

Fixed

• Static type-checkers can now resolve restgdf.FieldDoesNotExistError. It was already exported at runtime (__all__ and the lazy-export table), but missing from the TYPE_CHECKING import block that backs static analysis, so mypy/pyright reported an unresolved attribute even though the name worked fine at runtime.

Changed

• The mypy type gate now runs against real dependency types. A dedicated CI job installs every extra plus mypy and enables the pydantic.mypy plugin (via [tool.mypy] plugins), so it type-checks against actual aiohttp/pydantic types instead of reporting green over unresolved imports; pandas/geopandas are scope-silenced with ignore_missing_imports (their integration boundary is intentionally untyped). The internal type errors the un-defanged gate surfaced were fixed — drift alias-choice narrowing, the metadata field-row annotation, bounded-retry exception typing, and widening get_gdf’s session parameter to the AsyncHTTPSession transport protocol (a runtime contract that aiohttp.ClientSession satisfies at runtime but, under current aiohttp stubs, not as a static subtype).

• Raised the supported Python floor to 3.11 (3.9 is EOL 2025-10-31; 3.10 reaches EOL 2026-10-31); CI now tests 3.11–3.14.

• AGOLUserPass(referer=...) is now honoured at token-mint time. ArcGISTokenSession.__post_init__ threads the credential’s referer into the auto-built TokenSessionConfig, so token_request_payload switches the ArcGIS client field from "requestip" to "referer" and adds "referer": <url> to the /generateToken POST body. Previously this was silently ignored — the credentials-only constructor built its config without a referer, so AGOLUserPass(..., referer=...) had no effect on the minted token. See MIGRATION.md for the request-time Referer-header limitation this interacts with.

• Token-mint requests now send an explicit expiration field. token_request_payload emits AGOLUserPass.expiration (default 60 minutes) and the deprecated synchronous get_token helper sends expiration=60. Behaviourally equivalent to the ArcGIS server-side default of 60 minutes; the value is now explicit on the wire.

• The dev extra now composes restgdf[doc] and adds build+twine, so pip install -e ".[dev]" alone is enough to run every gate in CONTRIBUTING.md’s gate suite (docs build, packaging metadata sanity) without installing doc separately.

Security

• AUTH-01: a caller-supplied token in the request body is no longer serialized into the URL query string. On the documented FeatureLayer(token=...) / data={"token": ...} path, a short token-bearing request was routed via GET, leaking the ArcGIS token into the URL — where it lands in server / proxy / WAF access logs and Referer headers — on a plain aiohttp.ClientSession or a default header-mode ArcGISTokenSession. _arcgis_request now forces POST (token in the request body) whenever the outgoing body carries a token key, regardless of session transport. This complements the existing session-transport guard; ArcGIS /query and metadata roots already accept form-encoded POST bodies, so the wire contract is unchanged for servers. Tokenless requests keep the length-based GET/POST routing.

3.0.0 - 2026-05-02

restgdf 3.0.0 is a major, backwards-incompatible rewrite. The package splits into a light async core (aiohttp + pydantic v2) plus three opt-in extras — restgdf[geo] (GeoPandas/pandas/pyogrio), restgdf[resilience] (ResilientSession/ResilienceConfig retry and rate-limiting via stamina/aiolimiter), and restgdf[telemetry] (RestgdfInstrumentor OpenTelemetry instrumentation) — so a base install stays dependency-light. A typed streaming surface (iter_pages, stream_features, stream_feature_batches, stream_rows, stream_gdf_chunks) replaces the legacy row/feature helpers, with query-verb selection centralized in _choose_verb/ _arcgis_request (restgdf/utils/_http.py). restgdf.Config adds layered, frozen pydantic sub-configs resolved from RESTGDF_<CATEGORY>_<FIELD> env vars, and restgdf.errors grows a wider exception taxonomy including a five-member AuthenticationError hierarchy. See MIGRATION.md for the full breaking-change list; the detailed change set below was carried forward from the pre-release tranche.

Changed

• Gate-3 hardening follow-up. Three review-driven safety fixes land on top of the v3-followup tranche:

  • ArcGIS requests routed through _choose_verb now force POST whenever the effective session transport is "body" or "query" (including wrapped ResilientSession(ArcGISTokenSession(...)) stacks), preventing auth tokens from leaking into URL query strings on short requests.

  • restgdf.resilience._retry._RetriedCtx now mirrors aiohttp’s dual request-manager shape so await session.get(...) and async with session.get(...) both work against ResilientSession.

  • getgdf._advertised_max_record_count_factor() now rejects bool, NaN, and infinity inputs so malformed vendor metadata falls back to the byte-identical pre-T9 path.

  • Module-level get_gdf(..., session=None) now closes the temporary aiohttp.ClientSession it creates internally, eliminating the unclosed-session leak on direct helper usage.

  • _iter_pages_raw(..., max_concurrent_pages=N) now keeps at most N fetch tasks scheduled at once instead of pre-creating one task per page and only bounding execution with a semaphore, preventing pagination plans with many batches from exploding task memory.

  • Legacy streaming helpers (_feature_batch_generator, chunk_generator) now honor the repository-wide concurrency cap while they stream results and cancel outstanding work on early generator close, preventing abandoned page-fetch tasks from accumulating behind partially-consumed iterators.

  • Hypothesis-backed property tests now live behind a dedicated pytest --run-stress opt-in so the default suite remains a representative production-validation pass instead of mixing in a separate stress tier by default.

• Pagination planner wiring. When an ArcGIS layer advertises advancedQueryCapabilities.maxRecordCountFactor, get_query_data_batches now forwards that value to build_pagination_plan(advertised_factor=...). The wiring is strictly opt-in: servers that do not expose the field (or expose a non-positive / non-numeric value) get the previous byte-exact plan with no advertised_factor kwarg. This replaces the deferred-plumbing stub.

• Feature-count retry delegation. restgdf.utils.getinfo._feature_count_with_timeout now delegates its bounded timeout-retry loop to restgdf.resilience.bounded_retry_timeout (a new public helper) when the resilience extra is installed, giving restgdf a single stamina-backed source of truth for retry semantics. When the extra is absent, the previous inline loop is preserved byte-for-byte as a fallback. The retryable exception set (asyncio.TimeoutError, TimeoutError, aiohttp.ServerTimeoutError) and R-69 (ClientConnectionError propagates without retry) are preserved on both paths.

• GET/POST verb selection wiring. ArcGIS query requests now route through a single _arcgis_request helper in restgdf/utils/_http.py that consults _choose_verb (8,192-byte threshold on URL + urlencoded body). Previously every call site was hard-coded POST. Nine call sites across utils/getgdf.py, utils/getinfo.py, and utils/_query.py were migrated. The GET path coerces bool/None values in params to "true"/"false"/"" so yarl can serialize them; POST payloads are untouched. Zero behavior change for bodies above the threshold.

• Transport typing. ArcGISTokenSession now exposes close() and closed that delegate to its inner aiohttp.ClientSession, making it fully satisfy the restgdf._client._protocols.AsyncHTTPSession Protocol. Internal call sites previously typed aiohttp.ClientSession | ArcGISTokenSession were widened to AsyncHTTPSession across adapters/stream.py, directory/directory.py, featurelayer/featurelayer.py, utils/crawl.py, utils/getgdf.py, utils/getinfo.py, utils/_query.py, and utils/_stats.py. Zero runtime behavior change — widening to a superset Protocol is backwards-compatible for existing callers.

Added

Pagination

• restgdf.errors.PaginationInconsistencyWarning — new UserWarning subclass emitted by _resolve_page when a batch page returns zero features but the server still sets exceededTransferLimit=true. The warning fires regardless of the on_truncation mode ("raise", "ignore", or "split") so pathological server responses are always surfaced. Deliberately not included in restgdf.errors.__all__ or the top-level public API — warnings live outside the RestgdfError taxonomy; import via from restgdf.errors import PaginationInconsistencyWarning.

Domain resolution

• FeatureLayer.get_df(resolve_domains=False) — new kwarg on the pandas-first tabular accessor. When True, coded-value domain fields are replaced in-place with their human-readable names using a single cached pass over the layer’s metadata (no per-row HTTP). Defaults to False so the base code path is byte-identical for existing callers.

• restgdf.adapters.pandas.resolve_domains(df, fields) — public helper exposing the same resolution logic for callers already holding a pandas.DataFrame. Requires the geo extra (pandas is part of that install surface).

Resilience

• restgdf.resilience.bounded_retry_timeout — new public helper exposing a stamina-backed bounded retry loop for timeout-class exceptions. Used internally by _feature_count_with_timeout when the resilience extra is installed; safe for consumers on the same extra. The retryable exception set matches restgdf’s internal timeout policy (asyncio.TimeoutError, TimeoutError, aiohttp.ServerTimeoutError); aiohttp.ClientConnectionError propagates immediately (R-69 preserved).

Streaming

• FeatureLayer.iter_pages — low-level async generator yielding raw ArcGIS query-page envelopes with order ("request" default / "completion"), max_concurrent_pages (optional semaphore bound), and on_truncation ("raise" default / "ignore" / "split"). The "split" strategy bisects the predicate’s OID list via get_object_ids and recurses up to depth 32 before raising. Truncated pages under "ignore" log a structured warning on the restgdf.pagination logger and continue.

• FeatureLayer.iter_features / FeatureLayer.stream_features — flatten iter_pages into individual feature dicts. Deliberate aliases: stream_features is the canonical public entrypoint, iter_features the lower-level primitive.

• FeatureLayer.stream_feature_batches — yields one list[feature_dict] per page, mirroring iter_pages boundaries.

• FeatureLayer.stream_rows — yields row-shaped dicts (attributes merged with raw geometry). Pandas/GeoPandas-free; safe on a base install.

• FeatureLayer.stream_gdf_chunks — yields GeoDataFrame chunks over the optional geo stack; each chunk inherits attrs["spatial_reference"].

• iter_pages now emits exactly one feature_layer.stream INTERNAL parent span wrapping the per-page loop when telemetry is enabled; no restgdf-owned per-page spans are emitted. No-op when RESTGDF_TELEMETRY_ENABLED is unset. Constructed inside restgdf.utils.getgdf._iter_pages_raw.

• Spatial-reference propagation: restgdf.utils.getgdf.get_gdf, FeatureLayer.get_gdf, FeatureLayer.sample_gdf, FeatureLayer.head_gdf, and chunk_generator / FeatureLayer.stream_gdf_chunks all stamp gdf.attrs["spatial_reference"] with the raw dict from the layer’s metadata envelope (extent.spatialReference preferred, top-level spatialReference fallback). Normalization uses restgdf.utils._metadata.normalize_spatial_reference.

Adapters and tabular output

• restgdf.adapters subpackage (lazy-loaded via PEP 562): four submodules covering dict / stream / pandas / geopandas shapes. All submodules are base-install safe at import time; pandas and geopandas are required only at call time and raise OptionalDependencyError when missing.

  • restgdf.adapters.dict — feature_to_row, features_to_rows, plus as_dict / as_json_dict re-exports.

  • restgdf.adapters.stream — iter_feature_batches, iter_rows, iter_gdf_chunks.

  • restgdf.adapters.pandas — rows_to_dataframe (sync) + arows_to_dataframe (async).

  • restgdf.adapters.geopandas — rows_to_geodataframe + arows_to_geodataframe.

• FeatureLayer.get_df() — async pandas-first tabular accessor. Sibling to get_gdf() that returns a pandas.DataFrame from the same row stream and does not require the geo extra.

Configuration

• restgdf.Config — frozen pydantic 2.x aggregate of eight frozen sub-configs (TransportConfig, TimeoutConfig, RetryConfig, LimiterConfig, ConcurrencyConfig, AuthConfig, TelemetryConfig, ResilienceConfig) plus Config.from_env(env=None) classmethod. Sub-configs and the aggregate are immutable at both slot and nested-field level.

• restgdf.get_config() — process-wide cached Config accessor (functools.lru_cache(maxsize=1)).

• restgdf.reset_config_cache() — clears the cache; cascades bidirectionally with the existing reset_settings_cache so tests can refresh all configuration with a single call regardless of which accessor they use.

• Nested env-var surface RESTGDF_<CATEGORY>_<FIELD> wired through Config.from_env for every sub-config field (RESTGDF_TRANSPORT_USER_AGENT, RESTGDF_TIMEOUT_TOTAL_S, RESTGDF_RETRY_ENABLED, RESTGDF_LIMITER_RATE_PER_HOST, RESTGDF_CONCURRENCY_MAX_CONCURRENT_REQUESTS, RESTGDF_AUTH_TOKEN_URL, RESTGDF_TELEMETRY_LOG_LEVEL, …). Invalid coercions and validator rejections raise RestgdfResponseError with the underlying pydantic.ValidationError preserved as __cause__.

• Settings.max_concurrent_requests: int = 8 (field) + RESTGDF_MAX_CONCURRENT_REQUESTS env-var coercion (BL-01). Default matches aiohttp TCPConnector pool size.

Errors

• restgdf.errors module exposing the canonical exception taxonomy: RestgdfError, ConfigurationError, OptionalDependencyError, TransportError, RestgdfTimeoutError, RateLimitError, ArcGISServiceError, PaginationError, FieldDoesNotExistError, SchemaValidationError, AuthenticationError, and OutputConversionError. All re-exported from the top-level restgdf package via the lazy-import hook.

• PaginationError.batch_index / .page_size attributes carry pagination context when cursor-based iteration fails.

• RateLimitError.retry_after attribute carries the optional seconds-until-retry hint, populated from the server’s Retry-After header (integer seconds or RFC 7231 HTTP-date) by the resilience wrapper (Q-A12). New helper _parse_retry_after in restgdf.resilience._errors.

• Error-attribute population: RestgdfResponseError now carries optional url, status_code, and request_id attributes (kw-only, default None). TransportError gains url and status_code. RestgdfTimeoutError gains timeout_kind ("total", "connect", "read"). RateLimitError gains url and status_code alongside existing retry_after. All new attrs are backward-compatible — existing call sites that omit them get None defaults.

• Five AuthenticationError subclasses — InvalidCredentialsError, TokenExpiredError, TokenRequiredError, TokenRefreshFailedError, AuthNotAttachedError. All carry .context, .attempt, .cause attributes with SecretStr auto-redaction.

Auth runtime

• TokenSessionConfig.refresh_leeway_seconds (default 60) + TokenSessionConfig.clock_skew_seconds (default 30) — explicit integer fields (ge=0) replacing the implicit semantics of the previous single refresh_threshold_seconds knob (BL-04).

• ArcGISTokenSession.expires_at — tz-aware UTC datetime property computed from the epoch-ms expires field.

• _utc_now() shim for deterministic wall-clock test control.

• Structured auth.refresh.start / .success / .failure log events at DEBUG level on the restgdf.auth logger.

• Bounded /generateToken retry — transient errors retried up to 3× with exponential backoff; deterministic errors propagate immediately. After exhaustion raises TokenRefreshFailedError.

• Referer binding — token_request_payload honours config.referer and switches ArcGIS client to "referer" when set.

Normalization

• restgdf._models.responses.NormalizedGeometry, NormalizedFeature, and iter_normalized_features(response, *, oid_field=None, sr=None) — typed intermediate models plus iterator over FeaturesResponse.features. Wire-level features: list[dict] stays for perf; normalization is opt-in. Geometry type is heuristically inferred from shape; object_id is int-coerced from attributes[oid_field].

• restgdf._models.responses.AdvancedQueryCapabilities — typed PermissiveModel companion for the ArcGIS advancedQueryCapabilities sub-object, with camelCase / snake_case AliasChoices wiring and permissive extra="allow" preservation of unknown keys (BL-21).

• LayerMetadata.advanced_query_capabilities_typed: AdvancedQueryCapabilities | None — additive typed companion to the existing raw advanced_query_capabilities: dict | None field. Caller-opt-in; the raw dict stays the default representation.

• restgdf.utils._metadata.normalize_spatial_reference(sr) — pure helper returning (epsg_int | None, raw_dict | None) that prefers latestWkid over wkid for EPSG-consuming clients (R-28).

• concat_gdfs propagates GeoDataFrame.attrs["spatial_reference"] across concatenation.

• restgdf.utils._metadata.normalize_date_fields(features, fields) — converts ArcGIS esriFieldTypeDate epoch-ms integers to ISO-8601 UTC strings. Opt-in via normalize_dates=True on the adapter layer.

Pagination

• restgdf.utils._pagination.PaginationPlan (frozen dataclass) + build_pagination_plan(total_records, max_record_count, *, factor=1.0, advertised_factor=None) — pure-math pagination planner re-exported via restgdf.utils.getinfo. Emits (resultOffset, resultRecordCount) tuples byte-identical to the previous inline arithmetic in get_query_data_batches; clamps factor > advertised_factor with a warning via get_logger("pagination"). get_query_data_batches is rerouted through the planner with no public-signature change and all pinned fixtures preserved.

Observability

• restgdf._logging.get_logger(suffix) library-wide logger factory and build_log_extra standard extra= envelope helper. Existing get_drift_logger / restgdf.schema_drift contract unchanged.

• restgdf[telemetry] optional extra — RestgdfInstrumentor (dynamic subclass of AioHttpClientInstrumentor, R-58), feature_layer_stream_span async context manager (INTERNAL span, R-21), span_context_fields helper, and _SpanContextFilter auto-attached to the restgdf root logger for trace/span log correlation.

• docs/recipes/tracing.md — structured observability, error-attribute inspection, and OpenTelemetry integration.

• docs/recipes/streaming.md — the three streaming shapes, on_truncation options, order variants, and max_concurrent_pages knob.

Resilience

• restgdf.ResilienceConfig — frozen pydantic sub-config controlling the stamina-based retry wrapper and per-service-root token-bucket rate limiter. Fields: enabled, rate_per_service_root_per_second, respect_retry_after_max_s, fallback_retry_after_seconds, backend. Exposed via restgdf.Config.resilience and in top-level __all__. Disabled by default; opt in via RESTGDF_RESILIENCE_ENABLED=1.

• restgdf.resilience.ResilientSession — retry + rate-limit adapter implementing the AsyncHTTPSession protocol. Stamina-based retry with 429/5xx awareness. Pure pass-through when ResilienceConfig.enabled=False. Requires pip install restgdf[resilience].

• [project.optional-dependencies] resilience extra: stamina>=24.2, aiolimiter>=1.1.

• Per-service-root token-bucket rate limiting via LimiterRegistry and 429-cooldown via CooldownRegistry in restgdf.resilience._limiter. _service_root(url) derives the rate-limit key by truncating at the first FeatureServer / MapServer / ImageServer / SceneServer path segment.

Transport protocols and drift

• restgdf._client._protocols.AsyncHTTPSession — @runtime_checkable typing.Protocol capturing the get / post / close / closed surface restgdf transport sessions rely on; re-exported from restgdf._client.

• restgdf._models._drift.FieldSetDriftObserver — observer class that tracks attribute-key appearance / disappearance across feature-page batches and emits deduped field_appeared / field_disappeared records through the existing restgdf.schema_drift logger.

• Private restgdf.utils._http._choose_verb(url, body=None) seam returning "POST" for /query and /queryRelatedRecords, "GET" for bare service/layer metadata URLs, and "POST" as the conservative default. Call sites unchanged; forward-compatible stub for BL-50’s future ~1800-byte GET→POST auto-switch.

Internal helpers

• restgdf.utils._concurrency.bounded_gather(*aws, semaphore) — caps concurrent fan-out via an asyncio.BoundedSemaphore while preserving asyncio.gather result ordering and return_exceptions semantics (BL-01).

• restgdf.utils.getinfo._feature_count_with_timeout — inline bounded retry around get_feature_count with exponential backoff. Retries only on asyncio.TimeoutError, TimeoutError, and aiohttp.ServerTimeoutError; connection-level failures (aiohttp.ClientConnectionError) and deterministic errors (RestgdfResponseError, schema mismatches) propagate on the first attempt. Exhausted timeouts raise RestgdfTimeoutError with __cause__ preserved (BL-51).

• Directory.safe_crawl now routes its per-layer feature_count probe through a BoundedSemaphore sized from ConcurrencyConfig.max_concurrent_requests (Q-A7).

• restgdf.__getattr__ now consults a _REMOVED_EXPORTS extension point before raising AttributeError, letting future phases register removed top-level names with a DeprecationWarning + migration message. Mapping empty in this release (BL-57).

Tests + tooling

• Taxonomy + observability contract tests (tests/test_taxonomy_contract.py) asserting errors.__all__ shape and that get_logger(suffix) emits a structured record for every LOGGER_SUFFIXES entry (BL-37).

• Minimal-install contract test (tests/test_minimal_install.py) guarding against accidental import of pandas / geopandas / pyogrio when users install the base package without extras (BL-38).

• Streaming-recipe discoverability regression test (tests/test_streaming_recipe_discoverable.py).

• hypothesis, aioresponses, and opentelemetry-sdk added to the dev extra. New tests/test_crawl_property_hypothesis.py scaffold and tests/_mocks/aioresponses_helpers.py shared fixtures (BL-39, R-62 scope).

• bumpver pre_commit_hook (scripts/bumpver_stamp_date.py) auto-stamps CITATION.cff::date-released to the release date on every bumpver update, keeping the citation metadata in lock-step with version:. New test_citation_cff_date_released_is_iso_8601 pins the ISO-8601 date format (BL-40 follow-up).

• Install-combination CI matrix (R-62, v3-followup T5). New install_combinations job in .github/workflows/pytest.yml runs the test suite against six explicit pip install surfaces: base, [geo], [resilience], [telemetry], [geo,resilience,telemetry], and [dev]. Wired into the ci aggregator so regressions in any single extra fail PR checks before merge.

• Coverage-recovery tests (v3-followup T1–T4). Four targeted test modules lift measured coverage from 96.53% to 98.16%: tests/test_resilience_retry_coverage.py (17 tests; _retry.py 79% → 99%), tests/test_telemetry_coverage.py (9 tests; _correlation.py 100%, _spans.py 97%), tests/test_credentials_coverage.py (6 tests; credentials.py 91% → 100%), and tests/test_getgdf_coverage.py (10 tests; getgdf.py 95% → 99%). Coverage floor in pyproject.toml ([tool.coverage.report] fail_under) raised from 96 to 97 to match (v3-followup T11).

Changed

Breaking

• Default token wire transport flipped from "body" to "header" (BL-13). Tokens are now sent via the X-Esri-Authorization header. Set transport="body" in AuthConfig / TokenSessionConfig to restore the old behavior.

• refresh_leeway_seconds default raised 60 → 120 (BL-13).

• getgdf / _get_sub_features now raise restgdf.errors.PaginationError (not RuntimeError) on exceededTransferLimit=true. PaginationError carries batch_index and page_size (BL-08).

• PaginationError no longer multi-inherits RuntimeError (phase-3d consolidation under the BL-06 taxonomy). Callers catching RuntimeError around feature_count / pagination calls must widen to RestgdfError or narrow to PaginationError / ArcGISServiceError (BL-09, R-02).

Non-breaking

• FeatureLayer.where(new_where) now reuses the parent’s cached metadata so no second metadata GET (?f=json) is issued when the parent was already prepped via prep() / from_url(). A single feature-count POST (returnCountOnly=true) scoped to the refined where clause is still issued so refined.count remains correct for the refined filter. The new where_clause is threaded through data["where"] so subsequent query / streaming calls honour it (BL-46).

• ArcGISTokenSession.token_needs_update refactored to use expires_at and _utc_now() instead of inline epoch arithmetic (BL-16).

• Reactive 498/499 handling in _call_with_auth_retry — HTTP 498 triggers single-flight refresh + one retry; HTTP 499 raises AuthNotAttachedError immediately (BL-11).

• restgdf.utils.getinfo.service_metadata, restgdf.utils.crawl.fetch_all_data, and restgdf.utils.crawl.safe_crawl now route their internal asyncio.gather fan-out through bounded_gather with a per-call asyncio.BoundedSemaphore. Saturation semantics = wait (no new exception) (BL-01).

• ArcGISTokenSession.update_token_if_needed now collapses concurrent refresh attempts onto a single /generateToken POST via a lazily-initialized per-instance asyncio.Lock with a double-checked token_needs_update() inside the lock. The new _refresh_lock field is init=False, repr=False, compare=False (BL-03).

• RestgdfResponseError now inherits from restgdf.errors.RestgdfError in addition to ValueError. Class identity and the from restgdf._models._errors import RestgdfResponseError import path are preserved; except ValueError: call sites keep working (BL-06).

• restgdf.utils._optional._optional_dependency_error now returns restgdf.errors.OptionalDependencyError instead of a bare ModuleNotFoundError. Existing except ModuleNotFoundError: and except ImportError: handlers still catch the new exception because OptionalDependencyError multi-inherits ModuleNotFoundError (BL-07).

• HTTP timeouts are now plumbed through Settings.timeout_seconds into every library-maintained session.get / session.post call site (restgdf.utils._query, restgdf.utils._stats, restgdf.utils.getgdf._get_sub_features / get_sub_gdf, ArcGISTokenSession.update_token, and the ArcGISTokenSession.get / .post wrappers). The new restgdf.utils._http.default_timeout() helper returns an aiohttp.ClientTimeout sized from Settings.timeout_seconds (float, default 30.0, overridable via RESTGDF_TIMEOUT_SECONDS). Callers that already pass timeout= keep precedence (BL-02).

• ArcGISTokenSession.__post_init__ now respects a caller-supplied config=TokenSessionConfig(...) instead of overwriting it, and derives the TokenSessionConfig split fields from token_refresh_threshold internally (no longer via the deprecated refresh_threshold_seconds alias), so plain construction no longer fires a DeprecationWarning. token_refresh_threshold is resynced from the validated config after construction.

• pyproject.toml::[tool.coverage.report].exclude_also extended with if TYPE_CHECKING:, @overload, and bare-ellipsis stub lines (standard coverage.py idioms, pydantic / httpx / attrs precedent). Threshold fail_under=97 unchanged (R-63).

Deprecated

• FeatureLayer.row_dict_generator — use FeatureLayer.stream_rows. Emits DeprecationWarning and continues to delegate to the module-level row_dict_generator helper for backwards compatibility with existing unittest.mock.patch call sites.

• get_token() — emits DeprecationWarning on every call (BL-14). Migrate to ArcGISTokenSession for async token management. get_token now also accepts pydantic.SecretStr passwords.

• restgdf.Settings / restgdf.get_settings() — use restgdf.Config / restgdf.get_config(). get_settings() emits a single DeprecationWarning on first call and constructs its return value from get_config(); existing callers continue to work unchanged. Will be removed no earlier than restgdf 3.0 (BL-18).

• Six flat environment variables — RESTGDF_TIMEOUT_SECONDS, RESTGDF_TOKEN_URL, RESTGDF_REFRESH_THRESHOLD, RESTGDF_USER_AGENT, RESTGDF_LOG_LEVEL, RESTGDF_MAX_CONCURRENT_REQUESTS — in favour of their RESTGDF_<CATEGORY>_<FIELD> replacements. The old names continue to work but emit a DeprecationWarning when read via Config.from_env / get_config; when both old and new names are set the new one wins and the warning notes the override (BL-18).

• TokenSessionConfig.refresh_threshold_seconds — a read/write alias emitting DeprecationWarning. Reads return refresh_leeway_seconds + clock_skew_seconds; constructor writes split the supplied total into clock_skew_seconds = min(30, total) and refresh_leeway_seconds = total - clock_skew_seconds. Migrate to the explicit field pair.

Removed

• restgdf.utils._metadata.FIELDDOESNOTEXIST sentinel (and its re-export via restgdf.utils.getinfo). Call sites must now except FieldDoesNotExistError from the BL-06 taxonomy. Hard break — no compat shim (BL-09, R-02).

Fixed

• bumpver file pattern for CITATION.cff anchored to ^version: {version}$ (was unanchored version: {version}), preventing a latent release-time defect where the regex would silently rewrite cff-version: 1.2.0 to the new release version on every bumpver update. Discovered via bumpver update --dry during the CITATION auto-stamp work.

• ArcGISTokenSession.update_token now forwards the session’s verify_ssl flag as ssl= on the /generateToken POST. Previously the flag was honoured for feature / query requests but ignored during token refresh, so verify_ssl=False sessions could still fail TLS verification against self-signed ArcGIS Enterprise deployments.

2.0.0 - 2026-04-20

Major release — pydantic 2.13 integration. See MIGRATION.md for a complete breaking-changes table and migration recipes.

Breaking

• Public return and attribute shapes changed from plain dict / TypedDict to pydantic BaseModel classes:

  • FeatureLayer.metadata → LayerMetadata

  • Directory.metadata → LayerMetadata

  • Directory.services, Directory.services_with_feature_count, and Directory.crawl(...) → list[CrawlServiceEntry]

  • get_metadata(...) → LayerMetadata

  • safe_crawl(...) → CrawlReport

• AGOLUserPass.password is now pydantic.SecretStr; call .get_secret_value() at the HTTP-POST boundary.

• restgdf._types.* TypedDicts are replaced by lazy aliases that re-export the new pydantic models and emit DeprecationWarning on import. The shim will be removed in 3.x.

Added

• LayerMetadata, ServiceInfo, FieldSpec, Feature, FeaturesResponse, CountResponse, ObjectIdsResponse, TokenResponse, ErrorInfo, ErrorResponse, CrawlReport, CrawlServiceEntry, CrawlError — pydantic response models.

• AGOLUserPass, TokenSessionConfig — pydantic credentials / session config models.

• Settings, get_settings — process-wide runtime configuration backed by RESTGDF_* environment variables (CHUNK_SIZE, TIMEOUT_SECONDS, USER_AGENT, LOG_LEVEL, TOKEN_URL, REFRESH_THRESHOLD, DEFAULT_HEADERS_JSON).

• RestgdfResponseError — typed error raised when a strict-tier response fails validation; carries model_name, context, and raw payload attributes.

• restgdf.compat.as_dict / restgdf.compat.as_json_dict — migration helpers that convert any returned model (or passthrough any non-model) to a plain dict.

• restgdf.schema_drift logger — opt-in observability for vendor variance; NullHandler by default.

• Directory.report — the full CrawlReport (services, errors, root metadata) from the most recent .crawl() call.

Dependencies

• Added pydantic>=2.13.3,<3.

1.x

Earlier releases were not formally tracked here. See the Git tag history and PyPI release notes for pre-2.0 changes.