Skip to content

Accept and preserve string or integer user and location IDs #429

Description

@paco-barreras

Intended identifier rules

user_id and location_id should default to pandas string when NOMAD must create a schema without dtype information. User-provided string and integer identifiers, including nullable Int64, should both be accepted and preserved.

Required changes

  • Update nomad.io.base._is_traj_df and nomad.io.base._is_stop_df to accept string or integer user_id values.
  • Apply the same validation rule to location_id when that column is present. _is_stop_df currently does not validate location_id.
  • Update nomad.io.base._cast_traj_cols so integer user_id values are not converted to strings. Apply the same preservation rule to location_id if it is cast during ingestion.
  • Keep SCHEMA_DTYPES["user_id"] and SCHEMA_DTYPES["location_id"] as string; these remain the defaults, not the only accepted input types.
  • Update nomad.stop_detection.utils._get_empty_stop_df: preserve a typed empty identifier column such as pd.Series(dtype="Int64"); use the canonical string dtype when the identifier column is absent or is an untyped empty object column.

Synthetic identifiers

nomad.city_gen.City.blocks_gdf uses (coord_x, coord_y) tuples as an internal MultiIndex. nomad.visit_attribution.visit_attribution.poi_map falls back to poi_table.index when no usable location_id column is supplied. Passing a synthetic block table through that fallback can therefore expose coordinate tuples as location_id values.

Keep block-coordinate tuples as internal spatial keys, but do not emit them as public identifiers. Synthetic trajectory locations should use the scalar IDs from City.buildings_gdf["id"]. If block rows need public location IDs, create a stable scalar string or integer column before calling poi_map rather than using the tuple index.

Tests

  • Cover string, NumPy integer, and nullable Int64 identifiers in from_df, _is_traj_df, and _is_stop_df.
  • Check that integer identifiers survive ingestion without conversion to strings.
  • Check empty and populated stop summaries for both string and integer identifier dtypes.
  • Check that imported stop tables still do not require a cluster column.
  • Check that synthetic attribution does not return tuple-valued location_id values.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions