Skip to content

Refactor file serialization: suffix-driven formats, public to_dict/from_dict#148

Merged
raphaeltimbo merged 1 commit into
petrobras:mainfrom
raphaeltimbo:refactor-file-serialization
Jul 18, 2026
Merged

Refactor file serialization: suffix-driven formats, public to_dict/from_dict#148
raphaeltimbo merged 1 commit into
petrobras:mainfrom
raphaeltimbo:refactor-file-serialization

Conversation

@raphaeltimbo

Copy link
Copy Markdown
Collaborator

Summary

Follow-up to #145. This refactors how ccp objects are saved to and loaded from files, replacing the per-class toml/json branching with a single shared mechanism.

Design

  • The file extension decides the format. point.save("point.json") writes JSON, point.save("point.toml") writes TOML. The file_type parameter is removed, and an unsupported/missing extension raises a clear ValueError instead of silently truncating the target file (save) or leaving parameters unbound (load).
  • One shared moduleccp/data_io/serializers.py holds the suffix→(load, dump) registry, save_dict/load_dict helpers, and a Serializable mixin. Adding a future format is one line in the registry.
  • Public to_dict()/from_dict() on Point, Curve, Impeller, StraightThrough and BackToBack (replacing private _dict_to_save/_dict_from_load). This gives an in-memory serialization API — used by the app to embed impellers inside .ccp zips without temp files.
  • Atomic writes — data is dumped to a temp file and os.replaced onto the target, so a failed save never corrupts an existing file.
  • Versioned files — every saved file records ccp_version (same versioning scheme the app's convert() migration helper already uses), so future format changes can be migrated. Curve/Impeller files nest points under a points table; legacy flat files (and files without ccp_version) still load.

Bug fix

The app pages pass io.StringIO to Impeller.load / StraightThrough.load / BackToBack.load when restoring .ccp session files. For Impeller this has been broken since #145 (load now calls open() on its argument). Pages 1–4 and test_app now parse the toml in memory and use the public from_dict.

Breaking changes

  • save(file_name, file_type=...)save(file_name); the filename must have a .toml or .json extension (no silent extension appending).
  • load no longer accepts file-like objects — use from_dict(toml.loads(...)) for in-memory data.
  • _dict_to_save/_dict_from_load are replaced by public to_dict/from_dict.
  • StraightThrough.save/BackToBack.save gain json support via the mixin; their file parameter is now file_name.

Tests

  • Round-trip save/load parametrized over toml and json for Point, Curve (previously untested) and Impeller.
  • Unsupported-format error case, and a legacy flat-file load test.
  • Full suite: 159 passed (pytest ccp/tests -n 4 --dist loadfile), including the app integration tests against the real example .ccp files.

Notes for review

  • Part of the app-page diff is whitespace-only: ruff format applied to files that were not ruff-clean on main (hide-whitespace view recommended).
  • The 9 doctest failures in the fast tier (-m "not slow") pre-date this branch and are unrelated (verified on clean main); worth a separate fix.

…om_dict

Replace the per-class toml/json save/load logic with a single serializers
module (ccp/data_io/serializers.py):

- File format is decided by the file suffix (.toml or .json); the
  file_type parameter is removed. Unsupported suffixes raise ValueError
  instead of silently truncating or leaving variables unbound.
- Writes are atomic (temp file + os.replace), so a failed dump never
  corrupts an existing file.
- Point, Curve, Impeller, StraightThrough and BackToBack now expose
  public to_dict()/from_dict() (replacing _dict_to_save/_dict_from_load)
  and inherit save()/load() from a Serializable mixin.
- Saved files record the writing library version in a "ccp_version" key,
  enabling future format migrations; Curve/Impeller files nest points
  under a "points" table. Legacy flat files still load.
- Fix app pages passing io.StringIO to Impeller/StraightThrough/
  BackToBack.load (broken since petrobras#145 for Impeller): pages now use the
  public from_dict with toml parsed in memory.
- Tests: round-trips parametrized over toml and json for Point, Curve
  and Impeller, unsupported-format error case, and a legacy flat-file
  load test.

Part of the app-page diff is whitespace-only: ruff format applied to
files that were not formatted on main.
@raphaeltimbo
raphaeltimbo merged commit e247317 into petrobras:main Jul 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant