feat(moss-cli): add mapped CSV/JSON bulk import#392
Open
Mahin2076 wants to merge 1 commit into
Open
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds a bulk-import workflow to the Moss CLI that supports CSV/JSON/JSONL inputs where the source data doesn’t use the standard id / text field names, addressing the usability request in #379 and making non-code document loading more discoverable.
Changes:
- Added
moss documents import(andmoss doc import) to bulk-import documents with--id-column,--text-column, and metadata-column mapping. - Enhanced the document loader to support column mapping, better CSV handling (including multiline quoted fields), and more actionable validation errors.
- Added focused tests plus documentation and changelog entries describing the new workflow.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| packages/moss-cli/tests/test_documents_import.py | Adds comprehensive parser + CLI coverage for mapped imports across CSV/JSON/JSONL and stdin. |
| packages/moss-cli/src/moss_cli/main.py | Registers documents as an alias command group for doc to make the feature discoverable. |
| packages/moss-cli/src/moss_cli/documents.py | Implements mapping-aware parsing/validation for CSV/JSON/JSONL while preserving legacy document formats. |
| packages/moss-cli/src/moss_cli/commands/doc.py | Adds the import command and shared submission logic; parses mapping flags and validates input before client creation. |
| packages/moss-cli/README.md | Documents the new import workflow and examples for mapped column usage. |
| packages/moss-cli/CHANGELOG.md | Notes the new bulk import feature in Unreleased. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+176
to
+179
| columns: List[str] = [] | ||
| for value in values: | ||
| columns.extend(name.strip() for name in value.split(",")) | ||
| return columns or None |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request Checklist
Description
Adds a discoverable bulk-import workflow for CSV, JSON, and JSONL files whose source columns do not already use Moss's
idandtextnames.moss documents importcommand and the shortermoss doc importform.--id-column,--text-column, repeatable--metadata-column, and comma-separated--metadata-columnsmappings.Fixes #379
Testing
pytest packages/moss-cli/tests/test_documents_import.py -q— 29 passed on Python 3.13 and 3.14ruff check . --extend-exclude '**/*.ipynb'mypy packages/moss-cli/src/moss_cli/documents.py packages/moss-cli/src/moss_cli/commands/doc.py --ignore-missing-importsblack --checkandisort --check-onlyon changed Python filestwine checkpassed for bothmoss documents import --helppassedThe full Moss CLI suite reports 57 passed and 2 failed under the latest Typer. The same two untouched shell-completion tests (
test_completions_bash_outputs_scriptandtest_completions_zsh_outputs_script) also fail onorigin/mainin the same environment because Typer's completion API changed; this PR does not alter that code.Type of Change