-
Notifications
You must be signed in to change notification settings - Fork 4
DMD-922 - Add samples and security for llm export #2502
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
vojtabiberle
wants to merge
4
commits into
vb/DMD-921/llm-export-generator
Choose a base branch
from
vb/DMD-922/llm-export-samples-security
base: vb/DMD-921/llm-export-generator
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
DMD-922 - Add samples and security for llm export #2502
vojtabiberle
wants to merge
4
commits into
vb/DMD-921/llm-export-generator
from
vb/DMD-922/llm-export-samples-security
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
8fd0a2a to
60a1ec7
Compare
f8a3858 to
0e11410
Compare
60a1ec7 to
6c8da70
Compare
0e11410 to
fb8b1b3
Compare
6c8da70 to
2ca7d49
Compare
fb8b1b3 to
abdbe8f
Compare
2ca7d49 to
09f18af
Compare
abdbe8f to
4dc2192
Compare
09f18af to
ff88f73
Compare
0eb3386 to
cac5bcf
Compare
67c271f to
8dc0355
Compare
cac5bcf to
faaad27
Compare
8dc0355 to
8a3a7a2
Compare
faaad27 to
4b6ce89
Compare
8a3a7a2 to
0ed951b
Compare
0a8aa40 to
07790e0
Compare
0a816df to
a98c3ae
Compare
c9c4bd4 to
7a8c89b
Compare
a98c3ae to
3c2a2e9
Compare
7a8c89b to
0f454ba
Compare
3c2a2e9 to
99af6b7
Compare
0f454ba to
bd8a583
Compare
99af6b7 to
1cdd31a
Compare
bd8a583 to
f1f2c61
Compare
ddb7978 to
029e073
Compare
94c04b2 to
e2c2296
Compare
029e073 to
1dcec8e
Compare
e2c2296 to
a3ee0b0
Compare
1dcec8e to
9309d99
Compare
a3ee0b0 to
390a686
Compare
9309d99 to
ab1e155
Compare
390a686 to
de5829b
Compare
ab1e155 to
7653daf
Compare
de5829b to
d356a4c
Compare
7653daf to
fea7e7a
Compare
d356a4c to
e78dda4
Compare
fea7e7a to
a4bafec
Compare
e78dda4 to
3160b11
Compare
a4bafec to
3984869
Compare
3160b11 to
f24dcf7
Compare
3984869 to
efe95bb
Compare
f24dcf7 to
e585952
Compare
efe95bb to
3a65c44
Compare
e585952 to
77f1db7
Compare
3a65c44 to
a2418fa
Compare
77f1db7 to
7ae1037
Compare
a2418fa to
f56e998
Compare
7ae1037 to
d4e416b
Compare
- Add security.go with public repo detection and secret encryption - Add writer/csv.go for CSV sample output - Extend fetcher.go with FetchTableSample and FetchTableSamples methods - Extend generator.go with GenerateSamples, generateSamplesIndex, generateSampleFile methods - Add sample flags to export command (--with-samples, --without-samples, --sample-limit, --max-samples) - Update operation.go to integrate sample fetching and generation - Update options.go with sample-related options and helper methods Co-Authored-By: [email protected] <[email protected]>
- Add TestEncryptSecrets_SimpleMap for basic secret detection - Add TestEncryptSecrets_NestedMap for nested structure handling - Add TestEncryptSecrets_SliceWithMaps for array processing - Add TestEncryptSecrets_DeeplyNested for recursive encryption - Add TestEncryptSecrets_EmptyMap and TestEncryptSecrets_NoSecrets edge cases - Add TestDefaultSecurityOptions for configuration defaults 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>
The default behavior is already no samples, so --without-samples is redundant. Simplified to just --with-samples flag.
- Added DataQuality type with completeness, null counts, and distinct counts - Added computeDataQuality function to analyze sample data - Included data quality indicators in sample metadata output 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>
f56e998 to
3c18b07
Compare
d4e416b to
13270ab
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Release Notes
Add optional table data sampling to
kbc llm exportcommand with new flags--with-samples,--without-samples,--sample-limit, and--max-samples.Plans for customer communication
None.
Impact analysis
This PR extends the
kbc llm exportcommand with:The security module is implemented but not yet wired into the main export flow - it provides infrastructure for future PRs.
Change type
Feature
Justification
Part of the LLM twin format export feature (PR 6 of 7). Implements sample data export capability per RFC requirements.
Linear issue: DMD-922
Human Review Checklist
samples/index.jsonuses camelCase (tableId) whilesamples/{table}/metadata.jsonuses snake_case (table_id)Deployment
Merge & automatic deploy.
Rollback plan
Revert of this PR.
Post release support plan
None.
Link to Devin run: https://app.devin.ai/sessions/eecace21ad294aaabf3504f33fff40dc
Requested by: [email protected] (@vojtabiberle)