[Feature Request] [Epic] Clean up and sort repository

[mbuechse]

The repository is a curious blend of

• documents that have been officially archived (see Drafts, Operational-Docs, but also deprecated documents in Standards),
• documents that are normative and subject to a rigid formal process (see Standards: PR in a very certain format plus loose consensus),
• documents that are kind of informational and follow the classical PR-based process (see Community-Governance),
• a testing suite that also follows the classical PR-based process,
• a compliance monitor that has never left the state of being a prototype and that would benefit from a simplified process, not least because it's basically a one-man show (as unfortunate as that may be).

So, I propose to clean things up a bit.

• documents from Drafts and Operational-Docs that are still worth reading are moved to the docs repo, the directories deleted from the main branch,
• introduce subfolders draft and deprecated below Standards (we need a new version of scs-0001 for that, plus minor adjustments in the tooling), so it's more clear at first glance what's normative and effective,
• not sure about Community-Governance; could go to docs repo as well, or moved to the website, (or even to the minutes repo? it's not meeting minutes, but it's similarly related to current time),
• otherwise, keep Standards and Tests (although the latter could easily be moved into a separate repository),
• clearly document conceptual interfaces between Standards, Tests, and compliance monitor:
  • official report format for check scripts, main script, and potentially audits,
  • versioning of the test suite, the scope yaml format,
  • the business logic of how to evaluate the compliance status of a subject,
• move that business logic into a separate software module (to be located in the vicinity of Tests),
• move the compliance monitor into a separate repository where I can push to main.

Potentially if we decide to split this repo, we could also archive this one and create a new one for the core. (We could even contemplate moving to Codeberg, but OTOH let's not make this endeavor even more complicated.)

[mbuechse]

@toothstone @depressiveRobot @garloff @fkr Comments wanted.

[mbuechse]

@jklare maybe you want to comment as well :)

[jklare]

Looking at the repo and based on my own initial experience when joining the forum, I would strongly agree that it is a bit too cluttered currently to allow new contributors to find the relevant things. Personally, I would think everything apart from the Standards and Drafts folder should be either removed or moved somewhere else (following basically the criteria you proposed @mbuechse). Everything related to actual testing and compliance monitoring should be a separate repository to allow easier iteration and versioning for those things (while the standards repo does not need versioning, IMHO).

[fkr]

Indeed, I think this is a very welcome initiative and your proposal @mbuechse resonates well.

[mbuechse]

Makes me wonder if we even need a folder called Standards if that's basically the only one that's left. I would really archive this repo (maybe rename so the name can be reused) and put things into new ones.

[toothstone]

I'm generally also in favor of splitting up this project, to allow for more granular governance and hopefully also increase visibility of the distinct building blocks that make up the compliance flow.

Makes me wonder if we even need a folder called Standards if that's basically the only one that's left. I would really archive this repo (maybe rename so the name can be reused) and put things into new ones.

I'm not a fan of breaking git histories/blames and URLs just for the sake of aesthetics tbh. As someone who is late to the party it is often quite a task to reconstruct the history of specific ideas and decisions already, please don't make it even harder^^'

[jklare]

I agree with @toothstone in terms of "DO NOT BREAK HISTORY" :) For the clean-up and split, I would suggest the following workflow:

1. Remove everything from this repository that NOBODY needs.
2. Breakout the things that should live in new repositories by copying the full repo and all of its history and then doing an additional commit in the new repo, which removes everything, apart from the things that should now live in the new repo. The commit message should point to the original repo (this one) at the state before 3) happened.
3. After each breakout and creation of a new repo, create a commit in this repo which removes the parts that were moved to new repos, while using the commit message to point to those new repos.

[mbuechse]

We can try to keep the history of the main branch, but be advised that a clone of this repo is in the hundreds of megabytes, maybe even several GiBs, as Kurt pointed out recently (will look up the number shortly). So it would be good to prune the history.

[mbuechse]

I stand corrected. The clone is 8 MiB, almost the same with or without --single-branch. Curious, now I'm wondering what it was that Kurt had observed.

[jklare]

I think this is a reasonable amount for all the files in here and keeping the history shouldn't be a problem then.