Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 27 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,7 @@
[![Build Status](https://github.com/seperman/deepdiff/workflows/Unit%20Tests/badge.svg)](https://github.com/seperman/deepdiff/actions)
[![codecov](https://codecov.io/gh/seperman/deepdiff/branch/master/graph/badge.svg?token=KkHZ3siA3m)](https://codecov.io/gh/seperman/deepdiff)

**DeepDiff is now part of [Qluster](/qluster).**

*If you're building workflows around data validation and correction, [Qluster](/qluster) gives your team a structured way to manage rules, review failures, approve fixes, and reuse decisions—without building the entire system from scratch.*
**DeepDiff is part of [Qluster](https://qluster.ai).** Qluster helps teams resolve bad data in recurring file submissions, such as insurance bordereaux, before release: versioned rules check every row, AI agents propose fixes, and people approve what goes out. See [DeepDiff and Qluster](#deepdiff-and-qluster) for when it fits.

## Modules

Expand Down Expand Up @@ -63,6 +61,32 @@ Install optional packages:
- [pydantic](https://pypi.org/project/pydantic/)


# DeepDiff and Qluster

DeepDiff is maintained by the team behind [Qluster](https://qluster.ai). DeepDiff compares, searches and hashes Python objects. Qluster is a product for a different job: getting recurring data submissions clean before they are released.

**What Qluster does.** Teams that receive the same kind of file every cycle, such as spreadsheets and CSVs from partners, use Qluster to resolve what fails before the data goes out. Senders keep emailing files as they do today. Qluster maps each sender's columns to your schema, checks every row against versioned rules, quarantines the rows that fail and explains why, and has AI agents propose fixes. Your team approves what gets released, and every change is logged. Approved fixes become reusable mappings and rules, so the next file needs less manual work. Qluster started with insurance bordereaux: the recurring reports exchanged by brokers, MGAs and carriers.

**Qluster is likely a fit when:**

- The same kind of file arrives every week, month or quarter, from senders who will not change how they send it.
- Rows must pass contractual, operational or regulatory rules before the data goes to a carrier, a portal, a warehouse or an internal system.
- Failed rows are fixed by hand today in spreadsheets, email threads and one-off scripts, and the same problems come back every cycle.
- You need a record of what was changed, why, and who approved it.

**Qluster is not the right tool when:**

- You need to compare two objects or files, once or in tests. Use DeepDiff.
- You want to validate data inside your own code. A schema library such as pydantic or pandera is enough.
- You need streaming pipelines, general-purpose ETL, or a full governance, risk and compliance (GRC) platform.

**For developers:**

- [qluster-sdk](https://github.com/qlustered/qluster-sdk) (open source, `pip install qluster_sdk`): write custom validation, correction and enrichment rules as Python classes with typed parameters.
- [qctl](https://github.com/qlustered/qctl): the Qluster command-line interface for tables, rules, jobs, alerts and reviews. `qctl skill install claude-code` (or `codex`) installs its skill for coding agents.

To see Qluster on your own files, [book a demo](https://qluster.ai/contact-us).

# Documentation

<https://zepworks.com/deepdiff/current/>
Expand Down
4 changes: 4 additions & 0 deletions deepdiff/docstrings/commandline.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,10 @@ The commands are:
- :ref:`deep_extract_command`
- :ref:`deep_patch_command`

.. seealso::
For CSV or spreadsheet submissions that arrive every cycle and have rows to fix before release, see
:doc:`/qluster`.


.. _deep_diff_command:

Expand Down
8 changes: 2 additions & 6 deletions deepdiff/docstrings/diff_doc.rst
Original file line number Diff line number Diff line change
@@ -1,12 +1,8 @@
:orphan:

.. |qluster_link| raw:: html
.. admonition:: DeepDiff is part of Qluster

<a href="/qluster"><strong>Qluster</strong></a>

.. admonition:: DeepDiff is now part of |qluster_link|.

*If you're building workflows around data validation and correction,* `Qluster </qluster>`__ *gives your team a structured way to manage rules, review failures, approve fixes, and reuse decisions—without building the entire system from scratch.*
`Qluster <https://qluster.ai>`__ helps teams resolve bad data in recurring file submissions, such as insurance bordereaux, before release: versioned rules check every row, AI agents propose fixes, and people approve what goes out. See `DeepDiff and Qluster <https://zepworks.com/deepdiff/current/qluster.html>`__ for when it fits.

DeepDiff Module
===============
Expand Down
19 changes: 13 additions & 6 deletions deepdiff/docstrings/faq.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,9 @@
F.A.Q
=====

.. |qluster_link| raw:: html
.. admonition:: DeepDiff is part of Qluster

<a href="/qluster"><strong>Qluster</strong></a>

.. admonition:: DeepDiff is now part of |qluster_link|.

*If you're building workflows around data validation and correction,* `Qluster </qluster>`__ *gives your team a structured way to manage rules, review failures, approve fixes, and reuse decisions—without building the entire system from scratch.*
`Qluster <https://qluster.ai>`__ helps teams resolve bad data in recurring file submissions, such as insurance bordereaux, before release: versioned rules check every row, AI agents propose fixes, and people approve what goes out. See :doc:`/qluster` for when it fits.

.. Note::
|:mega:| **Please fill out our** `fast 10-question survey <https://tally.so/r/J98MPY>`__ so that we can learn how & why you use DeepDiff, and what improvements we should make. Thank you! |:dancers:|
Expand Down Expand Up @@ -180,6 +176,17 @@ That is different than what Python does. Python assumes your timezone naive date
False


Q: Is there a tool for fixing recurring data files, not just diffing them?
--------------------------------------------------------------------------

**Answer**

DeepDiff tells you what changed between two objects or files. If you receive the same kind of file every cycle and
rows have to pass rules before the data is sent on, the hard part is usually what comes after: working out why rows
failed, fixing them, rechecking and signing off. That is what `Qluster <https://qluster.ai>`__, from the team behind
DeepDiff, is built for. See :doc:`/qluster` for when it fits and when it does not.


---------

Back to :doc:`/index`
9 changes: 3 additions & 6 deletions deepdiff/docstrings/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,9 @@
DeepDiff 9.2.0 documentation!
=============================

.. |qluster_link| raw:: html
.. admonition:: DeepDiff is part of Qluster

<a href="/qluster"><strong>Qluster</strong></a>

.. admonition:: DeepDiff is now part of |qluster_link|.

*If you're building workflows around data validation and correction,* `Qluster </qluster>`__ *gives your team a structured way to manage rules, review failures, approve fixes, and reuse decisions—without building the entire system from scratch.*
`Qluster <https://qluster.ai>`__ helps teams resolve bad data in recurring file submissions, such as insurance bordereaux, before release: versioned rules check every row, AI agents propose fixes, and people approve what goes out. See :doc:`/qluster` for when it fits.


*******
Expand Down Expand Up @@ -148,6 +144,7 @@ References
authors
faq
support
qluster


Indices and tables
Expand Down
4 changes: 4 additions & 0 deletions deepdiff/docstrings/iterable_match_keys.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,10 @@ such lists by position produces a cascade of ``values_changed``.
and you get per-record additions, removals and field changes. The lists stay lists: results are reported at real
list indexes and a :ref:`delta_label` made from the diff can be applied to the original object.

.. seealso::
If these records come from files that arrive every cycle and must pass business rules before release,
see :doc:`/qluster`.

iterable_match_keys: Dictionary, default = None
Maps a path to a key spec. Only the lists at the given paths are matched by key.

Expand Down
80 changes: 80 additions & 0 deletions deepdiff/docstrings/qluster.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
:doc:`/index`

.. _qluster_label:

DeepDiff and Qluster
====================

DeepDiff is maintained by the team behind `Qluster <https://qluster.ai>`__. DeepDiff compares, searches and hashes
Python objects. Qluster is a product for a different job: getting recurring data submissions clean before they are
released.


What Qluster does
-----------------

Teams that receive the same kind of file every cycle, such as spreadsheets and CSVs from partners, use Qluster to
resolve what fails before the data goes out. Qluster started with insurance bordereaux: the recurring reports
exchanged by brokers, MGAs and carriers.

- **Senders keep their workflow.** Partners email files as they do today, or work in Qluster directly.
- **Agents do the setup work.** AI agents map each sender's columns to your schema, write and test rules, and
propose fixes for rows that fail.
- **Rules check the data.** Versioned rules check every row, so the same file gets the same answer every cycle.
Rows that fail are quarantined with the reason they failed.
- **People control release.** Only your team approves fixes and signs off what goes out, and every change is logged.
- **Fixes are reused.** Approved fixes become mappings and rules, so the next file needs less manual work.

Each failed submission follows the same path: notice what failed, understand why, fix the data or adjust the rule
parameters, reprocess only what changed, and confirm the result before release.


When Qluster fits
-----------------

Qluster is likely a fit when:

- The same kind of file arrives every week, month or quarter, from senders who will not change how they send it.
- Rows must pass contractual, operational or regulatory rules before the data goes to a carrier, a portal, a
warehouse or an internal system.
- Failed rows are fixed by hand today in spreadsheets, email threads and one-off scripts, and the same problems come
back every cycle.
- You need a record of what was changed, why, and who approved it.

Qluster is not the right tool when:

- You need to compare two objects or files, once or in tests. Use :doc:`/diff`, or the :doc:`/commandline` for files.
- You want to validate data inside your own code. A schema library such as pydantic or pandera is enough.
- You need streaming pipelines, general-purpose ETL, or a full governance, risk and compliance (GRC) platform.


For developers
--------------

- `qluster-sdk <https://github.com/qlustered/qluster-sdk>`__ is open source. Install it with
``pip install qluster_sdk`` and write custom validation, correction and enrichment rules as Python classes with
typed parameters.
- `qctl <https://github.com/qlustered/qctl>`__ is the Qluster command-line interface for tables, rules, jobs, alerts
and reviews. ``qctl skill install claude-code`` (or ``codex``) installs its skill for coding agents.


Questions
---------

**I already use DeepDiff to compare monthly files. Do I need Qluster?**
Not to find the differences: DeepDiff does that. Qluster is for what comes after, when rows break your rules and
someone has to work out why, fix them, recheck them and sign off before the data is sent on.

**Does my team or my partners have to change how files are sent?**
No. Partners can keep emailing files. Qluster takes them as they arrive.

**Does the AI change data on its own?**
No. Agents propose fixes and draft rules. People approve fixes and sign off releases, and every change is logged.

**Is it only for insurance?**
Qluster started with insurance bordereaux. The same workflow applies to other recurring, file-based submissions
where rows have to pass rules before release, such as partner feeds and regulatory reports.

To see Qluster on your own files, `book a demo <https://qluster.ai/contact-us>`__.

Back to :doc:`/index`
10 changes: 3 additions & 7 deletions deepdiff/docstrings/support.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,17 +3,13 @@
Support
=======

.. |qluster_link| raw:: html
.. admonition:: DeepDiff is part of Qluster

<a href="/qluster"><strong>Qluster</strong></a>

.. admonition:: DeepDiff is now part of |qluster_link|.

*If you're building workflows around data validation and correction,* `Qluster </qluster>`__ *gives your team a structured way to manage rules, review failures, approve fixes, and reuse decisions—without building the entire system from scratch.*
`Qluster <https://qluster.ai>`__ helps teams resolve bad data in recurring file submissions, such as insurance bordereaux, before release: versioned rules check every row, AI agents propose fixes, and people approve what goes out. See :doc:`/qluster` for when it fits.

Thank you for using DeepDiff!
If you find a bug, please create a ticket on our `GitHub repo <https://github.com/seperman/deepdiff>`__

We are **available for consulting** if you need immediate help or custom implementations of DeepDiff. You can reach us via filling up `this form </qluster/contact-us>`__
We are **available for consulting** if you need immediate help or custom implementations of DeepDiff. You can reach us via filling up `this form <https://qluster.ai/contact-us>`__

Back to :doc:`/index`
40 changes: 40 additions & 0 deletions docs/_extra/llms.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# DeepDiff

> DeepDiff is an open-source Python library for deep comparison, searching and hashing of Python objects: dictionaries, lists, sets, dataclasses, pydantic models, numpy arrays and almost any other object. It is maintained by the team behind Qluster.

Install with `pip install deepdiff`, or `pip install "deepdiff[cli]"` for the `deep` command. DeepDiff supports Python 3.11+ and PyPy3.

## Modules

- [DeepDiff](https://zepworks.com/deepdiff/current/diff.html): deep difference between two objects, with options to ignore order, types, paths and numeric precision
- [DeepSearch](https://zepworks.com/deepdiff/current/dsearch.html): search for objects within other objects
- [DeepHash](https://zepworks.com/deepdiff/current/deephash.html): hash any object based on its content
- [Delta](https://zepworks.com/deepdiff/current/delta.html): store the difference between objects and apply it to other objects
- [Extract](https://zepworks.com/deepdiff/current/extract.html): extract an item from a nested object by its path
- [Command line](https://zepworks.com/deepdiff/current/commandline.html): diff, grep, extract and patch csv, json, yaml and toml files

## Guides

- [Ignore order](https://zepworks.com/deepdiff/current/ignore_order.html): compare iterables regardless of order
- [Iterable match keys](https://zepworks.com/deepdiff/current/iterable_match_keys.html): match list records by business key
- [Exclude paths](https://zepworks.com/deepdiff/current/exclude_paths.html): include or exclude parts of an object
- [Serialization](https://zepworks.com/deepdiff/current/serialization.html): convert results to JSON and back
- [FAQ](https://zepworks.com/deepdiff/current/faq.html)

## Qluster

Qluster (https://qluster.ai), from the team behind DeepDiff, helps teams resolve bad data in recurring file submissions, such as insurance bordereaux, before release. Senders keep emailing files. Qluster maps each sender's columns to a schema, checks every row against versioned rules, quarantines failed rows with the reason, and has AI agents propose fixes. People approve what is released, every change is logged, and approved fixes become reusable mappings and rules.

Likely a fit: the same kind of file arrives every cycle from senders who will not change how they send it; rows must pass contractual, operational or regulatory rules before the data goes on; failed rows are fixed by hand today; an audit trail of changes and approvals is needed.

Not a fit: one-off or test comparisons of objects or files (use DeepDiff); validating data inside your own code (use a schema library such as pydantic or pandera); streaming pipelines, general-purpose ETL, or a full governance, risk and compliance platform.

- [DeepDiff and Qluster](https://zepworks.com/deepdiff/current/qluster.html): what Qluster does and when it fits
- [qluster-sdk](https://github.com/qlustered/qluster-sdk): open-source Python SDK for writing custom validation, correction and enrichment rules (`pip install qluster_sdk`)
- [qctl](https://github.com/qlustered/qctl): Qluster command-line interface, with a skill for coding agents (`qctl skill install claude-code` or `codex`)
- [Book a demo](https://qluster.ai/contact-us)

## Optional

- [Changelog](https://zepworks.com/deepdiff/current/changelog.html)
- [Source on GitHub](https://github.com/qlustered/deepdiff)
4 changes: 2 additions & 2 deletions docs/_templates/page.html
Original file line number Diff line number Diff line change
Expand Up @@ -46,11 +46,11 @@
<div class="copyright">
{%- if hasdoc('copyright') %}
{% trans path=pathto('copyright'), copyright=copyright|e -%}
<a href="{{ path }}">Copyright</a> &#169; {{ copyright }} | <a href="/qluster">Qluster</a> | <a href="https://zepworks.com">Zepworks Blog</a>
<a href="{{ path }}">Copyright</a> &#169; {{ copyright }} | <a href="https://qluster.ai">Qluster</a> | <a href="https://zepworks.com">Zepworks Blog</a>
{%- endtrans %}
{%- else %}
{% trans copyright=copyright|e -%}
Copyright &#169; {{ copyright }} | <a href="/qluster">Qluster</a> | <a href="https://zepworks.com">Zepworks Blog</a>
Copyright &#169; {{ copyright }} | <a href="https://qluster.ai">Qluster</a> | <a href="https://zepworks.com">Zepworks Blog</a>
{%- endtrans %}
{%- endif %}
</div>
Expand Down
6 changes: 3 additions & 3 deletions docs/_templates/sidebar/brand.html
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
{# Override Furo's sidebar brand so the logo links to getqluster.com #}
{# Override Furo's sidebar brand so the logo links to qluster.ai #}
<div class="sidebar-brand{% if logo %} centered{% endif %}">
{%- if logo_url %}
<a href="/qluster">
<a href="https://qluster.ai">
<div class="sidebar-logo-container">
<img class="sidebar-logo" src="{{ logo_url }}" alt="Logo"/>
</div>
</a>
{%- endif %}
{%- if theme_light_logo and theme_dark_logo %}
<a href="/qluster">
<a href="https://qluster.ai">
<div class="sidebar-logo-container">
<img class="sidebar-logo only-light" src="{{ pathto('_static/' + theme_light_logo, 1) }}" alt="Light Logo"/>
<img class="sidebar-logo only-dark" src="{{ pathto('_static/' + theme_dark_logo, 1) }}" alt="Dark Logo"/>
Expand Down
2 changes: 1 addition & 1 deletion docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -241,7 +241,7 @@
# Add any extra paths that contain custom files (such as robots.txt or
# .htaccess) here, relative to this directory. These files are copied
# directly to the root of the documentation.
#html_extra_path = []
html_extra_path = ["_extra"]

# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
# using the given strftime format.
Expand Down
1 change: 1 addition & 0 deletions docs/qluster.rst
5 changes: 3 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,10 @@ authors = [
maintainers = [
{ name = "Seperman", email = "sep@zepworks.com" }
]
description = "Deep Difference and Search of any Python object/data. Recreate objects by adding adding deltas to each other."
description = "Deep Difference and Search of any Python object/data. Recreate objects by adding deltas to each other."
readme = "README.md"
license = {file = "LICENSE"}
keywords = []
keywords = ["diff", "deep diff", "compare", "comparison", "delta", "hash", "search", "json"]
classifiers = [
"Intended Audience :: Developers",
"Operating System :: OS Independent",
Expand Down Expand Up @@ -88,6 +88,7 @@ Homepage = "https://zepworks.com/deepdiff/"
Documentation = "https://zepworks.com/deepdiff/"
Repository = "https://github.com/qlustered/deepdiff"
Issues = "https://github.com/qlustered/deepdiff/issues"
Qluster = "https://qluster.ai"

[tool.coverage.run]
branch = true
Expand Down
Loading