condense-json 1.0

Release: https://github.com/simonw/condense-json/releases/tag/1.0">condense-json 1.0

I'm trying to get braver at releasing 1.0 versions.

This little library is a year and a half old now - I've applied some sensible and non-disruptive fixes and shipped the big 1.0 for it.

Here's an example of what it can do, lifted from the README:

{
  "foo": {
    "bar": {
      "string": "This is a string with foxes in it",
      "nested": {
        "more": ["Here is a string", "another with foxes in it too"]
      }
    }
  }
}

Combine that with a replacements object:

{"1": "with foxes in it"}

And condense_json(input_json, replacements) produces the following:

{
  "foo": {
    "bar": {
      "string": {"$r": ["This is a string ", {"$": "1"}]},
      "nested": {
        "more": ["Here is a string", {"$r": ["another ", {"$": "1"}, " too"]}]
      }
    }
  }
}

It scans for strings or substrings that are present in that replacements object and replaces those with a special {"$r": ...} syntax in the output.

You can reverse the effect with uncondense_json(condensed, replacements).

The idea is to make it easier to store JSON that includes duplicated data from other related structures.

I use it to save space in the SQLite logs generated by https://llm.datasette.io/">LLM - see https://github.com/simonw/llm/pull/1586">PR #1586 for the latest iteration of that.

Tags: https://simonwillison.net/tags/json">json, https://simonwillison.net/tags/projects">projects, https://simonwillison.net/tags/python">python, https://simonwillison.net/tags/llm">llm