Data & Persistence overview

Durable writes

Staging bytes and replacing the active file are separate operations. This illustration prepares a same-directory temporary file; its owner must supply the replacement and recovery policy.

def stage_json_write(path, payload):
    directory = os.path.dirname(path)
    temp_path = path + '.tmp'

    encoded = json.dumps(payload, ensure_ascii=False, indent=2)
    if isinstance(encoded, unicode):
        encoded = encoded.encode('utf-8')

    with open(temp_path, 'wb') as stream:
        stream.write(encoded)
        stream.flush()
        os.fsync(stream.fileno())

    replace_file(temp_path, path)

This fragment needs json/os imports, Python 2 unicode and a project-supplied replace_file. The unused directory value does not create a parent directory. Serialization with ensure_ascii=False and conditional UTF-8 encoding finish before the temporary open; failures there prevent that open. The whole encoded value occupies memory, with no snapshot of mutable input, schema check, finite-number check, readback or size/latency bound.

The fixed .tmp name is opened in binary write mode, truncating any previous staged file. Write, flush and fsync precede context-manager close, then replacement. Failures propagate and prevent later stages; closing does not remove partial staged data. The skeleton has no cleanup or recovery handler.

replace_file is undefined here, so its name establishes no overwrite, backup, locking or durability contract. Its return value is ignored, including a false failure sentinel. A normal return leads to implicit None; an exception propagates. No persistence receipt, rollback or retry is supplied. The integrating owner must define replacement and diagnostics, supply module version attribution, and coordinate writers sharing the temporary name. See the backup-assisted illustration for a different explicit sequence; neither fragment establishes atomic or crash-durable replacement.

Python 2 text boundaries

  • Know whether each API gives you bytes or unicode.
  • Decode external bytes when they enter the program and encode when they leave.
  • Do not let JSON, log formatting, filenames, and network payloads silently mix types.
  • Use string keys consistently when JSON round-tripping numeric identifiers.

Archive and path safety

  • Resolve every archive destination and prove it remains beneath the intended root.
  • Limit download size, archive size, entry count, expanded size, and filenames.
  • Extract to staging, validate the expected files, then move them into place.
  • Do not trust remote filenames or executable helpers without integrity verification.