What Offline & Private AI Actually Means

Clawdbot and similar tools are a useful way to get ahead in understanding how LLMs work. What concerns me is how often "on-device" is treated as synonymous with "offline and private."

In default setups—and by that I mean the out-of-the-box configuration of most popular tools without toggling any privacy settings—your data is still:

• Logged and retained. Conversation histories, prompts, and outputs are stored on the provider's servers, often unencrypted at rest, with retention periods that range from 30 days to indefinite depending on the platform and tier.
• Stored at rest on third-party infrastructure. Even "local" tools that use a cloud API for inference send your input over the network; the computation happens on someone else's hardware, and the data touches someone else's storage layer.
• Likely used for training. Most free-tier and many paid-tier services reserve the right to use your inputs and outputs for model improvement unless you explicitly opt out, and the opt-out mechanism varies by provider.

This remains the default configuration for the majority of tools on the market.

Running models locally is not especially difficult if you stay within the OpenAPI-compatible pattern and use tools like LM Studio, Ollama, or equivalents—most can be set up in under an hour on consumer hardware. But locality alone does not imply privacy, security, or control. A model running on your machine can still phone home, log telemetry, or leak data through plugin integrations. You also need hardware with enough memory and compute to run the model at usable speeds, which currently means at least 16GB of unified memory for smaller models and significantly more for anything approaching frontier capability.

I work on local-first AI systems for regulated environments.

That work keeps reinforcing the same point: privacy depends on governance, not slogans. Local operation matters, but so do operator control, retention policy, and deployment discipline.

If you want to evaluate whether a tool is genuinely private, ask three questions: Does the data leave your hardware at any point during inference? Who has access to stored prompts and outputs, and under what retention policy? Is there a contractual or architectural guarantee against training on your data, or only a policy that can change with the next terms-of-service update?

Choose systems that let your data work for you without leaving your control. "Offline and private" should describe real control and accountable deployment, not a marketing claim you have to trust.