Your agent just finished a 40-minute run. It touched two dozen files, rewrote a config, deleted a module you didn’t expect it to touch, and reported “done.” Now you’re staring at a diff you didn’t watch happen, trying to reconstruct why — scrolling back through a terminal buffer that’s already scrolled off, or digging through a chat transcript that doesn’t quite line up with what actually landed on disk.

I hit this often enough running Claude Code, Codex CLI, and Gemini CLI back to back on the same repo that I built a tool to fix it. It’s called Halfhand, and it’s a local-first flight recorder for AI agents, written in Rust.

What Halfhand Actually Does

Halfhand ships as a single binary, hh. Point it at any agent command and it wraps that command in a PTY, recording terminal output and file changes as they happen — regardless of what you run. For agents it recognizes, it goes a level deeper and also captures the agent’s internal turns: prompts, tool calls, and tool results, as structured events.

hh run -- claude              # record a Claude Code session (or any command)
hh replay last                # faithfully play it back in an interactive TUI
hh inspect last                # non-interactive summary + step table
hh list                        # every recording, newest first
hh delete last --yes           # remove one

Everything — terminal bytes, file diffs, structured agent events — lands in a single local SQLite database plus a content-addressed blob store. There’s no server component, no account, no dashboard to log into. Recordings never leave your machine.

Recording a Session Looks Like This

Here’s hh inspect on a real Claude Code session — a step-by-step timeline of every file it touched, in order, with timestamps:

87 steps, 23 files changed, 36 minutes, one command. That table alone answers the question “what did it actually do” faster than re-reading the whole transcript.

Replaying It Faithfully

hh replay drops you into a keyboard-driven TUI: a timeline pane on the left, the detail for whatever step you’re on the right, and a keymap overlay (?) if you forget the bindings. j/k to move, J to jump to a timestamp, / to filter by kind or summary text, t to toggle terminal-output segments, d to jump straight to the next file diff.

Worth being precise about what this is: hh replay is a faithful transcript, not deterministic re-execution. It renders the agent’s text, tool calls, and file diffs in the exact order they happened, but the agent is never re-invoked, no API calls are made, and no side effects are reproduced. A session that got cut short (hh killed mid-run) replays the partial timeline and is honestly marked interrupted rather than pretending otherwise.

Zooming Into a Single Step

For scripting or quick audits, hh inspect <id> --step N --diff skips the TUI entirely and prints the exact unified diff for that step:

Add --json and the same data comes back structured, so you can pipe a session’s file changes into whatever review or CI tooling you already have. --failed filters down to steps where something went wrong — useful when an agent run silently regressed something and you don’t want to page through 80 steps to find where.

Installing It

Pick one, both install the hh binary:

# cargo, any OS with a Rust toolchain
cargo install halfhand

# shell installer (macOS & Linux), prebuilt binary + SHA-256 checksum
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/halfhandorg/halfhand/releases/latest/download/halfhand-installer.sh | sh

hh --help gives you the full command surface — every subcommand also takes --help with its own usage example:

A few worth calling out beyond the obvious ones:

Adapters: More Than Just a Terminal Recording

Every hh run captures terminal output and file changes no matter what you point it at. On top of that, Halfhand auto-detects a handful of agents and records their internal turns as structured events instead of just raw bytes: Claude Code, Claude Desktop, OpenAI Codex CLI, and Google Gemini CLI — force a specific one with --adapter if detection guesses wrong. If it can’t find a transcript for whatever you’re running, it degrades gracefully: you still get the complete terminal and file-change recording, just without the structured breakdown.

Secrets Are a First-Class Concern, Not an Afterthought

Recorded sessions can contain secrets present in prompts and tool outputs — that’s just the reality of recording an agent that has your API keys in its environment. Halfhand treats this as a first-class problem instead of a footnote:

• hh scan reports what it found, without ever printing the secret itself
• hh redact removes detected secrets from a recording in place, irreversibly
• hh export is redacted by default, whether you’re producing a JSON bundle, a portable .hh archive, or a self-contained HTML replay page

You can also turn on redaction at record time, before anything touches disk:

[redaction]
at_record = true   # default: false — scrub secrets before they hit disk
entropy = true     # default: true — conservative high-entropy detector
rules = [
  { name = "anthropic-api-key", pattern = "sk-ant-[a-zA-Z0-9\\-_]{32,}" },
  { name = "openai-api-key",    pattern = "sk-(?:proj|svcacct|[A-Za-z0-9]{2})-[A-Za-z0-9]{32,}" },
  { name = "stripe-secret-key", pattern = "sk_(?:live|test)_[a-zA-Z0-9]{24,}" },
]

Built-in detectors already cover AWS, GitHub, GitLab, Slack, PEM keys, and JWTs — the rules block just fills gaps for anything Halfhand doesn’t know about yet. A malformed rule is a hard error at startup, on purpose: silently dropping a detector would be a redaction hole, not a convenience.

Why Local-First

This is the part I care about most, and it’s not a marketing line — it’s enforced:

• Zero network calls. The hh binary links no HTTP client. A CI check fails the build if that ever changes.
• You own the data. One SQLite file plus a content-addressed blob store, in a data directory you control (HH_DATA_DIR). hh delete and it’s actually gone.
• Stable interface. The --json schema is frozen going into 1.0, CLI flags are additive-only, and database migrations only ever run forward.

No account, no telemetry, no “anonymous usage stats.” If you’re recording agent sessions that touch client code, production configs, or anything under an NDA, that constraint isn’t a nice-to-have.

What Developers Are Saying

“This is super cool! It’s like a VCR for LLMs.”
— Kenneth Reitz, creator of Requests, Certifi, Pipenv & more

Try It

cargo install halfhand
hh run -- claude
hh replay last

Halfhand is Apache-2.0 licensed and open to contributions — read CONTRIBUTING.md before opening a PR. Source, docs, and the full command reference live at github.com/halfhandorg/halfhand; the project site is halfhand.org.

I built this because I kept losing track of what my own agents were doing to my own repos. If you run Claude Code, Codex, or Gemini CLI against anything you’d want to audit later, it might save you the same headache it saved me.

The Modern Way to Install Python CLI Tools Globally Without Breaking PEP 668 Compliance

You’ve built a slick Python CLI tool—clean help messages, intuitive flags, everything just works. Time to install it globally:

pip install .

And then… this happens:

error: externally-managed-environment

× This environment is externally managed
╰─ To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.

I wrote a blog post about this error, explaining why it exists and how to work around it. But today, I want to focus on a more elegant solution that respects PEP 668 compliance while allowing you to install Python CLI tools globally.

Welcome to the PEP 668 era.

PEP 668 introduced “externally managed environments” to protect your OS from broken dependencies, especially on Linux distributions like Ubuntu 24.04. It’s a good safeguard—but it also means your old pip install habits for global CLI tools no longer work without hacks.

This post isn’t about fighting PEP 668. It’s about working with it—using modern, safe methods to install CLI tools globally without wrecking your system.

The Problem We All Face

Here’s the thing: PEP 668 isn’t trying to ruin your day. It’s actually solving a real problem that anyone who’s maintained Linux systems has experienced:

• Dependency hell: Your system’s Python packages conflicting with pip-installed ones
• Broken OS tools: That time apt stopped working because you installed the wrong version of requests
• The Friday afternoon panic: When your deployment scripts fail because someone globally installed a package that broke system dependencies

But here’s what PEP 668 didn’t solve: how to elegantly install the CLI tools we actually want to use globally. Tools like black, ruff, httpie, or that awesome script you just wrote.

What Actually Works in 2025

Let me save you some time. Here are the approaches that actually work, ranked by how much your future self will thank you:

1. pipx: The Tool You Should Be Using

If you’re not using pipx yet, you’re missing out. It’s specifically designed for this exact problem:

# Install pipx (once)
sudo apt install pipx  # Ubuntu/Debian
brew install pipx      # macOS

# Install CLI tools the right way
pipx install black
pipx install ruff
pipx install your-awesome-cli-tool

What makes pipx brilliant:

• Each tool gets its own isolated virtual environment
• Tools are globally accessible from ~/.local/bin
• Zero dependency conflicts between tools
• Dead simple management

pipx list                    # See what's installed
pipx upgrade black          # Update one tool
pipx upgrade-all            # Update everything
pipx uninstall black        # Clean removal

2. uv: The Speed Demon

If you haven’t tried uv yet, prepare to have your mind blown. It’s like pipx but written in Rust and ridiculously fast:

# Install uv
curl -LsSf https://astral.sh/uv/install.sh | sh

# Install tools at light speed
uv tool install ruff
uv tool install black
uv tool install your-cli-tool

# Management is equally fast
uv tool list
uv tool upgrade --all

The performance difference is noticeable. What takes pipx 30 seconds, uv does in 3.

3. pip –user: When You Need Simple

Sometimes you just need something installed quickly without the overhead:

pip install --user black

This installs to ~/.local/lib/python3.x/site-packages and puts binaries in ~/.local/bin. It’s fast, simple, and respects PEP 668.

The catch? No isolation. If two tools need conflicting dependencies, you’re back to dependency hell.

4. The Dedicated Virtual Environment Approach

For maximum control, create a dedicated virtual environment for your CLI tools:

# Set up once
python -m venv ~/.venv/cli-tools
source ~/.venv/cli-tools/bin/activate
pip install black ruff httpie your-awesome-tool

# Add to your shell config (~/.bashrc, ~/.zshrc)
alias black='~/.venv/cli-tools/bin/black'
alias ruff='~/.venv/cli-tools/bin/ruff'
alias http='~/.venv/cli-tools/bin/http'

This gives you complete control but requires more setup and maintenance.

Real-World Example: Setting Up a Development Environment

Here’s how I set up CLI tools on a new machine in 2025:

# Install uv (my current preference for speed)
curl -LsSf https://astral.sh/uv/install.sh | sh

# Essential development tools
uv tool install black          # Code formatting
uv tool install ruff           # Linting and more formatting
uv tool install mypy           # Type checking
uv tool install pytest         # Testing
uv tool install cookiecutter   # Project scaffolding
uv tool install httpie         # API testing
uv tool install poetry         # Dependency management

# Verify everything is working
uv tool list

Total time: under 2 minutes. Everything isolated, everything working.

The “But I Have a Script” Problem

“That’s great for published packages, but I just have a Python script I want to install globally.”

I hear you. Here’s the modern approach:

Option 1: Make it a proper package (recommended)

Create a minimal pyproject.toml:

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "my-awesome-script"
version = "0.1.0"
dependencies = []

[project.scripts]
my-script = "my_awesome_script:main"

Then install with pipx or uv tool install .

Option 2: Use pipx run for one-offs

pipx run --spec . my-script

This runs your script without permanently installing it.

Option 3: Direct installation with pipx

pipx install .  # From your script's directory

What About Docker and CI/CD?

In containerized environments, you can often ignore PEP 668 since you control the entire environment:

# In a Dockerfile, this is fine
RUN pip install --break-system-packages my-cli-tool

But even there, consider using uv for speed:

RUN curl -LsSf https://astral.sh/uv/install.sh | sh
RUN uv tool install my-cli-tool

The Migration Path

If you’re currently using --break-system-packages everywhere, here’s your migration strategy:

1. Audit what you have: pip list --user to see what’s installed
2. Start fresh: Use pipx or uv tool for new installations
3. Migrate gradually: Replace --break-system-packages installs one by one
4. Update your scripts: Change deployment scripts to use modern tools

Performance Comparison

I tested installing 5 common CLI tools on a fresh Ubuntu 24.04 system:

uv is clearly the speed winner, but pipx has better ecosystem maturity.

What’s Next?

The Python packaging ecosystem keeps evolving. Keep an eye on:

• PEP 704: Requiring virtual environments (might make this even more important)
• uv’s rapid development: New features land frequently
• pipx improvements: Still the most stable choice

Bottom Line

PEP 668 isn’t the enemy—it’s pushing us toward better practices. Instead of fighting it:

1. Use pipx or uv tool for CLI applications
2. Embrace isolation instead of global package soup
3. Update your deployment scripts to use modern tools
4. Never use --break-system-packages in production

Your future self (and your sysadmin) will thank you. Trust me on this one.

Rethinking the Command Line with AI

The terminal—long hailed as the developer’s power tool—has historically required an intimidating level of expertise. It demanded precise syntax, obscure flags, and near-religious mastery of text-based commands. But that era is evolving rapidly. AI is now embedding itself into the command-line interface (CLI), transforming it from a memorization gauntlet into an intuitive, intent-driven experience.

This change marks a deeper transformation: instead of focusing on how to do something, developers can now focus on what they want to achieve. It’s a major leap forward in developer experience, onboarding, and productivity—lowering the barrier for beginners and supercharging experts.

In this article, we examine three next-gen AI tools reimagining the terminal: Google Gemini CLI, Arkterm, and Warp.dev. Each offers a unique take on the AI-powered development environment, from large-context agents to fast command assistants and full multi-agent orchestration.

Google Gemini CLI: Your AI-Powered Terminal Assistant

The Gemini CLI from Google brings Gemini 1.5 Pro’s capabilities directly to the terminal. With its massive 1 million-token context window, it can reason over entire codebases, conversations, or documentation.

Key Features:

• Natural Language Interface: Ask it to generate code, fix bugs, or research APIs.
• Search Grounding: Integrates live Google Search results into its responses.
• Script-Friendly: Works in both interactive and non-interactive (scripted) modes.
• Open Source: Licensed under Apache 2.0 and open to community contributions.
• Generous Free Tier: 60 requests/minute and 1,000/day—free with a Google account.

Developers can provide custom context via Model Context Protocol (MCP) to tailor results to a project or company stack. It also shares functionality with Gemini Code Assist in VS Code, enabling multi-step plans, recovery from failures, and more.

ArkTerm: Fast, Safe, and Linux-Centric

ArkTerm is a lightweight, safety-first assistant for Linux users that translates natural language into precise CLI commands using LLAMA 3.1 via Grok’s LLM API.

What Sets It Apart:

• Sub-Second Responses: Delivers rapid results essential for terminal workflows.
• Command Safety: Never auto-executes commands. Warns users of destructive operations like rm -rf.
• Context-Aware: Detects your current directory/project type (e.g., Python, Rust) to fine-tune suggestions.
• Interactive Mode: Conversational follow-ups in-session.
• Whether you’re a sysadmin trying to compress logs or a dev managing Git repos, Arkterm reduces the time spent googling flags or parsing man pages—making Linux more approachable without sacrificing control.

Warp.dev: Terminal as a Multi-Agent Workspace

Warp.dev pushes the boundaries further by evolving the terminal into a multi-agent development environment. Rather than a single assistant, Warp uses multiple coordinated agents that collaborate across tasks.

Agentic Workflow:

1. Tell: Express your goal in natural language.
2. Agents Write: Different agents generate and modify code across repos.
3. Run in Parallel: Multiple agents execute workflows—debugging, building, deploying—all at once.

Warp’s Enterprise-Level Capabilities:

• Warp Drive & MCP: Share knowledge between agents for cohesive execution.
• Security First: BYO LLM, Zero Data Retention, and step-level user control.
• End-to-End Automation: From building full-stack apps to fixing bugs linked from Linear, Warp handles complete development lifecycles.

This agentic model hints at a near-future where developers orchestrate AI teams like engineers lead human ones—delegating tasks while maintaining high-level oversight.

Final Thoughts: The Terminal Becomes a Collaborator

AI-powered terminals mark a shift in how we write, test, and deploy code. Whether you prefer the raw power of Gemini CLI, the speed and safety of Arkterm, or the orchestration of Warp.dev, one thing is clear: the terminal is no longer just a tool—it’s becoming a partner.
This evolution could fundamentally reshape the path into software development. Learning to code might start with learning to communicate with AI agents rather than memorizing Bash syntax. That means more access, faster learning, and ultimately—smarter software development.
Welcome to the AI-native command line.

Developers building AI-powered apps need solid tools for productivity, observability, and deployment. In fact, a recent developer survey found 76% of respondents are using or planning to use AI tools in their workflow.

Open-source LLM projects have surged in 2024–2025, making it easier to experiment and ship with confidence. This post highlights 10 fully open-source, actively maintained LLM tools and libraries that deliver real developer value. Each section explains what the tool does, why it’s useful, and even shows a quick code snippet or GitHub link to get started.

Continue.dev – Custom AI Code Assistants

Continue.dev is an open-source platform for creating custom AI assistants inside your IDE, It provides free VS Code and JetBrains extensions (Apache-2.0 license) so that teams can build “AI-native” coding tools. With Continue, you can configure code autocompletion, in-editor chat, code edits, or even agentic tasks tailored to your codebase. For example, Continue stores its settings in ~/.continue/config.json.

A typical config might look like:

{
  "models": [
    {
      "model": "YourModelName",
      "title": "MyAssistant",
      "apiBase": "http://localhost:8080/v1/",
      "provider": "openai"
    }
  ]
}

Here you define which LLM endpoints the assistant should call. Once set up, you can press a hotkey to autocomplete code (tab-to-complete), ask questions about your code, or let an agent refactor it. The Continue GitHub has the VSCode/IDE plugin code and detailed docs. In short, Continue empowers developers to share and reuse AI-backed coding rules and prompts with familiar IDE tooling.

Langfuse – LLM Observability & Tracing

Langfuse is an open-source LLM engineering platform (Apache-2.0, self-hostable) that provides end-to-end observability for AI apps. It helps teams track, debug and improve LLM calls, logging each prompt and response as a trace. For example, Langfuse offers a simple decorator you wrap around your LLM calls:

from langfuse import observe
from langfuse.openai import openai

@observe()
def query():
    return openai.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": "Explain Langfuse"}],
    ).choices[0].message.content

query()

This @observe() decorator automatically logs the prompt, tokens, latency, and model parameters to Langfuse. The data appears in the Langfuse UI (local or cloud) so you can visualize sessions, spot errors, compare model performance, and manage prompts. Langfuse also includes versioned prompt management and evaluation tools, making it much easier to monitor costs and quality in production. See Langfuse on GitHub and try their quickstart for more integration examples.

Flowise – Visual LLM Flow Builder

Flowise’s drag-and-drop interface lets you connect LLM nodes, agents, and tools into a workflow. Flowise (MIT license) is a no-code/low-code LLM orchestration tool. It provides modular building blocks (“nodes”) so you can visually chain together models, prompt templates, retrieval tools, and decision logic. This is ideal for quickly prototyping AI assistants or autonomous agents without writing boilerplate code.

Getting started is easy: install the CLI, then launch the local UI with:

npm install -g flowise
npx flowise start

This opens a web UI at localhost:3000 where you can drag in nodes, set their parameters, and link them. For example, you might drop a chat model node, a function-calling node, and a data retriever into a flow. Flowise also supports multiple agents, human-in-the-loop review steps, and integrations with vector databases. In short, Flowise “provides modular building blocks … to build any agentic system, from simple workflows to autonomous agents”.

Check out the Flowise GitHub for code and deployment options (Docker, cloud, etc).

Dust – Multi-Agent AI Platform

Dust (MIT license) is a Rust/TypeScript-based platform for building AI agents and assistants. It started as a developer toolkit but now aims at enterprise AI deployments. Essentially, Dust lets you spin up multiple specialized agents (bots) that collaborate and use external tools, databases, or company knowledge bases. Unlike a single chatbot, Dust supports context-aware multi-agent workflows. For example, you might create a “Research Agent” to gather data and an “Analysis Agent” to interpret it, then orchestrate them in a pipeline. Dust includes a web UI and APIs for agent management. Developers can self-host it – the repo provides Docker Compose files to launch Dust’s services. In practice, using Dust involves running docker-compose up (per the repository) and then interacting with its REST API or UI to create agents. On GitHub the description summarizes it succinctly: “Dust – Custom AI agent platform to speed up your work.”. With 1.1k+ stars, Dust is battle-tested and integrates with tools like Slack, Notion, and more via connectors. See the dust-tt/dust GitHub for docs and quickstart.

OpenLLM – Deploy Any Open LLM as a Server

OpenLLM (Apache-2.0, by BentoML) makes running open-source LLMs in production a breeze. It provides a unified CLI/SDK so you can treat any model like an OpenAI API endpoint.

For example, installing OpenLLM is as simple as:

pip3 install openllm # install the package
openllm hello # try the example

You can then serve models with one command. For instance:

openllm serve llamacpp       # serves a local Llama model via HTTP
openllm serve starcoder:1.0  # fetches and serves StarCoder 1.0, GPU-optimized

Behind the scenes, OpenLLM integrates high-performance backends (Deepseek, vLLM, Ollama, etc.) and even offers a basic chat UI. It also handles Docker/Kubernetes deployment. Crucially, OpenLLM supports dozens of open LLMs (Llama 3.3, Qwen, Gemma, etc.) and will automatically download weights if needed. By conforming to the OpenAI API spec, your code doesn’t need to change if you swap a model from proprietary to open. OpenLLM is battle-tested (used in BentoCloud) and ideal for teams wanting to switch to local or custom models while keeping a familiar API. See the OpenLLM GitHub for more examples.

LangChain – LLM Integration Framework

LangChain (MIT license) remains the go-to Python framework for building LLM apps. It provides a standard interface for LLMs, embeddings, vector stores, and tools, making it easy to glue components together. LangChain’s core concept is “chains”: modular sequences of calls (to models, searches, tools, etc.) that handle complex workflows. For instance, a simple chain might read a question, query a vector DB, and then prompt an LLM with context – all with a few lines of code. Installation and use are straightforward:

pip install -U langchain   # install LangChain

from langchain.llms import OpenAI
llm = OpenAI(model_name="gpt-4o")
print(llm("Tell me a joke"))

LangChain shines in RAG (retrieval-augmented generation), agent simulation, and multi-step pipelines. It also supports dozens of integrations (OpenAI, HuggingFace, Chroma, Weaviate, etc.), so you can swap backends easily. Importantly, LangChain’s abstractions future-proof your code – if a better model emerges, you just change a config. For more, see the LangChain docs or the GitHub repo.

LlamaIndex – Data Augmentation & Knowledge Indexing

LlamaIndex (MIT license) is a “data framework” for building LLM applications over your data. It provides tools to ingest documents, create indices, and run queries that augment prompts with relevant context. For example, suppose you have a directory of PDFs.

LlamaIndex lets you load them and build a vector store index in minutes:

pip install llama-index-core llama-index-llms-openai llama-index-embeddings-huggingface

from llama_index.core import VectorStoreIndex, SimpleDirectoryReader

documents = SimpleDirectoryReader("path/to/docs").load_data()
index = VectorStoreIndex.from_documents(documents)
response = index.query("What does the report say about feature X?")

Under the hood, LlamaIndex handles tokenization, embeddings, and retrieval. You can use any LLM for querying (OpenAI, HuggingFace, Replicate, etc.) and any vector database. The library also supports other index types (trees, graphs). Essentially, LlamaIndex makes it trivial to add RAG to your app. Its rich ecosystem (300+ integrations) means it works with AWS, Pinecone, Supabase, and more. In short, LlamaIndex helps you “augment LLMs with private data, boosting accuracy and relevancy. Check out the LlamaIndex GitHub for guides and API details.

Guidance – Structured Prompting Language

Guidance (MIT license, by Anthropic) offers a new paradigm for prompt engineering. It introduces a domain-specific language embedded in Python to control generation. With Guidance, you can write templates with loops, conditionals, and even regex constraints, all inline. For example:

pip install guidance

from guidance import gen, models

llm = models.LlamaCpp("path/to/model.bin")
prompt = llm | gen("Tell me a poem about {topic}.", stop=".")
print(prompt(topic="sunrise"))

This code first loads an Llama model, then uses a template that injects a variable. Guidance ensures structured output and can drastically reduce off-topic responses. It even supports stateful control: you can programmatically manipulate generations, feed outputs back in, or use the #[ ] notation to “inscribe” values. Importantly, Guidance claims better cost and latency than naive prompting by optimizing tokens and reusing the same model session. It’s great for scenarios where precise output format is critical. See Guidance on GitHub for more examples and advanced features.

Text Generation Inference (TGI) – High-Performance LLM Server

Finally, Hugging Face’s Text Generation Inference (TGI) is a production-grade LLM server (Apache-2.0) written in Rust and Python. It’s optimized for low-latency, high-throughput inference on GPUs or CPUs. TGI supports models like Llama, Falcon, StarCoder, BLOOM, etc., and adds features like token streaming and batching. Getting started is simple with Docker. For example:

docker run --gpus all --shm-size 1g -p 8080:80 \
    ghcr.io/huggingface/text-generation-inference:3.3.2 \
    --model-id HuggingFaceH4/zephyr-7b-beta

This command launches a REST API on localhost:8080. You can then call the /generate_stream or OpenAI-compatible /v1/chat/completions endpoints to chat with the model. Under the hood, TGI uses parallelism across GPUs and supports tracing (Prometheus, OpenTelemetry) for metrics. In short, TGI lets you turn any open LLM into a scalable cloud endpoint, without crafting your own serving code. The TGI GitHub has extensive docs, performance tips, and examples.

Arkterm – AI-Native Shell Terminal

Arkterm’s AI-powered terminal interface, running an LLM inside your shell. Arkterm (MIT licensed) literally wires a large language model into your Linux terminal. You can speak natural language commands, ask for explanations, or have the LLM automate tasks. Arkterm is built by yours truly (inspired by Warp.dev). For example, after installing Arkterm, run:

git clone https://github.com/saadmanrafat/arkterm.git
cd arkterm
uv pip install -e .
arkterm --setup

This sets up a YAML config (~/.aiterm/config.yaml) where you specify an LLM and API key (e.g. via Groq). Once configured, your shell prompt becomes an AI assistant. Arkterm boosts productivity by letting devs query and extend their CLI with AI – great for scripting help or command automation. Check out Arkterm on GitHub for details.

Conclusion

Open-source LLM tooling is exploding — and these projects are leading the way. Whether you’re building apps, monitoring them, prototyping agents, or just playing around with new workflows, there’s likely a tool here you can adopt today.

More importantly, many of these are built by small teams or indie founders who care deeply about transparency, control, and developer experience. Supporting them means supporting the future of open AI infrastructure.

Shell Shocked: Wire an LLM Directly into Your Linux Terminal

The Philosophy: The Terminal Revolution You Didn’t See Coming

Inspired by a Wrap.dev engineer’s description of their interface as a way to “tell your computer what to do” rather than just chat, I explored Wrap.dev and developed my own minimal viable product for development. This project draws inspiration from Wrap.dev, aiming to create a comparable tool, though less sophisticated.

Picture this: You’re staring at a blank terminal, trying to remember the exact find command syntax to locate all Python files modified in the last week. Instead of fumbling through man pages or Stack Overflow, you type:

> find all python files changed in the last 7 days

And your terminal responds with the exact command you need, explains what it does, and offers to run it for you. Welcome to arkterm – where your Linux terminal becomes an AI-native workspace.

What Is arkterm?

arkterm is an open-source tool that transforms your terminal into an intelligent assistant by integrating large language models directly into your shell environment. Unlike traditional chatbots that live in browsers, arkterm brings AI into the very heart of your development workflow – the command line.

Built with Python and powered by Groq’s lightning-fast LLM API, arkterm bridges the gap between natural language and shell commands, making your terminal understand context, intent, and even your current project structure.

Why Your Terminal Needs AI (And Why Now)

The Problem with Traditional Command Lines

Modern developers juggle hundreds of commands across multiple tools, frameworks, and systems. The cognitive overhead of remembering syntax, flags, and edge cases slows down even experienced engineers. Traditional solutions like aliases and scripts only scratch the surface.

The AI Advantage

arkterm leverages the power of models like Llama 3.1 and Mixtral to:

• Understand Context: Knows your current directory, project type, and system environment
• Translate Intent: Converts natural language queries into precise shell commands
• Explain Actions: Shows you what commands do before executing them
• Learn Patterns: Adapts to your workflow and preferences

Getting Started: From Zero to AI Terminal in 60 Seconds

Prerequisites

• Python 3.10 or higher
• A free Groq API key (get one here)
• uv package manager (optional but recommended)

Installation

# Clone the repository
git clone https://github.com/saadmanrafat/arkterm.git
cd arkterm

# Install with uv (recommended)
uv pip install -e .

# Or with pip
pip install -e .

Initial Setup

# Create configuration file
arkterm --setup

This creates a YAML configuration file at ~/.aiterm/config.yaml:

API:
  api_key: "YOUR_GROQ_API_KEY_HERE"
  model: "deepseek-r1-distill-llama-70b"
  api_base: "https://api.groq.com/openai/v1"

SETTINGS:
  allow_command_execution: false
  max_tokens: 2048

Add your Groq API key and you’re ready to go!

Core Features That Make arkterm Shine

1. Natural Language Command Generation

Instead of memorizing complex syntax, describe what you want:

> arkterm "compress all log files older than 30 days"

Output:

I'll help you compress log files older than 30 days. Here's the command:

find /var/log -name "*.log" -type f -mtime +30 -exec gzip {} \;

This command:
- Searches in /var/log directory
- Finds files with .log extension
- Filters files older than 30 days (-mtime +30)
- Compresses each file with gzip

2. Interactive Mode for Continuous Assistance

Launch interactive mode for ongoing AI assistance:

arkterm -i

> check disk usage for directories over 1GB

Here's how to check disk usage for large directories:

```bash
du -h --max-depth=1 | grep '[0-9]G'


This command shows directory sizes and filters for those over 1GB.

> !exec du -h --max-depth=1 | grep '[0-9]G'
Executing: du -h --max-depth=1 | grep '[0-9]G'
2.1G    ./node_modules
1.4G    ./datasets
3.2G    ./models

3. Context-Aware Project Detection

arkterm automatically detects your project type and provides relevant suggestions:

def _detect_project_type(cwd: str) -> str:
    """Detects project type based on files in current directory"""
    files = os.listdir(cwd) if os.path.exists(cwd) else []
    
    if "package.json" in files:
        return "Node.js project detected"
    elif "requirements.txt" in files or "pyproject.toml" in files:
        return "Python project detected"
    elif "Cargo.toml" in files:
        return "Rust project detected"
    elif "go.mod" in files:
        return "Go project detected"
    elif ".git" in files:
        return "Git repository detected"
    else:
        return "General directory"

This context gets passed to the AI, enabling project-specific suggestions.

4. Safety-First Command Execution

arkterm prioritizes safety with built-in protections:

# Safety guidelines built into the system prompt
SAFETY_RULES = """
- Never execute commands that could damage the system
- Always warn about destructive operations (rm -rf, format, etc.)
- Prefer safer alternatives (mv to trash vs rm, etc.)
- Ask before modifying system files or configurations
"""

Advanced Use Cases

DevOps Automation

> arkterm "show me all docker containers using more than 500MB memory"

Response:

docker stats --format "table {{.Container}}\t{{.Name}}\t{{.MemUsage}}" --no-stream | awk 'NR>1 && $3 ~ /[5-9][0-9][0-9]M|[0-9]+G/ {print}'

Log Analysis

> arkterm "find all 404 errors in nginx logs from last hour"

Response:

grep "404" /var/log/nginx/access.log | awk -v cutoff=$(date -d '1 hour ago' '+%d/%b/%Y:%H:%M:%S') '$4 > "["cutoff'

System Monitoring

> arkterm "alert me if CPU usage goes above 80%"

Response:

while true; do
  cpu=$(top -bn1 | grep "Cpu(s)" | awk '{print $2}' | sed 's/%us,//')
  if (( $(echo "$cpu > 80" | bc -l) )); then
    notify-send "High CPU Usage" "CPU usage is ${cpu}%"
  fi
  sleep 60
done

The Technical Architecture

Core Components

To begin, establish a repository using UV, a Python package manager designed to streamline dependency management and virtual environments. Next, navigate to Groq (not Grok), register for their service, and get a free API key from their developer portal. Groq offers production-ready models such as gemma2-9b-it, meta-llama/Llama-Guard-4-12B, llama-3.1-8b-instant and more at no cost, though API rate limits and context token window restrictions apply. A comprehensive list of their supported models is provided.

The initial approach considered LangChain, but its intricate nature led to its exclusion from this project. As a result, we will use the requests library to communicate with Groq’s LLMs through API calls. It is important to note that Groq is not currently supported by LangChain. However, as engineers, we are adept at finding solutions.

# Core workflow
def fetch_response(query: str, config: Dict[str, Any]) -> str:
    """Main AI interaction pipeline"""
    
    # 1. Gather system context
    system_info = fetch_system_info()
    
    # 2. Create enhanced prompt with context
    system_prompt = get_enhanced_system_prompt(system_info)
    
    # 3. Call Groq API
    payload = {
        "model": config["API"]["model"],
        "messages": [
            {"role": "system", "content": system_prompt},
            {"role": "user", "content": query}
        ],
        "max_tokens": config["SETTINGS"]["max_tokens"],
        "temperature": 0.7
    }
    
    # 4. Parse and return response
    response = requests.post(api_endpoint, json=payload)
    return response.json()["choices"][0]["message"]["content"]

Command Block Parsing

arkterm intelligently extracts executable commands from AI responses:

def parse_command_blocks(response: str) -> list[str]:
    """Extract command blocks from markdown-formatted AI responses"""
    command_blocks = []
    current_block = []
    in_code_block = False
    
    for line in response.splitlines():
        stripped = line.strip()
        if stripped.startswith("```"):
            in_code_block = not in_code_block
            if not in_code_block and current_block:
                command_blocks.append("\n".join(current_block))
                current_block.clear()
        elif in_code_block:
            current_block.append(line)
    
    return command_blocks

Why Groq? The Speed Advantage

arkterm uses Groq’s inference engine for several key reasons:

1. Blazing Fast: Sub-second response times for real-time terminal interaction
2. Cost Effective: Free tier with generous limits perfect for CLI usage
3. Multiple Models: Access to Llama 3.1, Mixtral, and Gemma models
4. High Quality: Production-ready models with excellent code generation capabilities

Configuration and Customization

Model Selection

Switch between different models based on your needs:

# In interactive mode
> !model

Available Groq Models:
- llama3-8b-8192 (Fast, efficient)
- llama3-70b-8192 (More powerful)
- mixtral-8x7b-32768 (Mixtral model)
- gemma-7b-it (Google's Gemma model)
- deepseek-r1-distill-llama-70b (Distilled Llama model)

Currently using: llama3-8b-8192
Change model? (Enter model name or press Enter to keep current):

Safety Configuration

Control command execution permissions:

SETTINGS:
  allow_command_execution: true  # Enable command execution
  max_tokens: 4096              # Increase for longer responses

Best Practices and Pro Tips

1. Be Specific with Context

# Instead of: "delete old files"
# Try: "delete log files older than 7 days in /var/log"

2. Use Interactive Mode for Complex Tasks

For multi-step operations, interactive mode maintains context:

arkterm -i
> I need to set up a Python web scraper project
> create a virtual environment for it
> install requests and beautifulsoup4
> create a basic scraper template

3. Leverage Project Detection

arkterm automatically understands your project context:

# In a Node.js project
> arkterm "add a new dependency for testing"
# Suggests: npm install --save-dev jest

# In a Python project  
> arkterm "add a new dependency for testing"
# Suggests: pip install pytest

The Future of AI-Native Terminals

arkterm represents just the beginning of AI-native command-line interfaces. Future developments might include:

• Local Model Support: Running models like Code Llama locally for offline operation
• Shell Integration: Direct shell hooks for seamless AI assistance
• Team Knowledge Bases: Shared command libraries and best practices
• Automated Workflows: AI-generated scripts for complex multi-step operations

Getting Involved

arkterm is open-source and welcomes contributions:

• GitHub: saadmanrafat/arkterm
• Issues: Report bugs or request features
• Pull Requests: Contribute code improvements
• Discussions: Share use cases and ideas

Conclusion: Your Terminal, Supercharged

arkterm transforms the humble terminal from a command interpreter into an intelligent assistant that understands your intent, explains its actions, and helps you work more efficiently. By bringing large language models directly into the command line, we’re not just changing how we interact with our computers – we’re fundamentally rethinking what a terminal can be.

Whether you’re a DevOps engineer managing complex infrastructure, a developer navigating unfamiliar codebases, or a power user who lives in the terminal, arkterm offers a glimpse into the future of human-computer interaction.

The age of typing cryptic commands from memory is ending. The era of conversational computing has begun.

Seeing the World: A Beginner’s Guide to Convolutional Neural Networks using PyTorch

Welcome to the fascinating world of deep learning! If you’ve ever wondered how computers can recognize objects in images, distinguish between different types of clouds, or even power automated passport control systems, you’re about to uncover one of the key technologies behind it: Convolutional Neural Networks (CNNs).

These powerful neural networks are specifically designed to handle image data and have revolutionized computer vision over the past decade.

How Computers See Images

Before diving into CNNs, let’s understand how computers perceive images. Digital images are made up of tiny squares called pixels.

In a grayscale image, each pixel holds a numerical value representing a shade of gray, typically from 0 (black) to 255 (white). For color images, each pixel usually has three numerical values representing the intensity of Red, Green, and Blue (RGB) channels.

These values are organized into a tensor (like a multi-dimensional array) with dimensions for:

• Color channels (e.g., 3 for RGB)
• Height (number of pixel rows)
• Width (number of pixel columns)

The Problem with Traditional Neural Networks for Images

You might recall that traditional neural networks use linear layers where every input neuron is connected to every output neuron (fully connected networks). This architecture works well for data with a small number of features, but images pose a significant challenge.

Consider a simple grayscale image of 256×256 pixels:

• This single image has over 65,000 input features
• If you used a linear layer with even a modest 1,000 neurons, you’d end up with over 65 million parameters just in that first layer
• For color images, this number jumps significantly

Such a large number of parameters creates multiple problems:

1. Training becomes extremely slow
2. The risk of overfitting increases dramatically
3. Most critically, linear layers don’t inherently understand spatial patterns

If a linear layer learns to detect a feature, like a cat’s ear, in one corner of an image, it won’t automatically recognize the same ear if it appears in a different location. Images are all about patterns and their spatial relationships!

Introducing Convolutional Layers

This is where convolutional layers come in. CNNs use convolutional layers as a much more efficient and effective way to process images.

Instead of connecting every input pixel to every neuron, convolutional layers use small grids of parameters called filters (or kernels). These filters slide over the input image (or a feature map from a previous layer), performing a convolution operation at each position.

The convolution operation is essentially a dot product between the filter and a patch of the input data covered by the filter. The results of this sliding operation at each position are collected to create a feature map.

Key advantages of convolutional layers:

1. Parameter efficiency: They use far fewer parameters than linear layers for images
2. Location invariance: If a filter learns to detect a pattern, it can recognize that pattern regardless of its location in the input
3. Hierarchical feature learning: Early layers can detect simple features like edges and textures, while deeper layers combine these to detect complex features like shapes and objects

In PyTorch, you define a convolutional layer using nn.Conv2d. You specify the number of input and output feature maps (or channels) and the kernel size:

# 3 input channels (RGB), 32 output feature maps, 3x3 filter size
conv_layer = nn.Conv2d(in_channels=3, out_channels=32, kernel_size=3)

Essential CNN Operations: Padding and Pooling

Two other common operations in CNNs are zero padding and pooling.

Zero Padding

Often, zeros are added around the borders of the input before applying a convolutional layer. This technique helps:

• Control the spatial dimensions of the output
• Ensure that pixels at the border of the image are treated equally
• Prevent information loss at the edges

In PyTorch, you can specify padding using the padding argument in nn.Conv2d:

# Add 1 pixel of padding around the borders
conv_layer = nn.Conv2d(
    in_channels=3, 
    out_channels=32, 
    kernel_size=3, 
    padding=1
 )

Max Pooling

This operation typically follows convolutional layers. A non-overlapping window slides over the feature map, and at each position, the maximum value within the window is selected.

Using a 2×2 window, for instance, halves the height and width of the feature map. Max pooling helps to:

• Reduce the spatial dimensions
• Decrease the number of parameters and computational complexity
• Make the model more invariant to small shifts and distortions

In PyTorch, you implement max pooling with nn.MaxPool2d:

# 2x2 max pooling
pool_layer = nn.MaxPool2d(kernel_size=2)

Building a CNN Architecture

A typical CNN for image classification has two main parts: a feature extractor and a classifier.

1. Feature Extractor

This part is usually composed of repeated blocks of:

• Convolutional layers
• Activation functions
• Max pooling layers

Its purpose is to process the raw pixel data and extract relevant features.

2. Classifier

This part takes the flattened output of the feature extractor (which is now a vector) and passes it through one or more linear layers to make the final prediction. The output dimension of the last linear layer matches the number of target classes.

Here’s a simple CNN architecture in PyTorch:

import torch
import torch.nn as nn

class SimpleCNN(nn.Module):
    def __init__(self, num_classes=10):
        super().__init__()
        
        # Feature extractor
        self.features = nn.Sequential(
            # First block
            nn.Conv2d(in_channels=3, out_channels=16, 
                      kernel_size=3, padding=1),
            nn.ReLU(),
            nn.MaxPool2d(kernel_size=2),
            
            # Second block
            nn.Conv2d(in_channels=16, out_channels=32, 
                      kernel_size=3, padding=1),
            nn.ReLU(),
            nn.MaxPool2d(kernel_size=2),
            
            # Third block
            nn.Conv2d(in_channels=32, out_channels=64, 
                      kernel_size=3, padding=1),
            nn.ReLU(),
            nn.MaxPool2d(kernel_size=2)
        )
        
        # Classifier
        self.classifier = nn.Sequential(
            # Assuming input image was 32x32, 
            # after 3 pooling layers it's 4x4
            
            nn.Flatten(),  # Flatten the 4x4x64 feature maps
            nn.Linear(4 * 4 * 64, 128),
            nn.ReLU(),
            nn.Linear(128, num_classes)
        )
    
    def forward(self, x):
        x = self.features(x)
        x = self.classifier(x)
        return x

Activation Functions for CNNs

Like other neural networks, CNNs need nonlinearity to learn complex patterns. Activation functions are crucial for this.

For the hidden layers within the feature extractor, common choices include:

ReLU (Rectified Linear Unit)

• Outputs the input value if positive, and zero otherwise
• Avoids the vanishing gradients problem for positive inputs
• Available as nn.ReLU
• The most common choice for CNNs

Leaky ReLU

• A variation of ReLU that outputs a small non-zero value for negative inputs
• Prevents the “dying neuron” problem sometimes seen with standard ReLU
• Available as nn.LeakyReLU with a negative_slope argument

For the output layer:

• Sigmoid is typically used for binary classification
• Softmax is used for multiclass classification

Handling Image Data in PyTorch

To train a CNN, you need to prepare your image data. PyTorch’s torchvision library is very helpful here.

With a directory structure where each class has its own folder, you can use ImageFolder to create a dataset:

import torch
import torchvision
import torchvision.transforms as transforms

from torchvision.datasets import ImageFolder
from torch.utils.data import DataLoader


# Define transformations for images
image_transforms = transforms.Compose([
    transforms.ToTensor(),        # Convert PIL Image to PyTorch Tensor
    transforms.Resize((128, 128)) # Resize image to 128x128
])

# Create a dataset using ImageFolder
# Assumes data is in a directory structure like:
# cloud_train/
#   ├── class1/
#   │   └── img1.jpg
#   └── class2/
#       └── img2.jpg

train_dataset = ImageFolder(root='cloud_train',transform=image_transforms)

# Create a DataLoader for efficient batching and shuffling
train_loader = DataLoader(train_dataset,batch_size=32, shuffle=True)

Data Augmentation: Making Your Model Robust

A powerful technique for image data, especially to combat overfitting, is data augmentation. This involves applying random transformations to the training images, such as:

• Random Rotation: Exposes the model to objects at different angles
• Horizontal Flip: Simulates different viewpoints
• Color Jitter: Simulates different lighting conditions

These transformations artificially increase the size and diversity of your training set, making the model more robust to variations found in real-world images.

Implementation in PyTorch is straightforward:

train_transforms = transforms.Compose([
    transforms.RandomHorizontalFlip(p=0.5),
    transforms.RandomRotation(degrees=15),
    transforms.ColorJitter(brightness=0.1, contrast=0.1),
    transforms.ToTensor(),
    transforms.Resize((128, 128))
])

# Data augmentation only for training data, not validation/test
train_dataset = torchvision.datasets.ImageFolder(
    root='cloud_train',
    transform=train_transforms
)

Remember to choose augmentations that are appropriate for your specific task. Some augmentations could change the meaning of the image (e.g., flipping a “W” vertically might make it look like an “M”).

Training Your CNN

Training a CNN involves the standard deep learning training loop:

1. Define a loss function (e.g., nn.CrossEntropyLoss for multiclass classification)
2. Choose an optimizer (e.g., optim.Adam or optim.SGD)
3. Loop through multiple epochs (full passes through the training data)
4. Inside each epoch, process batches of data from the data loader

Here’s a complete training loop:

import torch.optim as optim

# Instantiate model, loss function, and optimizer
model = SimpleCNN(num_classes=10)
criterion = nn.CrossEntropyLoss()  # For multiclass classification
optimizer = optim.Adam(model.parameters(), lr=0.001)  # Adam optimizer

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device)

# Training loop
num_epochs = 10
for epoch in range(num_epochs):
    model.train()  # Set model to training mode
    running_loss = 0.0
    
    for inputs, labels in train_loader:
        # Move data to the same device as model
        inputs, labels = inputs.to(device), labels.to(device)
        
        # Zero the parameter gradients
        optimizer.zero_grad()
        
        # Forward pass
        outputs = model(inputs)
        loss = criterion(outputs, labels)
        
        # Backward pass and optimize
        loss.backward()
        optimizer.step()
        
        running_loss += loss.item()
    
    print(f"Epoch {epoch+1}/{num_epochs}, Loss: {running_loss/len(train_loader):.4f}")

Evaluating Your CNN

Evaluating your model’s performance is crucial. Data is typically split into training, validation, and test sets.

Key evaluation metrics for classification include:

• Accuracy: The overall frequency of correct predictions
• Precision: The fraction of correct positive predictions among all positive predictions
• Recall: The fraction of all positive examples that were correctly predicted
• F1 Score: The harmonic mean of precision and recall

Here’s an evaluation loop:

# Evaluation loop
model.eval()  # Set model to evaluation mode
correct = 0
total = 0

with torch.no_grad():  # Disable gradient calculation
    for inputs, labels in test_loader:
        inputs, labels = inputs.to(device), labels.to(device)
        
        # Forward pass
        outputs = model(inputs)
        
        # Get predicted class
        _, predicted = torch.max(outputs, 1)
        
        # Update statistics
        total += labels.size(0)
        correct += (predicted == labels).sum().item()

accuracy = 100 * correct / total
print(f"Test Accuracy: {accuracy:.2f}%")

Tracking training loss vs. validation loss (and accuracy) is key to detecting overfitting; if training loss keeps decreasing but validation loss starts to rise, your model is overfitting.

Fighting Overfitting in CNNs

Besides data augmentation, other strategies to fight overfitting include:

1. Dropout

Randomly deactivating a fraction of neurons during training, preventing over-reliance on specific features:

nn.Conv2d(32, 64, 3),
nn.ReLU(),
nn.Dropout(p=0.25),  # 25% dropout after activation
nn.MaxPool2d(2)

2. Batch Normalization

Normalizing the activations of the previous layer to speed up training and add some regularization:

nn.Conv2d(32, 64, 3),
nn.BatchNorm2d(64),  # Batch normalization after convolution
nn.ReLU()

3. Weight Decay

Adding a penalty to the loss function to encourage smaller weights:

optimizer = optim.Adam(model.parameters(), lr=0.001, weight_decay=1e-4)

4. Early Stopping

Monitoring validation performance and stopping training when it starts to degrade.

Modern CNN Architectures

While our example used a simple CNN, many powerful architectures have been developed:

• VGG: Uses very small 3×3 filters with many layers
• ResNet: Introduces skip connections to help train very deep networks
• Inception/GoogLeNet: Uses parallel paths with different filter sizes
• EfficientNet: Scales depth, width, and resolution together for efficiency

Many of these are available pre-trained in torchvision.models and can be used for transfer learning.

Conclusion

CNNs are the backbone of modern computer vision. By understanding how they process images through convolutional filters, pooling, and activation functions, you’ve taken a significant step in building powerful models that can truly “see” the world.

The key insights to remember:

• CNNs use sliding filters to detect patterns regardless of their location
• They build hierarchical representations from simple features to complex ones
• Techniques like pooling and padding help control spatial dimensions
• Data augmentation and regularization techniques like dropout are essential for robust models

Now it’s time to experiment and build your own CNN models! Whether you’re interested in image classification, object detection, or more advanced tasks like image segmentation, the principles covered here will serve as your foundation.

Deta Surf: Reclaim Your Digital World

We spend so much time in our web browsers today—they’ve become the central hub of our digital lives. But more often than not, it feels like we’re just navigating a scattered mess. We’re constantly switching between remote apps and services where our information lives, disconnected from context.

Enter Deta Surf — a new, invite-only alpha browser handcrafted in Berlin that rethinks what a browser can be.

Deta’s mission is to elevate minds through delightfully human computing—tools that enhance our intelligence and keep us in control.

Surf blends the power of a browser, a file manager, and an AI assistant into one coherent interface. It aims to bring back the “personal” in personal computing by keeping your data local, contextual, and always accessible.

Bringing Your “Stuff” Together

One of Surf’s key features is the concept of “Stuff.” This is a central place where you can collect and manage various types of digital content. This isn’t limited to just bookmarks; you can:

• Save websites as tabs
• Drag in files from your local machine
• Add images, YouTube videos, PDFs
• Capture super-powered screenshots

The idea is that it’s all multimedia, all in one place, and easily searchable.

“It’s a game changer for collecting wisdom, research, and notes,” said Limhi, a life guide and former law graduate, who appreciates that Surf keeps everything in one place and makes it easy to trust it to surface what’s needed.

Organizing with Contexts

To help manage the complexity of different projects or areas of your life, Surf introduces “Contexts.” These are like smart, focused environments—whether for work, reading, programming, or personal finances.

Within a context, you can:

• Open tabs (like a traditional browser)
• Add long-term storage items
• Visually arrange items on a “desktop”

Surf remembers where you left off in each context, and you can switch between them freely.

You can create contexts manually or use “Smart Contexts” by simply describing a topic—like “linear algebra” or “ramen recipes”—and Surf will automatically group related content from your Stuff. These are dynamic and update live.

AI That Actually Makes Sense

Surf’s standout feature is its thoughtful AI integration—not just AI for AI’s sake.

“What You See is What You Chat”

You can select any area on your screen (like a block of text, image, or a YouTube video), draw a rectangle around it, and ask a question. The AI takes visual and contextual input into account to explain technical concepts or summarize information.

Surf can process:

• Dozens of open tabs
• Thousands of pages of content
• Transcripts from YouTube videos

All without you needing to copy/paste or switch context.

Use AI with Any Source

You can ask questions based on:

• The active tab
• All open tabs
• A specific “Stuff” collection
• A particular Context

Surf will cite its sources so you can click and jump back to the original document or tab.

Currently supported LLMs include:

• Claude 3.7 Sonnet (default)
• GPT-4o
• Gemini Flash

(Some models may not support vision features yet.)

Spatial Browsing and Local Data

Taking cues from older desktop operating systems, Surf introduces spatial interfaces.

Each context can have its own desktop—an open canvas where you can arrange:

• Tabs
• PDFs
• Images
• Notes
• Other contexts

You can set a photo as the background to personalize each space. Think of it as infinite desktops, linked together like a mind map.

Local First. Cloud Second.

Deta Surf is built on a strong privacy foundation: your data lives locally.

• Files are stored on your device
• Local database powers search
• Local embeddings fuel the AI
• Even the local LLM can run offline

Cloud features will exist, but as a secondary option—similar to Obsidian’s sync model.

Who is Using Surf?

Deta Surf is already drawing attention from students, engineers, designers, and knowledge workers.

Limhi’s experience shows how Surf helps with:

• Organizing research
• Synthesizing large topics
• Revisiting trusted information

Alpha Status and the Road Ahead

Surf is still in its early days—invite-only, alpha-stage software. That means:

• You’ll encounter bugs
• It’s not ready as a full Chrome replacement (yet)
• It’s completely free for now

In the future, Deta plans to charge for cloud-based features but promises to keep the local-first experience intact. Their broader mission is to create joyful tools that enhance human intelligence—not replace it.

Try It Out

If you’re tired of feeling scattered—and excited by the idea of a browser that works with your context, understands your content, and respects your local data—then Deta Surf is absolutely worth watching.

🌐 Apply for early access

A no nonsense guide to UV (a Python Package Manager)

UV, a modern Python package manager, comes from an unexpected place — it’s written in Rust. It’s
10-100 times faster than traditional package managers. However, its true strength lies in addressing
long-standing issues in Python package managers, such as complex dependency resolution, inefficient
virtual environment handling, and fragmented tooling, offering a more reliable and streamlined experience
for developers. It’s designed to be a drop-in replacement for pip, virtualenv, pip-tools, and sometimes
even Poetry.

Why Use UV?

• Instant environment creation
• Fast, deterministic dependency resolution
• Built-in virtualenv management
• Lockfile support (uv.lock)
• Built-in CLI tool isolation (uvx)
• Python version management via .python-version
• Single binary, no runtime deps

Think of it as:
pip + virtualenv + pip-tools + pyenv + parts of Poetry + parts of Conda, all in one CLI tool
— but up to 100x faster.

Install UV with Astral’s Installer:

curl -Ls https://astral.sh/uv/install.sh | bash

Initialize a new project

uv init

This command creates a pyproject.toml and .venv with your current Python version. uv init
can prompt for additional project metadata.

Add Dependencies

uv add requests

Run your project in the environment

uv pip list
uv pip freeze
uv run main.py

You can also activate the venv manually

uv venv .venv  
source .venv/bin/activate

It’s compatible with existing pip projects or requirements.txt

You can still install traditional requirements.txt:

uv pip install -r requirements.txt

Install from lockfile

uv install

This installs all dependencies based on pyproject.toml and uv.lock

Upgrade dependencies

uv update

Resolves and upgrades all packages, updating the lockfile.

Okay so far what do we have?

$ tree
├── main.py
├── pyproject.toml
├── .python-version
├── README.md
└── uv.lock

Using Any Python Version with uv (even if it’s not installed)

One of UV’s most powerful features is how it handles Python versions seamlessly — even if you
don’t have that version installed locally.

uv venv -p 3.6 .venv

• This will create a virtual environment using Python 3.6.
• If you don’t have Python 3.6 installed, UV will:
  • Automatically detect that it’s missing.
  • Suggest installing it via pyenv, which UV supports natively.
  • Use .python-version to track which Python version the project is using.

Switching Python Versions Per Project

You can use .python-version manually:

echo "3.12.0" > .python-version
uv venv

UV reads the version, creates the venv with that interpreter, and tracks everything in
pyproject.toml

Removing Package

uv remove requests 

Cleanup

rm -rf .venv uv.lock

This makes UV fully self-contained and version-aware, solving one of the biggest pain points in Python development.

Why pip Suddenly Refuses to Install Globally?

Let’s try to recreate the error message.

$ pip3 install pandas
$ error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.

    If you wish to install a non\-Debian\-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3\-full installed.

    If you wish to install a non\-Debian\-packaged Python application,
    it may be easiest to use pipx install xyz, which will manage a
    virtual environment for you. Make sure you have pipx installed.

    See /usr/share/doc/python3.12/README.venv for more information.

note: If you believe this is a mistake, please contact your Python 
installation or OS distribution provider. 
You can override this, at the risk of breaking your Python installation or OS, 
by passing --break-system-packages. 

hint: See PEP 668 for the detailed specification.

“Errors should never pass silently—unless explicitly silenced.” — The Zen of Python

One such error is externally-managed-environment.

This message indicates that your Python environment is controlled by your operating system’s package manager (like apt, dnf, or yum), not by you. In simpler terms:

Your OS manages this Python installation. Hands off.

These system-managed environments are used to run critical tools, and direct modifications via pip could break essential components. That’s why these environments restrict you from installing or uninstalling packages freely. This isn’t a core Python change; it’s a coordinated effort between Python packaging tools and Linux distribution maintainers to improve system stability. Starting in 2022, distributions like Debian, Ubuntu 22.04+, and Fedora began enforcing this policy.

It’s also why tools like Docker can run into complications—making the transition to PEP 668 a frustrating experience, to say the least.

How Should You Install Python Packages in 2025?

$ python3 -m venv .venv
$ source .venv/bin/activate
$ pip3 install pandas # example

When creating virtual environments, check out UV — a Python package manager written in Rust, which is lightning fast. This gives you a self-contained environment with full control over what gets installed, without interference with the system Python and without the need to use sudo.

Why We’re Better Off With PEP668

Installing Python packages globally with pip has always been risky on Linux. It worked—until it didn’t. A single bad install could break tools like apt, disable automation scripts, or prevent Python from launching entirely. I've been on the wrong side of this a few times, and it’s not a pleasant experience.

This change doesn’t remove functionality—it just puts a guardrail in place. If you know what you’re doing, you can still override it. But for most users, it helps avoid subtle, frustrating bugs that only show up when it’s too late to undo them easily.

Break System Packages Flag

What happens when we --break-system-packages?

$ pip install <your-package> --break-system-packages

This tells pip:

Yes, I know I’m about to mess with a system-managed Python install. Let me do it anyway.

Not exactly safe—but it works if you know what you’re doing.

Risks Include

• You may overwrite system-critical packages like urllib3, certifi, or requests.
• It can break utilities like apt, dnf, or even the system’s python3 command.
• Uninstalling packages later may fail or remove components needed by your OS.
• Updates from your system package manager could conflict with or undo your changes.

Using System Package Manager to Install Python Packages

$ sudo apt install python3-requests

The downside of using apt for Python packages is that they are often several versions behind the official releases on PyPI. Some packages may not be available. Finally, dependency conflicts can arise when mixing apt and pip installations.

Final Thoughts

PEP668 isn’t here to ruin your workflow—it’s here to protect your system and nudge you toward better habits. Yes, the error was annoying the first time, but honestly, it forces us all to improve our Python hygiene—and that’s not a bad thing.