Debugging Production Systems
Four commands cover 95% of production debugging. Here's how they work together as a complete workflow — from noticing a failure to proving the fix.
Stream live requests — spot failures instantly
flux tail streams every request hitting your Flux functions in real time. Successes appear in green. Failures appear in red with an inline error summary.
The request ID on failures is your debugging handle. Copy it — you'll pass it to the next command.
Unlike log tailing, flux tail shows you the full execution context: method, path, status, latency, and error summary — all in one line.
Root-cause the failure — one command
flux why takes the request ID from flux tail and gives you everything you need to understand the failure:
- → Root cause — the first error in the span tree
- → Location — exact file and line in your code
- → Data changes — every row that was mutated (including rolled-back writes)
- → Suggestion — AI-generated fix hint based on the error pattern
This is enough to fix most failures. If you need more detail, continue to step 3.
Step through spans interactively
flux trace debug puts you in an interactive span-by-span walkthrough of the execution. Think of it like git bisect but for a single request's execution steps.
Each span shows you:
- → The exact input that was passed to that span
- → The exact output (or error) it returned
- → The duration and any DB state changes it caused
Use this when flux why identifies the failure but you want to understand exactly what the function received and returned at each step.
Verify your fix — diff two traces
After you fix the bug and deploy, trigger the same scenario and capture the new request ID. Then flux trace diff compares the two executions side-by-side:
- + New spans that were added
- − Spans that changed or were removed
- → Mutation changes (what the DB looks like now)
This is how you prove your fix worked — not just "it seems fine" but "here is exactly how the execution changed".