Frequently Asked Questions About Deming Live-Build Full Stack Iteration Method
22 answers covering everything from basics to advanced usage.
// Basics
What does 'write it yourself with Jarvis' actually mean in practice?
It means you can ask AI to outline a plan, explain a SQL concept, or suggest an approach — but you physically type every line of code into your editor yourself. The AI is the advisor; you are the engineer. This ensures you build a mental model of your code so you can fine-tune specific processes later without being lost.
What is a nightly sweep in this method?
A nightly sweep is the scheduled batch job that calls the third-party API, aggregates the full dataset of entities, computes statistics, and saves a snapshot to the database. It runs on a cron schedule such as daily or every 12 hours and is where expensive aggregate computations belong.
What is a run_id and why does it matter?
A run_id is a unique identifier for each nightly sweep execution. It matters because it scopes incremental updates — you fetch MAX(run_id) and only increment appearances for entities present in that latest run. Without run_id scoping you'd either recount everything or lose the ability to target only the currently active rows.
What is the count_appearances flag for?
The count_appearances flag is a boolean parameter on the build pipeline function that controls whether the incremental appearances update runs after saving a snapshot. It lets you reuse the same pipeline function for non-leaderboard data paths without triggering the increment, keeping one pipeline reusable across contexts.
// How To
How do I define a feature before I start coding?
State what you're adding in one sentence, like 'track how many times a leaderboard trader has appeared in our daily sweeps.' Then list every layer it touches: database schema, batch pipeline, API query, frontend table, and frontend sort/filter controls. Do not write any code until this map exists — it prevents half-wired features.
How do I structure a transaction with rollback for a save-then-increment sequence?
Use the pattern: try → save each user → if count_appearances is True call update_leader_appearances(run_id) → commit → log success. Except Exception: rollback → raise. Finally: close cursor. Wrapping the insert and update in one transaction means a partial failure rolls back cleanly instead of leaving the connection broken.
How do I write the incremental appearances SQL correctly?
First fetch the latest run with SELECT MAX(run_id) FROM leader_snaps. Then run UPDATE leaders SET appearances = appearances + 1 WHERE address IN (SELECT DISTINCT address FROM leader_snaps WHERE run_id = [max_run_id]). This touches only rows active in the current run and never recounts historical data.
How do I add a frontend sort button and table column for a new field?
Add the field name like 'appearances' to your sort_options array so the button renders automatically via your existing sort-button mapping. Then add a <th>Appearances</th> header and a corresponding <td>{row.appearances}</td> cell in the table body. A raw integer count needs no color formatting.
How do I test a new column end-to-end?
Drop the affected tables, restart the backend, and run the full pipeline script. After it completes, run SELECT appearances FROM leaders WHERE address = '[known_address]' to confirm the column populated. Then reload the frontend and confirm the column appears and the sort button works before committing.
// Troubleshooting
Why does my API keep failing on every request after adding a sort option?
This is almost always an aftershock error. The first request after adding the new sort field failed — often because the column didn't exist yet or ORDER BY hit the wrong table prefix — and was never rolled back, leaving the connection in an aborted state. Fix the root query, add rollback-on-exception, and restart the server.
My aggregate numbers keep growing wrong over time — what's happening?
You're likely doing a full-table recompute on every sync instead of an incremental update, or your incremental update isn't scoped to the latest run_id and is re-incrementing already-counted rows. Confirm you fetch MAX(run_id) and only increment rows where run_id matches the current run.
My database helper function name doesn't match what it does — is that a problem?
Yes. Naming a function save_user when it actually inserts into the leaders table creates confusion during refactoring and debugging. Name helpers after the abstraction they actually implement so future-you and any collaborators can reason about the code without tracing every line.
How do I recognize an aftershock error versus a real new bug?
Aftershock errors read like 'commands ignored until end of transaction block' and appear on every request, not just one. They're echoes of a single earlier failed query that was never rolled back. Before hunting anything else, find and fix that root failure and add rollback-on-exception — the aftershocks will disappear.
// Comparisons
How does the Deming Method compare to following a coding tutorial?
Tutorials walk you through building something once but often leave you unable to iterate independently — tutorial hell. The Deming Method is a repeatable process for shipping your own features incrementally across every layer, with AI as an advisor. It builds durable engineering ability rather than a single completed clone.
How does this compare to letting AI generate and paste entire features?
AI code-pasting is fast upfront but leaves you unable to debug or fine-tune later because you have no mental model of the code. The Deming Method uses AI to outline and explain while you hand-write everything, so you retain full control and can go back into specific processes without being completely lost.
How does precompute-then-serve compare to computing aggregates on demand?
On-demand computation runs expensive aggregation on every API call, which slows each request and scales poorly under load. Precompute-then-serve runs the heavy calculation once during the nightly batch and stores the result as a column, so the API just does a simple SELECT. It trades slightly stale data for dramatically faster, more scalable responses.
Why choose pure SQL over an ORM if ORMs are more popular?
ORMs abstract away the actual queries, creating a 'trust it works but have no idea what's happening' problem that compounds at every layer. Pure SQL means you always know exactly what's executing against the database, making performance tuning and debugging transparent. Popularity doesn't offset the loss of visibility for developers who want full control.
// Advanced
How do I handle schema changes in production versus development?
In development, drop and recreate affected tables to avoid migration headaches while iterating quickly. In production, use ALTER TABLE to add columns non-destructively — for example ALTER TABLE leaders ADD COLUMN appearances INTEGER DEFAULT 0 NOT NULL. Never drop production tables to add a column.
How do I keep the same pipeline function reusable across data paths?
Add a boolean flag like count_appearances to the build pipeline function. When True, it runs the incremental appearances update after saving the snapshot; when False, it skips it. This lets non-leaderboard paths reuse the same save logic without triggering the leaderboard-specific increment.
How do I avoid feature bloat while still capturing good ideas?
Identify the smallest shippable unit — one column, one sort option, one filter button — ship it, confirm it works end-to-end, then stop. Log every other idea (a secondary leaderboard page, view toggles, mobile layout tweaks) as future session starters. Note them, but do not build them until the current feature is confirmed working.
How do I design an allowlist and sort field map for safe dynamic sorting?
Maintain an allowed_sort_fields array of accepted string names and a sort_field_map dictionary mapping each name to its safe SQL fragment, like 'appearances' → 'leaders.appearances'. Validate user input against the allowlist first, then inject only the mapped fragment into ORDER BY. Reject anything not in the allowlist so raw user strings never touch your SQL.
How do I keep momentum across multiple build sessions?
Commit with descriptive messages that name both the feature and its mechanism, like 'add appearances to leaders table; auto-increment appearances on sync', push to GitHub, and check your contribution heatmap for visible progress. Ending each session with a logged list of follow-on ideas gives your next session a ready starting point.