# Microtables – TODO ## 1. Keyboard Navigation & Table Bounds - [ ] **1.1 Decouple table size from keyboard movement** Currently `press-enter-in-cell` uses `highest-row`/`highest-col` to decide where to move, coupling navigation to data extent. Intended design: - `highest-row`/`highest-col` should include the cursor's current position - `press-enter-in-cell` should only increment the position - Add a dedicated event handler for Tab (native Tab is no longer sufficient) - [ ] **1.2 Add arrow-key navigation** Implement `ArrowUp`, `ArrowDown`, `ArrowLeft`, `ArrowRight` handlers that move the cursor between cells, firing from `on-keyDown` (not `on-keyPress`). Navigation logic should live alongside Enter/Tab handling (1.1). - [ ] **1.3 Determine initial table size and scroll behaviour** `maxrow = 20` and `maxcol = "G"` are hardcoded. Decide whether the grid grows as the cursor reaches the edge or stays fixed with a scroll container. A fixed viewport with overflow scroll is likely right for the quick-calculation use case. - [ ] **1.4 Movement within a range selection** When a range is selected, Enter/Tab/arrow keys should cycle within the selection rather than the full table. Depends on the selection model (see section 5). --- ## 2. Formula Evaluation Engine - [ ] **2.1 Support lowercase cell references** `parse-variables` already uppercases symbol names, but confirm end-to-end that typing `=a1 + b2` in a cell evaluates correctly. - [ ] **2.2 Clarify dirty-forward-reference error semantics** `gather-variables-and-evaluate-cell` returns `:error` when forward references are dirty, but the intended semantics vs. `:insufficient-data` are not fully specified. Clarify and document the distinction. - [ ] **2.3 Verify `evaluate-all` handles cycles correctly** `evaluate-all` calls `evaluate-from-cell` which runs `find-cycle`, so cycles should be caught at initialization. Confirm this is sufficient and clean up the open question. - [ ] **2.4 Handle multiple cells modified simultaneously** The model processes one cell change at a time. A batch-update path will be needed for paste operations or future server sync. - [ ] **2.5 Consider incremental diff for back-reference updates** `denotify-references` + `notify-references` wipe and re-add all back-references on every edit. Diffing old vs. new `:refs` and updating only the delta may help at scale. - [ ] **2.6 Consider memoizing `find-cycle`** Evaluate whether a dynamic memoization strategy with cache invalidation on data change is worth the complexity. --- ## 3. Tests - [x] **3.1 Unit tests for formula evaluation** Implemented in `frontend/test/microtables_frontend/evaluation_test.cljs` using `cljs.test` + shadow-cljs `:node-test` target. Covers: - Basic arithmetic formulas - Cell references and chained references - Range functions (`sum`, `average`, etc.) - Cycle detection (mutual cycles, self-reference, three-cell cycles) - Error propagation (`:calc-error`, `:cycle-error`) - Edge cases: empty cells in ranges, non-formula cells - Run with `npm test` in `frontend/` - [x] **3.2 Integration tests for state transitions** Implemented in `frontend/test/microtables_frontend/events_test.cljs`. Tests the pure function pipelines that event handlers compose (not through re-frame dispatch). Covers: - Editing a cell and verifying downstream re-evaluation - `change-datum-value` marks dirty without evaluating - Reference graph updates when a formula changes (old refs removed, new refs added) - Deeply nested dependency chain propagation - [ ] **3.3 End-to-end tests** Browser-level tests covering full user flows: - Data entry and formula results visible in cells - Keyboard navigation - Range selection operations (once implemented) - Touch interactions (once implemented) --- ## 4. Formula Autocomplete - [ ] **4.1 Show available math functions after `=` is typed** When a cell value begins with `=`, display a dropdown of available math.js functions. Filter the list as the user continues typing. Selecting an item inserts the function name with a placeholder for arguments. - [ ] **4.2 Cell and range reference autocomplete** As the user types a column letter that could start a cell reference (e.g. `A`), suggest cell addresses that contain data. For ranges, trigger suggestions when `:` follows a cell reference. - [ ] **4.3 Argument hint tooltip** Inside a function call, show a tooltip indicating expected arguments and their types (e.g. `sum(value1, value2, ...)`). --- ## 5. Selection & Highlight Mode - [ ] **5.1 Implement the selection data model** The `:selection` key in `:position` is always `nil`. Implement `{:start {:col "A" :row 1} :end {:col "C" :row 3}}` as the selection shape. Set on shift+arrow or click-drag; clear on unmodified navigation. - [ ] **5.2 Wire up highlight rendering for selected cells** `highlight-cells` in `subs.cljs` already sets `:view :highlighted` on cells within a selection range, but the `::table-data` subscription never calls it when `:selection` is non-nil. Connect the two so selected cells receive the `highlighted` CSS class. - [ ] **5.3 Aggregate display for a selection** When a numeric range is selected, display the sum and average in the UI (e.g. a status bar or floating chip). Offer to insert the result as a formula into a user-chosen cell. --- ## 6. Cell & Range Control Buttons When a cell or range is selected, show a compact set of floating action buttons positioned around the selection. Buttons must be large enough to tap comfortably on a phone screen. Implement in this order: - [ ] **6.1 Expand / contract selection** — drag handles or +/− buttons to resize the selection boundary - [ ] **6.2 Calculate** — show sum/average inline for numeric selections (ties into 5.3) - [ ] **6.3 Clear** — delete cell contents and re-evaluate dependents - [ ] **6.4 Copy** — copy selection to the clipboard - [ ] **6.5 Move** — designate a new top-left cell; handle destination conflicts - [ ] **6.6 Delete** — remove cells and displace neighbours inward - [ ] **6.7 Intelligent fill** — trigger pattern inference (see section 10) --- ## 7. Column & Row Aggregates - [x] **7.1 Always-visible column totals row** A pinned row below the table showing the sum (and possibly average) of each column, updating reactively. Show only for columns that contain numbers. Make sure that it is not visually confused for another cell/row. - [ ] **7.2 Always-visible row totals column** Equivalent pinned column to the right of the table for row aggregates. Perhaps, due to concerns about mobile screen real estate, the row totals could be toggleable (not "always-visible" after all) - maybe behind the row header. --- ## 8. UI & Control Panel - [ ] **8.2 Add a "Help" modal** A small modal (triggered from the control panel) listing and explaining: - Supported formula syntax - Available math functions - Keyboard shortcuts - How cell references and ranges work - [ ] **8.4 Replace the logo** The current `logo.svg` is a placeholder. Design and replace with a better mark. --- ## 13. Visual Design & Styling The goal is intentional minimalism — not bare HTML. The current stylesheet has good structural foundations but is visibly unfinished: placeholder colours throughout, a content-template base, and unstyled interactive elements. Items are ordered so that each one builds on the previous. ### Foundational - [ ] **13.1 Establish a CSS design token set** Define all colours, spacing, and radii as CSS custom properties on `:root` before touching anything else. Suggested palette: - `--color-bg: #f9fafb` (page background) - `--color-surface: #ffffff` (cell / panel background) - `--color-border: #e5e7eb` (grid lines, panel edges) - `--color-header-bg: #f3f4f6` (row/column headers) - `--color-text: #111827` (primary text) - `--color-text-muted: #6b7280` (header labels, hints) - `--color-accent: #3b82f6` (focus ring, selection border) - `--color-accent-subtle: #eff6ff` (selection fill, hover tint) - [ ] **13.2 Strip out content-template CSS** The stylesheet was adapted from a blog template. Remove or replace: - `max-width: 600px` on `body` — clips the table on most screens; a spreadsheet should use available width - `h1`, `h2`, `h3` type scale — not used in the app - `line-height: 1.5em` on `body` — intended for prose, not a grid - `a` tag styles (`text-decoration`, `color: #09f`) — not relevant to the app chrome - [ ] **13.3 Replace all placeholder colours** Several elements have placeholder colours that were clearly never updated: - `#controls-left`: `background-color: pink` → `var(--color-surface)` - `#controls-bottom`: `background-color: lightblue` → `var(--color-surface)` - `#main-logo`: `background-color: lightblue` → transparent (let the SVG stand alone) - `.control-group`: `background-color: lightgreen` → remove; use spacing to separate groups - [ ] **13.4 Change grid lines from black to the border token** Replace `border: 1px solid black` on `td, th` with `border: 1px solid var(--color-border)`. This is the single highest-impact change to the table's appearance. ### Typography - [ ] **13.5 Switch to a system font stack** Replace the current `'Helvetica Neue', Verdana, Helvetica, Arial, sans-serif` with `system-ui, -apple-system, BlinkMacSystemFont, 'Segoe UI', sans-serif`. This makes the app feel native on each platform rather than web-circa-2010. - [ ] **13.6 Use monospace for cell values** Numbers in a proportional font look accidental; numbers in a monospace font look deliberate. Apply `font-family: 'SF Mono', ui-monospace, 'Fira Code', monospace` to `td input` and use a slightly smaller size (around `0.875rem`) relative to the header labels. ### Grid & Cells - [ ] **13.7 Make inputs fill their cells** `td input` currently has a fixed `width: 80px`, leaving dead space in the `` that does not respond to clicks. Set `width: 100%` and ensure `box-sizing: border-box` so the input expands to fill the full cell width. - [ ] **13.8 Refine hover and focus states** Current states (`#ccc` hover, `#ccf` focus background) are heavy and generic. Suggested approach: - Hover: `var(--color-accent-subtle)` — a very light tint - Focus: remove the background change entirely; rely on the `smartborder` ring (13.9) as the focus indicator, which avoids the "form field in a table" feeling - [ ] **13.9 Style the `smartborder` selection ring** The `smartborder` concept (an absolutely-positioned overlay extending ~2px beyond the cell boundary) is the right approach — it mirrors how professional spreadsheets handle selection. Replace the current `3px solid blue` with `2px solid var(--color-accent)` and ensure `pointer-events: none` so it does not interfere with clicking adjacent cells. - [ ] **13.10 Style the `smartborder` action buttons** The range-action buttons (extend, fill, move, empty, delete, copy) are currently unstyled blue circles. Redesign as small, clean pill buttons: white background, `var(--color-border)` border, `var(--color-text)` icon or label, with a shadow on hover. Size them to be comfortably tappable (minimum 32×32px touch target). - [ ] **13.11 Differentiate row and column headers** Apply `background: var(--color-header-bg)` and `color: var(--color-text-muted)` to `th`. Remove bold weight — a medium-weight muted label reads as a header without competing with cell content. ### Control Panel & Chrome - [ ] **13.12 Redesign the panel toggle buttons** `#left-controls-button` and `#bottom-controls-button` currently render as `xx-large` bold unicode arrows with a 2px border radius — too raw. Replace with small, refined pill or tab buttons: consistent padding, `var(--color-border)` border, `border-radius: 6px`, normal font weight, subtle hover state. --- ## 9. Touch Support - [ ] **9.1 Make cells tap-friendly** Ensure cells are large enough to tap accurately on mobile. Tap to select, double-tap or long-press to enter edit mode. The current `on-focus`/`on-blur` model may need adjustment for touch events. - [ ] **9.2 Touch-friendly range selection** Implement tap-and-drag or tap-then-drag-handle selection. The floating control buttons (section 6) should be comfortably tappable. - [ ] **9.3 Responsive layout** Adapt the layout to small screens: control panel sidebars should collapse by default on mobile; consider a bottom sheet instead of a side panel. - [ ] **9.4 Software keyboard handling** On mobile, the software keyboard resizes the viewport and may conflict with cell navigation. Ensure formula entry and cell-to-cell movement work correctly with a software keyboard open. --- ## 10. Intelligent Fill *Longer-term.* - [ ] **10.1 Pattern inference from a selection** When a user selects a range and triggers fill (via the control button in 6.7), infer the pattern (arithmetic sequence, geometric, repeated value, date sequence, etc.) and propose a continuation into adjacent empty cells. - [ ] **10.2 Fill preview and confirmation** Show a preview of the proposed fill before committing, allowing the user to accept, adjust the inferred pattern, or cancel. --- ## 11. Data Persistence *Long-term consideration — not yet committed.* - [ ] **11.1 Local storage persistence** Save the table to the browser's `localStorage` on change and restore on page load. Provides durable-but-local storage with no backend required. - [ ] **11.2 Server-side persistence** Extend the Express server to store table data, with considerations for: - Authentication / account model - Paid vs. free tier - Expiry / TTL on stored tables - Conflict resolution for future collaboration - [ ] **11.3 Table sharing** Allow a user to share a table with others via a link. Requires server-side persistence (11.2). Considerations: - Read-only vs. editable share links - Expiry / revocation of share links - Real-time collaboration (multiple users editing simultaneously) --- ## 12. Code Health & Refactoring ### Bugs & Correctness - [ ] **12.1 Fix duplicate `:inbound` entries** `notify-references` uses `conj` on a list with no deduplication. A formula like `=A1 + A1` adds A1's back-reference twice. `denotify-references` removes only the first match via `filter`, leaving one stale copy. Use a set for `:inbound`, or deduplicate on insert. - [ ] **12.2 Replace lazy seq in `:inbound` with a concrete collection** `denotify-references` stores the result of `(partial filter ...)` — a lazy sequence — into the app db. Lazy seqs in re-frame state can cause subtle equality and inspection issues. Replace with `(into [] (filter ...))`. - [ ] **12.3 Add a termination guard to `evaluate-all`** The retry loop — `(recur data (concat (rest queue) (list cur)))` — re-queues cells with unsatisfied dependencies indefinitely. Cycle detection in `evaluate-from-cell` should prevent an infinite loop in practice, but there is no explicit guard. Add a visited set or iteration counter as a safety net. - [ ] **12.10 Fix string comparison in `highest-col` and `order-two-cols`** Both functions call `(apply max ...)` on sequences of column-letter strings. ClojureScript's `max` is defined for numbers only — it does not perform lexicographic comparison on strings. This likely produces incorrect results for any column comparison and will break entirely once multi-letter columns (AA, AB, …) are introduced. Replace with an explicit comparator (e.g. `(apply max-key #(vector (count %) %) ...)` or a sort-based approach). - [ ] **12.11 Make `parse-range` fail loudly on malformed input** `parse-range` runs four independent `re-find` calls on the same string. If any one fails to match, it silently passes `nil` into `range->list`, which cascades into bad data with no error surfaced to the user. Either validate the full string with a single comprehensive regex up front, or add an assertion/guard that all four captures succeeded. ### Deprecations - [ ] **12.4 Replace deprecated `on-keyPress` with `on-key-down`** The cell's enter-key handler in `views/sheet.cljs` uses `:on-keyPress`, which is deprecated in both React and the underlying browser API. Move to `:on-key-down`. - [ ] **12.12 Replace deprecated `reagent/render` with `reagent.dom/render`** `core.cljs` calls `reagent/render`, which was deprecated in Reagent 1.x in favour of `reagent.dom/render`. Verify the installed Reagent version and update accordingly. ### Performance - [ ] **12.5 Address unbounded `evaluate-expression` memoization cache** `evaluate-expression` is memoized on `(expression, variables-map)`. Every unique combination ever computed is cached for the lifetime of the page. In a long editing session this becomes a gradual memory leak. Consider a bounded LRU cache or explicit cache invalidation. ### Cleanup - [ ] **12.6 Remove `println` debug statements** Every event handler and subscription (`::table-data` in particular) fires `println` on every call. This floods the console and makes debugging new code harder. Remove or gate behind a dev-mode flag. - [ ] **12.7 Consolidate hardcoded table dimensions** `maxrow = 20` and `maxcol = "G"` are hardcoded in `views/sheet.cljs` while `min-max-row` and `min-max-col` defined in `db.cljs` go unused. The view should read from the db values, leaving a single source of truth. (Related: 1.3.) ### Refactoring - [ ] **12.8 Break up `gather-variables-and-evaluate-cell`** At ~35 lines with six boolean flags and a multi-branch `cond`, the function is near its readability limit. Adding another error type or evaluation mode will tip it over. Consider extracting the disqualification checks and the variable-collection step into named helpers. - [ ] **12.9 Rewrite `create-all-references` to use `walk-modify-data`** Both functions perform a `reduce-kv` double-walk over the data map. Eliminate the duplication by having `create-all-references` call `walk-modify-data`. - [ ] **12.13 Convert `main-panel` to a form-2 Reagent component** `main-panel` in `views.cljs` is a form-1 component that calls `re-frame/subscribe` at the top level. Re-frame's subscription cache makes this work in practice, but the canonical pattern for components holding subscriptions is form-2 — subscriptions are set up in the outer `defn`, and only the inner `fn` renders. Form-1 subscription calls can fail to clean up correctly across hot reloads. - [ ] **12.14 Call `dev-setup` before `mount-root` in `core.cljs`** `init` currently calls `mount-root` before `dev-setup`. If `dev-setup` ever does real setup work (beyond a `println`), it will run after the first render. Swap the order so the dev environment is fully configured before any rendering occurs.