# Microtables – TODO ## 1. Keyboard Navigation & Table Bounds - [ ] **1.1 Decouple table size from keyboard movement** Currently `press-enter-in-cell` uses `highest-row`/`highest-col` to decide where to move, coupling navigation to data extent. Intended design: - `highest-row`/`highest-col` should include the cursor's current position - `press-enter-in-cell` should only increment the position - Add a dedicated event handler for Tab (native Tab is no longer sufficient) - [ ] **1.2 Add arrow-key navigation** Implement `ArrowUp`, `ArrowDown`, `ArrowLeft`, `ArrowRight` handlers that move the cursor between cells, firing from `on-keyDown` (not `on-keyPress`). Navigation logic should live alongside Enter/Tab handling (1.1). - [ ] **1.3 Determine initial table size and scroll behaviour** `maxrow = 20` and `maxcol = "G"` are hardcoded. Decide whether the grid grows as the cursor reaches the edge or stays fixed with a scroll container. A fixed viewport with overflow scroll is likely right for the quick-calculation use case. - [ ] **1.4 Movement within a range selection** When a range is selected, Enter/Tab/arrow keys should cycle within the selection rather than the full table. Depends on the selection model (see section 5). --- ## 2. Formula Evaluation Engine - [ ] **2.1 Support lowercase cell references** `parse-variables` already uppercases symbol names, but confirm end-to-end that typing `=a1 + b2` in a cell evaluates correctly. - [ ] **2.2 Clarify dirty-forward-reference error semantics** `gather-variables-and-evaluate-cell` returns `:error` when forward references are dirty, but the intended semantics vs. `:insufficient-data` are not fully specified. Clarify and document the distinction. - [ ] **2.3 Verify `evaluate-all` handles cycles correctly** `evaluate-all` calls `evaluate-from-cell` which runs `find-cycle`, so cycles should be caught at initialization. Confirm this is sufficient and clean up the open question. - [ ] **2.4 Handle multiple cells modified simultaneously** The model processes one cell change at a time. A batch-update path will be needed for paste operations or future server sync. - [ ] **2.5 Consider incremental diff for back-reference updates** `denotify-references` + `notify-references` wipe and re-add all back-references on every edit. Diffing old vs. new `:refs` and updating only the delta may help at scale. - [ ] **2.6 Consider memoizing `find-cycle`** Evaluate whether a dynamic memoization strategy with cache invalidation on data change is worth the complexity. --- ## 3. Tests - [x] **3.1 Unit tests for formula evaluation** Implemented in `frontend/test/microtables_frontend/evaluation_test.cljs` using `cljs.test` + shadow-cljs `:node-test` target. Covers: - Basic arithmetic formulas - Cell references and chained references - Range functions (`sum`, `average`, etc.) - Cycle detection (mutual cycles, self-reference, three-cell cycles) - Error propagation (`:calc-error`, `:cycle-error`) - Edge cases: empty cells in ranges, non-formula cells - Run with `npm test` in `frontend/` - [x] **3.2 Integration tests for state transitions** Implemented in `frontend/test/microtables_frontend/events_test.cljs`. Tests the pure function pipelines that event handlers compose (not through re-frame dispatch). Covers: - Editing a cell and verifying downstream re-evaluation - `change-datum-value` marks dirty without evaluating - Reference graph updates when a formula changes (old refs removed, new refs added) - Deeply nested dependency chain propagation - [ ] **3.3 End-to-end tests** Browser-level tests covering full user flows: - Data entry and formula results visible in cells - Keyboard navigation - Range selection operations (once implemented) - Touch interactions (once implemented) --- ## 4. Formula Autocomplete - [ ] **4.1 Show available math functions after `=` is typed** When a cell value begins with `=`, display a dropdown of available math.js functions. Filter the list as the user continues typing. Selecting an item inserts the function name with a placeholder for arguments. - [ ] **4.2 Cell and range reference autocomplete** As the user types a column letter that could start a cell reference (e.g. `A`), suggest cell addresses that contain data. For ranges, trigger suggestions when `:` follows a cell reference. - [ ] **4.3 Argument hint tooltip** Inside a function call, show a tooltip indicating expected arguments and their types (e.g. `sum(value1, value2, ...)`). --- ## 5. Selection & Highlight Mode - [ ] **5.1 Implement the selection data model** The `:selection` key in `:position` is always `nil`. Implement `{:start {:col "A" :row 1} :end {:col "C" :row 3}}` as the selection shape. Set on shift+arrow or click-drag; clear on unmodified navigation. - [ ] **5.2 Wire up highlight rendering for selected cells** `highlight-cells` in `subs.cljs` already sets `:view :highlighted` on cells within a selection range, but the `::table-data` subscription never calls it when `:selection` is non-nil. Connect the two so selected cells receive the `highlighted` CSS class. - [ ] **5.3 Aggregate display for a selection** When a numeric range is selected, display the sum and average in the UI (e.g. a status bar or floating chip). Offer to insert the result as a formula into a user-chosen cell. --- ## 6. Cell & Range Control Buttons When a cell or range is selected, show a compact set of floating action buttons positioned around the selection. Buttons must be large enough to tap comfortably on a phone screen. Implement in this order: - [ ] **6.1 Expand / contract selection** — drag handles or +/− buttons to resize the selection boundary - [ ] **6.2 Calculate** — show sum/average inline for numeric selections (ties into 5.3) - [ ] **6.3 Clear** — delete cell contents and re-evaluate dependents - [ ] **6.4 Copy** — copy selection to the clipboard - [ ] **6.5 Move** — designate a new top-left cell; handle destination conflicts - [ ] **6.6 Delete** — remove cells and displace neighbours inward - [ ] **6.7 Intelligent fill** — trigger pattern inference (see section 10) --- ## 7. Column & Row Aggregates *Under consideration.* - [ ] **7.1 Always-visible column totals row** A pinned row below the table showing the sum (and possibly average) of each column, updating reactively. Show only for columns that contain numbers. - [ ] **7.2 Always-visible row totals column** Equivalent pinned column to the right of the table for row aggregates. --- ## 8. UI & Control Panel - [ ] **8.2 Add a "Help" modal** A small modal (triggered from the control panel) listing and explaining: - Supported formula syntax - Available math functions - Keyboard shortcuts - How cell references and ranges work - [ ] **8.4 Replace the logo** The current `logo.svg` is a placeholder. Design and replace with a better mark. --- ## 13. Visual Design & Styling The goal is intentional minimalism — not bare HTML. The current stylesheet has good structural foundations but is visibly unfinished: placeholder colours throughout, a content-template base, and unstyled interactive elements. Items are ordered so that each one builds on the previous. ### Foundational - [ ] **13.1 Establish a CSS design token set** Define all colours, spacing, and radii as CSS custom properties on `:root` before touching anything else. Suggested palette: - `--color-bg: #f9fafb` (page background) - `--color-surface: #ffffff` (cell / panel background) - `--color-border: #e5e7eb` (grid lines, panel edges) - `--color-header-bg: #f3f4f6` (row/column headers) - `--color-text: #111827` (primary text) - `--color-text-muted: #6b7280` (header labels, hints) - `--color-accent: #3b82f6` (focus ring, selection border) - `--color-accent-subtle: #eff6ff` (selection fill, hover tint) - [ ] **13.2 Strip out content-template CSS** The stylesheet was adapted from a blog template. Remove or replace: - `max-width: 600px` on `body` — clips the table on most screens; a spreadsheet should use available width - `h1`, `h2`, `h3` type scale — not used in the app - `line-height: 1.5em` on `body` — intended for prose, not a grid - `a` tag styles (`text-decoration`, `color: #09f`) — not relevant to the app chrome - [ ] **13.3 Replace all placeholder colours** Several elements have placeholder colours that were clearly never updated: - `#controls-left`: `background-color: pink` → `var(--color-surface)` - `#controls-bottom`: `background-color: lightblue` → `var(--color-surface)` - `#main-logo`: `background-color: lightblue` → transparent (let the SVG stand alone) - `.control-group`: `background-color: lightgreen` → remove; use spacing to separate groups - [ ] **13.4 Change grid lines from black to the border token** Replace `border: 1px solid black` on `td, th` with `border: 1px solid var(--color-border)`. This is the single highest-impact change to the table's appearance. ### Typography - [ ] **13.5 Switch to a system font stack** Replace the current `'Helvetica Neue', Verdana, Helvetica, Arial, sans-serif` with `system-ui, -apple-system, BlinkMacSystemFont, 'Segoe UI', sans-serif`. This makes the app feel native on each platform rather than web-circa-2010. - [ ] **13.6 Use monospace for cell values** Numbers in a proportional font look accidental; numbers in a monospace font look deliberate. Apply `font-family: 'SF Mono', ui-monospace, 'Fira Code', monospace` to `td input` and use a slightly smaller size (around `0.875rem`) relative to the header labels. ### Grid & Cells - [ ] **13.7 Make inputs fill their cells** `td input` currently has a fixed `width: 80px`, leaving dead space in the `` that does not respond to clicks. Set `width: 100%` and ensure `box-sizing: border-box` so the input expands to fill the full cell width. - [ ] **13.8 Refine hover and focus states** Current states (`#ccc` hover, `#ccf` focus background) are heavy and generic. Suggested approach: - Hover: `var(--color-accent-subtle)` — a very light tint - Focus: remove the background change entirely; rely on the `smartborder` ring (13.9) as the focus indicator, which avoids the "form field in a table" feeling - [ ] **13.9 Style the `smartborder` selection ring** The `smartborder` concept (an absolutely-positioned overlay extending ~2px beyond the cell boundary) is the right approach — it mirrors how professional spreadsheets handle selection. Replace the current `3px solid blue` with `2px solid var(--color-accent)` and ensure `pointer-events: none` so it does not interfere with clicking adjacent cells. - [ ] **13.10 Style the `smartborder` action buttons** The range-action buttons (extend, fill, move, empty, delete, copy) are currently unstyled blue circles. Redesign as small, clean pill buttons: white background, `var(--color-border)` border, `var(--color-text)` icon or label, with a shadow on hover. Size them to be comfortably tappable (minimum 32×32px touch target). - [ ] **13.11 Differentiate row and column headers** Apply `background: var(--color-header-bg)` and `color: var(--color-text-muted)` to `th`. Remove bold weight — a medium-weight muted label reads as a header without competing with cell content. ### Control Panel & Chrome - [ ] **13.12 Redesign the panel toggle buttons** `#left-controls-button` and `#bottom-controls-button` currently render as `xx-large` bold unicode arrows with a 2px border radius — too raw. Replace with small, refined pill or tab buttons: consistent padding, `var(--color-border)` border, `border-radius: 6px`, normal font weight, subtle hover state. --- ## 9. Touch Support - [ ] **9.1 Make cells tap-friendly** Ensure cells are large enough to tap accurately on mobile. Tap to select, double-tap or long-press to enter edit mode. The current `on-focus`/`on-blur` model may need adjustment for touch events. - [ ] **9.2 Touch-friendly range selection** Implement tap-and-drag or tap-then-drag-handle selection. The floating control buttons (section 6) should be comfortably tappable. - [ ] **9.3 Responsive layout** Adapt the layout to small screens: control panel sidebars should collapse by default on mobile; consider a bottom sheet instead of a side panel. - [ ] **9.4 Software keyboard handling** On mobile, the software keyboard resizes the viewport and may conflict with cell navigation. Ensure formula entry and cell-to-cell movement work correctly with a software keyboard open. --- ## 10. Intelligent Fill *Longer-term.* - [ ] **10.1 Pattern inference from a selection** When a user selects a range and triggers fill (via the control button in 6.7), infer the pattern (arithmetic sequence, geometric, repeated value, date sequence, etc.) and propose a continuation into adjacent empty cells. - [ ] **10.2 Fill preview and confirmation** Show a preview of the proposed fill before committing, allowing the user to accept, adjust the inferred pattern, or cancel. --- ## 11. Data Persistence *Long-term consideration — not yet committed.* - [ ] **11.1 Local storage persistence** Save the table to the browser's `localStorage` on change and restore on page load. Provides durable-but-local storage with no backend required. - [ ] **11.2 Server-side persistence** Extend the Express server to store table data, with considerations for: - Authentication / account model - Paid vs. free tier - Expiry / TTL on stored tables - Conflict resolution for future collaboration - [ ] **11.3 Table sharing** Allow a user to share a table with others via a link. Requires server-side persistence (11.2). Considerations: - Read-only vs. editable share links - Expiry / revocation of share links - Real-time collaboration (multiple users editing simultaneously) --- ## 12. Code Health & Refactoring ### Bugs & Correctness - [ ] **12.1 Fix duplicate `:inbound` entries** `notify-references` uses `conj` on a list with no deduplication. A formula like `=A1 + A1` adds A1's back-reference twice. `denotify-references` removes only the first match via `filter`, leaving one stale copy. Use a set for `:inbound`, or deduplicate on insert. - [ ] **12.2 Replace lazy seq in `:inbound` with a concrete collection** `denotify-references` stores the result of `(partial filter ...)` — a lazy sequence — into the app db. Lazy seqs in re-frame state can cause subtle equality and inspection issues. Replace with `(into [] (filter ...))`. - [ ] **12.3 Add a termination guard to `evaluate-all`** The retry loop — `(recur data (concat (rest queue) (list cur)))` — re-queues cells with unsatisfied dependencies indefinitely. Cycle detection in `evaluate-from-cell` should prevent an infinite loop in practice, but there is no explicit guard. Add a visited set or iteration counter as a safety net. - [ ] **12.10 Fix string comparison in `highest-col` and `order-two-cols`** Both functions call `(apply max ...)` on sequences of column-letter strings. ClojureScript's `max` is defined for numbers only — it does not perform lexicographic comparison on strings. This likely produces incorrect results for any column comparison and will break entirely once multi-letter columns (AA, AB, …) are introduced. Replace with an explicit comparator (e.g. `(apply max-key #(vector (count %) %) ...)` or a sort-based approach). - [ ] **12.11 Make `parse-range` fail loudly on malformed input** `parse-range` runs four independent `re-find` calls on the same string. If any one fails to match, it silently passes `nil` into `range->list`, which cascades into bad data with no error surfaced to the user. Either validate the full string with a single comprehensive regex up front, or add an assertion/guard that all four captures succeeded. ### Deprecations - [ ] **12.4 Replace deprecated `on-keyPress` with `on-key-down`** The cell's enter-key handler in `views/sheet.cljs` uses `:on-keyPress`, which is deprecated in both React and the underlying browser API. Move to `:on-key-down`. - [ ] **12.12 Replace deprecated `reagent/render` with `reagent.dom/render`** `core.cljs` calls `reagent/render`, which was deprecated in Reagent 1.x in favour of `reagent.dom/render`. Verify the installed Reagent version and update accordingly. ### Performance - [ ] **12.5 Address unbounded `evaluate-expression` memoization cache** `evaluate-expression` is memoized on `(expression, variables-map)`. Every unique combination ever computed is cached for the lifetime of the page. In a long editing session this becomes a gradual memory leak. Consider a bounded LRU cache or explicit cache invalidation. ### Cleanup - [ ] **12.6 Remove `println` debug statements** Every event handler and subscription (`::table-data` in particular) fires `println` on every call. This floods the console and makes debugging new code harder. Remove or gate behind a dev-mode flag. - [ ] **12.7 Consolidate hardcoded table dimensions** `maxrow = 20` and `maxcol = "G"` are hardcoded in `views/sheet.cljs` while `min-max-row` and `min-max-col` defined in `db.cljs` go unused. The view should read from the db values, leaving a single source of truth. (Related: 1.3.) ### Refactoring - [ ] **12.8 Break up `gather-variables-and-evaluate-cell`** At ~35 lines with six boolean flags and a multi-branch `cond`, the function is near its readability limit. Adding another error type or evaluation mode will tip it over. Consider extracting the disqualification checks and the variable-collection step into named helpers. - [ ] **12.9 Rewrite `create-all-references` to use `walk-modify-data`** Both functions perform a `reduce-kv` double-walk over the data map. Eliminate the duplication by having `create-all-references` call `walk-modify-data`. - [ ] **12.13 Convert `main-panel` to a form-2 Reagent component** `main-panel` in `views.cljs` is a form-1 component that calls `re-frame/subscribe` at the top level. Re-frame's subscription cache makes this work in practice, but the canonical pattern for components holding subscriptions is form-2 — subscriptions are set up in the outer `defn`, and only the inner `fn` renders. Form-1 subscription calls can fail to clean up correctly across hot reloads. - [ ] **12.14 Call `dev-setup` before `mount-root` in `core.cljs`** `init` currently calls `mount-root` before `dev-setup`. If `dev-setup` ever does real setup work (beyond a `println`), it will run after the first render. Swap the order so the dev environment is fully configured before any rendering occurs.