# Microtables – TODO ## 1. Keyboard Navigation & Table Bounds - [ ] **1.1 Decouple table size from keyboard movement** Currently `press-enter-in-cell` uses `highest-row`/`highest-col` to decide where to move, coupling navigation to data extent. Intended design: - `highest-row`/`highest-col` should include the cursor's current position - `press-enter-in-cell` should only increment the position - Add a dedicated event handler for Tab (native Tab is no longer sufficient) - [ ] **1.2 Add arrow-key navigation** Implement `ArrowUp`, `ArrowDown`, `ArrowLeft`, `ArrowRight` handlers that move the cursor between cells, firing from `on-keyDown` (not `on-keyPress`). Navigation logic should live alongside Enter/Tab handling (1.1). - [ ] **1.3 Determine initial table size and scroll behaviour** `maxrow = 20` and `maxcol = "G"` are hardcoded. Decide whether the grid grows as the cursor reaches the edge or stays fixed with a scroll container. A fixed viewport with overflow scroll is likely right for the quick-calculation use case. - [ ] **1.4 Movement within a range selection** When a range is selected, Enter/Tab/arrow keys should cycle within the selection rather than the full table. Depends on the selection model (see section 5). --- ## 2. Formula Evaluation Engine - [ ] **2.1 Support lowercase cell references** `parse-variables` already uppercases symbol names, but confirm end-to-end that typing `=a1 + b2` in a cell evaluates correctly. - [ ] **2.2 Clarify dirty-forward-reference error semantics** `gather-variables-and-evaluate-cell` returns `:error` when forward references are dirty, but the intended semantics vs. `:insufficient-data` are not fully specified. Clarify and document the distinction. - [ ] **2.3 Verify `evaluate-all` handles cycles correctly** `evaluate-all` calls `evaluate-from-cell` which runs `find-cycle`, so cycles should be caught at initialization. Confirm this is sufficient and clean up the open question. - [ ] **2.4 Handle multiple cells modified simultaneously** The model processes one cell change at a time. A batch-update path will be needed for paste operations or future server sync. - [ ] **2.5 Consider incremental diff for back-reference updates** `denotify-references` + `notify-references` wipe and re-add all back-references on every edit. Diffing old vs. new `:refs` and updating only the delta may help at scale. - [ ] **2.6 Consider memoizing `find-cycle`** Evaluate whether a dynamic memoization strategy with cache invalidation on data change is worth the complexity. --- ## 3. Tests - [ ] **3.1 Unit tests for formula evaluation** Cover the core evaluation engine with unit tests: - Basic arithmetic formulas - Cell references and chained references - Range functions (`sum`, `average`, etc.) - Cycle detection - Error propagation (`:calc-error`, `:cycle-error`, `:insufficient-data`) - Lowercase cell references (see 2.1) - Edge cases: empty cells in ranges, self-references, deeply nested dependencies - [ ] **3.2 Integration tests for state transitions** Test re-frame event handlers end-to-end through the db: - Editing a cell and verifying downstream re-evaluation - Entering and leaving a cell - Reference graph updates when a formula changes - [ ] **3.3 End-to-end tests** Browser-level tests covering full user flows: - Data entry and formula results visible in cells - Keyboard navigation - Range selection operations (once implemented) - Touch interactions (once implemented) --- ## 4. Formula Autocomplete - [ ] **4.1 Show available math functions after `=` is typed** When a cell value begins with `=`, display a dropdown of available math.js functions. Filter the list as the user continues typing. Selecting an item inserts the function name with a placeholder for arguments. - [ ] **4.2 Cell and range reference autocomplete** As the user types a column letter that could start a cell reference (e.g. `A`), suggest cell addresses that contain data. For ranges, trigger suggestions when `:` follows a cell reference. - [ ] **4.3 Argument hint tooltip** Inside a function call, show a tooltip indicating expected arguments and their types (e.g. `sum(value1, value2, ...)`). --- ## 5. Selection & Highlight Mode - [ ] **5.1 Implement the selection data model** The `:selection` key in `:position` is always `nil`. Implement `{:start {:col "A" :row 1} :end {:col "C" :row 3}}` as the selection shape. Set on shift+arrow or click-drag; clear on unmodified navigation. - [ ] **5.2 Wire up highlight rendering for selected cells** `highlight-cells` in `subs.cljs` already sets `:view :highlighted` on cells within a selection range, but the `::table-data` subscription never calls it when `:selection` is non-nil. Connect the two so selected cells receive the `highlighted` CSS class. - [ ] **5.3 Aggregate display for a selection** When a numeric range is selected, display the sum and average in the UI (e.g. a status bar or floating chip). Offer to insert the result as a formula into a user-chosen cell. --- ## 6. Cell & Range Control Buttons When a cell or range is selected, show a compact set of floating action buttons positioned around the selection. Buttons must be large enough to tap comfortably on a phone screen. Implement in this order: - [ ] **6.1 Expand / contract selection** — drag handles or +/− buttons to resize the selection boundary - [ ] **6.2 Calculate** — show sum/average inline for numeric selections (ties into 5.3) - [ ] **6.3 Clear** — delete cell contents and re-evaluate dependents - [ ] **6.4 Copy** — copy selection to the clipboard - [ ] **6.5 Move** — designate a new top-left cell; handle destination conflicts - [ ] **6.6 Delete** — remove cells and displace neighbours inward - [ ] **6.7 Intelligent fill** — trigger pattern inference (see section 10) --- ## 7. Column & Row Aggregates *Under consideration.* - [ ] **7.1 Always-visible column totals row** A pinned row below the table showing the sum (and possibly average) of each column, updating reactively. Show only for columns that contain numbers. - [ ] **7.2 Always-visible row totals column** Equivalent pinned column to the right of the table for row aggregates. --- ## 8. UI & Control Panel - [ ] **8.1 Implement a compact, functional control panel** The current left and bottom panel slots exist but contain only placeholder content. Replace with a small, well-designed panel surfacing the most useful controls. The panel should stay visually aligned with the table as it scrolls (likely `position: sticky` or a scroll event listener). - [ ] **8.2 Add a "Help" modal** A small modal (triggered from the control panel) listing and explaining: - Supported formula syntax - Available math functions - Keyboard shortcuts - How cell references and ranges work - [ ] **8.3 Replace "About" with a popup modal** Remove the current inline disclaimer text. Add an About button that opens a conventional modal with app info, version, and credits. - [ ] **8.4 Replace the logo** The current `logo.svg` is a placeholder. Design and replace with a better mark. --- ## 9. Touch Support - [ ] **9.1 Make cells tap-friendly** Ensure cells are large enough to tap accurately on mobile. Tap to select, double-tap or long-press to enter edit mode. The current `on-focus`/`on-blur` model may need adjustment for touch events. - [ ] **9.2 Touch-friendly range selection** Implement tap-and-drag or tap-then-drag-handle selection. The floating control buttons (section 6) should be comfortably tappable. - [ ] **9.3 Responsive layout** Adapt the layout to small screens: control panel sidebars should collapse by default on mobile; consider a bottom sheet instead of a side panel. - [ ] **9.4 Software keyboard handling** On mobile, the software keyboard resizes the viewport and may conflict with cell navigation. Ensure formula entry and cell-to-cell movement work correctly with a software keyboard open. --- ## 10. Intelligent Fill *Longer-term.* - [ ] **10.1 Pattern inference from a selection** When a user selects a range and triggers fill (via the control button in 6.7), infer the pattern (arithmetic sequence, geometric, repeated value, date sequence, etc.) and propose a continuation into adjacent empty cells. - [ ] **10.2 Fill preview and confirmation** Show a preview of the proposed fill before committing, allowing the user to accept, adjust the inferred pattern, or cancel. --- ## 11. Data Persistence *Long-term consideration — not yet committed.* - [ ] **11.1 Local storage persistence** Save the table to the browser's `localStorage` on change and restore on page load. Provides durable-but-local storage with no backend required. - [ ] **11.2 Server-side persistence** Extend the Express server to store table data, with considerations for: - Authentication / account model - Paid vs. free tier - Expiry / TTL on stored tables - Conflict resolution for future collaboration - [ ] **11.3 Table sharing** Allow a user to share a table with others via a link. Requires server-side persistence (11.2). Considerations: - Read-only vs. editable share links - Expiry / revocation of share links - Real-time collaboration (multiple users editing simultaneously) --- ## 12. Code Health & Refactoring ### Bugs & Correctness - [ ] **12.1 Fix duplicate `:inbound` entries** `notify-references` uses `conj` on a list with no deduplication. A formula like `=A1 + A1` adds A1's back-reference twice. `denotify-references` removes only the first match via `filter`, leaving one stale copy. Use a set for `:inbound`, or deduplicate on insert. - [ ] **12.2 Replace lazy seq in `:inbound` with a concrete collection** `denotify-references` stores the result of `(partial filter ...)` — a lazy sequence — into the app db. Lazy seqs in re-frame state can cause subtle equality and inspection issues. Replace with `(into [] (filter ...))`. - [ ] **12.3 Add a termination guard to `evaluate-all`** The retry loop — `(recur data (concat (rest queue) (list cur)))` — re-queues cells with unsatisfied dependencies indefinitely. Cycle detection in `evaluate-from-cell` should prevent an infinite loop in practice, but there is no explicit guard. Add a visited set or iteration counter as a safety net. - [ ] **12.10 Fix string comparison in `highest-col` and `order-two-cols`** Both functions call `(apply max ...)` on sequences of column-letter strings. ClojureScript's `max` is defined for numbers only — it does not perform lexicographic comparison on strings. This likely produces incorrect results for any column comparison and will break entirely once multi-letter columns (AA, AB, …) are introduced. Replace with an explicit comparator (e.g. `(apply max-key #(vector (count %) %) ...)` or a sort-based approach). - [ ] **12.11 Make `parse-range` fail loudly on malformed input** `parse-range` runs four independent `re-find` calls on the same string. If any one fails to match, it silently passes `nil` into `range->list`, which cascades into bad data with no error surfaced to the user. Either validate the full string with a single comprehensive regex up front, or add an assertion/guard that all four captures succeeded. ### Deprecations - [ ] **12.4 Replace deprecated `on-keyPress` with `on-key-down`** The cell's enter-key handler in `views/sheet.cljs` uses `:on-keyPress`, which is deprecated in both React and the underlying browser API. Move to `:on-key-down`. - [ ] **12.12 Replace deprecated `reagent/render` with `reagent.dom/render`** `core.cljs` calls `reagent/render`, which was deprecated in Reagent 1.x in favour of `reagent.dom/render`. Verify the installed Reagent version and update accordingly. ### Performance - [ ] **12.5 Address unbounded `evaluate-expression` memoization cache** `evaluate-expression` is memoized on `(expression, variables-map)`. Every unique combination ever computed is cached for the lifetime of the page. In a long editing session this becomes a gradual memory leak. Consider a bounded LRU cache or explicit cache invalidation. ### Cleanup - [ ] **12.6 Remove `println` debug statements** Every event handler and subscription (`::table-data` in particular) fires `println` on every call. This floods the console and makes debugging new code harder. Remove or gate behind a dev-mode flag. - [ ] **12.7 Consolidate hardcoded table dimensions** `maxrow = 20` and `maxcol = "G"` are hardcoded in `views/sheet.cljs` while `min-max-row` and `min-max-col` defined in `db.cljs` go unused. The view should read from the db values, leaving a single source of truth. (Related: 1.3.) ### Refactoring - [ ] **12.8 Break up `gather-variables-and-evaluate-cell`** At ~35 lines with six boolean flags and a multi-branch `cond`, the function is near its readability limit. Adding another error type or evaluation mode will tip it over. Consider extracting the disqualification checks and the variable-collection step into named helpers. - [ ] **12.9 Rewrite `create-all-references` to use `walk-modify-data`** Both functions perform a `reduce-kv` double-walk over the data map. Eliminate the duplication by having `create-all-references` call `walk-modify-data`. - [ ] **12.13 Convert `main-panel` to a form-2 Reagent component** `main-panel` in `views.cljs` is a form-1 component that calls `re-frame/subscribe` at the top level. Re-frame's subscription cache makes this work in practice, but the canonical pattern for components holding subscriptions is form-2 — subscriptions are set up in the outer `defn`, and only the inner `fn` renders. Form-1 subscription calls can fail to clean up correctly across hot reloads. - [ ] **12.14 Call `dev-setup` before `mount-root` in `core.cljs`** `init` currently calls `mount-root` before `dev-setup`. If `dev-setup` ever does real setup work (beyond a `println`), it will run after the first render. Swap the order so the dev environment is fully configured before any rendering occurs.