diff --git a/PLAN.md b/PLAN.md index 3ad9776..4d4fa28 100644 --- a/PLAN.md +++ b/PLAN.md @@ -660,6 +660,12 @@ Before public launch: - Stats page — render pipeline stats (host count, icon coverage, crawl date) - Test across browsers and devices +### Lessons learned +- **Firefox CSS transform animation jitter (Bug 739176).** Firefox deliberately snaps transformed elements to device pixels as a performance optimization. At slow marquee speeds (~21 px/sec, <1px/frame at 60fps), this creates visible icon flickering as pixels snap between integer boundaries each frame. Chrome and Safari do smooth sub-pixel GPU interpolation, so they're unaffected. +- **Fix: `filter: blur(0px)` on Firefox.** A no-op CSS filter forces Firefox through a different compositing path that enables sub-pixel rendering during transform animations. Applied via `.browser-firefox .tab-row { filter: blur(0px); }` using the existing browser detection class on ``. +- **`image-rendering: pixelated` conflicts with sub-pixel transforms.** Once the filter hack enables sub-pixel rendering, `pixelated` (nearest-neighbor sampling) causes icons to flicker at sub-pixel positions. Fix: `.browser-firefox .tab-icon { image-rendering: auto; }` — bilinear filtering interpolates smoothly. Icons are very slightly softer on Firefox but don't flicker. Chrome/Safari keep `pixelated`. +- **Integer-pixel rAF animation is not a good alternative.** Attempted `requestAnimationFrame` with `Math.round()`/`Math.floor()` to control pixel snapping — eliminates flicker but introduces visible stepping at slow speeds. Text jumps are more distracting than the original jitter. The `filter` hack is strictly better. + ## Phase 10: Parallelization (if needed) Only pursue if single-machine time (~3-4 days) is unacceptable. @@ -690,6 +696,29 @@ Monthly pipeline triggered by new Common Crawl release. --- +## Before Next Run + +### Infrastructure +- **Run Postgres locally on EC2** instead of RDS. Eliminates the IOPS bottleneck that dominated this run + saves $12-15/run. Add Postgres install to `ec2-userdata.sh`. +- **Evaluate c5.xlarge vs c5.2xlarge** based on CPU utilization data from this run. May not need the extra cores. +- **Pre-size EBS/storage correctly** from the start — no mid-run resizes (RDS storage optimization causes I/O degradation). + +### Pipeline code +- **CC-Index: streaming dedup** (INSERT ON CONFLICT) to eliminate DuckDB memory pressure and swap thrashing. +- **WARC parser: tune writer count + batch size** based on observed IOPS from this run. With local Postgres, may be able to go back to 8 writers. +- **WARC parser: remove or gate debug log lines** (`[fetcher]`, `[writer]`) behind a `--verbose` flag. +- **Icon download: measure throughput** at 5000 concurrency, decide if worth going higher. +- **Bundle gen: bilinear downscaling** instead of nearest-neighbor for >128px icons. +- **SVG rasterization** via `rsvg-convert` (recovers ~3.5% of icons). + +### Measurement / validation +- **Measure final DB size** (tables + indexes) with `pg_total_relation_size` to right-size storage. +- **Check CPU utilization** across all stages to confirm actual bottlenecks. +- **Compare icon download rate** at 5000 vs 1000 concurrency. +- **Verify loss funnel numbers** add up end-to-end at 30M scale. + +--- + ## Future Improvements (Non-Blocking) ### Pipeline diff --git a/frontend/index.html b/frontend/index.html index dc0e0de..c4cda1c 100644 --- a/frontend/index.html +++ b/frontend/index.html @@ -46,6 +46,17 @@ will-change: transform; } + /* Firefox snaps transforms to device pixels (Bug 739176), causing + visible jitter. A no-op filter forces a different compositing path + that enables sub-pixel rendering during transform animations. */ + .browser-firefox .tab-row { + filter: blur(0px); + } + + .browser-firefox .tab-icon { + image-rendering: auto; + } + .tab { display: inline-flex; align-items: center; diff --git a/frontend/site.js b/frontend/site.js index c27a8db..3dae327 100644 --- a/frontend/site.js +++ b/frontend/site.js @@ -21,7 +21,8 @@ function detectBrowser() { if (ua.includes("Safari") && !ua.includes("Chrome")) return "safari"; return "chrome"; // Chrome, Edge, Opera, Brave, etc. } -document.body.classList.add(`browser-${detectBrowser()}`); +const browserName = detectBrowser(); +document.body.classList.add(`browser-${browserName}`); // How many tabs fit in one row? function tabsPerRow() { @@ -109,14 +110,12 @@ function createRow(entries, rowIndex) { row.className = "tab-row"; const speed = 90 + (rng() * 60); // 90-150s per cycle - row.style.setProperty("--speed", `${speed}s`); - - // Random direction const goLeft = rng() > 0.5; - row.classList.add(goLeft ? "scroll-left" : "scroll-right"); + const stagger = rng(); - // Stagger start so rows aren't synchronized - row.style.animationDelay = `${-rng() * speed}s`; + row.style.setProperty("--speed", `${speed}s`); + row.classList.add(goLeft ? "scroll-left" : "scroll-right"); + row.style.animationDelay = `${-stagger * speed}s`; // Add tabs twice so the marquee loops seamlessly (translate -50% = one full set) for (let copy = 0; copy < 2; copy++) { @@ -128,6 +127,7 @@ function createRow(entries, rowIndex) { return row; } + // Render entries into rows function renderEntries(entries) { const perRow = tabsPerRow();