|
|
1b665d1065
|
it's web, not Web according to APA style guides
|
2026-05-19 12:25:45 -04:00 |
|
|
|
4ceefeec3d
|
fixed typo
|
2026-05-19 11:50:49 -04:00 |
|
|
|
f7f564289c
|
drop the space, it's cleaner
|
2026-05-19 11:49:13 -04:00 |
|
|
|
3534f84b27
|
added about.html
|
2026-05-19 11:42:09 -04:00 |
|
|
|
1d5b7bd374
|
added random_order to host table schema
|
2026-05-19 10:47:05 -04:00 |
|
|
|
e6d5d5175c
|
fixed oom in bundle_gen and added randomOrder, still need a full redesign
|
2026-05-19 10:46:40 -04:00 |
|
|
|
cf17fc42b1
|
fixed icon downloading performance issues
|
2026-05-19 10:32:34 -04:00 |
|
|
|
2745e75408
|
updated infra README about pinning AMI
|
2026-05-19 10:23:29 -04:00 |
|
|
|
2f4e5b585d
|
updated PLAN.md for future plans
|
2026-05-19 08:34:42 -04:00 |
|
|
|
85b663a6e8
|
added logging to cloudfront
|
2026-05-18 13:42:49 -04:00 |
|
|
|
c7e33defa2
|
updated plan.md, finished integration test
|
2026-05-18 12:49:34 -04:00 |
|
|
|
5b3f6a6870
|
switched from s3 to disk for saving icons
|
2026-05-18 12:43:50 -04:00 |
|
|
|
113a261dae
|
updated duckdb and added a swap file
|
2026-05-18 02:10:15 -04:00 |
|
|
|
4436f43c6f
|
force destroy bucket with icons
|
2026-05-18 01:21:03 -04:00 |
|
|
|
ddeb8bc504
|
fix TOTAL_BUNDLES sed command in deploy script
|
2026-05-18 01:00:09 -04:00 |
|
|
|
21f2a75ed3
|
delete old tab bundles before making new ones
|
2026-05-18 00:49:50 -04:00 |
|
|
|
a977a8c0b3
|
added initial pipeline README
|
2026-05-18 00:40:57 -04:00 |
|
|
|
921f72d2aa
|
added deploy script
|
2026-05-18 00:40:27 -04:00 |
|
|
|
2bdb71a47a
|
added bot.html for scanning
|
2026-05-18 00:40:20 -04:00 |
|
|
|
f64b93b229
|
random favicon selection changing
|
2026-05-18 00:39:22 -04:00 |
|
|
|
e5035d9a28
|
updated PLAN.md, finished phase 5
|
2026-05-18 00:26:50 -04:00 |
|
|
|
4963866427
|
updated scanning useragent
|
2026-05-18 00:26:13 -04:00 |
|
|
|
77f69abf63
|
cloudfront setup
|
2026-05-18 00:25:54 -04:00 |
|
|
|
4492d225db
|
fancier frontend
|
2026-05-17 23:50:56 -04:00 |
|
|
|
1a584c8e50
|
basic frontend
|
2026-05-17 23:50:12 -04:00 |
|
|
|
771f5d76ab
|
updated PLAN.md for phase 4
|
2026-05-17 23:06:11 -04:00 |
|
|
|
f89883e745
|
added bundle generation
|
2026-05-17 23:02:34 -04:00 |
|
|
|
ca06a91dc6
|
don't allow 1 pixel favicons
|
2026-05-17 23:01:53 -04:00 |
|
|
|
b94427f200
|
don't use svg icons, they aren't supported in coversion
|
2026-05-17 22:34:27 -04:00 |
|
|
|
664197e287
|
added select.sql query
|
2026-05-17 22:22:44 -04:00 |
|
|
|
6cf6049698
|
rewrote icon selection in english rather than sql
|
2026-05-17 22:22:32 -04:00 |
|
|
|
5a2e37ae06
|
added icon downloader
|
2026-05-17 22:09:03 -04:00 |
|
|
|
8b5693b5c6
|
updated PLAN.md finished with phase 2
|
2026-05-17 20:37:38 -04:00 |
|
|
|
f45e4a6034
|
added warc parser
|
2026-05-17 20:25:59 -04:00 |
|
|
|
db81015e0b
|
added query.sh to read the cc-index from s3 parquet files and dump it into our psql db
|
2026-05-17 19:12:25 -04:00 |
|
|
|
65d2757527
|
allow ec2 to access common crawl s3
|
2026-05-17 18:22:41 -04:00 |
|
|
|
fcf203e1d8
|
added infra setup with terraform
|
2026-05-17 16:07:50 -04:00 |
|
|
|
64ae58494b
|
added timestamps, warc parser library, log files, progress bars, and testing the frontend with real data to the PLAN.md
|
2026-05-17 14:16:56 -04:00 |
|
|
|
c50be97fd7
|
added PLAN.md with initial dev plan
|
2026-05-17 14:00:14 -04:00 |
|
|
|
a327fb3db3
|
fixed diagram and last tweaks before we plan and code
|
2026-05-17 13:10:37 -04:00 |
|
|
|
01b5de040c
|
updated architecture, downloading all icons, millisecond seeds, mermaid diagram, and partial index
|
2026-05-17 13:03:37 -04:00 |
|
|
|
cf6d819f1f
|
initial ARCHITECTURE.md document
|
2026-05-17 12:19:06 -04:00 |
|
|
|
8ef465b2a4
|
don't use zdns, just use a local unbound to make things easier
|
2026-05-17 12:18:47 -04:00 |
|
|
|
f6ec08535f
|
initial commit
|
2026-05-15 17:32:55 -04:00 |
|
