|
|
0c9ad5bfd6
|
count iframes only if there isn't an error
|
2026-05-20 00:29:28 -04:00 |
|
|
|
56ae26cbef
|
added bmp decoder to bundler
|
2026-05-20 00:11:53 -04:00 |
|
|
|
7d24b406aa
|
redundant min
|
2026-05-20 00:10:04 -04:00 |
|
|
|
eb40995c60
|
just overwrite bundles, don't delete then re-add
|
2026-05-20 00:09:53 -04:00 |
|
|
|
2f1547a912
|
switched bundle host field to url to retain http
|
2026-05-19 23:38:14 -04:00 |
|
|
|
7f36e99443
|
updated random value to double precision float
|
2026-05-19 23:37:50 -04:00 |
|
|
|
a28cd2b056
|
updated pipeline README
|
2026-05-19 13:06:48 -04:00 |
|
|
|
3534f84b27
|
added about.html
|
2026-05-19 11:42:09 -04:00 |
|
|
|
1d5b7bd374
|
added random_order to host table schema
|
2026-05-19 10:47:05 -04:00 |
|
|
|
e6d5d5175c
|
fixed oom in bundle_gen and added randomOrder, still need a full redesign
|
2026-05-19 10:46:40 -04:00 |
|
|
|
cf17fc42b1
|
fixed icon downloading performance issues
|
2026-05-19 10:32:34 -04:00 |
|
|
|
5b3f6a6870
|
switched from s3 to disk for saving icons
|
2026-05-18 12:43:50 -04:00 |
|
|
|
ddeb8bc504
|
fix TOTAL_BUNDLES sed command in deploy script
|
2026-05-18 01:00:09 -04:00 |
|
|
|
21f2a75ed3
|
delete old tab bundles before making new ones
|
2026-05-18 00:49:50 -04:00 |
|
|
|
a977a8c0b3
|
added initial pipeline README
|
2026-05-18 00:40:57 -04:00 |
|
|
|
921f72d2aa
|
added deploy script
|
2026-05-18 00:40:27 -04:00 |
|
|
|
4963866427
|
updated scanning useragent
|
2026-05-18 00:26:13 -04:00 |
|
|
|
f89883e745
|
added bundle generation
|
2026-05-17 23:02:34 -04:00 |
|
|
|
ca06a91dc6
|
don't allow 1 pixel favicons
|
2026-05-17 23:01:53 -04:00 |
|
|
|
b94427f200
|
don't use svg icons, they aren't supported in coversion
|
2026-05-17 22:34:27 -04:00 |
|
|
|
664197e287
|
added select.sql query
|
2026-05-17 22:22:44 -04:00 |
|
|
|
5a2e37ae06
|
added icon downloader
|
2026-05-17 22:09:03 -04:00 |
|
|
|
f45e4a6034
|
added warc parser
|
2026-05-17 20:25:59 -04:00 |
|
|
|
db81015e0b
|
added query.sh to read the cc-index from s3 parquet files and dump it into our psql db
|
2026-05-17 19:12:25 -04:00 |
|
|
|
fcf203e1d8
|
added infra setup with terraform
|
2026-05-17 16:07:50 -04:00 |
|
