Files
bsky-social-app/bskyweb/static
bnewbold 6d9e23b1be bskyweb: update robots.txt (#595)
This is to make crawling more explicitly allowed, communicating
expectations.

If we ever end up with "expensive" routes on this service, will want to
add Crawl-Delay.
2023-05-07 12:19:56 -07:00
..
2023-03-20 14:41:15 -07:00
2023-05-07 12:19:56 -07:00