Distfile fetching problems in the AI age
I'm starting to notice an increase in unavailable distfiles for package builds. It seems to be blocks to prevent AI agents from downloading and we're getting caught in the blast radius. I'm curious if other BSD projects are having the same problem or if users have had problems downloading distfiles for ports builds across the BSDs due to this.
I don't want to blast specific projects here, but in one case, it seems IP based and I've seen others blocking fetch, curl, and wget user agents.
I'm not sure what the solution to this problem is. Might have to start using AWS spot instances for fetching, tor or VPNs. I'm not sure what to do about this problem. We do have some distfile caching on the cluster but some things we can't mirror on our own FTP.
(I'm with the MidnightBSD project)