r/AINewsHelpAndTips • u/crazy-creativ • 8d ago
Storage problem.
Hi! I am trying to create a database that can be closed and trained on certain pdfs. The issue I’m having is find a free storage that doesn’t just lock you out after a month. It’s web based so all workers can use it. I’m using GitHub and Vercel. I was using supabase, but now it locked and support won’t answer me (it’s been 2 weeks). I’m learning coding with this but using chats to help create it. The problem is now I need a new storage because supabase won’t even let me look at the website or do anything. I was recommended backblaze and neon but chats but idk what to do. It’s a work computer so it can be based on it and I can’t put a card down or download anything. Any tips would be great. Currently I only have about 1.5 GB of pdfs needed. In the future, there could be a server for this or a paid subscription but if I can’t prove it works, why would they pay?
1
u/karmatics29 8d ago
For 1.5 GB of PDFs, split the problem in two: file storage and the searchable part. Raw PDFs belong in object storage; Backblaze B2 is cheap but needs a card on file, while Cloudflare R2 has a free tier of 10 GB, though it also asks for a card. Since you can't add one, a practical route is to extract the text from each PDF, chunk it, and keep only the text and embeddings in Postgres. That usually shrinks to well under 1 GB, which fits free Postgres tiers.
Neon works for this because pgvector is supported, and free projects are small but enough for a prototype. The trade-off is that free tiers can suspend or limit you, so keep the original PDFs and a script that re-ingests them, so you can rebuild the database anywhere in minutes. Also export a pg_dump now and then, since the Supabase lockout shows why that matters.
Finally, skip fine-tuning for "training on PDFs". Retrieval over chunks with citations is simpler, cheaper and easier to update.
•
u/AutoModerator 8d ago
Take Note:
Carry on, and if you're sharing something cool, don't forget to flair your post!
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.