r/web_infrastructure • u/ManiBePoint • Mar 09 '17
The System Design Primer - Learn how to design large scale systems
r/web_infrastructure • u/ManiBePoint • Mar 09 '17
r/web_infrastructure • u/Pi31415926 • Oct 29 '19
r/web_infrastructure • u/[deleted] • Jun 25 '10
r/web_infrastructure • u/Dielectric • May 16 '16
r/web_infrastructure • u/1nf • Mar 26 '12
r/web_infrastructure • u/rahmad_d • Jun 09 '08
r/web_infrastructure • u/mexxmann • Aug 11 '17
Regarding the following article: http://highscalability.com/blog/2016/1/11/a-beginners-guide-to-scaling-to-11-million-users-on-amazons.html Does anyone know how #users are counted - is it concurrent, total, per hour, or other? Thanks!
ps - I have posted to r/aws and r/programming and gotten two different answers so far - 1. Monthly 2. Concurrent (makes a big difference)
r/web_infrastructure • u/[deleted] • Jul 19 '17
My team and I have a lot of content to work with regarding e-learning files. We need a system that can handle large storage sizes, especially with various video files. We like how Windows Explorer on our company network handles this, but we need to find a separate place to store these files in order to backup.
r/web_infrastructure • u/khcheung • Nov 19 '15
r/web_infrastructure • u/rahmad_d • Jun 26 '08
r/web_infrastructure • u/[deleted] • Jul 23 '24
I need to store 10gb+ of PDFs, along with their plain text and metadata, as well as some 1.5M vectors for a semantic retrieval system. The DB will almost only handle reads.
At first I went with Supabase, as they offer all that in a fully manages fashion, but given the size of the DB, I can't go with th free plan, and 25$/m seems overkill, especially since I will not be using auth or realtime functionalities, which are where Supabase shines.
So I took the cheap, dirty path with a $5/m Contabo VM where I'm self hosting a postgresql + pgvector. Problem is I'm not sure how reliable this infrastructure is, and the latency is not great since I'm in South America, and the closest Contabo servers are in NA.
Now, I don't need a super fast service, but I was wondering if there are better (affordable) options for my requirements, which basically boil down to low CPU, low memory, but (somewhat) bigger storage and reliability.
Thanks
r/web_infrastructure • u/anujtomar_17 • Jul 23 '24
r/web_infrastructure • u/ASamir • Aug 13 '21
We have a domain on GoDaddy and planning on routing the traffic to it through Route 53 and later will be creating subdomains using Route 53 too. So I wanted to know what are the pros and cons and also the security concerns for both scenarios of using GoDaddy for only the domain and Route 53 for subdomains and vice versa.
r/web_infrastructure • u/Bitru • May 27 '20
Hi,
So, I'm developing a python script that will eventually be hosted on a website. Users can enter different parameters to run script that is essentially a bot running 24/7. I've been looking for cloud solutions that are cost efficient (AWS, Google, Python anywhere) , but I've never used such service and don't know how to calculate the cost.
Essentially, every user that sign ups will have their own bot running by its own, so what would be the best way to deploy something like this and calculate cost?
r/web_infrastructure • u/[deleted] • May 05 '18
I have an app which sends a lot of real time data to my servers. Each packet is about 5 floats, but it is transmitted at the rate of 10-50 Hertz.
I want to store this data for a set period of time, like 30 minutes. I might need longer term storage in the future.
I want to process this stream of data in real time, and making sure that this has low latency is absolutely crucial.
How can I build a scalable architecture for dealing with these requirements ?
I was think about using Cassandra to store the time series data. Kafka to introduce redundancy, but I am not sure about it. Does having a Kafka layer introduce latency ?
One way I was thinking about doing this is to, do the necessary processing on the data, on the server that actually accepts the packets from the user. It is not complete heavy, so I think it is doable. And then from that server, also forward those packets to Cassandra.
Does this make sense ?
How would things change if I wanted to send more data. Rather than it been 5 floats, I send a whole image. Streaming images, basically a video. Would it still make sense to store the data in Cassandra ?
r/web_infrastructure • u/puemos • Sep 17 '17
Hi, All!
I've created lambda pack for cleaning up old Docker images from ECR/ECS I will be glad to get some feedback 👌
r/web_infrastructure • u/bullfish81 • Aug 22 '17
How to realize an autocomplete with hundreds of options? Should it be an API Endpoint with caching & invalidation? A Redis cache? Or a local storage with some form of invalidation? What else?
r/web_infrastructure • u/verydefunctional • Aug 08 '17
Hi everyone. I am new to the sub, but hope my topic is not "off-topic" here :).
I am looking for a web interface tunnel to any CLI application (just take telnet as example). I have stumbled across one a long while ago, before websockets (which would be great for this) were a thing and all. Of course I can't remember the name...
What it should be capable of doing is, start a CLI application and reroute the TTY to my browser, and in the first step I want to use it like my shell on the server. I'd like it if the proxy can be modified by me (I know a bit of css, javascript and various backend languages) so I can style and put alerts on the TTY output.
Anyone got some hints? Thanks!
r/web_infrastructure • u/yourbasicgeek • May 08 '17
r/web_infrastructure • u/xrkyx • Jan 07 '17
r/web_infrastructure • u/ScottIBM • Apr 25 '16
I'm not sure if this is the right subreddit or not but here it goes.
I'm working on a small IoT project and I'm in need of a dashboard that allows me to feed it data from a RESTful API. What I have in mind is something fairly out-of-the-box that gives components, such as tabular log views, charts, etc. all that can be fed from an API.
Does anyone know of any project that allows for this that is easy to host and work with? If this is the wrong subreddit let me know and I can move it to a more appropriate one.
-S
r/web_infrastructure • u/jamespaden • Sep 03 '15
r/web_infrastructure • u/jachreja • Mar 03 '15
Hey /r/web_infastructure! I was looking for any updates resources regarding the latest in security best practices with web development these days, specifically aws. I've read all of the whitepapers that they have available, but they're all circa 2013 off of their blog.
I meant to add that i've read through most of https://gist.github.com/chanj/6c48c059ad4b72a60bf3
Any other specific tools, resources, or case studies you would suggest looking into?
Thanks!
r/web_infrastructure • u/seevl • Nov 22 '13
r/web_infrastructure • u/dry_green_tea • Jun 21 '08