r/CloudandCode • • Oct 16 '26

Announcement Start Here: learn, build, or get help

12 Upvotes

Welcome to r/CloudandCode. We learn Python, AWS and SQL by building small projects, fixing errors and sharing what we learn.

Pick your next step

Posting a question?

Include what you are building, what you expected, what happened, and what you have tried. Add relevant code or an error message with secrets removed.

Use Help for a problem, Project for something you built, Guide for an explanation, Discussion for a question, and Resource for a useful link. Put the topic near the start of the title, such as [Python] or [AWS].

Be respectful, explain your reasoning, and disclose your connection to anything you promote. Read the community rules before posting.

Join the weekly project check-in for October 16 to share progress or a blocker.

New here? Introduce yourself below with what you are learning and one thing you want to build.

I run YourCloudDude and maintain this community. Creator resources are optional; the lessons and discussions here remain available without a purchase.


r/CloudandCode • • Oct 16 '26

Resource Project Library: choose your next small build

10 Upvotes

Pick one project and get a small version working. The links below lead to lessons and discussions in this community.

Python

Expense tracker: add an expense, save it between runs, and show totals by category. Then handle invalid amounts and missing files. Start the walkthrough.

File organizer: scan a test folder and preview destinations by file extension before moving anything. Test duplicate names and files without an extension. See the Python project ideas.

AWS and cloud

Static website: follow a request through S3 and CloudFront. Explain who can access the content and how you deploy an update. Build the website.

Task API: connect API Gateway, Lambda and DynamoDB. Test a missing permission and explain the error using logs.

Check account benefits and costs before deploying. Set a budget alert, keep the lab small, and clean up its resources when finished. An alert is not an instant spending cap. More AWS project ideas.

SQL

Orders analysis: use a small dataset to answer three questions: which products sell most, which customers return, and how sales change over time. Keep the queries and document any missing or duplicate data. Discuss SQL practice and projects.

Make it easy to review

Include setup steps, a short demo or sample output, and one failure you investigated. Use this README checklist.

Need the basics first? Choose a learning path.

Want feedback? Share what you are building in the Project Clinic.


r/CloudandCode • • 3d ago

Python 10 Python projects that teach you more than another 30 hours of tutorials

112 Upvotes

There's a strange problem with learning Python that nobody really talks about.

You can spend 30 hours watching tutorials, understand almost everything the instructor explains, and still struggle to build something on your own. You know what functions do, you've written a few loops, and you've probably followed someone building an entire application. Yet the moment you open an empty editor, everything feels different.

The problem isn't necessarily that you haven't learned enough. It's that you've spent more time watching someone solve problems than actually solving them yourself.

Most tutorials teach you how things work when everything goes right. The API responds, the database connects, the input is valid, and the code runs without errors. But real programming is about making decisions when things don't work as expected. You learn more from debugging one stubborn issue than from watching someone explain a concept you already understand.

I think that's why building a few projects properly can teach you more than constantly switching between courses. Not because projects are some magical shortcut, but because they force you to confront problems that tutorials usually solve for you.

Here are 10 Python projects I'd recommend, each designed to teach you a different part of real software development.

1. Expense Tracker: Learn how databases actually work

Start with something simple. Build an application that records your daily expenses, categorizes them, and generates monthly spending reports. You could initially store everything in a Python list or dictionary, but you'll quickly encounter a problem. What happens when you close the application? Your data disappears.

That's where SQLite comes in. Move your transactions into a database, let users update or delete entries, and generate reports using SQL queries. Once the basics work, introduce input validation, spending limits, and recurring expenses. You'll naturally learn CRUD operations, database design, and how applications manage information over time.

The real lesson: Learning SQLite isn't the goal. Understanding why persistent storage matters and how data flows through an application is what makes this project valuable.

2. Weather Dashboard: Learn APIs and error handling

Build a weather dashboard using Python and a free API like Open-Meteo. Let users enter a city and retrieve temperature, humidity, and weather conditions. The first version might take very little code, which is exactly why this is a good starting point.

Now make it more realistic. What happens when someone enters a city that doesn't exist? What if the API returns an error or takes too long to respond? What if the response is missing information your application expects? Add request timeouts, error handling, and caching to avoid unnecessary requests. You'll learn HTTP requests, JSON parsing, status codes, and how to deal with external services.

The real lesson: Calling an API is easy. Building something that continues behaving sensibly when the API fails is the actual engineering skill.

3. File Organizer: Learn practical automation

Look at your Downloads folder. There's probably a combination of PDFs, screenshots, ZIP files, documents, and random files you've forgotten about. Build a Python script that automatically organizes everything into folders based on file extensions.

The first version might only take 20 lines. But what happens when two files have the same name, a folder doesn't exist, or your script accidentally overwrites something important? Use pathlib and shutil, add logging, and implement a dry-run mode that shows what would change before moving anything.

The real lesson: Automation isn't just about making repetitive tasks faster. It's about making sure your program can perform those tasks safely without creating bigger problems.

4. URL Shortener: Learn backend development

Build a simple URL shortener using FastAPI or Flask. A user submits a long URL, your application creates a shorter identifier, and visiting that short link redirects them to the original website.

Start with basic routing and connect SQLite to store the URLs. Then handle invalid submissions, nonexistent links, and duplicate requests. Once everything works, introduce expiration dates, click tracking, and basic rate limiting. You'll learn API endpoints, HTTP methods, request validation, database integration, and how different parts of a backend communicate.

The real lesson: You're no longer writing a script that runs once and exits. You're building a service that needs to respond correctly to requests from other applications and users.

5. Website Change Monitor: Learn scheduling and reliability

Imagine you're waiting for an important documentation update or a public announcement. Instead of manually checking the website every day, build a Python application that monitors the page and alerts you when something meaningful changes.

Use requests and Beautiful Soup to retrieve the relevant content, normalize it, and compare it with a previously stored version. Schedule the script to run periodically and send an email notification when something changes. You'll eventually realize that websites update timestamps, advertisements, and other small elements without actually changing the information you care about. So you'll need to distinguish meaningful updates from irrelevant changes, handle failed requests, and respect the website's access rules.

The real lesson: Writing a script that works once is different from writing a script that can run unattended for weeks without constantly needing your attention.

6. CSV Data Cleaning Tool: Learn data processing

Find a messy dataset and build a tool that makes it usable. I'm talking about duplicate rows, missing values, inconsistent dates, unexpected whitespace, and columns containing multiple data types.

Use pandas to inspect the dataset, identify issues, clean the records, and export the corrected version. But don't blindly remove every row with missing information. Sometimes a missing value represents zero, and sometimes it means the information was never collected. Those situations require different decisions. Add a summary report that explains how many records were modified, removed, or flagged for review.

The real lesson: Data processing isn't just knowing pandas functions. It's understanding the information you're working with and recognizing how your assumptions affect the final result.

7. Background Job Processor: Learn concurrency

Suppose your application needs to process 1,000 images or send hundreds of notifications. Doing everything sequentially can become painfully slow, especially when tasks spend time waiting for external services.

Build a simple task queue using Python's queue module and worker threads for I/O-bound operations. Track whether tasks are pending, running, completed, or failed. For CPU-heavy processing, experiment with multiprocessing. Then deliberately stop a worker halfway through a task and figure out what happens to the unfinished work. Can you retry it? What if the same task gets processed twice?

The real lesson: Making software faster introduces new problems. Concurrency forces you to think about shared state, failures, retries, and whether operations can safely run more than once.

8. API Testing Suite: Learn automated testing

Take the URL shortener you built earlier and write an automated testing suite for it using pytest. Test whether valid URLs are accepted, invalid requests are rejected, redirects behave correctly, and nonexistent links return meaningful errors.

Use a separate test database so your tests don't interfere with actual records. Add tests for edge cases, mock external dependencies when necessary, and deliberately introduce a bug to see whether your tests catch it. This is where testing becomes less about following best practices and more about protecting your application from unexpected changes.

The real lesson: Tests aren't just there to prove your code works today. They give you confidence to modify your code tomorrow without breaking functionality you've already built.

9. Log Analyzer: Learn debugging and observability

Build a Python program that reads server logs and identifies useful patterns. For example, which endpoints are failing most frequently? How many requests return HTTP 500 errors? Are response times increasing during certain periods?

Start with a small log file and then test your application against a much larger dataset. You'll discover that loading an entire file into memory isn't always practical. Process logs line by line, handle malformed entries, and generate summary reports. Add filters for timestamps, HTTP status codes, and endpoints to make your analysis more useful.

The real lesson: Debugging doesn't always mean staring at source code. Sometimes the most important information is hidden in what your application was doing before something failed.

10. Deploy a Complete Python Application: Learn production fundamentals

Take one of your earlier projects, preferably the URL shortener, and deploy it so someone outside your laptop can actually use it. Package the application with Docker, configure environment variables, connect persistent storage, and deploy it to a hosting platform or cloud server.

Once it's running, add structured logging, health checks, and a basic backup strategy. Then start testing situations you normally wouldn't encounter in tutorials. Restart the application and check whether the data survives. Disconnect the database and observe what happens. Deploy a broken version and figure out how to roll back. You'll learn containers, deployment, configuration management, troubleshooting, and basic operational practices.

The real lesson: Making an application work locally is one achievement. Making it available, maintainable, and recoverable is an entirely different level of understanding.

The part most beginners get wrong

I wouldn't try to complete all 10 projects in a weekend. That's just another form of tutorial hell, except now you're collecting GitHub repositories instead of course certificates.

Here's what I'd do instead.

Pick one project and go deeper than you're comfortable with.

Build the simplest working version first. Then add requirements that force you to make decisions. Handle bad input, introduce persistent storage, write tests, add logging, and deliberately break something to understand how your application responds.

When you're stuck, resist the temptation to immediately copy a complete solution. Read the documentation, isolate the problem, experiment with smaller examples, and use AI or tutorials to clarify concepts rather than outsourcing the entire project.

I'd also document what went wrong and how I fixed it. Not because every project needs an impressive README, but because explaining your decisions forces you to understand them. If you can't explain why you chose a certain approach, there's probably something worth revisiting.

A simple expense tracker with proper validation, reliable storage, and meaningful tests can teach you more than a complicated AI application assembled from code you barely understand.

I think a lot of beginners underestimate this.

They keep looking for the next library, framework, certification, or course because learning something new feels productive. And sometimes it is. But there's a point where consuming more information stops being the thing holding you back.

You don't need to know every Python library to become a good developer. You need to become comfortable solving problems without already knowing the answer.

And that only happens when you spend enough time building, debugging, making mistakes, and figuring things out for yourself.

Curious what others think: Which Python project taught you the most, especially because something went wrong while building it?

P.S. If you're looking for resources to help you build Python projects, feel free to check out the Creator Resources button. No pressure, just there if you find it useful.


r/CloudandCode • • 10d ago

AWS & Cloud I wish beginners understood these 10 things before touching AWS

102 Upvotes

One thing I find frustrating about how AWS is taught is that beginners are often introduced to 50 different services before anyone explains how those services actually fit together. we learn how to launch an EC2 instance, create an S3 bucket, and configure a Lambda function. Everything looks pretty straightforward while you're following a tutorial.

Then you try building something without instructions, and suddenly you have no idea why your application isn't accessible, why permissions keep failing, or where that unexpected AWS charge came from.... i think there are a few things worth understanding before getting too deep into AWS. Most of them aren't even AWS-specific.

1. AWS won't save you from weak fundamentals.

If you don't understand how HTTP works, what DNS does, or how a server communicates with a database, AWS will probably confuse you even more. For example, when your EC2 application isn't responding, the problem could be your application, the listening port, a security group, or a route. Clicking around the console won't help much if you don't understand how a request reaches your server. Spend some time learning basic networking, Linux, and how web applications work. You'll thank yourself later.

2. The cloud isn't free just because you're learning.

This catches more beginners than it should. AWS offers free usage allowances and credits under certain conditions, but that doesn't mean everything you create is free. Even a stopped EC2 instance can leave behind billable EBS volumes. Public IPv4 addresses can cost money, and services like NAT Gateway can become expensive if you leave them running. Before launching anything, check its pricing. Set up billing alerts, understand that ordinary budget alerts aren't spending caps, and clean up resources after experimenting.

Learning cloud computing shouldn't come with a surprise bill.

3. Your root account is not your everyday login.

The root user has enormous control over your AWS account. Using it for everyday experiments is an unnecessary risk. Enable MFA, protect those credentials, and use a separate identity with appropriate permissions for your regular work. Also, never put AWS access keys directly into your code or push them to GitHub. Learn to use IAM roles and temporary credentials wherever possible. These habits might seem boring when you're starting out, but they're part of being a responsible cloud engineer.

4. IAM is more important than memorizing AWS services.

A surprising amount of AWS troubleshooting eventually comes down to permissions.

Imagine your Lambda function needs to read an object from S3. You could give it full administrative access and make the error disappear. But you've also given that function far more power than it needs. A better approach is to understand which permissions are required and grant only those. That's the idea behind least privilege. Learn the difference between users, roles, policies, and temporary credentials early.

It'll make everything else easier to understand.

5. Networking is probably going to be your biggest headache.

A lot of beginners treat VPCs, subnets, route tables, and security groups as settings they just need to configure once. But networking determines whether your application can communicate with anything. An EC2 instance being in a public subnet doesn't automatically make it reachable from the internet. You also need the correct addressing, routing, and security rules. Try deploying a small web server and deliberately changing its networking configuration. Break connectivity, inspect what's wrong, and fix it.

You'll learn more from that exercise than from memorizing definitions of every VPC component.

6. Regions and Availability Zones aren't the same thing.

A Region is a separate geographic area. Availability Zones are isolated locations within that Region. This matters because putting your entire application on a single EC2 instance in one Availability Zone creates a single point of failure. If that instance or its Availability Zone becomes unavailable, your application might go down. AWS gives you the infrastructure to build reliable systems, but it doesn't automatically make every architecture highly available.

Understanding redundancy and failure recovery is far more useful than simply knowing that AWS has data centers around the world.

7. You don't need Kubernetes for every application.

There's a tendency to assume that a more complicated architecture is automatically a better one.

It isn't. If you're hosting a small application, EC2 might be enough. If you have short, event-driven tasks, Lambda could make sense. If you're working with containers, ECS may be worth exploring. Every option has trade-offs involving cost, scaling, maintenance, and operational complexity. Before choosing a service, ask yourself what problem you're actually trying to solve.

Sometimes the simplest architecture is the most sensible one.

8. Not all storage services solve the same problem.

Beginners often hear about S3, EBS, EFS, and RDS without understanding why AWS needs different storage options.

Think about what you're storing and how your application needs to access it.

S3 works well for things like uploaded images, backups, and other objects. EBS provides block storage commonly used with EC2. RDS helps you run relational databases without managing all the underlying database infrastructure yourself. You wouldn't normally replace a relational database with S3 just because both can store data. understanding the workload matters more than remembering the service name.

9. Deploying an application doesn't mean you've finished building it.

Getting an application to run is one thing. Understanding what happens when it fails is another. Let's say your application suddenly starts returning 500 errors. Do you know where to check the logs? Can you tell whether the application is failing, the database is unavailable, or a dependency is timing out? Get comfortable with CloudWatch logs, metrics, and alarms. Learn how to investigate failures instead of immediately restarting everything.

A working application is good. An application you can troubleshoot is much better.

10. Watching tutorials isn't the same as understanding AWS.

This is probably the biggest one. It's easy to finish a course, collect a certificate, and feel like you've learned a lot. Then someone asks you to design a simple application from scratch, and you don't know where to begin. Try building one complete project without following a step-by-step tutorial.

A basic URL shortener is a good example. You could use API Gateway, Lambda, and DynamoDB. Add appropriate IAM permissions, monitoring, error handling, and documentation. Then take it a step further. Ask what happens if the database becomes unavailable, how you'd handle increased traffic, and what the application might cost. Make changes. Break something. Figure out why it broke.

That's where a lot of the real learning happens.

One last thing.

You don't need to understand every AWS service before getting started. Nobody does. But you should understand what you're building, why you chose certain services, how the pieces communicate, what could go wrong, and how much it might cost. AWS becomes much less intimidating when you stop treating it like a collection of services to memorize and start treating it like a set of tools for solving problems.

For people who've been working with AWS for a while, what's one thing you wish someone had explained to you when you were just starting out?


r/CloudandCode • • 17d ago

Community Discussion Google just dropped Gemini 4 Carbon. Here's what it can actually do for developers

5 Upvotes

Google just announced Gemini 4 Carbon, and ngl, this one looks pretty interesting if ur into Python, coding, cybersecurity, or building actual projects. Instead of focusing only on generating code from prompts, Google is pushing this model toward solving harder engineering problems that can take multiple steps, require understanding large codebases, and involve finding bugs that aren't obvious at first glance.

What caught my attention is the focus on long-running engineering tasks. Most of us have used AI to generate a Python function, explain an error, or write some SQL. That's useful, but real development is rarely just one prompt followed by perfect code. Sometimes u have to understand an existing project, trace how different files connect, identify what's actually broken, make changes, and test whether those changes affected something else.

That's the kind of work Gemini 4 Carbon is designed to handle better. Google says its internal teams have used the model on engineering tasks lasting days, including investigating subtle bugs across large codebases and producing fixes. That doesn't mean it'll magically finish ur entire project without mistakes, but the direction is pretty clear. AI tools are moving beyond answering coding questions toward helping with longer development workflows.

So how could this actually help u?

Imagine ur working on a Python backend with FastAPI, PostgreSQL, and a few external APIs. Everything worked initially, but after adding more features, ur application starts getting slower and occasionally returns errors. Instead of asking AI to rewrite random functions, u could give it the relevant code, logs, and project structure, then ask it to investigate where the bottleneck might be and propose changes with tests.

The useful part isn't just getting a code suggestion. It's having a model that can reason through multiple connected components and help u narrow down what's actually causing the problem. U would still need to verify the diagnosis, run benchmarks, and review the changes, but that could save a lot of time compared with debugging everything blindly.

Cybersecurity is another big focus of this release.

Google is reporting some pretty strong benchmark results. Carbon scored 88.2% on SEC-bench Pro for vulnerability discovery and 71.0% on CWE-bench for patching security weaknesses. Those are benchmark results, not guarantees that the model will find every vulnerability in ur project, but they show where Google is investing heavily.

For developers, that could eventually mean stronger assistance with reviewing code for security issues, identifying risky patterns, suggesting safer implementations, and understanding why a particular piece of code might be vulnerable. If ur building APIs, authentication systems, or cloud applications, having another tool that helps examine security decisions could be genuinely useful.

I can also see this being valuable for learning. Let's say ur studying Python and don't understand why a certain implementation is inefficient or unsafe. Rather than just asking for the corrected code, u could ask the model to explain the original mistake, suggest alternative approaches, identify edge cases, and help create tests to check the behavior. That turns AI into something closer to a code review partner instead of just a code generator.

Another interesting improvement is around full-stack development. Google says Carbon has made gains in coding, agentic workflows, web development, and multimodal understanding. So think about tasks like understanding an existing frontend, making coordinated backend changes, updating tests, and helping troubleshoot why different parts of an application aren't working together properly.

For someone building a portfolio, that could make the development process more approachable. U might be able to spend less time on repetitive setup and more time understanding architecture, improving reliability, testing failure cases, and making the project useful. But I wouldn't let an AI tool make every decision for me, especially when the whole point of building projects is learning how those decisions work.

There's one important catch, though.

Gemini 4 Carbon was announced on October 28, but it isn't generally available to everyone yet. Google is starting with a limited group of cybersecurity defenders and plans to expand access gradually. So if u open Gemini today and don't see Carbon, that's expected. Gemini 4 Argon, released earlier this month, is already available through supported paid Gemini plans and developer access, and it's another option for trying Google's newer reasoning and coding capabilities.

I also wouldn't blindly trust any model with production code, sensitive credentials, or security changes. Even a strong coding model can misunderstand requirements, introduce bugs, or suggest something that works locally but creates problems after deployment. I would still review the diff, run tests, check permissions, and understand what changed before merging anything important.

The bigger thing I find interesting is how the role of AI in development keeps changing. Earlier, the main use case was asking it to write a function or explain a concept. Now we're seeing models increasingly designed to investigate larger problems, work across files, follow longer workflows, and help with tasks that previously required a lot more manual coordination.

For beginners, I still think the best approach is to use AI to understand things faster, not skip understanding them completely. Ask it why ur code failed, what alternatives exist, which edge cases ur missing, and how to test ur implementation. If u can explain the final code without the AI sitting next to u, then the tool probably helped u learn something.

Personally, I'm more interested in how models like Carbon could improve debugging, code reviews, and security analysis than in seeing another demo where AI generates an entire website from one sentence. Building the first version is often the easy part. Understanding why it breaks, keeping it secure, and maintaining it as the project grows is where things get interesting.

What would u actually want to use Gemini 4 Carbon for when it becomes available: building projects, debugging Python code, reviewing security, or automating repetitive development work?

And do u think AI is making developers better at building things, or just making it easier to ship code they don't fully understand?


r/CloudandCode • • 17d ago

swe here, I'm looking to learn more about the cloud, I'm an expert in web dev, but not so much in building cloud architecture/ maintenance/ deployment/ cybersec. anyone interested in friendly networking?

1 Upvotes

thanks, feel free to dm


r/CloudandCode • • 19d ago

Discussion [Projects] Five checks that make a small project easier to review

3 Upvotes

A useful project review starts with evidence someone else can reproduce. Here is a short checklist you can apply to a Python script, SQL report or small cloud application.

1. Can someone run it from a clean starting point?

Write down the required runtime, dependencies, configuration names and exact start command. Include sample input. Use placeholders for secrets.

Evidence: a short setup section that works without your local editor settings or remembered console clicks.

2. Is the expected result concrete?

For a CSV tool, show input rows and expected output. For an API, show one request, status and response. For SQL, state what one result row represents.

Evidence: a small example with an answer that can be checked.

3. What happens when input is wrong?

Try one missing field, one invalid value and one empty input. Record whether each is rejected, skipped or given a documented default.

Evidence: an understandable error or an explicit result, rather than an unexplained crash.

4. What happens on the second run?

Does it append, overwrite, update or create duplicates? There is no universal right behavior; it must match the intended workflow.

Evidence: two runs against the same sample and a note explaining the difference.

5. Can you locate one intentional failure?

Break a harmless part of a practice setup: use a nonexistent local file or an invalid test configuration. Follow the error back to its cause.

Evidence: a brief note with symptom, cause, fix and verification. For a cloud project, also document how to clean up the lab.

Share a reviewable update

Use this template if you want feedback:

Building:
Input and expected result:
What works:
One failure tested:
What I want reviewed:

A small code snippet, sanitized output or repository is optional. Share only public-safe examples. The most useful review request points to one decision or behavior you are uncertain about.

Which of these checks exposed something you had missed in your project?


r/CloudandCode • • 20d ago

Guide [AWS] Private S3 + CloudFront: the origin endpoint changes the design

4 Upvotes

Design exercise: visitors should load a static site through CloudFront, while anonymous direct access to the S3 objects stays blocked.

The important distinction

An S3 bucket origin and an S3 website endpoint behave differently. CloudFront Origin Access Control (OAC) supports an S3 bucket origin. An S3 website endpoint is configured as a custom origin and cannot use OAC or OAI.

Sketch the request path

Visitor -> CloudFront -> S3 bucket origin
               |
         signed origin request

For this design, use the regular S3 bucket origin, configure OAC to sign requests, and grant the CloudFront service principal access through a bucket policy scoped to the intended distribution. Keep S3 Block Public Access enabled. OAC configuration and the bucket policy must work together.

Review these checks

  1. A known object loads through CloudFront.

  2. An anonymous direct request to the same S3 object is denied.

  3. Another distribution is not accidentally allowed by an overly broad bucket policy.

  4. The default root object is configured if requests to / should serve index.html.

  5. A genuinely missing asset produces the error behavior you intend.

Avoid masking the wrong failure

Mapping every 403 to index.html may hide permission problems and make missing JavaScript files return HTML. For a single-page application, define a deliberate route-handling strategy and test routes separately from assets.

Practice question: CloudFront returns a cached page after you break its S3 permission. Does that prove the origin access configuration works?

Answer: no. A cache hit can be served without fetching that object from the origin. Verify an origin fetch when checking origin permissions.

You can do the architecture review on paper. If deploying a lab, check service pricing and remove practice resources when finished.

References: S3 origin access, default root objects, CloudFront caching.


r/CloudandCode • • 21d ago

Guide [Python] Turn a website checker's CSV into a small incident summary

2 Upvotes

If you tried the website checker, the next useful step is reading its output.

Here is a local report that counts recorded outcomes. You can also run it using just the sample CSV below.

1. Create health_checks.csv

status,error
200,
404,HTTP_404
,ReadTimeout
200,
302,

2. Save this as summarize.py

import csv
from collections import Counter

counts = Counter()
total = 0
with open("health_checks.csv", newline="", encoding="utf-8") as source:
    reader = csv.DictReader(source)
    required = {"status", "error"}
    if not required.issubset(reader.fieldnames or []):
        raise ValueError("CSV needs status and error columns")
    for row in reader:
        status = (row.get("status") or "").strip()
        error = (row.get("error") or "").strip()
        label = error or (f"HTTP_{status}" if status else "NO_STATUS")
        counts[label] += 1
        total += 1

print("Checks:", total)
for label, count in sorted(counts.items()):
    print(f"{label}: {count}")

Run python summarize.py in the folder containing the CSV.

Expected output:

Checks: 5
HTTP_200: 2
HTTP_302: 1
HTTP_404: 1
ReadTimeout: 1

3. Interpret it carefully

These are counts of recorded checks. They are not counts of unique incidents, and they are not a measured uptime percentage.

An HTTP 404 can be the correct response for a deliberately missing URL. A 302 means a redirect was returned, not that its destination worked. A timeout tells you this check did not receive the expected response in time; it does not explain the root cause.

The script prefers the recorded error label when present and otherwise labels the HTTP status. A blank status with no error is counted as NO_STATUS rather than silently called successful.

Portfolio extension

Group results by URL and UTC date. Give each URL an expected status, then report mismatches. Preserve the raw CSV so someone can reproduce the summary.

Test a header-only file, a missing error column and a blank status/error row. Decide how malformed rows should be handled before using an imported CSV in a real workflow.

References: csv.DictReader, collections.Counter.


r/CloudandCode • • 22d ago

Guide [AWS] Three SQS failure drills that explain retries better than a happy-path demo

3 Upvotes

A worker receiving a message is only the start of the workflow. Here are three tabletop drills for a standard SQS queue with a custom polling worker. You can reason through them without provisioning anything.

Drill 1: receive, then crash

The worker receives a message and crashes before completing the job or deleting it.

Expected behavior: receiving does not remove the message. After the visibility timeout expires, it can become available again. A very long timeout can delay recovery after a crash.

Design question: how will you distinguish a slow worker from a failed one?

Drill 2: complete, then crash before deletion

The worker completes a side effect, but crashes before deleting the message.

Expected behavior: the message may be processed again. Standard queues use at-least-once delivery; duplicate delivery is also possible independently of this crash.

Design question: what identifies the business operation across attempts? Generating a new ID on every retry defeats duplicate detection. Recording completion separately from an external side effect can leave a crash window; an external service's idempotency support or another coordination pattern may be needed.

Drill 3: one message always fails

A malformed payload repeatedly fails while valid jobs keep arriving.

Expected behavior: with a configured redrive policy, messages exceeding the receive threshold move to a dead-letter queue. The DLQ isolates failures for investigation; it does not repair the data or code.

Design question: what evidence will you inspect before redriving it? Replaying unchanged bad data can repeat the same failure.

Write a tiny failure table

For each drill, record the message's state, the completed side effect, the next attempt's behavior and the log evidence you need. Use job IDs and attempt information, keeping sensitive payloads out of logs.

Visibility timeout is not an exactly-once guarantee. Choose it from measured processing behavior and extend it when appropriate. Lambda event-source mappings add their own batching and retry behavior, so check that integration separately.

References: at-least-once delivery, visibility timeout, dead-letter queues.


r/CloudandCode • • 23d ago

Guide [SQL] A LEFT JOIN can make zero orders look like one: COUNT(*) vs COUNT(id)

3 Upvotes

You want every customer in a report, including customers with no paid orders. A LEFT JOIN is useful here, but the aggregate you choose matters.

Run this complete example in SQLite

It creates sample data inside the query, so it does not change existing tables:

WITH customers(id, name) AS (
    VALUES (1, 'Ada'), (2, 'Lin'), (3, 'Sam')
), orders(id, customer_id, status) AS (
    VALUES (101, 1, 'paid'), (102, 1, 'pending'), (103, 2, 'paid')
)
SELECT c.name,
       COUNT(*) AS joined_rows,
       COUNT(o.id) AS paid_orders
FROM customers AS c
LEFT JOIN orders AS o
  ON o.customer_id = c.id AND o.status = 'paid'
GROUP BY c.id, c.name
ORDER BY c.id;

Expected result:

name  joined_rows  paid_orders
Ada   1            1
Lin   1            1
Sam   1            0

Why Sam has one joined row

The LEFT JOIN retains Sam even though there is no matching order. The right-side columns are NULL in that retained row.

COUNT(*) counts the row. COUNT(o.id) counts non-NULL order IDs, so it returns zero for Sam. Use a column that is guaranteed non-NULL on a real matching row.

The second trap: moving the filter

Move o.status = 'paid' out of ON and into a WHERE clause. Sam disappears, because the NULL-extended row does not satisfy that condition.

Keep the condition in ON when the requirement is: retain every customer, but count only their paid orders.

Two practice changes

  1. Add another paid order for Ada. Both counts for Ada become 2.

  2. Remove Lin's paid order. Lin remains, with joined_rows 1 and paid_orders 0.

Before fixing a report with DISTINCT, write down what one output row represents. Joining orders to multiple order items can multiply rows and inflate counts again; you may need to aggregate at the order level first.

References: SQLite aggregate functions, SELECT and joins.


r/CloudandCode • • 24d ago

Guide [Python] DEBUG="false" can still turn a flag on: parse configuration explicitly

1 Upvotes

A small deployment bug to watch for:

bool("false")  # True
bool("0")      # True

These are non-empty strings. Python is testing whether the string is empty, rather than interpreting a configuration value.

Environment variables arrive as strings, so bool(os.getenv("DEBUG")) is an unsafe way to parse a true/false flag.

A small parser you can reuse

Save this as config_demo.py. It uses only the standard library:

import os

def read_bool(name, default=False):
    raw = os.getenv(name)
    if raw is None:
        return default
    values = {
        "true": True, "1": True, "yes": True,
        "false": False, "0": False, "no": False,
    }
    key = raw.strip().lower()
    if key not in values:
        raise ValueError(f"{name} must be true/false, 1/0 or yes/no")
    return values[key]

print("DEBUG:", read_bool("DEBUG"))

Run it with python config_demo.py. With DEBUG unset, the output is DEBUG: False.

Test these cases

DEBUG valueResult
unsetFalse
false, 0, noFalse
true, 1, yesTrue
TRUETrue
empty string or maybeValueError

Rejecting an invalid value helps expose a deployment typo instead of silently choosing a surprising behavior. The error names the setting without printing its raw value.

Extend it

Add a separate PORT parser that accepts only integers from 1 to 65535. Decide explicitly whether a missing setting uses a default and whether an empty setting is an error.

Keep configuration parsing near startup so a bad setting fails before the application starts accepting work. Only log settings you have deliberately classified as safe to disclose.

References: Python truth value testing, environment variables.


r/CloudandCode • • 25d ago

Guide [AWS] S3 AccessDenied: a five-step checklist before changing permissions

3 Upvotes

An S3 403 is easier to investigate when you record the exact request that failed. Use this checklist before changing any policies.

1. Confirm the caller

Run aws sts get-caller-identity in the same environment and with the same profile as the failing command. For an application, identify its actual workload role. A successful command from your laptop does not establish what the application's role can do.

2. Write down the operation and resource

Listing a bucket and reading an object are different operations:

OperationIAM actionExample resource
List objectss3:ListBucketarn:aws:s3:::example-bucket
Read one objects3:GetObjectarn:aws:s3:::example-bucket/reports/demo.csv

The names above are placeholders. Confirm the real bucket, object key and request context.

3. Read the full error

If AWS includes a policy type or denial reason, start there. Some requests return only a generic error; a message can also report only one of several causes.

4. Trace the applicable controls

Check the relevant identity and bucket policies, then boundaries, organization controls or endpoint policies if they apply. An explicit deny can override an allow. For SSE-KMS objects, check the relevant KMS permissions and key policy too.

5. Make one targeted change and retry

Document the cause, the smallest justified change, and the exact request used to verify it. Avoid making the bucket public as a troubleshooting shortcut.

Five-minute exercise

A role can list example-bucket, but downloading reports/demo.csv fails. What have you established, and what remains unproven?

Answer: listing works for that caller and request. It does not establish permission to read the object or use its encryption key. Check the failed action and resource separately.

For a help request, share a redacted error, caller type, action and resource pattern. Leave credentials, tokens and private data out.

References: S3 403 troubleshooting, IAM policy evaluation.


r/CloudandCode • • 27d ago

Guide [Python] A small website checker that logs HTTP errors and timeouts to CSV

5 Upvotes

If you're comfortable with Python loops, try this as your next project: a URL checker that records each result in a CSV and keeps going when a request fails.

You'll practise exceptions and file handling. The whole exercise runs locally, so you don't need an AWS account.

1. Save the checker

Install Requests in the Python environment you'll use for the script:

python -m pip install requests

Save this as checker.py:

import csv
import sys
from datetime import datetime, timezone
from time import perf_counter

import requests

urls = sys.argv[1:] or ["https://example.com"]

with requests.Session() as session, open(
    "health_checks.csv", "a", newline="", encoding="utf-8"
) as output:
    writer = csv.writer(output)
    if output.tell() == 0:
        writer.writerow(["checked_at_utc", "url", "status", "elapsed_ms", "error"])

    for url in urls:
        checked_at = datetime.now(timezone.utc).isoformat()
        started = perf_counter()
        status, error = "", ""
        try:
            with session.get(
                url, timeout=(3, 5), stream=True, allow_redirects=False
            ) as response:
                status = response.status_code
                if status >= 400:
                    error = f"HTTP_{status}"
        except requests.RequestException as exc:
            error = type(exc).__name__

        elapsed_ms = round((perf_counter() - started) * 1000, 1)
        writer.writerow([checked_at, url, status, elapsed_ms, error])
        output.flush()
        print(url, status or "no response", elapsed_ms, error or "headers received")

Pass URLs as command-line arguments. If you leave them out, python checker.py checks example.com once. Each run adds rows to health_checks.csv in your current folder.

2. Make it fail on purpose

Create an empty practice folder and start a local server there:

python -m http.server 8000 --bind 127.0.0.1

Leave it running. Open a second terminal in the folder containing checker.py and run:

python checker.py http://127.0.0.1:8000/ http://127.0.0.1:8000/missing-page

Open the CSV. You should see one row for each request: 200 for the directory page, then 404 with HTTP_404 for the missing path.

Now stop the server with Ctrl+C and run the checker again. The new rows should have an empty status and ConnectionError. The earlier 404 was a reply from the server; this time, the checker didn't receive an HTTP response.

3. Read the results

  • timeout=(3, 5) sets connect and read timeouts. The request doesn't have a guaranteed eight-second deadline.

  • This script reads the response headers and closes the response without downloading the full body. elapsed_ms includes that work; it isn't a full page-load measurement.

  • Redirects are logged without being followed. A 302 row doesn't tell you whether the destination works.

  • A 200 response doesn't tell you whether login, checkout or a database query works. A 403 or 429 could mean an access restriction or rate limit, so neither automatically means the whole site is down.

Local checks covered 200, 404, 503, an unfollowed 302, a delayed-header timeout, a refused connection and an invalid URL. A second run also appended rows without repeating the CSV header.

For the next version, give each URL an expected status and log any mismatch. If a URL is supposed to redirect, what else would you check before calling it healthy?

References: Requests timeouts and streaming responses.


r/CloudandCode • • Oct 16 '26

Discussion Weekly project check-in: what are you building? | Oct 16

8 Upvotes

What are you working on this week? A small improvement, a broken experiment or your first working version all count.

Share a short update:

  • Building: what the project should do and the stack you are using.

  • Progress: one thing you changed or learned.

  • Blocker: the specific part you are unsure about, with an error or example if useful.

  • Next step: the smallest thing you plan to finish next.

For example: “I’m building a Python expense tracker. CSV saving works now, but I need to handle invalid amounts. Next I’m adding two input checks and documenting how to run it.”

A repository or screenshot is optional. Remove secrets and private data before sharing. If you can help someone else, explain why you would try your suggested approach.

Need an idea? Choose a small build. For a more detailed debugging question or design review, use the Project Clinic.

What would make this a useful week for your project?


r/CloudandCode • • Oct 10 '26

Guide Case Study: I built the same Python API three ways, Flask vs FastAPI vs Django

18 Upvotes

Clarification: this is a conceptual walkthrough of the same task API in three frameworks. It does not report three completed implementations or measured benchmark results. Reddit does not let me change the original title, so I’m making that distinction explicit here.

I’d choose a framework by the work around the endpoints: validation, database access, authentication, administration and maintenance. Here is how I would reason about Flask, FastAPI and Django for one small application.

Keep the requirements the same

Imagine a task API backed by PostgreSQL. A task has an ID, title, description, completion status and creation date.

The routes would be:

  • POST /tasks: create a task.

  • GET /tasks: list tasks.

  • GET /tasks/{id}: fetch one task.

  • PATCH /tasks/{id}: change selected fields.

  • DELETE /tasks/{id}: remove a task.

I’d also require input validation, predictable errors, database migrations, tests, configuration outside the code and clear setup instructions. If users are added, each user should only access tasks they are allowed to see.

Keeping that contract fixed makes the comparison useful. A different endpoint spelling tells me little about how a framework handles the rest of the application.

Flask: choose the pieces yourself

Flask gives me a lightweight starting point for routing requests to Python functions. For a small internal API, that can be easy to follow: I can see the request arrive, run the application logic and return a response.

As the project grows, I would need to choose more of the surrounding stack. That includes validation, database integration, serialization, migrations, authentication and a project structure. Flask’s extension ecosystem supports adding functionality as the application needs it.

That flexibility suits a project where I already know which components I want. I can keep a small service focused, choose my database layer and avoid introducing features I do not need.

The trade-off is decision-making. A beginner may spend as much time comparing libraries as learning the request flow. A team also needs conventions so every route does not handle validation and errors differently.

For this task API, I’d settle on one validation approach, one database access pattern and one error format early. Then I would test an invalid title and a missing task before adding more endpoints.

FastAPI: make the API contract explicit

For a standalone API, I like having request and response shapes close to the endpoint definition. FastAPI combines Python typing with validation models and generated API documentation. Its features documentation explains those built-in API tools.

For POST /tasks, I would define which fields are accepted, whether a title can be empty and which fields the response exposes. For PATCH, I would distinguish an omitted field from a field explicitly supplied with a value.

That gives another developer a clearer contract. Interactive documentation also makes it easier to inspect the available requests while developing, although I would still document setup, authentication and examples in the README.

FastAPI does not remove database design or application decisions. I still need migrations, transactions, authorization, dependency choices and a maintainable layout.

I would also choose synchronous or asynchronous code according to the libraries doing the work. Adding async to a function does not make a blocking database or HTTP call non-blocking. The official async guidance is useful before mixing those styles.

For this API, I’d start with a consistent database approach and test it under a realistic workload before making performance claims. Clear validation and documentation are useful even without a benchmark win.

Django plus DRF: consider the whole product

For an API comparison, I would consider Django together with Django REST Framework. Django brings a larger application foundation, including its ORM, migrations, authentication system and admin interface. DRF adds API-focused tools such as serializers and permissions.

That is more structure to learn for five CRUD endpoints. It becomes easier to justify if the application also needs user accounts, staff administration, related data models or an established way to organize a larger codebase.

Imagine the task app gaining organizations, memberships, role-based access and an internal support team. An admin interface and consistent model/migration workflow could save substantial custom work.

I would still test access deliberately. Authentication identifies a caller; authorization decides what that caller can do. Adding a framework or a permission class does not automatically prove that every query and object operation respects task ownership.

The question I’d ask is which Django features the product will actually use. Endpoint count alone would not decide it for me.

What I would choose for this example

  • A small service with specific library choices and limited surrounding features: I’d consider Flask.

  • A standalone API where explicit schemas, validation and interactive docs matter: I’d consider FastAPI.

  • A product with relational models, administration, accounts and growing application workflows: I’d consider Django with DRF.

These are starting points for a decision. Existing team knowledge, integrations, deployment constraints and maintenance requirements can change the answer.

Test the same behavior in every version

I’d write the acceptance checks before comparing frameworks:

  1. A valid create request returns a task that can be fetched later.

  2. Invalid input returns a useful error without saving a bad record.

  3. Fetching or changing a missing task behaves consistently.

  4. If the app has users, one user cannot access another user’s private tasks.

  5. A database failure produces useful logs without exposing credentials.

  6. Another developer can run the app and migrations from the README.

If performance matters, I’d measure the same workload, database state and deployment resources. Slow queries, external requests and configuration deserve investigation alongside framework overhead.

A useful next exercise

I’d learn one framework well enough to build, test and deploy this complete API. Then I would implement one endpoint in a second framework and compare the validation, database call, error handling and tests.

The Project Library has small builds to start with. Bring a specific framework decision or blocker to the Project Clinic.

For your next backend, which requirement would have the biggest effect on the choice: API contracts, administration, existing team experience or something else?


r/CloudandCode • • Oct 09 '26

DevOps Case Study: I rebuilt a Terraform-style AWS setup with Pulumi to see where it actually feels different

8 Upvotes

I have been using Terraform as the default example whenever I talk about Infrastructure as Code because it is one of the easiest tools to recognize in cloud and DevOps discussions. But I wanted to look at Pulumi from a more practical angle, not as another “Terraform killer” post. The question I wanted to answer was much simpler: if I already understand AWS infrastructure, what changes when I describe the same infrastructure using a programming language instead of HCL?

For this case study, I imagined a fairly normal small AWS application rather than a huge enterprise environment. The requirement was to create a VPC, networking, security groups, an Application Load Balancer, an ECS service, an RDS database, an S3 bucket, and the IAM permissions required by the application. This is complex enough to expose the differences between the tools without turning the comparison into a 5,000-resource platform.

With Terraform, I would normally describe the infrastructure using HCL. A resource is declared, its arguments are configured, outputs from one resource are passed into another, and Terraform builds the dependency graph. Once you understand the Terraform workflow, the model is predictable: write configuration, run terraform plan, review the proposed changes, and then run terraform apply.

Pulumi keeps the Infrastructure as Code idea, but changes the way I express it. Instead of learning a configuration language specifically for infrastructure, I can write the infrastructure using languages such as TypeScript, Python, Go, C#, Java, or JavaScript. For someone who already spends most of the day writing application code, that feels immediately different.

A tiny example makes the difference obvious.

With Terraform, an S3 bucket might look roughly like this:

resource "aws_s3_bucket" "uploads" {
  bucket = "example-app-uploads"
}

With Pulumi using Python, the same idea can look closer to normal application code:

import pulumi_aws as aws

uploads = aws.s3.Bucket(
    "uploads",
    bucket="example-app-uploads"
)

At this level, the difference is mostly syntax. I do not think choosing Pulumi because Python looks more familiar is enough reason by itself to replace Terraform. The more interesting difference appears when the infrastructure starts containing real logic, reusable abstractions, validation, loops, configuration, and relationships between many resources.

Imagine I want to create several similar S3 buckets, security rules, or application components based on configuration. In Pulumi, I can use the same language features I already use in normal software: functions, classes, loops, conditionals, packages, testing tools, and type systems where the language supports them. I do not have to mentally switch between “programming code” and “infrastructure configuration” as much.

That can make reusable infrastructure feel very natural. I could create a Python function or TypeScript component that represents something like a standard application service, and that component could create the load balancer configuration, ECS resources, IAM role, logging, and other pieces required by that service. The team can then reuse that abstraction instead of copying the same infrastructure definitions into multiple places.

This is probably the part of Pulumi I find most interesting.

Infrastructure begins to feel like a software library.

But that strength can also become a weakness.

If I give developers a full programming language, they can build much more sophisticated abstractions. They can also build abstractions that are much harder to understand than the infrastructure underneath them. A simple Terraform directory with explicit resources can sometimes be easier to review than a Pulumi project where resources are created through several classes, helper functions, conditions, and internal libraries.

So I would not describe Pulumi as automatically cleaner than Terraform.

It gives me more expressive power.

Whether that becomes cleaner or more complicated depends heavily on how the team uses it.

State is another area where I initially expected Pulumi to feel completely different, but the underlying problem still exists. An Infrastructure as Code tool needs some way to understand the relationship between the resources defined in code and the resources that actually exist in the cloud. Pulumi maintains state for this reason, just as Terraform has state as a central part of its workflow.

Pulumi can use Pulumi Cloud for state and related management features, and there are also options for using other supported backends depending on how you want to operate it. The important lesson for me is that using a general-purpose programming language does not remove the infrastructure state problem. You still need to think about where that state lives, who can access it, how teams collaborate around it, and how environments are separated.

The workflow is also familiar enough that the transition is not as strange as I expected.

With Terraform, I am used to:

Write configuration
        ↓
terraform plan
        ↓
Review changes
        ↓
terraform apply

With Pulumi, the mental flow is similar:

Write infrastructure code
        ↓
pulumi preview
        ↓
Review changes
        ↓
pulumi up

That similarity matters because one of the most valuable parts of Infrastructure as Code is not the syntax. It is the ability to inspect what is about to change before I change the environment.

For this AWS example, I would probably structure the Pulumi project around reusable pieces instead of putting the entire architecture into one massive Python or TypeScript file. Networking could be one component, the application service another, the database another, and shared configuration could define differences between development and production.

That gives me another benefit: application developers can understand more of the infrastructure without learning an entirely separate language first. If a team already uses TypeScript heavily, a TypeScript Pulumi project may feel much more approachable than introducing another syntax solely for infrastructure.

But I also think Terraform has a major advantage here that should not be ignored: familiarity.

A huge number of cloud engineers already know Terraform. There is an enormous amount of existing Terraform code, documentation, examples, community discussion, modules, and organizational knowledge around it. If I join a company and they already have hundreds of Terraform modules working reliably, replacing that environment with Pulumi just because I prefer Python would probably create more problems than it solves.

This is why I think the “Pulumi vs Terraform” discussion becomes much more useful when the question changes from which tool is better? to which workflow fits this team better?

If I have a software-heavy team that wants to build internal infrastructure abstractions using TypeScript or Python, reuse normal testing patterns, and treat infrastructure code like the rest of the software stack, Pulumi becomes very attractive.

If I have an infrastructure team with years of Terraform experience, established modules, existing automation, and engineers who value a specialized declarative configuration format, Terraform may remain the simpler decision.

There is also a testing angle I like with Pulumi. Because the infrastructure lives inside a real programming language, I can use familiar testing patterns around some parts of the infrastructure code. That does not mean unit tests magically prove the cloud environment will behave perfectly, but it gives me more familiar tools for validating infrastructure logic and reusable components before deployment.

The biggest mistake I would avoid is choosing Pulumi because it lets me say, “Now I can write infrastructure in Python.”

That is a nice developer experience benefit, but it is not the architectural reason to use it.

The stronger reason is that a full programming language gives me a different way to build abstractions, share infrastructure components, test logic, and integrate infrastructure development into the rest of the software engineering workflow.

For a beginner, I would still recommend understanding the infrastructure manually before going too deep into either tool. If you do not understand what a VPC, security group, load balancer, IAM role, or ECS service is doing, writing it in Python instead of HCL does not suddenly make the architecture understandable.

The tool should automate infrastructure you already understand.

It should not hide infrastructure you never learned.

If I were learning Pulumi from scratch, I would start with a very small AWS project. I would create something like an S3 bucket, then add IAM configuration, then create a small EC2 or Lambda setup. I would run pulumi preview, inspect exactly what is going to change, deploy it, modify the code, observe the update, and finally destroy the practice environment.

Only after understanding that lifecycle would I move into reusable components and larger architectures.

My takeaway from this case study is that I would not call Pulumi a direct “better Terraform.” I would call it a different approach to the same Infrastructure as Code problem.

Terraform asks me to describe infrastructure primarily through a purpose-built configuration language and its surrounding ecosystem.

Pulumi lets me describe infrastructure using the programming languages and software engineering tools I may already be using elsewhere.

Both approaches can create good infrastructure.

Both can also create terrible infrastructure if I do not understand what I am building.

For me, the real decision would come down to the team, existing infrastructure, abstraction needs, language preferences, ecosystem requirements, and how much value we actually get from treating infrastructure more like application code.

If you had to start a completely new AWS project today with no existing IaC codebase, would you choose Terraform or Pulumi, and what would be the reason behind that choice?


r/CloudandCode • • Oct 08 '26

Guide If I had 30 days to become comfortable with cloud computing, this is what I would do

68 Upvotes

I’d use 30 days to build a small AWS application, investigate its failures, and explain each part of the request path. The plan below is the order I would follow.

Before starting: this assumes you can run a small application locally and use a terminal. If HTTP, basic programming or ports are new, spend longer on the first section. Thirty days is a suggested pace, not a deadline.

Use a practice environment. Check service costs and account benefits before deploying, set a budget alert, and remove resources after each lab. Alerts can lag and do not act as an instant spending cap.

Days 1 to 5: follow a request

For the first few days, I would focus on understanding what cloud computing actually changes. I would learn the difference between running infrastructure myself and using a cloud provider, then spend time on scalability, elasticity, high availability, regions, Availability Zones, pay-as-you-go pricing, and the shared responsibility model. I would also learn enough basic networking to understand IP addresses, ports, DNS, public versus private networks, and what a request is actually doing when it travels across the internet.

I would not try to become a networking expert in five days. I would just make sure that terms like subnet, route, firewall, DNS, and port are not completely abstract anymore. A lot of AWS becomes easier once you can look at an architecture and follow the traffic instead of memorizing boxes.

I would also enable billing visibility and understand that cloud resources can cost money even when I am only experimenting.

Checkpoint: draw the path from a browser, through DNS and a network connection, to a process listening on a port.

Days 6 to 10: connect IAM, EC2 and S3

This is where I would start using AWS properly. I would learn IAM first because almost every AWS project eventually becomes a permissions problem. I would focus on identities, roles, policies, least privilege, and the basic mental model of who is making a request, what action they are trying to perform, and which resource they are trying to access.

Then I would launch one EC2 instance and deploy something simple to it. I would learn what an AMI is, what instance types mean, what a security group does, how ports work, and why an application running on a server is not automatically reachable from the internet. I would deliberately close the application port or make the process listen only on localhost, then try to figure out why the website stopped working.

I would also create an S3 bucket and learn the difference between files on a server and objects in object storage. I would upload files, work with object keys, keep the bucket private, and give a workload only the permissions it needs. At this stage, I would already have enough knowledge to build something small instead of continuing to collect theory.

Checkpoint: explain the identity, action and resource behind one successful AWS request and one denied request.

Days 11 to 15: add data, networking and monitoring

Once I can deploy a basic application, I would add a database. I would probably use RDS with PostgreSQL because it gives me a reason to understand relational data, database endpoints, credentials, private access, and security groups between application and database layers. I would not expose the database publicly just because it is easier.

I would learn how the application should reach it properly.

This is also where I would spend time on VPC basics. I would not try to memorize every networking component. I would build a simple mental model around public and private subnets, route tables, internet gateways, security groups, and the path traffic takes through the system.

I would keep asking which component needs to talk to which other component and whether that communication should be public.

Then I would introduce CloudWatch. I would look at logs when my application fails, inspect basic metrics, and create at least one useful alarm. I think monitoring should be learned early because a deployed application is much easier to understand when you can see what it is doing instead of guessing every time something goes wrong.

Checkpoint: show the permitted application-to-database path and find an application failure in the logs.

Days 16 to 20: build a small serverless API

At this point, I would switch models and build something with API Gateway, Lambda, and DynamoDB. I would not do this because serverless is automatically better. I would do it because I want to understand what changes when I stop managing a long-running server.

I would build a small API where API Gateway receives requests, Lambda handles the logic, and DynamoDB stores the data. I would practise IAM roles again, look at Lambda logs in CloudWatch, and intentionally remove a permission so I can troubleshoot an AccessDenied error. I would also spend some time understanding DynamoDB access patterns instead of treating it like a relational database with a different name.

By the end of these five days, I would want to be able to explain the difference between running an application on EC2 and running logic with Lambda. I would want to understand what responsibilities move to AWS, what new limits appear, and why one option might fit a workload better than the other.

Checkpoint: demonstrate one valid request, one invalid request and one missing record. Explain the response for each.

Days 21 to 24: investigate failures

This is probably the part I would spend more time on if I were learning again. I would take the applications I already built and deliberately break them. I would remove IAM permissions, close security group ports, use the wrong database credentials, stop an application process, misconfigure an environment variable, and see what evidence AWS gives me.

I would also take the EC2 application and put it behind an Application Load Balancer with more than one instance. Then I would stop one of the instances and watch what happens. That gives me a practical reason to understand health checks, load balancing, high availability, and why running multiple instances across Availability Zones can matter.

I think this kind of practice is much more useful than spending another four days watching videos. When something fails, I am forced to follow the request, inspect logs, check permissions, understand networking, and build a mental model of how the system actually works.

Checkpoint: keep a short incident note with the symptom, evidence, cause, fix and a successful retest.

Days 25 to 27: sample queues, containers and IaC

I would not try to master all three topics in three days. I would only learn enough to understand why they exist. For SQS, I would build one simple background workflow where an API places work into a queue and another component processes it later.

That would help me understand asynchronous processing, retries, visibility timeouts, and why one slow task should not always block a user request.

For containers, I would take one application I already understand, package it with Docker, push the image to ECR, and run it through ECS or Fargate. The goal would not be to jump into Kubernetes. I would simply understand how container images, registries, tasks, and managed compute fit together.

Then I would take one small piece of infrastructure and describe it using Terraform or CloudFormation. Maybe an S3 bucket, security group, or EC2 instance. I would create it, modify it, destroy it, and recreate it so I understand why Infrastructure as Code is useful for repeatability.

Checkpoint: explain which problem each tool solves. Choose one to practise further instead of rushing all three.

Days 28 to 30: finish one explainable project

For the final three days, I would stop learning new services and build one project from requirements to deployment. It could be a task manager, URL shortener, file-processing app, or simple SaaS backend. I would choose the smallest architecture that solves the requirements instead of trying to use everything I learned during the month.

I would make sure the project has compute, storage, networking, permissions, monitoring, and a clear deployment path. If it needs relational data, I would use RDS. If it needs file storage, I would use S3.

If something can happen in the background, I might use SQS. If one service is not necessary, I would leave it out.

Then I would write down the architecture and explain why each component exists, what happens if it fails, what I would monitor, what might become a bottleneck, and how I would improve the design if the application had more users. I would also document how to run the project so another person could understand it without me sitting beside them.

Checkpoint: another person should be able to follow your README, understand the diagram and see how to clean up the lab.

Review the finished project

Trace one request from the client to storage and back. Explain who can access each component, what evidence helps when it fails, and which change you would make if usage grew. Keep any performance claims tied to tests you actually ran.

Start with the ordered AWS lessons, or choose a build from the Project Library. If you hit a blocker, share the error and what you tried in the Project Clinic.

Which part would you spend extra time on: networking, IAM, deployment or debugging?


r/CloudandCode • • Oct 07 '26

AWS & Cloud I think beginners understand cloud better when they learn the shared responsibility model early

14 Upvotes

One thing that confused me when I first started learning cloud was the idea that once an application moves to AWS, Azure, or another cloud provider, the provider somehow becomes responsible for keeping everything secure. It sounds reasonable at first because they own the data centers, networking hardware, physical servers, and a huge part of the infrastructure underneath the services we use.

But using the cloud does not mean handing over all responsibility. It changes which parts of the system I manage and which parts the cloud provider manages. That distinction is basically what the shared responsibility model is trying to explain, and I think understanding it early prevents a lot of confusion later.

Take EC2 as a simple example. AWS manages the physical data center, the hardware, and the virtualization layer that allows the virtual machine to exist. But once I launch an EC2 instance, I still have responsibilities inside that machine. I need to think about the operating system, patches, application configuration, network access, credentials, and what software I install.

If I accidentally configure the security group to expose a sensitive port to the entire internet, AWS did not make that decision for me. The infrastructure might be working exactly as designed, while my configuration is still insecure. This is one of the reasons cloud security is often less about whether AWS itself is secure and more about whether I configured the resources correctly.

Now compare that with something like RDS. I am still responsible for things such as who can connect to the database, how credentials are handled, what data I store, and which network paths are allowed. But AWS manages more of the underlying database infrastructure than I would normally manage if I installed PostgreSQL myself on an EC2 instance.

Then compare both of those with Lambda. I do not manage an operating system for every function invocation or patch the servers underneath Lambda. AWS handles much more of that layer. I still have to secure my function code, IAM permissions, secrets, dependencies, event sources, and whatever data the function accesses.

That is why I find it useful to think of cloud services as different points on a responsibility spectrum. The more managed the service becomes, the more infrastructure work moves to the provider, but my responsibility does not disappear. It moves higher up the stack.

A rough mental model I use is:

More control                         More managed

EC2  →  ECS/Fargate  →  RDS  →  Lambda

More infrastructure                 Less infrastructure
I manage                            I manage

That diagram is intentionally simplified, but the idea is useful. If I choose EC2, I get a lot of control over the machine, but I also accept more operational responsibility. If I choose a managed service, I usually give up some control while AWS takes responsibility for more of the underlying platform.

This is also why I do not think questions like “Is serverless more secure than EC2?” have a simple yes or no answer. Serverless removes some responsibilities, but I can still give a Lambda function AdministratorAccess, leak a secret in the code, expose an API incorrectly, or process sensitive data without proper controls.

Managed does not mean impossible to misconfigure.

The same idea applies to S3. AWS is responsible for keeping the storage infrastructure operating, but I am responsible for deciding who can access my bucket and objects. If I create an overly permissive bucket policy or give an IAM role far more access than it needs, that is part of my side of the responsibility.

I think this becomes especially important because cloud resources are so easy to create. On physical infrastructure, provisioning a new server might involve a long process. In AWS, I can create resources within minutes. That speed is powerful, but it also means I can create insecure infrastructure very quickly if I do not understand the configuration.

IAM is probably one of the clearest examples. AWS gives me a powerful identity and permissions system, but it does not know the exact permissions my application should have. I have to decide whether a Lambda function really needs access to every S3 bucket or only one bucket, whether a human user needs administrator access, and whether a particular workload should even be allowed to perform an action.

This is why least privilege matters so much in cloud environments. I try to give an identity the permissions required for its job rather than giving broad access because it is easier. Broad permissions often make the first deployment quicker, but they also make the potential impact of a mistake much larger.

The same responsibility model applies outside security too. Cloud providers can give me highly available building blocks, but they cannot automatically make my application highly available. If I put everything on one EC2 instance in one Availability Zone, the architecture still depends on that one location because I designed it that way.

AWS gives me multiple Availability Zones, load balancers, Auto Scaling, managed databases, backups, and other tools. It is still my job to decide whether my application needs them and configure the architecture appropriately.

Backups are another good example. I should not assume that because my data is in the cloud, it can never be lost. I still need to understand what backup behaviour the service provides, what I configured, how long data is retained, and whether I have ever tested how recovery would actually work.

For me, this is the bigger lesson behind shared responsibility. The cloud provider gives me infrastructure and managed services, but architecture, permissions, configuration, data handling, and application security still require decisions from me.

The exact boundary changes depending on the service.

That is why learning the model once is not enough. Whenever I use a new cloud service, I want to understand what AWS is managing for me and what is still my job.

If I cannot answer that, I probably do not understand the service as well as I think I do.

The more cloud I learn, the less I think of AWS as “a company running my infrastructure for me” and the more I think of it as a collection of services where I choose how much operational responsibility I want to keep.

That decision affects security, control, flexibility, cost, and how much infrastructure I eventually need to manage myself.

If you are currently learning cloud, which responsibility surprised you the most when you realized AWS does not automatically handle it for you?


r/CloudandCode • • Oct 03 '26

AWS & Cloud Cloud computing made more sense to me once I stopped thinking of it as “someone else’s computer”

16 Upvotes

When I first started learning cloud computing, the explanation I kept hearing was that “the cloud is just someone else’s computer.” That is not completely wrong, but I do not think it is very useful either. It makes cloud sound like nothing more than renting a remote server, when the more interesting part is really about how quickly you can get infrastructure, how easily you can change it, and how much of the underlying work you can hand off to a provider.

If I buy a physical server for an application, I have to think about the machine itself, where it lives, how much capacity I need, what happens if it fails, how long replacement takes, and whether I bought too much or too little. In the cloud, I can request compute, storage, networking, databases, and other services when I need them, then change or remove them much faster than I could with physical infrastructure.

That difference matters because applications rarely stay exactly the same. A small project might have almost no traffic today and suddenly need more capacity later. With cloud infrastructure, I can increase resources, run more instances, distribute workloads, or move parts of the system into managed services without buying and installing new hardware every time the requirements change.

This is where concepts like scalability and elasticity started making more sense to me. Scalability is about whether the system can handle more work as demand grows. Elasticity is about adjusting resources as that demand changes. If traffic increases for a few hours and then drops again, I do not necessarily want to permanently run enough infrastructure for the busiest possible moment.

Another thing I misunderstood early was the idea of “pay only for what you use.” Cloud pricing is not automatically cheap. It simply changes the model. Instead of buying infrastructure upfront, you generally pay based on the resources and services you consume. That can be very efficient when the architecture matches the workload, but it can also become expensive if resources are oversized, left running, or designed without any thought about cost.

The cloud also gives you different levels of responsibility. With a virtual machine like EC2, you still manage the operating system, application runtime, patches, and much of the software stack. With a managed database like RDS, AWS takes over more of the database infrastructure work. With something like Lambda, AWS manages even more of the underlying compute while you focus mainly on your function and its configuration.

I think this is one of the most useful ways to understand cloud services. Every managed service is really a decision about which responsibilities you still want to own and which responsibilities you want the cloud provider to manage. More managed does not automatically mean better, but it can reduce operational work when the trade-off fits the application.

Availability is another reason cloud architecture looks different from running one machine in one room. Cloud providers divide infrastructure into regions and availability zones so applications can be designed to avoid depending completely on one location. That does not make failures disappear, but it gives you building blocks for designing systems that can continue operating when individual components or locations have problems.

The same idea applies to storage. Instead of thinking only in terms of a hard drive attached to one machine, cloud platforms give you object storage, block storage, managed databases, distributed caches, and other options. The important skill is not memorizing every storage service. It is understanding what kind of data you have, how it needs to be accessed, how durable it needs to be, and what happens if the compute running your application gets replaced.

Security also changes in the cloud because the infrastructure is programmable. Permissions, network rules, encryption, identity, logging, and access controls can all be defined through configuration. That creates a lot of flexibility, but it also means bad configuration can be just as dangerous as bad code. A powerful cloud account with overly broad permissions can create a much bigger problem than a small local project.

This is why I do not think learning cloud computing should start with memorizing service names. I would first understand compute, networking, storage, databases, identity, availability, monitoring, and cost. Once those ideas make sense, services like EC2, S3, RDS, IAM, CloudWatch, and Lambda stop feeling like random products and start looking like different ways to solve familiar infrastructure problems.

The biggest shift for me was realizing that cloud computing is not mainly about where the computer is located. It is about how infrastructure is provisioned, managed, scaled, automated, and replaced. The remote server is only one part of that.

If someone asked you to explain cloud computing without using the phrase “someone else’s computer,” how would you explain it?


r/CloudandCode • • Oct 02 '26

Community Discussion I think one of the most underrated skills in tech is knowing when not to add another tool

10 Upvotes

When I was newer to building projects, I used to assume that adding more technology automatically made a project more impressive. If the application already used Python and PostgreSQL, maybe I should add Redis. If I was deploying on AWS, maybe I should also add Docker, SQS, Lambda, Terraform, and a load balancer. The architecture looked more advanced, but a lot of the time I was creating problems that the project did not actually have.

Now I try to ask a much simpler question before adding anything: what problem is this tool solving for me right now? If I cannot answer that clearly, I probably do not need it yet. A small application with one backend, one database, and a simple deployment can already teach you a huge amount if you understand how the pieces work together and what happens when something fails.

Caching is a good example. Redis is useful, but I would not add it to a project just because I learned that large systems use caching. I would first look for an actual expensive or repeated operation. Maybe thousands of requests are repeatedly asking the database for the same information and that query is becoming a bottleneck. Now caching has a reason to exist, and I can measure whether adding it actually improves anything.

Queues are similar. SQS is useful when I have work that does not need to happen inside the user's request, or when I need a buffer between systems operating at different speeds. But if my tiny API has three users and every request finishes in a few milliseconds, adding a queue may simply give me another service to configure, monitor, secure, and debug without improving the application.

The same thing applies to microservices. Splitting a small project into six services can look impressive on an architecture diagram, but those services now need to communicate over a network. That introduces timeouts, retries, authentication between services, separate deployments, more logs, and more places where failures can happen. If one application can handle the requirements cleanly, I would rather keep it simple until there is a real reason to separate something.

Even Docker can fall into this trap. Containers are genuinely useful, especially when I want consistent deployments or need to run an application through ECS or another container platform. But if I am still learning how my Python API works, adding Docker before I understand the application itself can sometimes make debugging harder. I would rather understand the basic deployment first, then containerize it and understand exactly what problem the container is solving.

I think this mindset also makes cloud architecture much easier to learn. Instead of trying to memorize when to use fifty AWS services, I can start with a basic system and let the problems introduce the services naturally. When one server becomes a single point of failure, load balancing starts making sense. When background work slows down requests, queues start making sense. When files should survive server replacement, object storage starts making sense.

That is also why I like improving an existing project instead of constantly starting new ones. A project with real limitations gives you reasons to learn new technology. You can measure what is slow, see what fails, and understand what becomes difficult to manage. When you eventually add another component, you know exactly why it is there because you experienced the problem without it.

I think this makes projects easier to explain too. Saying, “I used Redis, SQS, ECS, and Terraform” tells me which tools were used. Saying, “database reads became repetitive, so I introduced caching,” or “report generation was blocking API responses, so I moved it behind a queue,” explains the engineering decision. The second explanation is much more interesting because it shows that the technology came after the requirement.

There is nothing wrong with experimenting with tools just because you want to learn them. I do that too. But I try to separate learning experiments from architecture decisions. If I add Kubernetes because I want to learn Kubernetes, that is completely fine. I just would not pretend the small application actually required Kubernetes in order to work.

The more I learn, the more I think good architecture is often about removing unnecessary things rather than adding more boxes. Every new service creates another dependency, another configuration, another security surface, another cost, and another thing that can fail. Sometimes that complexity is absolutely worth it, but I want the requirement to justify it.

So when I look at a project now, I am trying to ask a different question. Not “What technology can I add next?”, but “What is the biggest real limitation of the current version?” Once I know that, the next tool usually becomes much easier to choose.

What is one technology you added to a project and later realized you probably did not need?


r/CloudandCode • • Oct 01 '26

10 AWS projects I would build if I were learning cloud from scratch today

41 Upvotes

If I were learning AWS again, I would not try to memorize fifty services before building anything. I would pick projects where every new AWS service solves a problem I can actually see. That way, EC2 is not just “a virtual server,” SQS is not just “a queue,” and CloudWatch is not just “monitoring.” Each service becomes connected to something I built, broke, and had to understand.

I also would not try to make every project production-grade from day one. I would start with the smallest working version, then improve the architecture when the current version creates a reason to do so. These are the ten projects I would use to gradually move from basic AWS concepts to real architecture decisions.

1. Deploy a simple website on EC2

I would start with one EC2 instance running a basic website or API. The interesting part is not the website itself. I would learn how an instance gets a public address, how security groups control access, how ports work, how SSH or Session Manager access works, and why an application listening on localhost may not be reachable from outside the machine.

Then I would intentionally break the deployment. I would close port 80, run the application on the wrong port, or change the security group and try to figure out why the website stopped working. That troubleshooting teaches me more about EC2 than simply launching ten instances from the console.

2. Build a private S3 file storage system

Next I would create an application that uploads and retrieves files from S3. I would keep the bucket private instead of making everything public, then give the application only the permissions it actually needs. That forces me to understand buckets, objects, keys, IAM permissions, and the difference between storing files on a server and storing them in object storage.

I would later add versioning and lifecycle rules, then deliberately remove one required IAM permission and investigate the resulting AccessDenied error. That connects S3 and IAM through a real problem instead of learning them as completely separate AWS topics.

3. Host a static website with S3 and CloudFront

Once I understand S3, I would take a static frontend and place CloudFront in front of it. I would keep the S3 origin private where appropriate and let CloudFront become the delivery layer. This gives me a practical reason to learn caching, distributions, HTTPS, DNS, origins, and how content reaches users.

I would also change a file and see why CloudFront might continue serving an older version. That one problem gives me a much better understanding of caching and invalidation than memorizing that a CDN “makes websites faster.”

4. Build a small application with EC2 and RDS

Then I would build something that actually needs persistent relational data, maybe a task manager or job application tracker. The application could run on EC2 while PostgreSQL runs on RDS. The important part would be making the application reach the database without making the database publicly accessible to everyone.

This project would force me to connect VPC networking, security groups, database endpoints, credentials, backups, and application configuration. I would also intentionally break the database security group or credentials and practise following the connection until I find the real problem.

5. Build a serverless API with API Gateway, Lambda, and DynamoDB

After working with servers, I would build the same kind of backend without managing an EC2 instance. API Gateway would receive requests, Lambda would run the application logic, and DynamoDB would store the data. This gives me a direct comparison between a server-based architecture and a serverless one.

I would focus heavily on DynamoDB access patterns instead of treating it like a relational database. I would decide how items should be keyed based on the queries the application actually needs, then use CloudWatch logs to understand what happens when Lambda fails or lacks permission to access the table.

6. Build an event-driven image processor

I would create a small workflow where uploading an image to S3 triggers processing automatically. Lambda could resize or transform the file and save the result into another location. The architecture is small, but it introduces event-driven thinking because nothing needs to manually call the processing function.

This project would also give me a reason to think about permissions, failures, retries, file naming, duplicate processing, and logs. I would upload an unsupported file or remove a permission and see exactly how the workflow behaves when the happy path breaks.

7. Build a background processing system with SQS

Then I would build an API that receives work quickly but processes slower tasks in the background. Maybe a user requests a report, image conversion, or notification. Instead of forcing the user to wait, the application would put a message on SQS and another worker or Lambda function would process it separately.

This is where I would learn why queues exist. I would experiment with retries, visibility timeouts, duplicate processing, and a dead-letter queue. Rather than memorizing the phrase “decoupled architecture,” I would actually see how a queue prevents one slow component from blocking another.

8. Make an EC2 application highly available

Once I understand a single EC2 deployment, I would rebuild it with multiple instances behind an Application Load Balancer. I would place the instances in an Auto Scaling group, spread capacity across more than one Availability Zone, and configure health checks so unhealthy instances stop receiving normal traffic.

Then I would intentionally stop one application instance while sending requests to the load balancer. Watching the application continue working while one server fails makes high availability much easier to understand than reading a definition of it.

9. Containerize an API and deploy it with ECS and Fargate

Next I would take an API I already understand, package it into a Docker image, push the image to ECR, and deploy it through ECS using Fargate. I would place an Application Load Balancer in front and run multiple tasks so I could observe how ECS manages the desired number of containers.

This project would teach me the difference between building a container and actually operating containerized workloads. I would see how ECR, ECS, Fargate, networking, IAM, CloudWatch, and load balancing fit together instead of treating “Docker on AWS” as one vague concept.

10. Rebuild one of the projects entirely with Terraform

For the final project, I would not invent another application. I would take one architecture I already understand and recreate the infrastructure using Terraform. Maybe the EC2 and load balancer project, or the S3 and CloudFront project. The point would be learning how to describe resources instead of manually clicking through the console every time.

I would create the environment, inspect the Terraform plan, make a small change, apply it, destroy the practice environment, and recreate it again. Once I can reproduce the architecture reliably, I would put the Terraform configuration in Git and document the decisions behind it.

The progression matters more to me than the number of projects. The early projects teach me individual AWS building blocks, while the later ones force me to think about availability, asynchronous processing, containers, monitoring, security, and repeatability. Each new service appears because the project creates a problem that needs solving.

I would also keep every architecture diagram and README focused on decisions instead of just listing services. I want to be able to explain why I used RDS instead of DynamoDB, why a queue exists, why a bucket is private, what happens when an instance fails, and what I would remove if the application only had ten users.

That is what would make these projects useful in a portfolio. The impressive part is not having fifteen AWS icons in one diagram. It is being able to defend every box in that diagram.

If you were starting with AWS today, which one of these would you build first?


r/CloudandCode • • Sep 30 '26

Python 10 Python projects I would build if I were learning Python from scratch today

79 Upvotes

If I were learning Python again, I would not spend months building calculators, number guessing games, and tiny scripts that I never touch again. Those projects are fine for learning syntax, but after the basics I would start building things that force me to work with files, APIs, databases, errors, automation, and real user input.

The goal would not be to build all ten as quickly as possible. I would build one, improve it until I understood the decisions behind it, then move to the next project when I needed a new problem to solve.

  1. File Organizer

I would start with a script that scans a folder and automatically organizes files into categories such as images, PDFs, videos, and documents. The first version can be extremely small, but then I would add duplicate handling, custom folder rules, logging, a dry-run mode, and safer error handling. This teaches file handling, pathlib, functions, exceptions, and automation while producing something I could genuinely use.

  1. Expense Tracker

I would build a command-line expense tracker that saves the amount, date, category, and description of every expense. I would begin with CSV storage, then add monthly summaries, category filters, budgets, and eventually move the data into SQLite. This is a good way to connect Python fundamentals with data handling and SQL without making the first version unnecessarily complicated.

  1. CSV Data Cleaner

I would take a deliberately messy CSV file containing missing values, inconsistent dates, duplicate records, and badly formatted text, then build a tool that cleans it automatically. Later I could add configurable cleaning rules and generate a report describing what was changed. This project teaches pandas, validation, transformations, and something beginners often miss: real data is rarely clean.

  1. API Health Monitor

I would create a script that checks a list of websites or APIs every few minutes and records whether they are responding, how long they take, and which status code they return. Once the basic version works, I would add logging, retries, timeouts, historical results, and notifications when something becomes unavailable. This project teaches HTTP requests, scheduling, error handling, monitoring, and the difference between a successful request and a healthy service.

  1. Log Analyzer

I would build a program that reads application or server logs and answers useful questions about them. Which errors happen most often? When do failures usually occur? Which endpoints are producing the most errors? Instead of just reading text files, I now have to parse information, count patterns, handle large inputs, and turn raw logs into something useful.

  1. Job Application Tracker

I would build a small application for tracking companies, roles, application dates, statuses, interview rounds, and notes. I would start with SQLite and a CLI, then gradually add searching, filtering, reminders, and perhaps a small FastAPI or Flask interface later. This gives me a reason to learn CRUD operations, database design, validation, and application structure rather than practising those concepts separately.

  1. Image Processing Automation Tool

I would create a program that takes a folder of images and automatically resizes, compresses, renames, or converts them. Then I would add batch processing, output folders, duplicate protection, configuration, and progress reporting. This is a good automation project because the first version is simple, but improving it forces you to think about reliability and what happens when one file is corrupted or has an unexpected format.

  1. URL Shortener API

This is where I would start moving beyond scripts. I would build an API where a user sends a long URL and receives a short code, then anyone visiting that short code gets redirected to the original address. I would use FastAPI, add a database, handle duplicate URLs, validate input, track click counts, and eventually deploy it. This project teaches APIs, routing, databases, HTTP, deployment, and how backend components work together.

  1. Personal Dashboard

I would build one small dashboard that pulls information from several sources and displays it in one place. It could combine tasks, GitHub activity, spending, weather, or anything else I actually care about. The interesting part is connecting multiple APIs, dealing with authentication, caching responses, handling failures, and deciding what information is actually worth showing instead of simply calling APIs for the sake of it.

  1. Automated Backup Tool

For the final project, I would build something that backs up selected folders automatically. The first version might simply copy files to another location, but then I would add timestamps, incremental backups, compression, logging, configuration, and eventually cloud storage such as S3. I would also deliberately test what happens when files are missing, storage becomes unavailable, or the same backup runs twice. This turns a simple script into a much more realistic reliability project.

I would not try to make every one of these "production ready." I think that phrase makes beginners add Docker, Redis, queues, cloud services, and authentication to projects before they understand the basic application. I would rather build the smallest useful version first, then add complexity only when the current version creates a reason for it.

What I would focus on across all ten projects is progression. Make version one work, deliberately break it, improve the error handling, reorganize the code, add tests where they make sense, document how to run it, and make sure another person could clone the repository without needing you to explain every step.

If you build projects that way, even a simple expense tracker can teach you more than a much larger project copied directly from a tutorial.

If I turn one of these into a complete Python Project #4 walkthrough next, which one should I pick?


r/CloudandCode • • Sep 29 '26

I do not consider a project finished until someone else can run it without asking me for help

10 Upvotes

One thing I have started noticing with beginner projects is that we often stop at the exact moment the code starts working on our own machine. The API responds, the Python script runs, the database connects, and the project gets pushed to GitHub. Technically the application works, but if another person clones the repository and immediately has to message you asking which Python version to use, which environment variables are required, how to create the database, or which command starts the application, I do not think the project is really finished yet.

I think a much better final test is simple: can someone who has never seen this project before get it running from the repository alone? That question exposes a completely different set of problems. Maybe your requirements.txt is missing dependencies, maybe your code assumes a folder already exists, maybe database credentials are hard coded, or maybe the application depends on something installed globally on your laptop that you forgot was even there. These problems are not as exciting as adding another feature, but fixing them makes a project feel much closer to something another developer could actually work with.

For a Python project, I would start by making the environment predictable. I would use a virtual environment and keep the dependency list intentional instead of blindly dumping everything installed on my computer into the project. I would also check that the project works from a fresh environment, because that is the easiest way to discover dependencies I accidentally relied on. If the application needs Python 3.12 or another specific version, I would document that too instead of assuming everyone has the same setup as me.

Configuration is another place where projects often become difficult to reproduce. If my application needs a database URL, API key, AWS region, or another environment-specific value, I do not want those secrets committed directly into GitHub. I would load them from environment variables and include something like a .env.example containing only the names and safe example values. That way someone looking at the repository understands what configuration is required without me exposing actual credentials.

Then I would test the startup process itself. If running the project requires someone to execute seven commands in the correct order while also manually creating three directories, there is probably room to improve the developer experience. I do not necessarily need a complicated deployment system, but I would try to make the steps predictable. Install the dependencies, configure the environment, initialize anything the application needs, then run one clearly documented command.

The README becomes much more important at this stage too. I used to think of a README as a place to write a short project description and maybe list the technologies I used. Now I think it should answer the questions someone would naturally have when they first open the repository: what problem does this solve, how is the project structured, how do I run it, what configuration does it need, what does the architecture look like, and what decisions did I make while building it?

I would also include a small example of the project actually working. For an API, that could be a few sample requests and responses. For a CLI application, it could be terminal output. For a data project, it might be a small sample dataset and an example result. That makes the repository easier to understand because someone does not need to read every file before they know what the application is supposed to do.

Databases are another easy place to accidentally make a project dependent on your own machine. Maybe the application expects a table you created manually six weeks ago, or it depends on sample records that only exist inside your local database. If the project uses a database, I would think about how another person initializes the schema and gets enough sample data to test the application. Even a simple migration or setup script is much better than a README saying, "Create the required tables manually."

Tests become more valuable here as well. Even a small collection of tests can give someone confidence that they configured the project correctly. If they clone the repository, install everything, run the tests, and those tests pass, they immediately know a lot more about whether the environment is working. The tests do not need to cover every possible edge case before the repository becomes useful, but having a few tests around the core behaviour makes the project much easier to trust and change.

I think this is also a useful portfolio signal. Lots of repositories contain working code, but fewer repositories feel like they were prepared for another developer to use. A clean setup guide, sensible configuration, clear project structure, useful examples, and reproducible dependencies tell me that the person building the project thought beyond the moment when the code first ran successfully.

The nice thing is that none of this requires adding another AWS service or building a more complicated application. You can take a very small project you already have and improve the experience around it. Clone it into a completely new directory, create a fresh environment, follow only your own README, and pretend you know nothing about the project. Every time you have to rely on something you remember but never documented, you have found something worth fixing.

That is one of the tests I want to use more often with my own projects. Not just "does this work?", but "could another developer reproduce this without me sitting next to them?" A project that passes the second test usually feels much more complete than one with five extra features and no reliable setup process.

If you cloned one of your current GitHub projects onto a completely fresh machine today, do you think you could get it running using only the README?


r/CloudandCode • • Sep 28 '26

AWS & Cloud I think beginners are learning microservices way too early

23 Upvotes

When I first started looking at cloud architecture, microservices seemed like the obvious next step after building a basic application. Most architecture diagrams I saw had multiple services, queues, APIs, containers, databases, caches, and a lot of arrows connecting everything together. It looked more "production ready" than a simple application, so it was easy to assume that splitting a project into microservices automatically made the architecture better.

The more I build and study these systems, the less I agree with that idea. If I were building a new project today, I would start with a well-structured monolith unless I had a real reason not to. One application, one deployment, and clearly separated modules inside the codebase can already teach you APIs, databases, authentication, testing, Docker, AWS, CI/CD, monitoring, and plenty of other useful engineering concepts.

The important word there is well-structured. A monolith does not have to mean putting your entire project inside one massive file where everything depends on everything else. You can still separate users, payments, orders, notifications, and other responsibilities into clean modules. The difference is that those modules run together instead of immediately becoming separate networked services.

Imagine I am building a small ecommerce backend. I could create separate services for users, products, orders, payments, notifications, and inventory from day one. Now I need to think about how those services communicate, how authentication moves between them, what happens when one service is unavailable, how deployments are coordinated, and how I trace one request across several systems.

For a project with almost no users, I have just created a distributed systems problem before I had a scaling problem.

If I keep the first version as a modular monolith, a request might simply enter the application, execute the required logic, update the database, and return a response. I can spend my time making the actual product work, improving the database design, writing tests, understanding failure cases, adding useful logging, and deploying the application properly.

Then I can watch where the real boundaries appear.

Maybe notification processing starts taking too long and does not need to happen inside the main request. I might move that work behind a queue. Maybe image processing becomes resource intensive and needs to scale independently. Maybe one part of the application receives far more traffic than everything else. Now I have an actual reason to separate something.

That is very different from creating six microservices because a diagram told me that scalable systems use microservices.

There is also a hidden cost beginners sometimes miss. Once two functions inside the same application become two separate services, a normal function call may become a network request. Network requests can fail, time out, retry, return partial responses, or become unavailable entirely. Now I need to think about service discovery, authentication between services, retries, observability, deployment, version compatibility, and possibly asynchronous communication.

The application did not necessarily become more scalable just because I added those problems.

This is also why I would not use the number of AWS services in a project as a measure of how advanced it is. A small application running on ECS with RDS and S3 can involve plenty of meaningful architecture decisions. Adding Kafka, Kubernetes, Redis, five databases, and twelve microservices does not automatically make the project better if I cannot explain why any of them are necessary.

I think the stronger question is: what problem am I solving by separating this component?

If the answer is that it needs independent scaling, separate deployment, stronger fault isolation, a different technology stack, or clear ownership by another team, then splitting it might make sense. If the answer is simply "because real companies use microservices," I would probably keep it inside the application for now.

That does not mean beginners should avoid microservices completely. I think they are worth learning because they expose you to some of the hardest and most interesting problems in modern systems. But I would learn them after I have built something simple enough to understand first.

A really useful exercise would be to build the same application twice. Start with a modular monolith and get the whole system working. Then choose one component that actually has a reason to become independent, move it out, and see what new problems appear.

Maybe notifications become a separate worker consuming messages from SQS. Maybe file processing moves into an event-driven Lambda workflow. Maybe a high-traffic read path gets separated so it can scale independently.

Now you are not just learning what a microservice is.

You are learning why you would create one.

That distinction matters a lot to me because architecture is mostly about trade-offs. Microservices can give you independent deployment and scaling, but they also introduce distributed systems complexity. A monolith can be simpler to operate, but eventually certain parts may become difficult to scale or change independently.

Neither architecture is automatically the "professional" one.

The right architecture depends on what the application actually needs.

If I were building a portfolio project today, I would much rather build a clean monolith, explain its boundaries, identify where it would eventually struggle, and then show how I would extract one component than start with twelve services I barely understand.

What do you think beginners should build first: a modular monolith, microservices from day one, or both versions of the same project?