r/devsecops • • Oct 17 '25

Our AI project failed because we ignored prompt injection. Lessons learned

76 Upvotes

Just wrapped a painful post mortem on our GenAI deployment that got pulled after 3 weeks in prod. Classic prompt injection attacks bypassed our basic filters within hours of launch.

Our mistake was relying on model safety alone and no runtime guardrails. We essentially treated it like traditional input validation. Attackers used indirect injections through uploaded docs and images that we never tested for.

How are you all handling prompt injection detection in production? Are you building custom solutions, using third party tools, or layering multiple approaches?

Really need to understand what works at scale and what the false positive rates look like. Any lessons from your own failures would be helpful too.

Thanks all!


r/devsecops • • Jan 23 '25

Opengrep - a truly Open Source fork of Semgrep - Announced

Thumbnail opengrep.dev
71 Upvotes

r/devsecops • • May 24 '25

Security team dumped another 500 "critical" alerts on us today

63 Upvotes

'm so tired of this shit. Every week it's the same thing, it's 12am on friday i'm still at it on a long weekend.

opsec sends over this massive spreadsheet of vulnerabilities that need to be "fixed immediately." Half of them are in containers that ran for 30 seconds during builds. The other half are in services nobody uses anymore but we're too scared to delete. We're fighting the wrong battles. I want to secure our stuff but this approach is driving me fking up the walls.


r/devsecops • • Mar 02 '22

DevSecOps Playbook - An open-source step-by-step guide

63 Upvotes

I have been working on this project for about 6 months and am excited to let it finally see the light of day. Please meet the DevSecOps Playbook, a step-by-step guide to building a DevSecOps practice inside your software delivery organization.

This playbook is meant to be highly prescriptive and each task has a priority and a difficulty. So if you are starting your DevSecOps journey please start with the priority 1 tasks and when you are done with those circle back to the priority 2 tasks.

In addition to being a step-by-step playbook, this document also maps to a number of compliance frameworks including NIST 800-53, NIST SSDF, ISO27001, SOC2, CIS 8, APRA 234, and the brand new Australian ISM Guidelines for Secure Development.

I hope you enjoy and feel free to ping me here or raise a PR if you want to add something. This is meant to be a community project!

https://github.com/6mile/DevSecOps-Playbook


r/devsecops • • Aug 05 '26

We found a production server that hadn't been patched in 14 months. It wasn't in any of our tools

61 Upvotes

I'm at a logistics company. Was running a routine network sweep last month and found a server on our internal subnet handling live customer shipment data. Fully prod, responding to real requests.

It wasn't on our CMDB, wasn't on crowdstrike and Qualys has never scanned it. Our monitoring platform had no clue it exists, so it had no alerts, coverage or patching history going back at least 14 months based on the OS patch level.

Turns out someone on the app team spun it up for a migration project two years ago. The project got deprioritized and the original engineer had left, but then the server kept running. Handling customer data. With no security controls on it at all.

What bothers me isnt that we found this one but that finding it was basically luck. If I hadn't run that sweep on that subnet that week, itd still be sitting there.

Were now pulling asset lists from every tool we have and cross referencing them. The gap list so far is uncomfortable. Found 40 more devices this week that exist in EDR but not CMDB, and about 20 that exist nowhere but respond to ping.

How do you verify youve found everything? Not what your tools report, but what theyre missing.


r/devsecops • • Jan 24 '23

"Machine identities are scattered across hybrid multi-cloud environments, and need to be able to connect to other workloads or services to fulfill their task. But the access policies that control access privilege levels should be consistent, no matter which cloud a workload runs on."

Thumbnail
techopedia.com
59 Upvotes

r/devsecops • • Apr 29 '25

Wiz Launches MCP Server: Smarter AI Context Meets Real-Time Cloud Security

Thumbnail
wiz.io
57 Upvotes

r/devsecops • • Sep 29 '26

Critical RCE Alert: Full takeover of HashiCorp Vault and OpenBao. OpenBao is patched. Vault remains exposed

Thumbnail
control-plane.io
46 Upvotes

OpenBao engineers at ControlPlane have chained 4 vulnerabilities to show how under certain conditions, an OpenBao or Vault server can be completely compromised from an unauthenticated position. This is only the second RCE ever found in the Vault codebase.

The exploit is highly plausible in real-world environments, requiring only an unauthenticated entry path and a defined Raft snapshot policy to trigger a complete server compromise.

If you are impacted, upgrade as soon as possible to OpenBao 2.6.3 or 2.7.0

While OpenBao is fully patched, HashiCorp Vault remains exposed as of writing. Unfortunately, IBM's unwillingness to coordinate a mutual disclosure policy means Vault users currently lack an official mitigation


r/devsecops • • Jul 15 '24

Wiz expands SIEM integration with Google Security Operations

Thumbnail
wiz.io
46 Upvotes

r/devsecops • • Mar 01 '26

Trivy Github repository is empty?

45 Upvotes

I have some automation that pulls Trivy binary from Github and runs scans using it. Today my automation failed all of a sudden as it was not able to download the Trivy binary from Github. I checked the releases page on Github and it was empty. I navigated the acquasecurity/trivy repo and entire repo is empty. I am not sure if this is just a temporary Github glitch or something else. Anyone observing same issue?

https://github.com/aquasecurity/trivy


r/devsecops • • Mar 21 '26

Your security scanner was the attack vector: Trivy GitHub Action tags force-pushed to malware

43 Upvotes

A few days ago, someone force-pushed malicious code into nearly every version tag of aquasecurity/trivy-action - one of the most widely used security scanning actions on GitHub, referenced by 10,000+ workflows. 75 out of 76 tags were compromised.

The payload silently exfiltrates CI/CD secrets (AWS/GCP/Azure creds, SSH keys, K8s tokens) by dumping runner memory BEFORE running the real Trivy scan. Your builds look green, your scans pass, and meanwhile your cloud keys are walking out the door.

Details: https://socket.dev/blog/trivy-under-attack-again-github-actions-compromise

No words... I've seen SHA-pinned actions in repositories like OpenFGA, and I remember thinking it looked awkward - not having the ability to easily manage GitHub Actions versions, stuck with those annoying SHA hashes instead of clean version tags. But now I see that this has to be one of the essential prevention steps for all GitHub Actions (maybe excluding GitHub's own first-party actions), along with mandatory configuuration that prevents using any actions not pinned by SHA: https://github.blog/changelog/2025-08-15-github-actions-policy-now-supports-blocking-and-sha-pinning-actions/

It seems that you're lucky if you download and run Trivy directly in your CI instead of using their GitHub Action, but who knows.

I also hope that industry will start widely adopting GitHub Immutable Releases.

Who's actually enforcing SHA-pinned GitHub Actions across their entire org? Does anybody use tools like:

- github.com/sethvargo/ratchet

- https://github.com/suzuki-shunsuke/pinact


r/devsecops • • Nov 17 '25

Which DevSecOps certifications are worth it in 2024/2025?

43 Upvotes

Hey everyone,

I'm looking to get into DevSecOps and already have some hands-on experience with common tools and understand the mindset at a junior level. I'm familiar with OWASP principles and various security practices in the CI/CD pipeline.

However, I'd like to get a certification to boost my chances when applying for roles. I'm wondering which certifications are actually valued by employers in the DevSecOps space?

I've come across several options like:

  • Certified DevSecOps Professional (CDP)

  • GIAC Security Essentials (GSEC) or other GIAC certs

  • Certified Kubernetes Security Specialist (CKS)

  • AWS/Azure/GCP security certifications

  • OWASP

For those already working in DevSecOps or hiring for these roles which certifications actually made a difference for you? Are there any that are considered more credible or worth the investment?

Would appreciate any advice or experiences you can share!

Thanks in advance!


r/devsecops • • Mar 21 '23

Mitigating SSRF in 2023

Thumbnail
blog.includesecurity.com
43 Upvotes

r/devsecops • • Sep 06 '24

What is DevSecOps (Coming From Someone with 4 Years DevSecOps Experience in 2 Companies)

40 Upvotes

Looks like people are very confused about the role DevSecOps engineer. Allow me to hopefully help people out.

Short answer is DevSecOps is like a combination of application security and cloud security.

Longer answer is DevSecOps is DevOps with focus on security, ideally sole focus is on security and minimal devops tasks. Like DevOps connects devs and cloud engineers, and DevSecOps handles the security of DevOps. General tasks of devsecops are SAST, SCA, DAST, application security monitoring, application monitoring, cloud security monitoring, security incident response, application security architecture, cloud security architecture.

As people with experience will know, DevOps has different meanings to different companies of different sizes and needs, and DevSecOps is the same. DevSecOps is even newer than DevOps, so companies are still trying to figure it out and out how to integrate it to their setup. Several recruiters contact me every month, and each of them have different job descriptions for DevSecOps. So I'm sure pretty much everyone is confused what it really is. LOL

Here's my background. I'm currently a senior DevSecOps engineer in my current company. Before this, I was a DevSecOps engineer in another one for 3 years. So total is 4 years DevSecOps experience. Before being in a DevSecOps role, I've been in DevOps for around 2.5 years. Before DevOps, I worked in helpdesk, network admin, sys admin, and security engineer roles for 9+ years.


r/devsecops • • May 14 '26

How to secure your GitHub Actions against supply chain attacks

39 Upvotes

We've all been seeing the news and it's clear that GitHub Actions isn’t just CI anymore. It's now part of your supply chain. It builds packages, publishes releases, deploys infra, and often has access to the credentials attackers want.

We put together a practical checklist for locking it down, but the highest-impact controls are pretty simple:

  1. Set default GITHUB_TOKEN permissions to read-only.

  2. Pin third-party actions to full commit SHAs, not tags like u/v4.

  3. Be very careful with pull_request_target, especially on public repos and fork PRs.

  4. Treat PR titles, branch names, issue bodies, labels, comments, and commit messages as untrusted input.

  5. Use OIDC for cloud access instead of long-lived AWS/GCP/Azure secrets.

  6. Don’t put untrusted code and privileged credentials in the same workflow context.

  7. Avoid broad artifact uploads like path: ..

  8. Don’t use self-hosted runners for public repos unless you really know what you’re doing.

  9. Add CODEOWNERS/review requirements for .github/workflows/.

  10. Continuously lint workflow YAML for risky triggers, unpinned actions, and script injection.

Full checklist here:
https://corgea.com/learn/github-actions-security-checklist


r/devsecops • • Oct 07 '26

Coding agents can now edit files and run commands, so where does IDE security start?

38 Upvotes

We rolled out AI assistants to about 120 engineers and the policy is two paragraphs long. The assistants can read the repo and write files and some of them shell out. Our appsec review covered the model vendor and nothing about what the agent does locally. Im trying to work out where the control points even are, since the laptop is now executing code nobody reviewed. Secrets in env files are the part keeping me up. How are people scoping this?


r/devsecops • • Jun 12 '26

One of our devs almost cooked our prod DB

38 Upvotes

Two devs on our team wanted to spin up an open source LLM image in a Kubernetes sandbox, just messing around to see if it could help with some internal automation. Totally reasonable thing to want to do. They grabbed an old deployment config to save time because who wants to write yaml from scratch. The old config had an IAM role attached. Full read access to our production database, nobody noticed.

So for a few hours we had an unvetted third party container image with a wide open path straight to our database. If that image had been compromised, or if it was phoning home to some external endpoint, we would have had no idea until it was way too late. Standard image scan showed nothing because the image itself was clean, no CVEs, no vulnerable packages, totally fine on paper since the problem wasn't the code it was the permissions it inherited from a config.

Caught it at 1am because our security tool flagged a new unverified entity holding a privileged path to the database and paged me. Tore down the deployment and fixed the IAM policy before anything actually happened.

ONE lazy copy paste, there was no malicious intent or anything,, just two devs doing their work and not reading the config. You can have all the right processes and it just takes one person in a hurry to completely undo it. This job is just way too fkn stressful at times.


r/devsecops • • Mar 27 '26

CI/CD security checklist after the Trivy GitHub Actions compromise

37 Upvotes

75 Trivy Action tags got repointed to malware in a single push. Every pipeline using u/ v1 or u/ main references ran attacker-controlled code with access to repository secrets. Then CanisterWorm used stolen npm tokens to infect 140+ downstream packages through postinstall scripts.

I maintain an open-source security scanner (Ship Safe) and I spent a few days hardening our own pipeline after studying the attack. Here's the checklist we came out with:

GitHub Actions:

  • Pin every third-party action to full commit SHA (replace u/ v1 with @<sha> # v1)

  • Add explicit permissions block to every workflow (default is write-all)

  • Never use pull_request_target with actions/checkout (gives fork PRs write access)

  • Audit run: blocks for ${{ github.event }} interpolation (script injection vector)

npm / package publishing:

  • npm ci --ignore-scripts in all pipelines (blocks postinstall payloads)

  • .npmrc with ignore-scripts=true for local dev

  • OIDC trusted publishing (no long-lived npm token to steal)

  • npm publish --provenance for verifiable builds

  • Strict files allowlist in package.json (no test files, no configs published)

  • Sensitive file gate: npm pack --dry-run | grep -iE '\.env|\.key|credentials'

Access control:

  • CODEOWNERS on action.yml, package.json, .github/, and publish configs

  • Require PR reviews for protected paths

  • FIDO-based 2FA on npm (not TOTP -- it's phishable)

  • Rotate all CI tokens after any suspected compromise

Detection:

  • Run a security scanner in CI that checks for the above

  • Self-scan: your own scanner runs against your own code before publish

Ship Safe's CICDScanner agent checks for all the GitHub Actions issues automatically:

npx ship-safe audit .

We also run ship-safe audit . against ourselves in our own CI pipeline. If a supply chain attack injects code into our repo, our scanner catches it before it ships to npm.

What's your pipeline hardening look like? Are you SHA-pinning actions or still on tags?


r/devsecops • • Mar 05 '26

Java keeps having critical auth library vulnerabilities. Is this a pattern or am I imagining it?

37 Upvotes

This week: CVE-2026-29000 - CVSS 10.0 auth bypass in pac4j-jwt.

2022: CVE-2022-21449 - psychic signatures, blank ECDSA sigs passed verification in the JDK itself.

Before that: Spring Security and Apache Shiro auth bypasses.

Is the Java ecosystem uniquely bad at this, or does every language have this problem and Java just gets more scrutiny because it runs more enterprise backends?

Some links to help:

1/ https://www.codeant.ai/security-research/pac4j-jwt-authentication-bypass-public-key

2/ https://nvd.nist.gov/vuln/detail/C%20then%20then%20automatically

3/ https://www.cve.org/CVERecord?id=CVE-2026-29000

What's your go-to JWT library in Java right now? How confident are you in it?


r/devsecops • • Nov 05 '25

Any good tools for finding duplicate code in big monorepos?

37 Upvotes

Our monorepo has years of copy-pasted utils scattered across projects. Searching manually is impossible. Is there a reliable way to detect duplicates and suggest consolidation?


r/devsecops • • Feb 16 '25

Devsecops interview questions

37 Upvotes

Hello all, I gave a devsecops interview recently and wanted to share type of questions asked by interviewer. It was a good 30 minutes of interview.

Encryption algorithms What is Symmetric, Asymmetric, hashing, encoding? What is the difference between above? Which one is more secure? Why do we still use symmetric and not only hashing or Asymmetric if its more secure than symmetric?

What is transitive dependency in devsecops?

How do you integrate security tools in jenkins? Explain the processes/stage included.

What is sql injection and how can you prevent it?

What is XSS and how can you prevent it?

What is the process of threat modelling you followed?

How did you make sure that team is adhering to threat modelling framework?

In SCA scan the tool has given that spring jcl 1.0.0 dependency is missing..but its not present in pom.xml is this a false positive or not?

How can you find correct libraries to fix vulnerabilities? Example: there maybe hundreds of libraries for an application.

In transitive dependency how can you find out the correct parent library to update ?

Explain me any type of vulnerabilities (high low medium)you have worked on and fixed while working in devsecops.

What is the secure approach dev team can use to store passwords?


r/devsecops • • Feb 21 '26

Hot take: hardened container images are a lie if your devs keep asking for emergency patches

36 Upvotes

this keeps coming up on our side and I’m curious if others are seeing the same pattern. we talk a lot about hardened container images, but in practice security teams keep chasing cve after images ship, devs file constant requests to patch base images, CI pipelines slow down because images arent actually minimal or stable, and the list goes on... at some point it feels like we’re pretending images are hardened when they’re really just bloated base images with scanners slapped on top. If hardened container images are the answer, why do so many teams still operate in permanent patch mode?


r/devsecops • • 20d ago

We stopped fixing dependency CVEs that our code never reaches

35 Upvotes

Our dependency alerts hit 340 open last month across 9 services. Most of it was transitive stuff in packages we pull in for one function. We were patching in order of CVSS score and never getting below 300, because new ones showed up faster than we closed them.

What changed it was splitting the list by whether our code can actually reach the vulnerable function. We already had coderabbit's security scan on, and its findings now come with a reachability and exploitability badge, so we filtered on that. 340 became 41 that were reachable from something we expose. We fixed those 41 in about two weeks, mostly through the fix PRs it opens.

The part I'm still not comfortable with: reachability is a guess. One finding marked not reachable was reachable through a background job that imports the whole package dynamically. We caught it in a manual pass, the tool didn't.

So now the rule is reachable gets fixed this sprint, not reachable gets a quarterly sweep. Is anyone else triaging dependency CVEs this way, or do your auditors still want every critical closed no matter what?


r/devsecops • • May 31 '26

Best tools for SAST + SCA + Image Scan + IaC Scan + DAST

35 Upvotes

Hi experts,
New to sec tools. What are the best tools in the market for SAST + SCA + Image Scan + IaC Scan + DAST?
Over the search I found multiple tools, bit confused what to choose.

My choice of tools:

SAST - SonarQube

SCA - Snyk

Image Scan - Trivy

IaC Scan - Trivy

DAST - OWASP ZAP