r/softwaretesting • u/Ok-Motor-9812 • 19d ago
6,715 unit tests stayed green while every upload in the app was denied
The code looked up an organization like this:
admin.from('organizations').select('is_active').eq('id', orgId)
The key of that table is organization_id. So every upload failed its check.
The test for it was green, because it asserted on a mock recorder:
expect(recorder).toContainEqual({
table: 'organizations', op: 'eq', args: ['id', ORG],
})
That 'id' was copied from the code under test. tsc passed too, the query builder accepts any string for a column it doesn't know.
I added two checks at commit time in my coding agent setup. Column names in queries and in those mock assertions get compared with the real schema from the local database. And a function that gains a new query needs one test with no database mock, run against the local stack, with an allowed case and a denied case.
2
u/Future_AGI 18d ago
a production-side check could have surfaced this one quickly too: alert per permission check on an unusual jump in its deny rate after a deploy, and an every-upload-denied bug shows up in the first window, including the cases no test author thought to write.
1
1
1
3
u/hoteloscar 19d ago
And this is why all DAO code and schema should be tested against a running DB at dev and build time.