r/mongodb • u/ReSpectacular • Jan 09 '24
Mongodb throughputs are nowhere close to official benchmarks.
Inspired by Through the Looking Glass: Analyzing the interplay between memory, disk, and read performance. numbers. I decided to give `mongodb` a test. However, I achieved only 2,000 readouts and 3,000 OPS at peak. Humanly speaking, it took 50 seconds to randomly access 100,000 records (2,000 records per sec throughput), by an index from a database whose cache fully fits into a memory.
How to re-create the environment:
Find M1 Mac, 16 Gb RAM, 1 TB SSD;
Install nodejs;
Download Docker Desktop. Allocate 8 GB of RAM and 10 CPUs in settings;
Spin up a mongo server on docker with enough RAM to cover it's cache size: `docker run --name some-mongo -d mongo --wiredTigerCacheSizeGB 3`
Use the script bellow to prepopulate a database locally with 100,000 documents, 20 kb doc size, and index it by `ref_id`:
import { MongoClient, ServerApiVersion } from "mongodb";
import crypto from "node:crypto";
const uri = `mongodb://mongoadmin:secret@localhost:27017/`;
export const client = new MongoClient(uri, {
serverApi: {
version: ServerApiVersion.v1,
strict: true,
deprecationErrors: true,
},
});
const content = crypto.randomBytes(10000).toString("hex");
const article = {
content,
};
export const fillColl = async coll => {
let batch = [];
for (let i = 1; i <= 100000; i++) {
const _newArticle = JSON.parse(JSON.stringify(article));
_newArticle.ref_id = i.toString();
batch.push(_newArticle);
}
await coll.insertMany(batch);
};
await client.connect();
const db = await client.db("articles_test_indexed");
await db.dropCollection("general");
const articlesColl = await db.collection("general");
articlesColl.createIndex(
{
ref_id: 1,
},
{
unique: true,
},
);
console.log(`Filling articles_test_indexed`);
await fillColl(articlesColl);
await client.close();Use the next script to initiate 100,000 random queries using `ref_id` index:
import { MongoClient, ServerApiVersion } from "mongodb";
import { performance } from "node:perf_hooks";
const uri = `mongodb://mongoadmin:secret@localhost:27017/`;
export const client = new MongoClient(uri, {
serverApi: {
version: ServerApiVersion.v1,
strict: true,
deprecationErrors: true,
},
});
await client.connect();
const db = await client.db("articles_test_indexed");
const articlesColl = await db.collection("general");
console.log(`Reading 100,000 random records, in articles_test_indexed by 'ref_id'`);
const t0 = performance.now();
const promises = [];
for (let i = 1; i <= 100000; i++) {
const randomNumber = Math.floor(Math.random() * 100000).toString();
const promise = articlesColl.findOne({ ref_id: randomNumber });
promises.push(promise);
}
await Promise.all(promises);
const t1 = performance.now();
console.log(`Total Execution time: ${t1 - t0} milliseconds`);
// Total Execution time: 52492.09483242035 milliseconds, ~52 seconds
await client.close();Results: Total readout time: ~52 seconds. 2000 reads per 1 second.


