r/DistributedComputing • • Jun 15 '25

MiniClust: a lightweight multiuser batch computing system

MiniClust : https://github.com/openmole/miniclust

MiniClust is a lightweight multiuser batch computing system, composed of workers coordinated via a central vanilla minio server. It allows distribution bash commands on a set of machines.

One or several workers pull jobs described in JSON files from the Minio server, and coordinate by writing files on the server.

The functionalities of MiniClust:

  • A vanilla minio server as a coordination point

  • User and worker accounts are minio accounts

  • Stateless workers

  • Optional caching of files on workers

  • Optional caching of archive extraction on workers

  • Workers just need outbound http access to participate

  • Workers can come and leave at any time

  • Workers are dead simple to deploy

  • Fair scheduling based on history at the worker level

  • Resources request for each job

3 Upvotes

0 comments sorted by