r/DistributedComputing • u/elmariac • Jun 15 '25
MiniClust: a lightweight multiuser batch computing system
MiniClust : https://github.com/openmole/miniclust
MiniClust is a lightweight multiuser batch computing system, composed of workers coordinated via a central vanilla minio server. It allows distribution bash commands on a set of machines.
One or several workers pull jobs described in JSON files from the Minio server, and coordinate by writing files on the server.
The functionalities of MiniClust:
A vanilla minio server as a coordination point
User and worker accounts are minio accounts
Stateless workers
Optional caching of files on workers
Optional caching of archive extraction on workers
Workers just need outbound http access to participate
Workers can come and leave at any time
Workers are dead simple to deploy
Fair scheduling based on history at the worker level
Resources request for each job