r/DistributedComputing • • Nov 08 '23

CUDOS Promotes Sustainability 🍀

3 Upvotes

CUDOS promotes sustainability through its innovative approach to cloud computing. By leveraging blockchain technology, CUDOS aggregates computing resources from data centers powered by renewable energy sources, reducing the carbon footprint of digital operations. This approach not only curtails environmental impact but also enhances cost-efficiency and accessibility. CUDOS ensures that AI, decentralized technologies, and other compute-intensive tasks can thrive in a sustainable ecosystem. Its decentralized network guarantees flexibility for data security and compliance, offering a responsible pathway to technological advancement. In essence, CUDOS spearheads a transformative shift toward sustainable digital infrastructure, allowing businesses and individuals to embrace cutting-edge technology while safeguarding our planet's resources. Embrace the future with r/Cudos_Official! https://cudos.org


r/DistributedComputing • • Nov 02 '23

Choreography, or Orchestration, That is the Wrong Question

Thumbnail
diagrid.io
3 Upvotes

r/DistributedComputing • • Oct 29 '23

Important update of BOINC for MacOS Sonoma

Thumbnail
x.com
3 Upvotes

r/DistributedComputing • • Oct 19 '23

CUDOS: Decentralized Cloud Computing Power for the Future

3 Upvotes

CUDOS presents a new frontier in cloud computing, offering a decentralized platform that harnesses the power of distributed networks for computational purposes. This platform can potentially revolutionize how we access and use computing resources, with its scalable architecture capable of handling various workloads, including AI and scientific simulations. What sets CUDOS apart is its marketplace, which enables users to buy and sell computing power, creating a dynamic and decentralized ecosystem. Security is a top priority, with blockchain technology and other robust measures in place to protect data and privacy. The CUDOS token drives this innovative system, incentivizing users to contribute their computing prowess. Ultimately, RCUDOS is the gateway between the vast, underutilized computational power and those who need it, paving the way for a future where computing is accessible to everyone. To learn more about r/Cudos_Official, visit: https://cudos.org


r/DistributedComputing • • Sep 19 '23

Book recommendations

3 Upvotes

Hello!

I'm a fairly senior software engineer (10y,) and I want to get more into distributes systems. I'm aware of paradigms like the actor model and some basic principles like idempotency. But I want to learn more and become more confident in distributed systems design and architecture.

I'm looking for recommendations for "canonical" literature on the subject, that will give me a solid understanding on the topic, preferably well balanced between CS and industry, if you know what I mean.

Thanks!


r/DistributedComputing • • Aug 30 '23

Decentralized Cloud Computing and CUDOS: A Transformative Duo

3 Upvotes

Decentralized cloud computing, exemplified by platforms like CUDOS, redefines how we approach data processing and resource allocation. CUDOS, a decentralized computing network, empowers users to securely share and monetize their idle computing power.

This innovative concept offers numerous benefits. It enhances the efficiency of resource utilization, reducing wastage in data centers. CUDOS also incentivizes individuals and businesses to participate in a global network, promoting a more inclusive digital economy.

Decentralized cloud computing, embodied by CUDOS, represents a paradigm shift in computing. It leverages the untapped potential of distributed resources, ushering in a new era of efficiency, accessibility, and collaboration in the digital realm.


r/DistributedComputing • • Aug 24 '23

A Simple Proof of Sybil-Proof

Thumbnail
github.com
3 Upvotes

r/DistributedComputing • • Jun 22 '23

BOINC 7.22.2 Release

Thumbnail
self.boinc
3 Upvotes

r/DistributedComputing • • Apr 18 '23

Create a distributed database cluster with Kubernetes in two easy steps

Thumbnail opensource.com
3 Upvotes

r/DistributedComputing • • Mar 29 '23

Application data sharding techniques and examples

3 Upvotes

Let’s say you have a list of tasks and the size of the list is huge > 200 mln elements.

Tasks need to be loaded into memory(cache) when application(s) is running. Let’s say the size of one task is 50KB and for 200 mln tasks we will need a machine with 10 terabyte of memory. Even if there is a single machine with that amount of memory, running the application in one machine is not safe and there are many other problems related to that like scalability, resource utilization, etc.

But we can shard the tasks and distribute among many smaller machines.
How to implement that sharding part? Obviously, the implementation requires adding more components like membership/peer discovery services, consensus algorithms and others to the stack which is ok. Is there any open source project which implements the similar functionality?


r/DistributedComputing • • Mar 22 '23

Top distributed systems conferences/journals

3 Upvotes

I'm looking for cutting edge research work in distributed systems for my research synopsis. Can someone recommend me some journal/conference.

Thank you


r/DistributedComputing • • Feb 10 '23

OpenFaas workflow engines

3 Upvotes

Folks, looking for open-source alternatives to AWS Step Functions in Kubernetes + OpenFaas land. Have come across faas-flow, yet the project does not seem to have state built in.

What serverless orchestration engines are available on the market today, and which event brokers (kafka/sqs/amqp) do they support?

Am hoping to hear that Apache Airflow is not the only option, not faas-flow the only open-source orchestrator.

Thanks!!


r/DistributedComputing • • Feb 03 '23

Daskqueue: Dask-based distributed task queue

3 Upvotes

I started working on a distributed task queue library a few months back. The library is available as a python package to install a start using : daskqueue - pypi package

For all its greatness, Dask implements a central scheduler (basically a simple tornado event loop) involved in every decision, which can sometimes create a central bottleneck. This is a pretty serious limitation when trying to use Dask in high-throughput situations.

Daskqueue is a small python library built on top of Dask and Dask Distributed that implements a very lightweight Distributed Task Queue. Daskqueue also implements persistent queues for holding tasks on disk and surviving Dask cluster restart.

I also wrote an article about implementation details: https://medium.com/@aminedirhoussi1/daskqueue-dask-based-distributed-task-queue-6fb95517dfea

Hope you enjoy it, can't wait to hear about your feedback :) !


r/DistributedComputing • • Jan 12 '23

Gridcoin was going up while Bitcoin was going down, Why?

3 Upvotes

Anyone have any ideal's

I've been watching GRC since november. Almost sold a bundle in early december for a special grow project i started. Lucky I didn't. I couldn't believe irt started popping in mid december doing the exact opposite of the whole crypto market, now it's still riding high.

I know the volume is low, today it says 60k but there has always been speculation on bot action on the exchanges. It will be interesting to see it it will take a ride with BTC or or what.

Anyone heard any real news on GRC, i'll do some reading over the next week and see if I can find anything. Glad been crunching it for a while now.

https://www.coingecko.com/en/coins/gridcoin-research

All the best in 2023 : Let's go Gridcoin!

SystemViper

XtremeSystems

XS Medical Cannabis Grower Project


r/DistributedComputing • • Dec 09 '22

Pythonic Supercomputing: Combine Taichi and MPI4Py to implement parallel + distributed computing and achieve satisfactory scalability

Thumbnail docs.taichi-lang.org
3 Upvotes

r/DistributedComputing • • Jun 30 '22

BoincGames.com BOINC fun and interesting for all involved.

3 Upvotes

Hey Ya'll, Today is the last day of Q2 for BoincGame.com
I've been busy trying to get XS points in all available projects.
Very interesting trying to run everything a little.
I also didn't want to speak that out loud because of the off chance
that we could get 3rd for for Q2 and not wake up any people not watching.
So as we go into the final Q2 results day, XS has a chance for 3rd, with P3D right on our tail.
One of the best things to come out of this is that other XS members have signed up.
Including myself we are 6 strong, I don't know how much they will be involved,
but just by the act of signing up is fantastic! XS will get any points they choose to run
Another great thing is it will really test BG for load and usability.
I look at the 2022 BoincGames.com Season as it's pre-release testing phase.
Management is very open to discussions to make it better and I am
hoping to help with that , So join, help out and mold a better and
more fun Distributed Computing experience.

Nuff for now, waiting on today's results / The Q2 final results.

https://ibb.co/album/8NtRPP

Enjoy your day!
SV

XtremeSystems

www.XS4S.com

PS: let's go WCG we miss you!

www.BoincGames.com


r/DistributedComputing • • Apr 14 '22

Message durability and quality of service across a large-scale distributed system

Thumbnail
ably.com
3 Upvotes

r/DistributedComputing • • Aug 03 '21

When ReactiveX meets Ray

3 Upvotes

I am currently working on parallelizing RxPY computations on a ray cluster. I published a dedicated library to integrate both in a ReactiveX pipeline:

https://github.com/maki-nage/rxray/

It supports distributing the computation of each even either in a round-robin way or with partitioning (this is needed for stateful transforms).

I am interested in some feedback from people working on python stream processing on top of ray. Is there anybody working on similar topics?


r/DistributedComputing • • Jul 18 '21

Market for renting out personal GPU power when not using it?

3 Upvotes

I'm considering buying a few RTX 3090s for personal use, with the idea that I could rent out the resources by hour/day/week for parts of the year. In theory, I don't see why not. In practice, I can see many reasons it people wouldn't want to use it -- privacy, access security, uptime & reliability, reputation, and general customer support issues.

However, I don't need a reliable customer pool, and I would simply be doing it occasionally to offset some of my costs, not to turn it into a profitable business. I imagine researchers at universities might a be a prime customer source for a couple months at a time, if they can convince their overlords to pay me. I'd put it all under an LLC to make it a B2B-like transaction.

If I were to get 3x RTX 3090s, it would have 50% more compute (based on CUDA cores) with slightly more total memory than a p3.8xlarge on AWS (4x v100s), which costs $12.24/hr on-demand. I'd be happy to rent it out for a fraction of that price.

For reference, it would be an isolated system (running nothing else), probably behind a VPN with port forwarding and some routing table magic to keep it isolated. I'm sure I could combine docker and/or user accounts to further isolate the environment. I don't have it all those details worked out yet, but I have the background to figure it out.

So my questions are:

  1. Has anyone else done this (either buyer or seller)?

  2. Is there a service where you could add your computing power to a pool so I wouldn't have to do this myself?

  3. I'm not offering it yet, but I'd be interested to hear if this is intriguing to anyone here.


r/DistributedComputing • • Jul 13 '21

Serverless Kafka Stream Processing with Python

3 Upvotes

r/DistributedComputing • • May 17 '21

Easy Autoscaling of Clusters with Ray's Python API

Thumbnail
anyscale.com
3 Upvotes

r/DistributedComputing • • Apr 07 '21

Ray Summit 2021 is a free conference on distributed machine learning and python

Thumbnail
anyscale.com
3 Upvotes

r/DistributedComputing • • Mar 23 '21

Kiwi.com: Nonstop Operations with Scylla Even Through the OVHcloud Fire

Thumbnail
self.nosql
3 Upvotes

r/DistributedComputing • • Mar 23 '21

Executing a distributed shuffle without a MapReduce system

Thumbnail
medium.com
3 Upvotes