Introduction

PeerMR is a service that makes it easy to process large datasets in a distributed fashion across a cluster of computers. PeerMR, along with connected peer or worker nodes, provides a distributed file system and compute layer over a managed pool of web browsers. Functionality is exposed via a JavaScript SDK that allows data scientists and practitioners to execute distributed machine learning and data processing jobs without the need to manage any infrastructure.

Use Cases For PeerMR

• Processing large datasets without the need to manage any infrastructure.
• Customers of cloud providers that are running Hadoop, Spark or other map-reduce-like jobs who want to reduce costs by porting them to PeerMR.
• Training multiple machine learning models in parallel using TensorFlow.js, mljs or other Javascript/WASM-based machine learning code.
• Training a single Tensorflow.js model across multiple CPU or GPUs in parallel using our fork of TensorFlow.js which provides primitives for distributed training without any code changes to the TensorFlow.js-based code.
• Distributed WebGPU compute. WebGPU is a new web standard that provides low-level access to the GPU. You can specify minimum constraints for the WebGPU adapters if your application requires a certain level of performance.

Non-Use Cases For PeerMR

• Training a model in parallel using Python PyTorch or TensorFlow code. These frameworks do not run in web browsers. It is, however, possible to convert an existing PyTorch or TF model to ONNX and use the ONNX.js library with PeerMR to distribute inferencing. You can also port the training code to TensorFlow.js and distribute training with PeerMR. Our fork of TensorFlow.js provides primitives for distributed training of a single model across multiple browsers with no code changes to the TensorFlow.js-based code.
• Distributing CUDA, ROCm or other native GPU code. However, any existing GPU code can be converted to WebGPU. WebGL is another option, but it's harder to determine if a particular web browser is leveraging an actual GPU or a software renderer and so speedups are not guaranteed.

Contact Us

Hire our team of data science and ML practitioners to help you onboard a new use case or port your existing PyTorch or Tensorflow models to PeerMR. https://www.peercompute.com/contact