How to Buy Distributed Training (SN38) on LBank

Distributed Training (sn38) is a decentralized subnetwork within the Bittensor ecosystem focused on the collaborative training of large language models. The project aims to provide a trustless alternative to the centralized approach currently dominated by major technology corporations. By utilizing a global network of independent participants, it seeks to lower the high cost and resource barriers that typically prevent smaller entities from developing state-of-the-art artificial intelligence models. The technical foundation of the project relies on a distributed approach to machine learning. It uses a specific library called Hivemind to facilitate the process of linking thousands of individual computers across the internet to train a single, unified model. This system employs a method known as butterfly all-reduce for gradient averaging. In this setup, individual miners perform local training on assigned segments of a dataset, such as the Fineweb dataset from Hugging Face. Once local updates are calculated, the network coordinates to average these updates, resulting in a global model that reflects the collective work of all participants. There are two primary roles within this ecosystem: miners and validators. Miners provide the high-end computing power, bandwidth, and low-latency connections necessary to execute training iterations. They are responsible for processing data and contributing to the shared model state. Validators manage the integrity of the network by ensuring that miners are performing legitimate work. They coordinate the training steps, handle model state updates, and evaluate the quality of contributions based on deterministic validation methods. The broader vision of the Distributed Training project is to democratize the development of foundation models. By creating an incentivized landscape where computing resources are pooled together, the project hopes to create models that are not governed by a single entity. This approach addresses concerns regarding AI alignment and censorship by allowing for a more transparent and open-source path toward high-level intelligence. The project remains actively developed on platforms like GitHub, where the community works on improving network stability, reducing communication overhead, and scaling the size of the models being trained. Learn more