A neural network data multiplexing system for simultaneous processing and disentanglement of multiple input instances

Background

The field of deep learning has witnessed remarkable progress in recent years, with neural networks achieving state-of-the-art results across natural language processing, computer vision, and other domains. As models have grown in size and complexity, there has been a corresponding increase in computational demands, particularly during training and inference. This has led to a pressing need for methods that can improve the throughput of neural networks—allowing them to process more data in less time—without requiring significant hardware upgrades or sacrificing accuracy. Efficient utilization of computational resources is critical for both research and deployment, especially as applications scale to larger datasets and real-time requirements.

Current approaches to increasing neural network throughput primarily rely on batch processing, where multiple input instances are processed in parallel but independently within the same model. While batching leverages hardware parallelism, it does not fundamentally reduce the number of forward passes required, nor does it address the redundancy in processing similar patterns across inputs. Other methods, such as data augmentation or mixup, combine inputs for regularization but are not designed for simultaneous, disentangled prediction of multiple instances. These techniques often face limitations in scaling, can introduce interference between inputs, and may degrade model performance when pushed beyond modest batch sizes. As a result, there remains a significant gap in methods that can truly compress and process multiple independent inputs together in a single forward pass while reliably recovering accurate, individual predictions.

Technology

Data multiplexing in neural networks is a technique designed to significantly boost the throughput of deep learning models by allowing them to process multiple independent input instances simultaneously within a single forward pass. This is achieved by first applying fixed linear transformations—such as random orthogonal matrices or Hadamard products—to each input, mapping them into distinct subspaces, and then aggregating them into a single, mixed representation. This combined input is processed by a standard neural network backbone (such as a Transformer, MLP, or CNN), after which a demultiplexing layer recovers the individual outputs. The demultiplexing can be performed using specialized multi-layer perceptrons or index embeddings, which help disentangle the mixed data. To further enhance the model’s ability to separate the multiplexed inputs, a self-supervised retrieval warmup task is used during training, encouraging the network to reconstruct the original features from the compressed representation. This architecture leverages the overparameterization typical of modern neural networks, enabling substantial increases in throughput across a variety of tasks and model types.

What differentiates this technology is its ability to compress and process multiple inputs as a single representation, going beyond traditional batch processing or data augmentation techniques like mixup. Unlike batch processing, which simply processes multiple instances in parallel, data multiplexing actually combines the data into a single computational path, allowing for dramatic increases in throughput—up to 20 times more inputs per computation in some cases—without significant loss in accuracy. The use of fixed linear transformations ensures that each input occupies a distinct subspace, minimizing interference, while the retrieval warmup task guarantees effective disentanglement during training. This approach is validated across NLP and computer vision domains, showing minimal performance degradation even as the number of multiplexed instances increases. By reducing computational time, energy, and cost, this technology provides a scalable and efficient solution for deploying large-scale neural networks in real-world applications.

Benefits

• Significantly increases neural network throughput by enabling simultaneous processing of multiple inputs in a single forward pass.
• Reduces computational time, energy consumption, and operational costs without substantial accuracy loss.
• Applicable across diverse neural architectures including Transformers, MLPs, and CNNs, and various domains such as NLP and computer vision.
• Uses fixed linear transformations to map inputs into distinct subspaces, minimizing interference and preserving input order for effective disentanglement.
• Incorporates a self-supervised retrieval warmup task to improve model convergence and accurate recovery of individual inputs from multiplexed representations.
• Supports multiplexing of up to 40 inputs in Transformers with minimal performance degradation, enabling up to 18x throughput gains.
• Flexible demultiplexing strategies (index embeddings or MLPs) allow scalable recovery of individual outputs from the combined representation.
• Leverages model overparameterization and theoretical foundations to maintain performance while dramatically increasing processing efficiency.

Applications

• High-throughput NLP inference services
• Real-time image classification pipelines
• Efficient large language model serving
• Accelerated autonomous vehicle perception
• Scalable multi-task learning platforms

Additional info

This framework enables simultaneous processing of multiple inputs by compressing them into a single representation. Multiplexing layers apply fixed linear transformations, mapping inputs into distinct subspaces. After a single forward pass, demultiplexing layers recover individual outputs using index embeddings or MLPs. A self-supervised retrieval warmup facilitates disentanglement, significantly increasing throughput in Transformers and CNNs by leveraging overparameterization to process multiple instances concurrently.