DGX spark is a dev or poc box. It was meant to develop, test, deploy solutions-before deploying it on gpu clusters. The reason they made it only two as a cluster is for you to test concurrency not to inference some model.

Here is what DGX was designed for “CUDA kernel prototyping and tuning , end-to-end model training and fine-tuning, large-batch hyperparameter search, distributed training experiments with NCCL, data preprocessing and ETL acceleration with GPU-aware Spark, feature engineering at scale on GPU dataframes, building and testing custom PyTorch/TensorFlow ops, GPU-accelerated graph analytics (GraphX/GraphFrames/RAPIDS cuGraph), simulation workloads (Monte Carlo, physics, finance, risk), reinforcement learning training loops, synthetic data generation and augmentation, GPU-accelerated SQL / lakehouse query benchmarking, experimentation with new parallelization strategies (tensor/model/pipeline parallel), multi-GPU performance profiling and bottleneck analysis, I/O and data-pipeline stress testing for future production clusters, validating mixed-precision and quantization strategies, developing and testing custom CUDA libraries for internal use, benchmarking different storage formats and compression schemes on GPU, building internal GPU-accelerated analytics dashboards and reports, prototyping real-time streaming pipelines with GPU operators, running large-scale unsupervised learning (clustering, PCA, topic models) on GPUs”