CUDA Neural Network Inference.
A neural network built from scratch to compare sequential CPU inference with CUDA GPU inference on large synthetic datasets.
Implementation
A 2–8–1 neural network predicts whether a student passes a subject from attendance hours and time spent on a final project. The synthetic dataset is a GPU-parallelisation case study, not a realistic predictive model.
Training uses backpropagation on the CPU. For GPU inference, each CUDA thread processes one sample, with the trained network held in constant memory. Measurements distinguish kernel execution from total GPU inference, including allocation and memory transfers.
Benchmark
The repository’s benchmark results report 34.16× end-to-end inference speedup at 200 million samples on an Intel Core i7-9700F and NVIDIA GeForce RTX 2080 Ti.
| Samples | CPU inference | Total GPU inference | Speedup |
|---|---|---|---|
| 1 million | 84.102 ms | 67.056 ms | 1.25× |
| 10 million | 812.631 ms | 87.003 ms | 9.34× |
| 200 million | 16,507.000 ms | 483.276 ms | 34.16× |
These timings exclude CPU training. Transfer and allocation overhead make small datasets slower on the GPU; the benefit increases with dataset size.
Source code and Gnuplot scripts are available in the CUDA neural inference repository.