NVIDIA Interview Question
What is Tensor Parallelism (TP)?
Interview Answer
Anonymous
Jun 5, 2026
It is a distributed computing technique used to split the parameters (weights and biases) and computations of a single neural network layer across multiple GPUs.