PyTorch is one of the two main frameworks in which Neural Networks are built, the other one being Tensorflow.
The fast.ai course uses PyTorch.
PyTorch vs Tensorflow
Backed by Facebook
Backed by Google
slightly less popular, but growing quickly
slightly more popular, but growing less quick
The main difference between PyTorch and Tensorflow is that PyTorch builds and evaluates a dynamic graph, whilst Tensorflow builds a static graph. A dynamic graph is especially useful in Recurrent Neural Networks (RNN's), used often in Natural Language Processing and other context dependent tasks.
There are some other additional cosmetic differences (naming conventions and such are generally regarded to be better in PyTorch), but the performance of both frameworks is nearly identical; they both use Numpy to perform the underlying calculations.