All the code snippets in the previous section were written in Python. Developers prefer Python for its clean syntax, whereas C++ runs much faster and provides direct control over memory and hardware. To combine both strengths, frameworks pair a Python frontend with a C++ backend.

Python is an interpreted language: instead of being compiled ahead of time, a Python script (.py) is read and executed line by line by the Python interpreter at runtime, as shown in the following figure. This makes interactive development and debugging easy, but the interpreter is too slow for heavy tensor math and cannot directly control GPU hardware.

In contrast, C++ (along with CUDA) is a compiled language. Instead of interpreting code line by line at runtime, a compiler translates the source files ahead of time into a binary shared library (.so), as shown in the following figure. This .so binary contains both compiled CPU machine code and compiled GPU kernels, which execute directly on the CPU and GPU at runtime without an interpreter.

In a machine learning framework, the C++ and CUDA backend is pre-compiled into .so shared libraries before the framework is distributed. At runtime, as the Python interpreter steps through user code line by line, it calls into these .so binaries via language bindings instead of performing the tensor operations in Python, as shown in the following figure.