Tensors - Part I: What is a Tensor?

the word “tensor” is shrouded in mystery. the same term is applied to many different things that don’t appear to have much in common with each other.

you might have heard that a tensor is a box of numbers. just as a matrix is a rectangular collection of numbers, a tensor could be a cube of numbers or even some higher-dimensional set of numbers.

you might also have heard that a tensor is something that has upper and lower indices, such as the riemann tensor above, things that have arcane manipulation rules such as “einstein summation.”

or you  might have heard that a tensor is something that changes coordinates like a tensor. a tensor is as a tensor does. something that behaves the right way under certain changes of variables is a tensor.

and then there are things that aren’t called tensors, but they have tensor products . these seem simple enough in some cases—you think “i didn’t realize that has a name. so it’s called a tensor product. good to know.” but then in other cases, tensor products seem more elusive. if you look in an advanced algebra book hoping for a simple definition of a tensor product, you might be disappointed and feel like the book is being evasive or even poetic because it describes what a tensor product does rather than what it is . that is, the definition is behavioral rather than constructive.

what do all these different uses of the word “tensor” have to do with each other? do they have anything to do with the tensorflow machine learning library that google released recently? that’s something i’d like to explore over a series of blog posts.