The values at the second last layer of a classifier serve as the vector of the object.
A classifier classifies an object. Millions of food images go in and four outputs come out: fruit, vegetable, cooked food or something else.
Inside are layers of neurons, each connected to the next:
- The last layer. It gives the probabilities of the four classes and spikes the highest to one.
- The second last layer. The image has been compressed to its useful features, and the numbers here decide the classification.
The values of these neurons, put in a one-dimensional array, are the vector. An image of 100 KB becomes an array of numbers.
A cat and a dog share features, so their vectors are similar. A chimpanzee is not a pet, so its vector is not.
A classifier of pet or wild keeps the vectors in a 1D space. All animals by type need n dimensions.
This is the logic of why vectors work. Word2vec was inspired by it, and more efficient algorithms minimize loss in the representation directly.