Exercise 4 (Pitfalls in t-SNE)
By Anoushka Jain, Ehsanullah Hafizyar, and Christian Bungartz
The perplexity can be interpreted as how many points in the neighbourhood(whose distance) the algorithm tries to preserve, setting perplexity to be equal to the number of points it will try to preserve the whole structure of the dataset and not just neighbours. By trying to preserve the structure it does a terrible job.
This happens because t-SNE tends to expand denser regions of the data. The algorithm magnifies the area in the middle of the square since the there is less empty space around them than the ends, so the algorithm magnifies them.
This happens because very low values of perplexity tries to preserve local effects. Small perplexity suggests that the algorithm focuses on smaller neighbourhoods and this breaks the algorithm into small meaningless clusters
For perplexity 80, the visualisation resembles the input data. Since the shape and distribution of points is pretty complex, larger values of perplexities tend to help in preserving neighbourhoods by improving global connectivity of the topology.