Just noticed that you can preview files when selecting a file and pressing the space bar on the keyboard. It works for images, docs, audio, videos and even PDF files. You can simply close the preview using ESC. Really handy!
The responsible tool is called nemo-preview.