NOTE: not maintained (by me) since 3.8 or so, looking for new maintainers! Contact me through the repo. You may have some luck with changing the version in 'metadata.json' to your gnome-shell version. Shows workspace names in the workspace switcher popup. Edit stylesheet.css to change the look. Note - doesn't work with the 'workspace grid' extension, that has its own option to show the labels.

Extension Homepage

https://bitbucket.org/mathematicalcoffee/workspace-labels-gnome-shell-extension

Download

Shell version… Extension version…