Yeah you can do it through the Steam Settings.
https://help.steampowered.com/en/faqs/view/4DBA-E6A9-1115-7852
How to add non-Steam music to the music player? If you have music files on your computer, you can sync them with your Steam Music Player.
First, go to your music settings: Steam > Settings > Music : There you can specify a folder on your hard drive for Steam to scan for any supported music files. Once scanned they will be available in your music player.
What music file formats does Steam support Currently we support MP3, FLAC, OGG Vorbis, and M4A (both AAC and Apple Lossless) files.
Thank you for the response. I think that help faq is outdated. I can’t find where to specify a folder. I’m not sure if they removed it or moved it else where.