Hello.
I have been attempting to find a way to automate the generation of m3u8 URLs from streaming sites which require you to click on the video player to initiate loading the media.
I’ve found some information relating to Selenium, but haven’t used that before and haven’t had any success so I’m not sure if there are other solutions.
I’d considered generating URLs for successive videos based on apparent naming conventions, iterating over them to access one at a time, [figure out how to automatically initiate the video so the m3u8 requests get made], capture the m3u8 URL, initiate download with that URL and name each appropriately with something like yt-dlp’s autonumber.
I’ve figured out and tested options for most of these steps, but I haven’t had luck with the automated loading/initiation of the video stream in order to load the m3u8 requests. I’m still doing that step manually.
My laptop is crazy old and struggles to play video in a browser, seemingly it fills up its memory and it has crashed before. So I grab the m3u8 URLs to either load them into a local media player for streaming or download them for later, the latter especially if my internet connection is struggling as it often does.
Any advice or direction is greatly appreciated.
Thank you very much!
I have used selenium in the past. The trick is to capture all cross-loaded connections and then go through them and pattern match to the m3u8. Now selenium out of the box can’t do that, but you will be able to use seleniumwire to capture those connections from selenium.
The drawback of this approach is it takes some time to tune the script and the pattern, and although selenium can be run headless, it’s still a full browser, making it very memory- inefficient.
Depending on the popularity of the site, jdownloader might be able to capture it as well and even if it’s not fully supported, you can use their url filters to strip down the urls to only capture the m3u8. However this is kind manual, so it’s just more efficient than clicking but not automated. I guess maybe if you get all the urls in a big list and then let jdownloader iterate over all of them it’s more efficient.
Maybe yt-dlp has some options for fetching the m3u8 from a base url but I’m not so sure.
Basically the issue is you are fighting against the developers, who want you to view but not download the content, and thus they pour some time into making sure that you are actually visiting the site and not ripping the content. So it’s always gonna be difficult and the seleniumwire method is kind of the best I ever got.
This is not easy, so if you figure it out do please open source it so others can benefit too :)
I usually use VideoDownloadHelper addon in Firefox to get the download url, and then use yt-dlp to download it. However, this still requires me to navigate the page.
