Revision history for StreamFinder 1.00 Date/time First version, released on an unsuspecting world. 1.01 2018/08/10 - Fixed issue with some Facebook videos on Youtube. - Removed Tunein as a valid site (can no longer extract streams). 1.02 2019/01/31 - Make StreamFinder::Youtube the default and handle pretty much anything youtube-dl handles, ie. brighteon, vimeo, etc. 1.03 2019/08/15 - Fix StreamFinder::Reciva now failing to connect to server (added wget as fallthrough as LWP::UserAgent and curl fail with unsupported SSL protocol errors). - Fix StreamFinder.IHeartRadio now failing to grab images / icons. - Strip off "?autoplay=true" from StreamFinder::Youtube urls. - Minor doc fixes. 1.04 NOT RELEASED - Make debug option work for all stream types, and allow specifying as either "debug", "-debug", "debug => 1", or "-debug => 1" in parameter list for new() (ie. my $station = new StreamFinder($url, -debug => 1); - Better handle ".m3u8" streams returned by brighteon.com (Youtube). 1.05 2019/09/03 - Add StreamFinder::BannedVideo (https://banned.video/) to handle their video streams (which youtube-dl doesn't currently handle). 1.10 2019/09/07 - Add user-configuration files (~/.config/StreamFinder/config, ~/.config/StreamFinder//config (see docs). User can now specify / override most LWP::UserAgent options, ie. "agent", "timeout", etc. in the new configuration files. User must create these files, if needed with a text editor. - Greatly reduce falling back to wget by adding default user-agent and checking for wget availability. Specify an "agent" => "your agent" line in the user-configuration file to change. - Add "-random" and "-noplaylists" options to $station->getURL() (see docs) - StreamFinder::BannedVideo: Add "-keep" option to better control type and order of streams returned (see docs). - StreamFinder::Facebook: Depreciated ~/.config/.fbdata for new user-configuration files. - StreamFinder::IHeartRadio: Add podcast-playing capability. - StreamFinder::IHeartRadio: Depreciated old stream order options, replacing them with new -keep and -skip arguments and configuration file entries. - StreamFinder::RadioNet: (https://www.radio.net/) ADDED - StreamFinder::Tunein: RESTORED (now uses StreamFinder::Youtube for stream fetching), but adds metadata loading (which youtube-dl does not currently fetch for these streams. - StreamFinder::Youtube: Better handle "chunky" brighteon.com .m3u8 video streams. Also add "-notitle" option, streamlining use by StreamFinder::Tunein (see docs). 1.11 2019/09/07 - StreamFinder::Youtube: bugfix for brighteon.com m3u8 video URLs. - Docs: fix a cpl. typos, actually include the updated README file. 1.12 2019/09/10 - StreamFinder::Tunein: Add podcast-playing capability. (Will look for "playUrl" before falling back to youtube-dl for stream). - StreamFinder::Reciva: Fix to make LWP::Useragent fetch reciva URLs using the ANCIENT TLSv1 to allow to work without falling back to wget. See the StreamFinder::Reciva docs for creating a one-line custom config file to do this - I didn't make it the default since they could FIX this at ANY time, which would break things! - Add URI::Escape and HTML::Entities as DEPENDENCIES - in order to ensure that the returned stream titles are un-escaped. - Fix $station->getID('fcc') to work properly (return the station's FCC call-letters for sights and stations that support it. - Set $station->{artist} and $station->{year} where applicable. - Minor doc updates. - Fix some sight-specific submodules to properly accept just a station / video ID rather than a full URL (see docs). - Add "desc" option go getTitle() to fetch station's (long) description (for sites that provide that field), otherwise, the title is returned. 1.13 2019/09/11 - Fix missing (recently-added) file (RadioNet.pm) in MANIFEST file. - More doc. fixes, mostly to the individual submodules. 1.20 2019/09/16 - Removed usage and all references to WWW::Youtube::Download - This module no longer fetches the "author" (Youtube user) nor does anything else we can't more easily do manually. - StreamFinder::Apple: (podcasts.apple.com) ADDED! - StreamFinder::Brighteon: (brighteon.com) ADDED (split off from StreamFinder::Youtube). - StreamFinder::Vimeo: (vimeo.com) ADDED (split off from StreamFinder::Youtube. We separated these for better support - ie. ability to get the "author" fields and access some additional streams. We also can often avoid having to call youtube-dl. - StreamFinder::Youtube: Removed "notitle" option, added "fast" option, and add "description" field fetch to youtube-dl. Fixed url-fetching order to call youtube-dl FIRST to prevent infinite stall if called with an already-valid stream-url (Fauxdacious mediaplayer). StreamFinder::IHeartRadio - Fix sometimes-corrupted ID field. StreamFinder::Reciva - Fix failure to always capture ID field if given wrongly-formatted full url. - Add $station->{'genre'} field to IHeartRadio, RadioNet, Reciva, and Tunein (the sites that support it). - Various small code and doc. tweaks / cleanups. - DEBUG was accidently left set to ON, now turned OFF by default! 1.21 2019/09/18 - Significant documentation improvements & polishing. - StreamFinder::BannedVideo: Fix incorrect IDs returned. - StreamFinder::Tunein: (new()) - Fix mishandled URL if only station-ID specified (non-full URL). - StreamFinder::Apple: (new()) - Handle if only podcast-ID specified. 1.22 2019/10/07 - Minor bugfixes, namely handling double-quotes in titles in some submodules. - Minor doc. improvements. 1.23 2019/11/02 - StreamFinder::BannedVideo: Fix to handle change in their site - to execute extra step now needed for infowarsmedia.com videos and make the retrieved mp4 stream the "best" stream. 1.24 2019/11/06 - StreamFinder::Youtube: Eliminate express page scan for metadata for non-Youtube sites processed to prevent ocassional infinite loop caused by attempting to scan a media stream instead of a website. - Unescape HTML entities in the Genre string in Reciva & Tunein. - Minor doc. improvements. 1.25 2019/11/08 - StreamFinder::Brighteon: Fix another site-change that caused blank titles to be returned, and broaden title and description searching ability. 1.26 2019/11/19 - Correctly handle "HOME" directory in M#-Windows. - StreamFinder::IHeartRadio - Handle non-US site URLs. Change default "keep" priority to "secure_shoutcast, shoutcast, secure, any" for easier use by Fauxdacious. 1.27 2020/04/23 - Added StreamFinder::Spreaker for https://widget.spreaker.com podcasts. - Minor doc. improvements. 1.28 2020/04/25 - Fix failure to include StreamFinder::Spreaker in MANIFEST and to update change log. 1.29 2020/07/02 - Split out video's channel page (that was previously appended onto the "artist" field) into a new, separate "albumartist" field. This affects the video sites: Brighteon, Vimeo, and Youtube. (Old way was: artist:="Artist - Artist's video channel URL"). These are now put into separate fields. - Fix .pls playlists returned by some Tunein stations by stripping off some garbage Tunein appends in the format: "stream.pls?DIST=TuneIn&TGT=TuneIn&maxServers=..." and more accurately obtain title, artist, and album fields for stations vs podcasts. - Fix issue causing some vimeo videos to fail to fetch stream. - Clean up some "dirty" Spreaker titles (convert invalid characters). - Marked StreamFinder::Radionomy depreciated - site appears defunct. 1.30 2020/07/05 - Add "-nopls" option to getURL() to parse ".pls" playlists and return only a single entry, like "-noplaylists", but pass others (".m3u" and ".m3u8") through unchanged. This needed because most ".m3u*" are playable "HLS" streams, whereas ".pls" playlists are actually playlists and many players will not play them as-is. - StreamFinder::RadioNet: Fix issue with no streams being fetched for some stations & clean up titles a bit. - StreamFinder::IHeartRadio: Add year to podcasts, fix some title and description issues. - StreamFinder::Spreaker: Fix some streams not working, fix "-debug" option to work. 1.31 2020/08/10 - Add StreamFinder::Blogger (www.blogger.com) for Blogger videos. - Add StreamFinder::Bitchute (www.bitchute.com) for Bitchute videos. 1.32 2020/10/16 - Remove StreamFinder::BannedVideo as banned.video has become so paranoid as to add sophisticated anti-bot/scraping detection that I have been unable to figure out how to circumvent. NOTE/RANT: They claim to be pro freedom / anti-big tech spying, etc. but their site has become unusable without third-party / Google javascripts, and other malware nasties we and our users are trying to avoid for our own safety! Most of their videos can now also be found on Bitchute anyway! - StreamFinder::Youtube: Fix minor bug that sometimes returned no streams. 1.33 2020/11/01 - StreamFinder::Apple: Fix some icon images not being downloaded. - Add StreamFinder::SermonAudio. - Remove depreciated StreamFinder::Radionomy (site now defunct). 1.34 2020/11/02 - Add StreamFinder::Castbox (https://castbox.fm podcasts) - StreamFinder::Spreaker: Enhance to accept podcast page URLs (like other podcast-ish modules rather then just the bare widget.spreaker... URLs. These pages also have artist and albumartist fields! Also, properly return the total no. of streams found ($podcast->count()). - "internationalize" several sites (eliminate the ".com" from the required matching regices (allow for ".eu", ".net", etc. to match"). 1.35 2020/11/04 - Fix regex glitch breaking the sites we "internationalized" in v1.34. 1.36 2020/11/16 - StreamFinder::Apple: Return correct description for podcast episodes. - StreamFinder::Spreaker: Fix minor title and image issues. - StreamFinder::Youtube: (For now): convert www.youtube.com to youtube.be to get around youtube.com seeming to block youtube-dl. 1.37 2020/11/25 - Add StreamFinder::Rumble for videos on rumble.com - StreamFinder::Castbox: Return full description for podcast episodes. 1.38 2021/01/01 - StreamFinder::IHeartRadio: Handle streamless custom radio-station urls, ie. "https://station.iheart.com" (revealed by CPAN bug# 133982). These reference the correct "live" url that contains the steams! Also fix description, artist and albumartist fields failing to return values for (some?) podcasts. - StreamFinder::Rumble: Strip the podcast ID field to just it's unique ID. - StreamFinder::Reciva will likely be removed next release after 01/30/2021 due to that site's announcement to go offline after that date! 1.40 2021/01/04 - Add StreamFinder::Google for Google (podcasts.google.com) podcasts. (closes CPAN feature# 133993). 1.41 2021/01/06 - StreamFinder::Google, StreamFinder::Castbox, StreamFinder::Spreaker, and StreamFinder::IHeartRadio: Add ability to accept a podcast / channel page without a specific episode and return the first (latest) episode for that page. NOTE: StreamFinder::Apple already does this but returns all episode streams in a list with the first episode the first stream, but the metadata returned is for the Apple podcast page, and left this way for backward compatability. 1.42 2021/04/11 - StreamFinder::Rumble: Fix failure to find any streams (Rumble chgd. their html slightly), and Add "-keep" option to limit / order the streams fetched (default: mp4,webm,any) - see updated docs. - StreamFinder::Apple: Remove music.apple.com (music samples no longer scrapable? (Much more complicated and I never use it - you only ever could get short sample clips - not full songs anyway! Also, misc. fixes/tweaks, also see last item below: - StreamFinder::Castbox: Added Album field. - StreamFinder::Spreaker: Fix failure to fetch anything (Spreaker chgd. their html a bit!). Also, added Album field, and additional check for possible AlbumArtist url. - StreamFinder::Google: Misc. fixes/tweaks, also see below: - Apple, Castbox, and Google: Add ability to fetch the first (latest?) podcast, and return all podcasts as an extended MP3 playlist (new "-playlist" option to the $station->get() function) - for podcast author's main page urls. 1.43 2021/04/11 - StreamFinder::Rumble: Fix doc glitch in SYNOPSIS section specifying the "-keep" option as a hash reference (must actually be an array reference, causing user to not get any streams if using the SYNOPSIS example as a program. Also fixed this in StreamFinder::IHeartRadio! - StreamFinder::Spreaker: Fix failure to fetch icon image for some podcasts. 1.44 2021/04/19 - StreamFinder::Rumble: If only a podcast "ID" is given, check to see if it's either the long ID for the html page or the short one used for the embed page and append the appropriate url (prev. only the long html page ID string was supported. Either way, the correct Rumble ID will be returned. - Add some default values for "genre", such as "Podcast" for Spreaker, Castbox, Apple, and "Video" for Youtube, Vimeo, Brighteon and Bitchute. - Initialize some variables that should be. - StreamFinder::Apple: Better checks for album, genre and year. - Misc. doc-fixes all over. 1.45 2021/05/04 - Add StreamFinder::Anystream - as a "fall-back" module to search any specified webpage URL for streams (streams are matched by searching for URLs matching specific known stream types, ie. ".mp3", ".ogg", etc. (The default "-keep" list is: "mp3,ogg,flac,mp4,m4a,mpd,m3u8,m3u,pls"). - Added StreamFinder option "-omit" to specify submodules that you have installed to NOT be used, ie. -omit => "Anystream,Youtube". 1.46 2021/05/05 - Oops - add StreamFinder::Anystream to MANIFEST. - Add & document new -secure option (all) to allow user to limit streams returned to (secure) https streems. - Minor doc. cleanups. 1.47 2021/05/10 - Fix potential infinite loop if unrecognized option arg. passed in. - StreamFinder::Youtube - Add fallback ability to check html page for embedded iframe containing a valid StreamFinder page, and new option: "-noiframes" to not do this (if set to 1/true). 1.48 2021/05/19 - StreamFinder::Youtube, StreamFinder::Anystream - Only look for streams in actual HTML pages, NOT media URLs - to prevent potentially very long or even infinite delay trying to fetch the URL. - StreamFinder::Spreaker - Unescape artist and album fields, remove some extraineous debug output. 1.50 2021/05/22 - Refactor all modules to combine all common methods into a separate internal module: StreamFinder::_Class (new) to greatly simplify and eliminate redundant code. This should make creating new site modules much simpler (now most if not all functions other than "new()" should not be required in site modules unless it's necessary to override)! This better follows proper object-oriented programming standards. - StreamFinder::Tunein - Add "-notrim" option to NOT trim extra "ad", misc. stuff from stream URLs (previously, and by default URLs are "trimmed" removing this stuff (after and including the first "?"). - StreamFinder::Tunein, StreamFinder::IHeartRadio - Attempt to grab full description of podcasts (may include HTML, ie. links). - Misc. minor code cleanups found while converting. - StreamFinder::Anystream - Fix bug that mangled some streams returned. Also make sure URLs returned containing "\/"s are unescaped to "/"s. 1.51 2021/07/01 - Add "-log" and "-logfmt" options to append a line to a specified log file for each successful stream-url fetch (see docs). - Fix bug that caused config file arguments with unescaped Perl regex characters in their values to be ignored. - StreamFinder::Rumble - Fix bug that caused description to be cut short. - StreamFinder::Spreaker - Handle their multiple URLs, and also fetch video streams. - StreamFinder::SermonAudio - Handle additional "alternate" URLs as well as video streams they can contain. Also try harder to find icon. 1.60 2021/10/02 - Add 'artist' option to getIconURL() and getIconData() functions, (Currently only Apple, Brighteon, and Youtube) to return the "channel" (AlbumArtist) icon instead of the individual podcast icon. We may add this to more of the other sites later, but this will likely require an additional, separate fetch of the channel page. - Small doc. touchups. 1.61 2021/10/03 - Add 'artist' icon option (see v1.60 changes) to StreamFinder::Rumble, StreamFinder::Castbox, StreamFinder::Spreaker and StreamFinder::Vimeo. 1.62 2021/10/08 - Fix getImageData() returning icon instead of image if wget used to fetch. - getImageUrl() and getImageData() can now also accept the 'artist' argument and return a separate artist / channel image, if one larger / different from the corresponding icon exists. Currenty, only StreamFinder::Odysee provides these. - Add StreamFinder::Odysee to handle odysee.com videos (requires youtube-dl). 1.63 2021/10/27 - NOT RELEASED - *** Youtube seems to now throttle video download/playback making most videos virtually unplayable through third-party players, ie. youtube-dl. The currently-suggested workaround is to use a forked version called yt-dlp, which seems to have figured out a workaround! - Update the default -agent (browser user-agent version) string to: "Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0". - StreamFinder::Anystream: Properly handle the base-url for relative links, and add first word of url following the domain-name for the "ID" to make it more unique. - StreamFinder::Youtube: Add several new config options: -youtube-dl: Allow specifying youtube-dl alternative program, such as yt-dlp (See headline above). -format, -userid, -userpw (added to new() parameter list). -formatonly, -user-agent, -youtube-dl-args, -youtube-dl-add-args: (See pod docs for descriptions and values). - Remove conversion of www.youtube.com to youtube.be (no longer seems to be needed). - Misc code and doc. cleanups. 1.70 2021/10/28 - Includes all changes made in unreleased v1.63 (rc) above. - Add ability to specify alternate location of config. files - via new environment variable: STREAMFINDER (see docs). - Refactor all youtube-dl calls to eliminate redundant code and make use of StreamFinder::Youtube configurations (primarily to allow for specifying alternate youtube-dl program) by forcing all youtube-dl usage to go through StreamFinder::Youtube. This change affects StreamFinder::Bitchute, ::Blogger, ::Brighteon, ::Odysee, ::Tunein, and ::Vimeo. This also now allows configuring individual modules to override the default Youtube module's youtube-dl parameters, of which "-format" will likely be the most overridden. This should be done in module's config files, adding "-youtube" in front of them, ie. "-youtube-format" for the above example. - StreamFinder::Odysee: Add "-youtube" option similar to some other video modules (values: yes|no|first|last|only) to enable usage of the Youtube link at the bottom of the description for (most) Odysee videos instead of the embedded info in the Odysee page. - StreamFinder::_Class: Fix bug handling option parameters, particularly ones containing a dash between words. - StreamFinder::Blogger & ::Brighteon: Change -youtube option default from "yes" to "no" to match other modules using this option. 1.71 2021/10/30 - StreamFinder::Youtube: Add ability to scrape additional metadata from Youtube pages referenced by initial non-Youtube page URLs via youtube-dl & friends' "--get-id" option, if one is returned. 1.72 2021/11/03 - StreamFinder::Youtube: Fix bug that prevented access to some videos (namely ones with non-alphanumeric characters in the youtube-id) introduced in previous release. Also fix issue with failure to fetch Youtube metadata for Youtube videos embedded in iframes in non-Youtube sites (also broke by v1.71). - Also add a -youtubeonly option to tell ::Youtube to NOT return embedded Rumble.com or other videos found in non-Youtube pages. Modified docs to more clearly explain this. - StreamFinder::IHeartRadio: Add ability to return first (latest) episode if only a podcast channel page (without episode) URL is used. Also add proper channel metadata (name, channel artist's icon) to metadata returned for podcasts. - StreamFinder::IHeartRadio: Fix Perl character-encoding issue. Also try harder to ensure podcast channel pages return the first (latest) episode, instead of the oldest. 1.80 2021/11/10 * This turned into a major release with many improvements to handling podcasts, particularly metadata fetching and handling of podcast pages and the multi-episode playlists they can return. - Refactor podcast modules to always return full metadata for the first (latest) episode for podcast pages (as if they had been called with the id/URL for the first episode itself - ie. the "description" field, etc. often had that of the podcast page itself instead of that of the first episode). Also have the podcast modules always return a full extended m3u playlist when $podcast->get('playlist') is called. This affects StreamFinder::Apple, ::Castbox, ::Google, and ::Spreaker. - Also make "genre" and "year" fields work in all of the above, and ensure the genre and album fields are included in the m3u playlist (#EXTGENRE and #EXTALB lines) in all of the above. - Also reverse "artist" and "album" fields in all of the above: I was never sure which field was which for podcasts (as most appear similar) but some actually used them for the artist(s)' names and the podcast name, and we had them reversed. - Add new "playlist" option to the $podcast->count() function to return the number of episodes in the playlist instead of the number of streams returned (for the first episode) in all of the above. - StreamFinder::Apple - Make podcast pages work again (Apple changed their code some) - episode pages were still working though. - StreamFinder::IHeartRadio (Podcasts), ::Tunein (Podcasts), and ::SermonAudio (non-Sermon pages): Add ability to accept podcast pages and return the first (latest) episode when given a podcast page URL: (IHeartRadio returned the oldest (last) one rather than latest (first) due to their podcast pages sometimes having them listed in reverse order. Tunein did not return anything for a podcast page, only for specific podcast episode pages). 1.81 2021/11/13 - Added StreamFinder::Podbean (www.podbean.com podcasts)! - StreamFinder::Bitchute, ::Brighteon, ::Rumble, & ::Youtube - Add ability to handle channel page URLs and return the first (latest) video (the same feature we just added for all the podcast sites). NOTE: Not added to ::Odysee or ::Vimeo due to those sites requiring JavaScript to obtain the video links. - Also added scraping for genre (previously hardcoded to "Video") for StreamFinder::Bitchute & StreamFinder::Youtube. 1.82 2021/11/21 - StreamFinder::Rumble - Add -quality option to limit maximum video quality (ie. 720 for 720p, 1080 for 1080p, etc. for users with limited internet bandwidth. Also fix very minor issue to ensure that non- classed streams (classed via extension) are rated in the same order as specified in the -keep option (was previously always mp4, then webm regardless of -keep order). 1.83 2022/04/28 - Add StreamFinder::BrandNewTube (brandnewtube.com videos) 1.84 2022/04/29 - Add StreamFinder::PodcastAddict (podcastaddict.com podcasts) - Improve module-selection speed/efficiency (only require in module matching URL, instead of all modules). 1.85 2022/05/06 - StreamFinder::PodcastAddict - Fix failure of some video URLs to play. - StreamFinder::SermonAudio - Fix failure to fetch some icons properly. - StreamFinder::SermonAudio - Add a "-quality" option to limit streams returned to "audio", "low", "high", or "any" (default) - see docs. - StreamFinder::BrandNewTube - Several metadata-fetching improvements. 1.86 2022/05/09 - StreamFinder::Podbean - Fix failure to fetch many current Podbean URLs, namely ones that are not custom site-hosted by Podbean (ie. https://www.podbean.com/... as opposed to https://.podbean.com/...). This appears due to Podbean possibly expanding the types of valid URLs at some point. - Update default user-agent to Firefox 99. 1.87 2022/06/07 - StreamFinder::BrandNewTube - Add support for Ugetube.com videos. Note: User must specify full ugetube.com URLs. 1.88 2022/07/30 - StreamFinder::PodcastAddict - Fix failure to capture "year" field. - StreamFinder::Rumble - Fix not capturing full video descriptions. - StreamFinder::Youtube - Add -formats_by_url => {hash} option to allow specifying different video formats allowed based on URL matching. 1.90 2022/10/04 - Add StreamFinder::SoundCloud (soundcloud.com songs) - StreamFinder::Bitchute - Fix not capturing full video descriptions, and fix failure to capture proper genre. - StreamFinder::Anystream - Handle extended M3U (HLS) playlist URLs. 1.91 2022/10/04 - Add StreamFinder::SoundCloud to MANIFEST file. 1.92 2022/10/13 - StreamFinder::Bitchute - Handle "embedded" Bitchute pages, ie. https://www.bitchute.com/embed/[id] - StreamFinder::BrandNewTube - Fix some UGetube.com sites not fetching artist or albumartist. NOTE: May get renamed to StreamFinder::UGetube in future, as it appears that BrandNewTube.com seems to be withering. - StreamFinder::SoundCloud - Fix up documentation a bit. 1.93 2022/10/21 - StreamFinder::Tunein - Use https://opml.radiotime.com/Tune.ashx JSON API instead of youtube-dl/yt-dlp to extract radio streams. (Removes dependency on youtube-dl) 2.00 2022/11/03 - Add StreamFinder::InternetRadio (www.internet-radio.com stations) - Add StreamFinder::OnlineRadiobox (onlineradiobox.com stations) - StreamFinder::AnyStream - Add .aac streams to search. - StreamFinder::Tunein - Cpl. minor cleanups. - StreamFinder::Youtube - Accept urls containing "/c/" as channel urls. 2.01 2022/11/05 - StreamFinder::OnlineRadiobox - Add -notrim option and fetch the station's callsign, ie. the "fccid". - Misc. minor code & doc. cleanups and update some copyright years. 2.02 2022/11/10 - Add -hls_bandwidth option to specify a numeric (BITS per second) limit on HLS streams returned by getURL() when possible - useful for users with limited bandwidth. - StreamFinder::Brighteon - Add ability to obtain the video's "year". 2.03 2023/02/24 - Fix debug messages that were displaying without debugging turned on. - StreamFinder::Brighteon - Convert "embed" URLs to standard ones, so that if passed an embed url, the proper stream data will be returned. 2.04 2023/02/28 - Added StreamFinder::Rcast to fetch rcast.net radio stations. - StreamFinder::BrandNewTube - Fix several issues with this rapidly- depreciating video site (more recent videos seem to be wrapped in javascript and becoming unscrapable). These fixes restore some fields for which the site has changed the code. Note: This module also handles ugetube.com videos, which seem to still work properly, and may be renamed to StreamFinder::UGETube in the future if brandnewtube.com support is eventually dropped. - StreamFinder::Odysee - Add new "-youtube=ifneeded" option to only fetch Youtube streams if youtube-dl fails to find a valid Odysee stream. Also add a new option "-nohls" to ignore HLS (.m3u/.m3u8). StreamFinder class - Fix undefined variable warning when attempting to fetch a stream URL when there are no streams matching restricted criteria, even if a valid StreamFinder object was returned. 2.10 2023/03/02 - Added StreamFinder::Goodpods to fetch goodpods.com podcasts. - StreamFinder::SermonAudio - Add "-speakericon" to reverse the two artist/channel images - SermonAudio usually provides 2 separate artist/ channel images (one for the church/channel, and one a portrait of the preacher). The -speakericon option (if set) reverses the ones returned by getIconURL('artist') and getImageURL('artist'). Also try harder to find these icon images in the sermon pages. - StreamFinder::Vimeo - Fix missing "description" (getTitle('desc')) and icon (video-preview) image due to site changes. Also do better job selecting this image, which usually comes in multiple sizes: make getIconURL() return the smallest and getImageURL() the largest. 2.11 2023/03/03 - Added StreamFinder::Podchaser to fetch podchaser.com podcasts. - Fix failure to fetch channel art on first episode if given a podcast page; misc. cleanups. 2.12 2023/03/06 - StreamFinder::Podchaser - When given a podcast page, refactor to 1) avoid a 2nd fetch of the latest episode page if we are able to get all the needed metadata from the podcast page code; and 2) Fix failure to fetch with just a podcast-ID or podcast-ID/episode-ID (without a full URL). Also misc. tweaks to try to get all metadata, properly handle sorting the playlist (getURL('playlist')), and doc. tweaks. 2.13 2023/04/08 - StreamFinder::Apple - Fix sometimes failure to fetch proper title. 2.14 2023/04/20 - StreamFinder::PodcastAddict - Fix failure to fetch podcasts and episodes due to slight changes in their site. 2.15 2023/07/28 - Added StreamFinder::LinkTV to fetch linktv.org video shows. 2.16 2023/08/24 - Added StreamFinder::Zeno for zeno.fm radio stations & podcasts. - StreamFinder::Brighteon - Add "formats_by_channel" option keyed by albumartist. This needed b/c some video formats (ie. .m3u8) don't seem to work on some sites but are the only ones that work on others. - StreamFinder::Podchaser - Add "-notrim" option to trim trailing garbage off stream URLs. - StreamFinder::Podchaser - Fix sometimes fetching wrong stream on episode pages, and fix podcast pages not returning any episodes due to recent changes in their site (more stupid javascripting). - StreamFinder::PodcastAddict - Mark DEPRECIATED since they recently javascripted & crapped up their podcast pages making it now impossible to fetch a playlist or first episode from them (correct episode pages still seem to work for now, if you get them from your browser first). 2.17 2023/08/25 - StreamFinder::Rumble - Further Cleanup artist and description fields of excess leading & trailing spacing likely introduced by possible tweaks in their site. This issue was causing Fauxdacious mediaplayer to fail to properly load display metadata for Rumble streams. 2.18 2023/09/04 - StreamFinder::Anystream - Add -maxstreams option to limit number of streams that can be returned - some sites contain tons. Also add ability to fetch additional metadata (ie. iconurl, description, channel (artist), etc. from sites that contain them in standard HTML tags. One site that does is: www.theepochtimes.com/epochtv - StreamFinder::Goodpods - REMOVED due to them javascripting up their site to where it's no longer possible to scrape any streams. 2.19 2023/09/12 - StreamFinder::Youtube - 1) Fix failure to fetch artist, icon, etc. sometimes on embedded IFRAME urls (slight site changes) and first episode from some channels. 2) Add -youtube-site argument to specify a different default youtube site (default https://www.youtube.com). 3) Add ability to parse youtube channel URLs containing an at-sign, ie.: https://www.youtube.com/@channelID. - StreamFinder::Subsplash - Restore as EXPERIMENTAL, as this site seems to now work again, at least for audio streams on some sites. - StreamFinder::Anystream - doc. touchups. 2.20 2023/10/24 - Update default user-agent to: "Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0" - StreamFinder::Apple - Handle "embedded" Apple podcasts (https://embed.podcasts.apple.com/). Also fix embedded subsplash.com stream URLs that contain "\u00##" characters and were being trimmed when encountered. - StreamFinder::Rumble - Convert one of their hosting sites that no longer works (hugh.cdn.rumble.cloud/video) to the standard one that still does (sp.rmbl.ws). - StreamFinder::Youtube - Try harder to get the first (most recent) video from channel pages (by skipping any "banner" video before it). 2.21 2023/11/16 - StreamFinder::Anystream - Handle "escaped" URLs embedded in some sites, ie. crap like: https:\/\/stupidsite\u002fstream.mp3 and ones that start with "//", ie. "//stupidsite/stream.mp3". - StreamFinder::Bitchute - Fix failure to fetch artist and albumartist (channel name and page URL) due to a slight tweak to their site. - StreamFinder::Brighteon - Hide extraneous debug message (when not running in debug mode). - StreamFinder::Rumble - Warn if had to convert new "bad" site: "hugh.cdn.rumble.cloud.video" to "sp.rmbl.ws". Also: try harder to fetch artist, albumartist, and icon (from "embed" URLs), since some channels allow public access to the "embed" version of their URLs but NOT always to their main (html) ones (go figure). 2.22 2024/01/20 - Remove StreamFinder::LinkTV (site no longer hosts videos onsite). - Mark StreamFinder::SoundCloud DEPRECIATED (now requires cookies and trackers to be enabled onsite to browse, but song URLs still work here. - Minor module doc touchups (to note recent depreciations, etc.) - StreamFinder::Rumble - Add -bitrate (max kbps) option. 2.23 2024/01/28 - StreamFinder::Rumble - Stop converting Rumble's "new" streaming URLs to their "old" format (initiated by v2.20) since the new URLS now seem to work properly and Rumble seems to be bandwidth-throttling the old ones, I assume to force usage of the new ones and close the old hosts. - StreamFinder::Bitchute - Silence cpl. stray debug messages when debug mode is not on. - Update copyright year to 2024. 2.30 2024/04/25 - StreamFinder::Castbox - Refactor to scrape episode data from the podcast page as now Castbox episode pages are no longer scrapable (now dynamically generated by javascript), but for now, all required episode data is embedded in the corresponding podcast page (for now). Also now append "lyrics" (transcript) to the "description" field (for Fauxdacious, etc.) for Castbox pages that provide this in a "lyrics" (.lrc) link! - StreamFinder::Google - REMOVED since Google has officially closed down it's podcast service. - StreamFinder::Podchaser - Fix failure to fetch episode data for the first episode (when a podcast page given) and failure to fetch all due to minor changes in their site. - StreamFinder::Spreaker - Refactor since Spreaker has now javascripted and cookied up their pages, but we're still able to scrape it's podcast pages for podcast metadata, required playlist data and 1st episode ID, and they provide a separate api site for fetching all episode data when given the episode-ID. - StreamFinder::Subsplash - Fix extraction of the artist & album fields due to minor changes in their site. Also add missing "Url" (first stream) field (not required for fetching streams). 2.31 2024/04/27 - StreamFinder::Castbox - Document new feature (in v2.30) that appends any transcript (.lrc) link into the "description" field. - StreamFinder::Podchaser - Fix failure to fetch the proper iconurl for the first episode and the podcast's album fields when given a podcast URL. - docs - Remove StreamFinder::Google (removed in v2.30) from supported podcast list. 2.32 2024/05/18 - Welcome epochtimes.com/epochtv videos by adding StreamFinder::EpochTV. Site was previously supported by youtube-dl, but was often missing some metadata & often returned wrong HLS stream quality, particularly for users with limited bandwidth. 2.33 2024/06/03 - StreamFinder::Brighteon - Redo to handle their redesigned site (which now requires JavaScript in order to view pages). Channel pages are no longer scrapable (so no more video playlist or fetching latest video), but individual video pages still contain all the same data embedded in Json format (at least for now). - StreamFinder::EpochTV - Few minor tweaks to try harder to fetch streams, the artist, the artist's channel icon, and the year. 2.34 2024/07/03 - StreamFinder::Bitchute - Refactor due to Bitchute's major recent redo of their site (more Javascript & crap), but they have a second site URL (api.bitchute.com) which still works well, so we autoconvert non-working URLs from www.bitchute.com to api.bitchute.com. 2.40 2024/07/26 - StreamFinder::Youtube - Refactor to add "-format-fallback" option to allow specifying a second "-f " option for use if no streams match the "-format" option. Defaults have been changed from "mp4" to "best", and the "-format-fallback" defaults to "bestaudio". These changes are required after 07/24 due to Youtube making (nearly) all of their public streams either audio-only or video-only, which are pretty much useless for streaming via your own media-player without actually downloading the video (and remuxing to combine the audio and video streams) to a file first! Note: The ability to do this is only possible due to the Herculean efforts of the yt-dlp developers and users are strongly encouraged to upgrade to the latest version 2024.07.25 to avoid likely just getting audio streams, as the internet continues to get more crapped up and locked down every day. :( 2.41 2024/09/14 - StreamFinder::AnyStream - Intercept & handle all .m3u8 (HLS) urls in order to apply any "hls_bitrate" option limits and select the highest one from the stream list if the HLS url is a master list. This seems to be necessary as ffmpeg, used in Fauxdacious for HLS streams always seems to select the highest bitrate stream and provided no known argument for limiting stream selection by bitrate. - StreamFinder::SermonAudio - Massive redo due to them completely overhauling their site, which now only offers a single video stream in HLS format in place of multiple bitrate choices, obsoleting the "high" and "low" "quality" options. The choices are now: "audio" or "any". Users should now use the "hls_bitrate" option to limit bandwidth for slower internet connections. Also add new "nowebp" option to download icon images in their native formats instead of webp (needed by Fauxdacious GTK versions and perhaps others). - StreamFinder::Youtube - Change default external Youtube-fetching program from "youtube-dl" to "yt-dlp", as the former may no longer properly handle many youtube streams. 2.42 2024/09/22 - StreamFinder::SermonAudio - Fix excess padding sometimes left on both sides of Genre text. Also update copyright year & docs. - StreamFinder::Podchaser - Fix failure to fetch failure to fetch episode icon, and sometimes other fields for the latest episode when given a podcast url due to slight changes in their site. - StreamFinder - Fix some issues with handling HLS (.m3u8) streams, particularly in StreamFinder::Anystream, but effecting all modules, mostly for better handling by Fauxdacious Mediaplayer and the ffmpeg library when using internet connections with limited bandwidth. The biggest change is we no longer attempt to parse out HLS master playlists when neither hls_bandwidth (limit) or -noplaylists parameters are set. Instead, just return the master playlist stream as-is. This allows for better control over selected streams within bandwidth limits within Fauxdacious itself. To function the old way, simply specify an hls_bandwidth limit (in the config file) or "-noplaylists => 1". 2.43 - Update copyright year to include 2025 - StreamFinder::SermonAudio - Fix failure to return description field and sometimes incorrect icon and image URLs due to slight changes to their site. Also try harder to find description. 2.44 - StreamFinder::Rumble - Add "-order" option & refactor to better deal with their recent addition of m3u8(hls) streams; and, if accepted, force audio-only (aac) stream-option to last (for use if no streams fall within the user's max. bandwidth/quality limit options. Also exclude their new "video-only" streams ("*.[A-Z]aa.mp4*" urls). - StreamFinder::Youtube - Strip "?list=..." part off Youtube video urls to prevent fetch "hanging" issue. 2.45 2025/07/22 - StreamFinder::Youtube - Add ability to fetch the "New & Improved xxxx Screwed Up" Bitchute streams which can no longer be scraped except by the very latest (version 2025.05.22) of yt-dlp! We still scrape the Bitchute video page for metadata (so StreamFinder::Bitchute remains the default and best way to fetch Bitchute videos). NOTE: Users must *NOT* have the "-youtube" option set to 'No', else Bitchute video streams can no longer be retrieved. Also, users can no longer supply a Bitchute channel page and get the latest video. Internet Encrapification continues at full speed ahead here in 2025 as more and more streaming sites continue to invest tons of money to hire developers to obfuscate their sites to prevent a handful of users like us avoiding their javascripts, trackers, spam, excess ads, etc. :( - StreamFinder::Rumble - Set default "-order" to "quality". 2.46 2025/11/14 - StreamFinder::Rumble - Add "-notrim" option, and strip off "?ets=..." from urls unless set. Also fix a Perl uninitialized variable warning. - StreamFinder::Youtube - Properly strip off excess bs from URLs, such as "&source=..." unless "-notrim" option set. Also now search for any Youtube urls embedded in webpages' JSON code as a fallback. Update copyright year to 2025. 2.47 2026/01/09 - StreamFinder::Odysee - Fix failure to fetch artist field and streams from their webpages (they've tweaked their site and have added back stream-fetchability directly from their pages). We also no longer call StreamFinder::Youtube if -youtube => 'no', or if one or more streams are successfully fetched from their webpage(s) and -youtube => 'ifneeded' (how it was meant to work). - StreamFinder::Rumble - Fix failure to fetch artist name and full description text (slight changes in their site). Also reject streams named "*.[A-Z]aa.rec.{mp4|webm}" as these seem to be video-only / "dash"? streams, and any stream with non-numeric "quality" (except "hls" or "auto" (ie. "timeline", "pause_ads_params", etc. as these are not real streams. - StreamFinder::Youtube - Fix sometimes failure to fetch the proper image URL and/or the youtube video-url often embedded in many Odysee videos as the last line of their description via StreamFinder::Odysee. Also allow the -format-fallback option to be set to "" or 0 to allow fallback to any available format (no -f option appended to yt-dlp or youtube-dl) - unless -formatonly option is set. 2.50 2016/01/29 - Add StreamFinder::PragerU to handle www.prageru.com videos. - StreamFinder::Subsplash - Refactor to optionally use yt-dlp to fetch video streams (-youtube => {yes|no|first|last|ifneeded}) similar to several of the other fetcher modules, as Subsplash pages pretty much only return audio streams (if at all) without javascripts/cookies now. Also fixed & greatly improved metadata fetch from their pages. - StreamFinder::Brighteon - Fix failure when -youtube => 'only' is set. - StreamFinder::(BitChute,Brighteon,Odysee) - Try harder to get full "description" field, tweak a cpl. regices to better handle non-ascii characters sometimes found in certain text and/or URL fields. - StreamFinder::Rumble - Filter out some more now non-playable .mp4/.webm streams from consideration. - StreamFinder::Youtube - Prevent sometimes non-URLs being returned as valid streams causing failure to fallback to StreamFinder::Anystream when no valid streams found. 2.51 2016/01/31 - StreamFinder::EpochTV - Fix returning mostly invalid stream URLs. Also add "-speakericon" (boolean) to return artist's icon instead of the default channel icon (EpochTV offers separate ones, like StreamFinder::SermonAudio). Also improved several other metadata values returned. - StreamFinder::Bitchute - Change default -youtube value from "no" to "yes" as yt-dlp is now required to fetch Bitchute streams. Also clean up code & docs a bit. StreamFinder::(Anystream,Subsplash) - Touch up docs a bit.