I have tested latest Youtube API V3 ruleset, and it seems its not working as intended when using Chrome / Edge browser.
When i open youtube video that supposed to be blocked in Chrome / Edge, i get Youtube custom page saying im offline.
When i open the same page in mozilla / ie, i get the expected proxy blockpage
When analyzing logs i can see that on chrome / edge, after the block event there is additional activity, seems that application for chromium is more advanced and tries to circumvent the 403 using several different players, and after it fails to do so, shows internal custom block page.
Is there any hope to fix that?
the "you are offline" is expected when you try to play a video that is blocked. If you go to a video that is allowed it should play.
Is this the behaviour you see?
The problem is that a Service Worker was implemented for Youtube, which seems to work for Edge and Chrome (where Edge actually is Chrome under the hood...). The service worker is sort of a cache that sits between the Browser and the Proxy, so the Browser is no longer sending a request for a video to MWG, but it talks to the service worker. The service worker replies before we actually see the request, so it was possible in the old version to still play the video (since we block the page showing the video player, not the video itself).
The updated rule set blocks the request that queries the video URLs for a specific video. When this request does not come through (blocked by MWG), the service worker replies with "You are offline".
I think there might be a way to not block the query for the video URLs but modify them to deliver some fake response, but I did not find the time to look into this.
Sure, this is the behavior, and yes, it's affecting chromium browsers, the platform edge is build on.
The problem for me was that i had custom block pages with button to create service desk tickets.
Now this functionality is useless.
I'm not familiar with service workers, but perhaps you can do a quick check on this, and post your thoughts?
Maybe it can be somehow circumvented, perhaps just by returning something other than 403?
you can block the service worker by blocking youtube.com/sw.js, but this caused some other behaviour in my tests (back button in browser not working as expected, etc.).
There is not much that can be done. There are two items on Youtube, there is the page you see with the comments and the window which plays the video. This is the HTML response the browser renders, which in the past was replaced by the 403 Error Template by MWG.
The other piece is the video itself, which sits on googlevideo.com URLs. The page previously mentioned fetches the video from the googlevideo.com URLs and the video player on this site plays the video.
In the past when you went to a Youtube URL the browser called the page playing the video from Youtube. Once it was rendered the video content was obtained from googlevideo.com and the video started. In case MWG blocked the video the browser requested the page and MWG replaced the content.
The service worker now runs in the browser and caches the page which plays the video. So if you go to a video the browser does not query MWG for the page with the video player any longer, but it queries the service worker. The service worker returns the cache version of the page. The video is obtained and displayed, without allowing MWG to block the request.
I found a call to Youtube that returns the meta data, such as title, comments, etc. It might be possible to intercept this call and make changes, so that - for example - the page holding the video player can tell that the video was blocked for some specific reason.
Also it might be possible to not return a 403 but some modified response to the service worker to make it believe it received a valid response and update the page on the client.
There are two problems with this:
1.) This requires not only blocking some specific URLs but look into some traffic and apply modifications. The problem here is, as soon as Youtube makes some changes on the API, this breaks again and needs to be reviewed and adjusted. Customers have to update their rule sets manually. In the end everyone is unhappy.
2.) I need to find out what exactly are possible responses and a good way to modify them. This takes some time which I was not able to spent yet.