Where does this fit in the Linux stack?
Posted by karland90@reddit | linux | View on Reddit | 12 comments
So I was reading the issue-thread about KDE Plasma adapting to the recent EU requirements about accessibility. And avoiding users accidentally creating situations that could trigger photosensitive epilepsy sounded difficult.
This made me think - hypothetically speaking - in which part of a modern (e.g. KDE-based) Linux distro could an OS-level universal photo sensitivity filter be implemented 🤔? I.e. an optional tool where successive frames are analyzed and if a danger level threshold is crossed, a mitigation procedure is triggered. That procedure could be freezing/skipping frames, morphing between frames more slowly, or displaying a warning overlay/watermark).
Can this be a regular user app? Does it require changes to some part of the rendering stack?
Based on googling for 5 min, I found:
- this mention of University of Maryland having a fully open-source detection tool in the works:
We are working on a new fully-open-source version that will be updated for new technologies (the current version is open-source except for a proprietary analysis engine we purchased the rights to use). It will also be free to use. No ETA for it as yet.
- some Github repo searches: 1 2
- one of the more promising results: 3
- that searching for "epilepsy detection" gives a lot of "noise" in projects doing health tracking for detection of an epileptic fit.
I'm hoping someone is inspired to dig into making this or I get pointers which issue tracker or forum to take this towards 🙏
Maybe Linux can get another trailblazer win, Apple can copy it and get admired as innovative for it, and we get the smug "um akshually ☝️". But the world would still be better than before 😌
Misicks0349@reddit
generally not flashing content is just done by.... not doing it I suppose?
I think the problem with some kind of epilepsy detector thats "desktop wide" is that the "desktop" is a much more unconstrained and performance sensitive environment compared to something like a video, for a epilepsy detector on waylands side you'd have to have it run for every frame on every surface, which is simply taxing.
The tools you linked also aren't particularly promising either, PEAT hasn't been updated since 2017 and seems to be a post hoc solution (i.e. it doesnt run in real time) and the javascript extension you linked just compares the brightness of the previous frame to the current frame and checks if it reaches some kind of threshold, I'd imagine its prone to many a false positive and false negatives (as well as only working for a subset of photosensitive epileptic people as not all of them have flashing images as their trigger).
sobe3249@reddit
I think people with serious epilepsy would be ok with a few % performace hit to make sure it safe for them. Everyone else can just not enable it.
Just not doing it sounds cool, but even a bug can cause it or a random youtube video without a warning. If it would be preventable on wayland level, I don't see the downsides.
Misicks0349@reddit
depends, for something as basic as that javascript plugin you might only see a few percentage drops, but I very seriously doubt the effectiveness of just comparing the last frames brightness to the current frames brighgtness and then blotting out the screen (and it might lead to situations where it causes epilepsy instead of helping it)
for more involved solutions like PEAT I'd imagine the cost is far greater and would only increase in cost the higher resolution your monitor is.
there will (always) be bugs unfortunately, but the solution is to fix them. I very seriously doubt the capacity of a wayland compositor to actually be able to mitigate these kinds of bugs, and it might be completely unable to if the epileptic display is caused by a bug "lower down" on the chain (in linux's DRM manager for example). The wayland compositor itself could also introduce epileptic bugs through this feature as well, e.g. rapidly flashing the screen on and off.
karland90@reddit (OP)
Regarding the more-harm-than-good argument: instead of flashing to black, it could alternatively dim (black semi-transparent overlay), freeze (continue showing the same frame) or morph (display an average of multiple frames).
The performance cost of running it may indeed be considerable, but it might be worth it as u/sobe3249 suggested.
I do agree much more with your suggestion that it is not likely to very effective. False positives and negatives.
So in summary, if you can't rely on it 100%, the enormous development cost (since it might be a full new research project) might not be justified. To justify the cost, it would be better if it is as low-level as possible - so something written for Vulkan, optionally enabled in the Linux DE or some other way.
kolorcuk@reddit
Compositor in Wayland
X in x
Able-Reference754@reddit
Why would you think there needs to be an OS level solution? It just doesn't make any sense. KDEs job is to ensure that KDE is safe and accessible, it has no part in making software/media used on a KDE desktop safe and accessible.
Booty_Bumping@reddit
Sure, but it can be done at that level, so why not make such an option available? For example global colorblindness filters get a lot of flak, but they can be really helpful if you just can't configure an app to work for you.
From what I understand about accessibility from those who need it, there's not one way to provide good accessibility, and it's often better to just provide a lot of different options for different requirements. An inelegant solution is a bandaid, but at least it covers more scenarios when individual apps tend to ignore accessibility.
Able-Reference754@reddit
Can it? You're looking to prevent flashing, so we're talking repeated brightness change over time, from a compositor level it's quite hard to see into the future, predict and prevent it from occurring. A video decoder on the other hand can quite easily decode ahead. Unless you're talking about literally clamping down every single big brightness change you're going to have very limited opportunity to implement any heuristics worth a damn in the compositor.
Booty_Bumping@reddit
True. It would have to compromise between either displaying a few frames and then cutting it off, or delaying the entire screen at all times, and neither of these options are particularly great. But photoepilepsy is a bit unique from other disabilities in that it could literally be lifesaving to prevent it, so I could see why someone would be fine with adding latency just to avoid the possibility.
zlice0@reddit
not reading anything beyond the bit on preventing flashes... sounds like when peopel want to make internet laws and have no clue how networking works.
Zamundaaa@reddit
That would be possible to implement in a compositor plugin - KWin has an effects API (C++ and js) that could do such things.
Booty_Bumping@reddit
This sounds like something a shader could accomplish in a Wayland or Xorg compositor. A compositor has access to the pixels on the screen, so it could use some heuristic to freeze the image if it's changing too often.