How come Linux doesn't seem to support handwriting to text?

[-]

GJT11kazemasin@reddit

This might be a solution: running Google Input Tool alongside your desktop https://github.com/Saren-Arterius/google-chinese-handwriting-ime

[-]

mecha_monk@reddit

Besides xournal there’s also apps like write

[-]

It is possible to use the drawing feature on Google Keep to handwrite your note and then use the menu (three dots) option: 'Grab Image Text' to digitize. While it seems to work well, it is not exactly ideal for integration with other applications.

[-]

Booty_Bumping@reddit

Even the proprietary offerings for this work incredibly poorly. There's a reason it never became a big thing.

[-]

BandicootSilver7123@reddit

Chrome os is already gnu/linux. Why not just install the linux apps you want on your chrome os so you keep the handwriting to text feature close in the process.

[-]

anh0516@reddit

Lack of interest among developers.

But yeah, it's a glaringly missing feature.

[-]

MatchingTurret@reddit

Lack of interest among developers.

I think this goes deeper.

A successful open source project usually starts small: Linus releasing Freax 0.1 written all by himself. Perl developed by Larry Wall, Peter Mattis and Spencer Kimball starting GIMP and GTK. Once it has proven itself useful, it attracts outside contributions and starts to grow.

Problem: The first version of certain features is beyond the reach of a single developer or even a small team working in their spare time. It needs long term investment and a big team to get over the hump to become useful. And I don't see how this can be done by developers working in their spare time alone.

[-]

hobo_stew@reddit

With the modern AI advances it should be feasible for a single person

[-]

MatchingTurret@reddit

With the modern AI advances it should be feasible for a single person.

Did you just volunteer?

[-]

hobo_stew@reddit

I‘m a mathematician. I‘m satisfied with showing feasibility

[-]

Taldoesgarbage@reddit

But not a programmer, as is evident.

[-]

hobo_stew@reddit

I've trained a deep neural network for reconition of a basic set of handwritten letters myself some years ago in my free time. i'm sure a professional developer can handle this task just fine.

[-]

MatchingTurret@reddit

For handwriting recognition you need more. Your NN needs to be able to predict the next word to make sense of unintelligible characters, e.g. a language model.

[-]

TesNikola@reddit

i'm sure a professional developer can handle this task just fine.

I think you're missing the point. Our time is worth something too, you know. Perhaps it's easy to overlook the motivations that tend to drive OSS contributions.

Ergo, without clear demand, and more over, a personal investment, no great reason for it to be done.

[-]

hobo_stew@reddit

yes, but I am suprised that there has not been one that wanted handwriting recognition enough to work on it, given the amount of software developers world wide.

thats more or less what i expressed in my previous comments with less words

[-]

NeverMindToday@reddit

I wouldn't expect AI to help experienced developers do something where existing code or documentation/examples isn't available for training on.

[-]

hobo_stew@reddit

I've trained a deep neural network for reconition of a basic set of handwritten letters myself some years ago in my free time. i'm sure a professional developer can handle this task just fine

[-]

whatstefansees@reddit

It's not concerning the operating system or Linux' fault. It's a program you use and you might try to do so on Linux through WINE

[-]

Unslaadahsil@reddit (OP)

I'm not sure how it would work in Linux, but Wine does not solve this.

In windows, it's a function that's part of the OS (at the very list it comes pre-installed) and it's not a singular program that does it, but rather a function of the OS that work on most writing inputs.

No writing program comes with this. Running word or office through Wine does not make it magically appear. It's a function installed on windows and many android devices separately from Word, or onenote, or what have you.

[-]

Fofeu@reddit

Define "OS" so that we can compare comparable things.

At the minimum, an OS is a kernel + the bare minimum to run an executable with it. I doubt that Microsoft bundled a handwriting recognition solution inside their kernel or libc-equivalent. Linux does the same here.

If you mean by OS "extensive set of programs designed to offer a decent user experience", then yes Linux is lagging behind. After a quick google search, Windows exposes a service that handles it (how?).

Microsoft had the money to hire experts in the relevant field (AI?) to design a performant handwriting recognition service and then money again to hire people to integrate it with their existing input framework.

For Linux, you probably have a good base to do the second part, but it is worthless without the first. From yet another search, it seems the state-of-the-art in open source is Tesseract OCR which you have to train for your specific handwriting (otherwise it works only with printed text).

[-]

anotheridiot-@reddit

There are better handwriting recognition algorithms in papers with code, tesseract is quite bad from my experience.

[-]

Fofeu@reddit

There are better handwriting recognition algorithms in papers with code

I wouldn't be surprised. It's not my area of expertise, so I don't know what there is out there. Do you have an example ?

tesseract is quite bad from my experience

At handwriting recognition or at OCR in general ? I've heard that it's okay for OCR for printed text (it's old, but was designed for that), but that by default handwriting is horrible (2% detection rate at best) and you need to manually tune it to your own handwriting which is quite tedious.

[-]

anotheridiot-@reddit

https://paperswithcode.com/sota/handwriting-recognition-on-kohtd

Tesseract was bad at even regular OCR from what I've tested.

[-]

iBlag@reddit

I don’t think Tessaract does real-time/online handwriting recognition, only offline.

And you just proved OP’s point: Linux has no similar solution for the real-time/online handwriting recognition feature that Windows does.

But when you point this out to other Linux fans, they get defensive, blame the user for wanting the thing they can get elsewhere, and redirect.

Which is a toxic way to ensure the problem is never solved. 🙃

[-]

iBlag@reddit

I don’t think Tessaract does real-time/online handwriting recognition, only offline.

And you just proved OP’s point: Linux has no similar solution for real-time/online handwriting recognition.

[-]

pikecat@reddit

It sounds like OS bloat, something that Linux users want yo be away from.

Also Linux users tend to be be better with the keyboard,so have less interest in this. I'd definitely rather type.

[-]

Neoptolemus-Giltbert@reddit

"Any feature I don't personally need is bloat"

[-]

_oohshiny@reddit

Gentoo user

[-]

mrvictorywin@reddit

and many android devices

If you know an app with this functionality built in, you could use it with Waydroid. Not all Android devices have the same feature set, an app may need to implement this on its own to work.

[-]

kwyxz@reddit

Just because it does not require a 3rd party does not mean it's not a program running. It's probably embedded in the Windows graphical environment and windowing system, so I'd assume on Linux it would be something that needs to be handled by the desktop environment. So not Linux itself, but a program you use.

[-]

Rusty-Swashplate@reddit

I got 2 Chromebooks and 1 Windows laptop and all 3 have a touch screen and pens. They came with it. No choice. I don't mind the touch screen. The pens I never used except for testing.

I don't know a single person who uses a pen. Well, 1 I know who's using it as a mouse replacement, but no one uses it to hand-write anything.

On a tablet though, this is far more common. But since there's not many tablets with Linux (excl. Android), that makes the need to develop handwriting/drawing capabilities on Linux to almost zero.

[-]

MeanEYE@reddit

I'd hazard a guess that interest is low. But at the same time it's not an easy thing to do well. It's pretty much machine learning in combination with UI/UX development.

[-]

jimicus@reddit

It's part of a bigger issue: The Linux desktop environment is first and foremost built by and for software engineers.

And software engineers (on the whole) have a remarkable tolerance for pretty abysmal user experiences. Anything that might improve this - particularly something niche and hard to implement like handwriting recognition - is basically never going to happen without commercial sponsorship.

[-]

Hans_Wurst_42@reddit

I really dig this on my iPad. Miss it in Linux.

[-]

MatchingTurret@reddit

Is there just no interest in this feature among the Linux community?

Obviously you are interested. Go ahead and contribute an implementation! Or do you expect someone else to do it for you?

[-]

Unslaadahsil@reddit (OP)

I would have no idea where to even start. If I knew how to do it, I would have done it already.

[-]

Odd_War853@reddit

Just an idea for until it is a native feature. Use waydroid and with that something like nebo. Works perfect for me

[-]

wbeater@reddit

Is there just no interest in this feature among the Linux community?

The other way around, the users have no interest in Linux. Especially since the main/primary user, I claim without knowing it, uses proprietary software from Adobe, which only exists for Windows or Mac.

[-]

Neoptolemus-Giltbert@reddit

Why do you believe a designer would be using handwriting to text -conversion? Also what kind of "peripheral device" are you talking about? A touchscreen? Which is widely available on a lot of laptops?

[-]

wbeater@reddit

Also what kind of "peripheral device" are you talking about?

stylus

[-]

Realistic_Bee_5230@reddit

I hope there are many of us that are like me but use linux as their os for school, I have a 2in1 laptop and I write on it all the time, I use note for that but I hope note gains the abilities of onenote soon

[-]

Cool-Curve2346@reddit

Прастити, «Дайсон подарил», это он жене компанию подарил или что-то из продукции, например пылесос?

[-]

BujuArena@reddit

Does Xournal++ do this?

[-]

Unslaadahsil@reddit (OP)

Alas. no. Or it didn't last I checked, though I doubt such a big update would have gone unnoticed.

Xournal++ is great for writing through handwriting directly, and amazing for taking notes when you also need to put quick drawings or geometry in, but it doesn't do the "handwriting to text" conversion I'm looking for.

[-]

CaglanT@reddit

Maybe this fork?

https://github.com/PellelNitram/xournalpp_htr

[-]

CaglanT@reddit

Maybe this fork?

https://github.com/PellelNitram/xournalpp_htr

[-]

djustice_kde@reddit

yea the IME standards are a serious obstacle.

then again.. we built this stuff with our hands on keyboards. apple did a palmpilot flop with handwriting to text at one point. i like my compose key. i use shell aliases extensively. i have a 2-in-1 but i'm a tattoo artist/programmer, not a poet. not to mention it sounds like the most painful and boring thing to have to code without any bulk input-output dataset to opencv to. basically, it's not something a programmer would use. it would really require a multitude to convene to solve the gap. like autocad or procreate. for a lone 20ft beard it's simply impossible.

i started an opencv based ASL->text project for android yeears back.. i couldn't make progress fast enough to validate the effort.

[-]

OrseChestnut@reddit

If it doesn't exist it's probably because:

1) It's a 'hard' problem to solve. It requires more than your standard coding skills to develop this type of recognition algorithm.

2) It's niche. Usually stuff for Linux is developed out of 'benevolent self interest,' i.e. I made this because I wanted it, but I don't mind sharing. The popularity of this app is limited, so the pool of potential developers is also limited.

If you want it, getting proactive is your best bet. Either create it yourself if that's within your skill set, or otherwise scout for interest from potential developers. Since nobody seems to be developing it already it will probably require substantial funding to employ someone. Get some quote then launch a kickstarter to raise the funds maybe?

[-]

_Proud-Suggestion_@reddit

Could you explain your requirements? Like more clearly so someone can pick it up or maybe even you could help out but ya a clear definition of the problem would be nice. Like what do you expect.

[-]

brownphoton@reddit

This is a super niche feature. Frankly, you’re the first person I have come across who wants handwriting support for text entry at an OS level on a computer, so my guess would be that nobody else really cares enough about it.

Clearly you seem to care a lot about it, so go ahead and add it yourself, or pay someone to do it for you.

[-]

kansetsupanikku@reddit

I don't think that it should be included in Linux. More likely, it should be an optional input method for desktop sessions, perhaps also some gpm-like software for VTs.

I see no reason why it couldn't be implemented, much less one related to Linux. If anything, ChromeOS comes with a proof of concept that it can be done.

[-]

triemdedwiat@reddit

It is really just a form or OCR and the underlying experience with OCR is that, except in a few situations . it is easier to just have someone type it up directly.

It has too be trained in the first place and then what ever is produced has to be corrected.

[-]

CompetitionSquare240@reddit

Because that’ll be too good

Linux only focuses on fuckery for some reason

[-]

Tough-Boat-2601@reddit

This problem has gotten substantially easier in the last few years. Would you be willing to pay for this feature to cover the cost of hosting a model that does this type of inference?

[-]

Unslaadahsil@reddit (OP)

Probably, though it would depend on the price/quality ratio.

[-]

Tough-Boat-2601@reddit

There is this API that Google provides https://cloud.google.com/vision/docs/handwriting

A solution would involve sending an image to the API and getting the response.

You could spin up a terrible version of a handwriting app with Python.

[-]

spezdrinkspiss@reddit

IMEs are hard, and linux users are woefully belligerent to any kind of machine learning algorithms (and you need that for handwriting inputs)