So you really want a second phone/smaller tablet. Here's why: It already has all the features you need. Older modrls allow for sd card removal, which means that even if the phone bricks, you can pull your data out of the card if you store it there. Backing up files is very easy using android phones. A lot of note apps available. Older models, but still working normally are affordable, especially second hand. If you uninstall all but what you need from it, it should run fairly well and be reliable. Bloatware and filled storage clunk our phones down, your note taking phone would have the privilege of not going through that. In case your main phone decides to give up on you randomly, you have a backup ready.

Getting a phone or small tablet (or using an old one you probably have) is the best option for you in my opinion. If money isn't an issue, there are models that allow for freehand writing with a stylus that's provided with the phone. I know you asked for a device SPECIFIC to note-writting, but with how customizable phones are, you can designate one as a note-taking device and min-max it to fit your needs.