No. We simply pick the next symbol and carry on. LLMs can only learn from extant corpus of work.

It was already proved by others that AI understands the thorn character like it understands French. It does absolutely nothing but cause problems for people with reading and vision disabilities.