r/dotnet • u/Mindless-Arachnid-74 • 1d ago

Question Need help with encoding

/r/csharp/comments/1x5wtt0/need_help_with_encoding/
1 Upvotes

67% Upvoted

2 comments sorted by

1

u/AutoModerator 1d ago

Thanks for your post Mindless-Arachnid-74. Please note that we don't allow spam, and we ask that you follow the rules available in the sidebar. We have a lot of commonly asked questions so if this post gets removed, please do a search and see if it's already been asked.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

0

u/ggende 1d ago

I've never dealt with this as a technical challenge, but I don't think this is really a technical challenge anyway. It's a requirements challenge. On the technical side, you just get a list of transcription targets and translate on the fly (check out .net Rune struct, if you're not already familiar with it... that will be your best friend). You'll need a default case for the ones that slip through (presuming you won't be making several million entries in the table), or if you need to be really thorough, add something that flags unknown reference characters and sets those aside to be handled later once the translation tables are updated.

Like I said above though, this is really a requirements problem. That is something I do have experience with. Ideally, ask the customer what they want... or even better, create a suggested translation table and ask them to review any contentious choices. Em-dash to regular dash is easy, as is <insert different width space here> to regular space. Don't waste their time on those. If there's anything you're unsure of though, ask them.

If asking them isn't possible for some reason, think about who is going to complain when you're wrong, and ask them. If that also isn't feasible for some reason, at least let your boss know where you had to make judgement calls.

Do you have a representative sample you could use to capture the likely input scope? Or is it something where the entire UTF8 scope is likely, including stuff like Chinese? If it's the former (i.e. you have a representative sample), start working with that.

Or... are you asking if a library already exists? If so, sorry, I'm not aware of any. :/