I never stop to be amazed of the amount of tools that you have on linux.
I had to compare two tables from dos, one generated from excell the other from a custom database.
Both were supplied as csv files.
I wanted to create a tool in my favorite language FPC/Lazarus.
But alas, I got strange characters in the fields, just because of the Swedish letters öäå.
Tried to cat the files and that didn’t help.
Vi, of course, showed everything correct. What would you expect from the best editor in the world.
I determined that the issue was that the files were using unicode for “strange” characters, and linux like UTF-8.
Searched the net for tools in FPC to convert the strings, but didn’t find anything usefull.
Then I thought about finding a command line tool to convert the files before I used FPC/Lazarus to do my stuff.
Lo and behold – I found iconv, already installed in my mint.
First I ran dos2unix to get rid of the crls sequense.
Then I ran iconv -f ISO8859-1 -t UTF-8 > and all my problems disappeared into thin air
I love linux!