What an encoding is
A file is bytes. An encoding is the agreement about which bytes mean which characters. Plain letters and digits are the
same in almost every encoding; accents, umlauts, the euro sign and every non-Latin script are not. In
UTF-8, ü is two bytes, C3 BC. In Windows-1252 — what Windows has long used for
Western European text — it is one byte, FC.
Reading the symptoms
| You see | Instead of | What happened |
|---|---|---|
Zürich, Genève, Café | Zürich, Genève, Café | UTF-8 read as Windows-1252 or Latin-1 |
€ | € | UTF-8 read as Windows-1252 |
Z�rich or Z?rich | Zürich | Windows-1252 read as UTF-8 |
Z√ºrich | Zürich | UTF-8 read as Mac Roman |
The first row is by far the most common: a modern system writes UTF-8, and a program that expects a regional encoding reads it. Excel, for instance, generally needs a byte-order mark at the start of a UTF-8 CSV to recognise it — which is why it offers a separate CSV UTF-8 format when saving.
Opening the file correctly
Intact CSV Editor detects the encoding when a file opens — over 80 of them, including UTF-16, Windows code pages, Mac Roman, DOS pages, East Asian encodings and EBCDIC — and says in the status bar what it found and how sure it is. If a file is ambiguous, pick the encoding in the Format inspector and the file is read again. Re-reading changes nothing on disk.
For a folder that always receives the same kind of file, a rule in Settings ▸ Rules fixes the encoding once —
~/Exports/Bank is Windows-1252, for example — and detection is set aside for those files.
Saving so that it arrives intact
By default the file is saved in the encoding it came in, with its byte-order mark if it had one. To send it to a system that wants something else, choose the encoding in the export sheet — and if a value contains a character that encoding cannot hold, such as a € in a Latin-1 file, the save stops and names the character and the cell. It is never quietly written as a question mark.
When the garbling is already in the file
If somebody opened the file wrongly and saved it, the garbled characters are now real text in the file. Opening it
with another encoding will not help, because the bytes now spell ü. Repair it with
Find and Replace on the common pairs — ü → ü, ö → ö,
ä → ä, é → é — and preview the changes before replacing them all.
Try it on your own file
Intact CSV Editor is free for seven days, fourteen with an account — every feature, no card. Download the free trial.
