Zürich instead of Zürich: what went wrong, and how to fix it.

What an encoding is

A file is bytes. An encoding is the agreement about which bytes mean which characters. Plain letters and digits are the same in almost every encoding; accents, umlauts, the euro sign and every non-Latin script are not. In UTF-8, ü is two bytes, C3 BC. In Windows-1252 — what Windows has long used for Western European text — it is one byte, FC.

Reading the symptoms

The first row is by far the most common: a modern system writes UTF-8, and a program that expects a regional encoding reads it. Excel, for instance, generally needs a byte-order mark at the start of a UTF-8 CSV to recognise it — which is why it offers a separate CSV UTF-8 format when saving.

Opening the file correctly

Intact CSV Editor detects the encoding when a file opens — 81 text encodings, among them UTF-16, the Windows code pages, Mac Roman, the DOS pages and the East Asian encodings, and 52 EBCDIC mainframe pages — and says in the status bar what it found and how sure it is. If a file is ambiguous, pick the encoding in the Format inspector and the file is read again. Re-reading changes nothing on disk.

The single-byte encodings are the hard case: they all hold the same 256 values and differ only in what they mean by them, so Polish names in Windows-1250 read without a single error as Windows-1252 and come out as ŁódĽ. Nothing in the bytes says which is meant — but a person sees it at once. The magnifier beside the encoding in the Format inspector opens Which Character Set Is It?: every character set the app knows, each with a line of your file read that way, the likely ones first. Click the one that reads properly.

For a folder that always receives the same kind of file, a rule in Settings ▸ Rules fixes the encoding once — ~/Exports/Bank is Windows-1252, for example — and detection is set aside for those files.

Two encodings in one file

Stack two exports together — one from a system that writes UTF-8, one from a system that writes Windows-1252 — and the result is a file no single encoding can read: whichever is chosen, half the accents break. Intact CSV Editor notices when a file holds both, reads each character as what its own bytes can only be, and says so under Notes in the Format inspector. Saving the file writes it in one encoding.

Saving so that it arrives intact

By default the file is saved in the encoding it came in, with its byte-order mark if it had one. To send it to a system that wants something else, choose the encoding in the export sheet — and if a value contains a character that encoding cannot hold, such as a € in a Latin-1 file, the save stops and names the character and the cell. It is never quietly written as a question mark.

When the garbling is already in the file

If somebody opened the file wrongly and saved it, the garbled characters are now real text in the file. Opening it with another encoding will not help, because the bytes now spell ü. Choose Table ▸ Tidy Up ▸ Repair Text Written Twice Over: it finds the values that were encoded twice, shows you examples — Müller → Müller — and repairs them only when you say so, as one step you can undo. Only you can tell whether ü was meant, so the app asks rather than guesses.

A few stray pairs can also be fixed with Find and Replace — ü → ü, é → é — with the changes previewed before replacing them all.