Code Yarns ‍👨‍💻
Tech BlogPersonal Blog

How to check if file is UTF-8

📅 2019-Dec-29 ⬩ ✍️ Ashwin Nanjappa ⬩ 🏷️ utf8 ⬩ 📚 Archive

The war of the character encodings for text files is over and UTF-8 has won. Occasionally, I need to check if a text file is UTF-8 or some other encoding like ASCII or UTF-16.

$ file foobar.json
foobar.json: ASCII text
$ isutf8 foobar.json

$ isutf8 /bin/ls
/bin/ls: line 1, char 1, byte offset 25: invalid UTF-8 code

Notice how it is a quiet tool that returns exit code 0 if the file is UTF-8 and prints nothing. If the file is not UTF-8, then it prints some details.

$ sudo apt install moreutils