dump method from the Python json package can be used to write a suitable Python object, usually a dictionary or list, to a JSON file. However, I discovered that Unix shell programs have problems working with such a JSON file. This turned out to be because this dump method does not end the last line with a newline character! According to the POSIX definition of a line in a text file, it needs to end with a newline character. (See here).
I replaced this:
json.dump(json_data, open("foobar.json", "w"), indent=4)
with open("foobar.json", "w") as json_file:
json.dump(json_data, json_file, indent=4)
json_file.write("\n") # Add newline cause Py JSON does not
Many Linux tools and compilers that process text files require the text file to end with a newline. This is simply because they are designed to process a series of lines, each ending with a newline. That means, the last line of the file should also end in a newline.
If the file does not end with a newline, then C++ compilers and other tools complain with errors such as
No newline at end of file.
How do you end up with a file that does not end in newline? If you use tools that are written for other platforms like Windows or are cross-platform, those might not follow the rule of ending the last line with a newline. The latest culprit who I found doing this mistake was the Visual Studio Code editor. Editors like Vim, which come from a Unix heritage, just do not allow you to save without a newline.
In Visual Studio Code, I set the
files.insertFinalNewline option to true to fix this mistake.
I have a Python script named foo.py. Since, I would like to run it directly from the shell, I have added a shebang to run it through Python:
print "Hello world"
If I run it directly at the shell, I get a strange error:
: No such file or directory
However, if I run it explicitly with the Python interpreter, it works fine:
$ python foo.py
This strange error turned out to be due to the newline markers in the file. The file was created on Windows and had CR+LF as the newline marker. But, I was running the script on Linux, which only uses LF as the newline marker.
So, this must have led the env program to run the character CR as a command. And obviously, it could not find any such file or directory.
The solution is simple. Convert the newline markers in the file to that of the host system, in this case to Linux. This can be done easily in Vim by using the fileformat option.
Tried with: GNU CoreUtils 8.12.197, Python 2.7.3 and Ubuntu 12.04 LTS
It is pretty common to create, edite and exchange source files between Windows, Linux and Mac computers. The newline markers in text files are different on all these three systems. Sometimes, this leads to funny looking characters when files are created on one system and opened on another. Typically, the
unix2dos tools are used to convert text files between these formats.
In Vim, it is easy to convert the end-of-line markers of a file to that of Windows, Linux or Mac. This is done using the fileformat option.
To view the end-of-line markers of the currently open file:
To set the end-of-line markers of the file to Windows:
To set the end-of-line markers of the file to Linux:
To set the end-of-line markers of the file to Mac:
After changing the newline markers, save the file.
Tried with: Vim 7.3 and Ubuntu 12.04 LTS
A text file with multiple lines of text appears concatenated into a single line when opened in Notepad on Windows.
This happens because of the difference in the way the end-of-line is written by Linux (or other Unix-based OS) and Windows. If this text file was created on Linux and opened on Windows using Notepad, this problem occurs.
Linux uses the newline character (ASCII:
0x0A) to indicate end-of-line. Windows uses two characters: carriage return (ASCII:
0x0D) followed by newline (ASCII:
0x0A) to indicate end-of-line.
Almost any Windows editor other than Notepad, like Notepad++ for example, can understand both types of end-of-line indicators and can open such text files correctly.