If you try to
mv a large number of files or directories, you will encounter this error:
$ cd big_dir $ cp * ../../some_other_dir Failed to execute process '/bin/cp'. Reason: The total size of the argument and environment lists 3.2MB exceeds the operating system limit of 2MB. Try running the command again with fewer arguments.
Note that you can copy or move one directory that contains a million files or any number of files for that matter. This error happens only if the shell has to pass a large number of input arguments to the
mv programs. So, if you run the above command from inside a directory containing 100K files, you will surely get this error.
This is because there is a limit to the size of the arguments and environment strings that can be passed to a program. That limit can be queried:
$ getconf ARG_MAX 2097152
The result will vary on different computers. But they always have a limit and it is encoded in the Linux kernel. There does not seem to be an userspace method to increase this size.
Instead of looking to increase the
ARG_MAX size, examine the real problem. Why make the shell expand all the filenames and pass them as one gigantic list of strings to the programs? Instead these alternate solutions can be tried:
- See if you can instead
mva parent directory, instead of a million files.
Move the files one by one by writing a loop in shell script or Python.
Use other programs like rsync to copy a directory to the destination.
Reference: Search for
ARG_MAX in the execve(2) manpage
Tried with: Ubuntu 16.04