Size of argument and environment lists exceeds limit

Problem

If you try to cp or mv a large number of files or directories, you will encounter this error:

$ cd big_dir
$ cp * ../../some_other_dir
Failed to execute process '/bin/cp'. Reason:
The total size of the argument and environment lists 3.2MB exceeds the operating system limit of 2MB.
Try running the command again with fewer arguments.

Note that you can copy or move one directory that contains a million files or any number of files for that matter. This error happens only if the shell has to pass a large number of input arguments to the cp or mv programs. So, if you run the above command from inside a directory containing 100K files, you will surely get this error.

This is because there is a limit to the size of the arguments and environment strings that can be passed to a program. That limit can be queried:

$ getconf ARG_MAX
2097152

The result will vary on different computers. But they always have a limit and it is encoded in the Linux kernel. There does not seem to be an userspace method to increase this size.

Solution

Instead of looking to increase the ARG_MAX size, examine the real problem. Why make the shell expand all the filenames and pass them as one gigantic list of strings to the programs? Instead these alternate solutions can be tried:

• See if you can instead cp or mv a parent directory, instead of a million files.

• Move the files one by one by writing a loop in shell script or Python.

• Use other programs like rsync to copy a directory to the destination.

Reference: Search for ARG_MAX in the execve(2) manpage

Tried with: Ubuntu 16.04