Size of argument and environment lists exceeds limit


If you try to cp or mv a large number of files or directories, you will encounter this error:

$ cd big_dir
$ cp * ../../some_other_dir
Failed to execute process '/bin/cp'. Reason:
The total size of the argument and environment lists 3.2MB exceeds the operating system limit of 2MB.
Try running the command again with fewer arguments.

Note that you can copy or move one directory that contains a million files or any number of files for that matter. This error happens only if the shell has to pass a large number of input arguments to the cp or mv programs. So, if you run the above command from inside a directory containing 100K files, you will surely get this error.

This is because there is a limit to the size of the arguments and environment strings that can be passed to a program. That limit can be queried:

$ getconf ARG_MAX

The result will vary on different computers. But they always have a limit and it is encoded in the Linux kernel. There does not seem to be an userspace method to increase this size.


Instead of looking to increase the ARG_MAX size, examine the real problem. Why make the shell expand all the filenames and pass them as one gigantic list of strings to the programs? Instead these alternate solutions can be tried:

  • See if you can instead cp or mv a parent directory, instead of a million files.

  • Move the files one by one by writing a loop in shell script or Python.

  • Use other programs like rsync to copy a directory to the destination.

Reference: Search for ARG_MAX in the execve(2) manpage

Tried with: Ubuntu 16.04


GCP: An alternate to cp

gcp showing progress of a file copy
gcp showing progress of a file copy

cp is the tool used in Linux to copy files and directories. GCP aims to be an alternate to this ubiquitous tool.

Installing it is easy:

$ sudo apt install gcp

One of its compelling features is that it shows a bar and stats of the progress of the copy operation. This can be useful when copying lots of files or to a slow disk.

An alternate to this program is Advanced Copy tool. However, that website seems to be down for sometime now.

Tried with: GCP 0.1.3 and Ubuntu 14.04

How to copy or move from shell with progressbar

When I am copying or moving a big file or a large number of small files, I wish I could see some progress of the operation and the speed at which the operation is being performed. This can be done by using the Advanced Copy mod of the cp and mv commands, which have been patched to display a progressbar with all this information during the operation. You can either get the source code or pre-built binaries from here.

I like to use these commands by default, so I move them to /usr/local/bin:

$ sudo cp ./cp /usr/local/bin
$ sudo cp ./mv /usr/local/bin

Typically, your PATH puts /usr/local/bin ahead of /bin, so the patched versions of the commands will be executed now. Else, change your PATH to accomplish this.

To view the progressbar for a operation, use the -g option:

$ cp -g -R foo-big-file.bin /media/joe/backup/

I use the fish shell, where I have added aliases for both of these commands with progressbar display:

function cpg
    cp -g $argv

function mvg
    mv -g $argv

Tried with: Ubuntu 14.04