How do I copy an entire directory of files into an existing directory using Python? – Dev

The best answers to the question “How do I copy an entire directory of files into an existing directory using Python?” in the category Dev.

QUESTION:

Run the following code from a directory that contains a directory named bar (containing one or more files) and a directory named baz (also containing one or more files). Make sure there is not a directory named foo.

import shutil
shutil.copytree('bar', 'foo')
shutil.copytree('baz', 'foo')

It will fail with:

$ python copytree_test.py 
Traceback (most recent call last):
  File "copytree_test.py", line 5, in <module>
    shutil.copytree('baz', 'foo')
  File "/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/shutil.py", line 110, in copytree
  File "/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/os.py", line 172, in makedirs
OSError: [Errno 17] File exists: 'foo'

I want this to work the same way as if I had typed:

$ mkdir foo
$ cp bar/* foo/
$ cp baz/* foo/

Do I need to use shutil.copy() to copy each file in baz into foo? (After I’ve already copied the contents of ‘bar’ into ‘foo’ with shutil.copytree()?) Or is there an easier/better way?

ANSWER:

This limitation of the standard shutil.copytree seems arbitrary and annoying. Workaround:

import os, shutil
def copytree(src, dst, symlinks=False, ignore=None):
    for item in os.listdir(src):
        s = os.path.join(src, item)
        d = os.path.join(dst, item)
        if os.path.isdir(s):
            shutil.copytree(s, d, symlinks, ignore)
        else:
            shutil.copy2(s, d)

Note that it’s not entirely consistent with the standard copytree:

  • it doesn’t honor symlinks and ignore parameters for the root directory of the src tree;
  • it doesn’t raise shutil.Error for errors at the root level of src;
  • in case of errors during copying of a subtree, it will raise shutil.Error for that subtree instead of trying to copy other subtrees and raising single combined shutil.Error.

ANSWER:

Here’s a solution that’s part of the standard library:

from distutils.dir_util import copy_tree
copy_tree("/a/b/c", "/x/y/z")

See this similar question.

Copy directory contents into a directory with python

  • Reference – https://docs.python.org/3/distutils/apiref.html#distutils.dir_util.copy_tree

ANSWER:

In slight improvement on atzz’s answer to the function where the above function always tries to copy the files from source to destination.

def copytree(src, dst, symlinks=False, ignore=None):
    if not os.path.exists(dst):
        os.makedirs(dst)
    for item in os.listdir(src):
        s = os.path.join(src, item)
        d = os.path.join(dst, item)
        if os.path.isdir(s):
            copytree(s, d, symlinks, ignore)
        else:
            if not os.path.exists(d) or os.stat(s).st_mtime - os.stat(d).st_mtime > 1:
                shutil.copy2(s, d)

In my above implementation

  • Creating the output directory if not already exists
  • Doing the copy directory by recursively calling my own method.
  • When we come to actually copying the file I check if the file is modified then only
    we should copy.

I am using above function along with scons build. It helped me a lot as every time when I compile I may not need to copy entire set of files.. but only the files which are modified.

ANSWER:

Python 3.8 introduced the dirs_exist_ok argument to shutil.copytree:

Recursively copy an entire directory tree rooted at src to a directory named dst and return the destination directory. dirs_exist_ok dictates whether to raise an exception in case dst or any missing parent directory already exists.

Therefore, with Python 3.8+ this should work:

import shutil

shutil.copytree('bar', 'foo')
shutil.copytree('baz', 'foo', dirs_exist_ok=True)