Data Migration using emcopy

emc’s emcopy (part of emc’s rather elusive but freely downloadable if you can find it CIFS_Tools.zip package) is akin to Microsoft’s own robocopy, only faster with extra benefits.  The command syntax is much like robocopy so if you’re familar with that, then using emcopy will be easy.


emcopy lets you copy a file or directory (and included subdirectories) from and to an
NTFS partition, keeping security the same on the copy as on the original.
It allows you to back up the file and directory security—ACLs, owner information,
and audit information—from a source directory to a destination directory without copying
the file data.  It defaults to 64 threads but can be told to use up to 256 threads, massively speeding up copy operations.

In practice, emcopy vs robocopy – I have found it to be much faster than robocopy.

Download it from here…

https://download.emc.com/downloads/DL32449_CIFS_Tools.zip.zip

Official documentation here…

https://mydocs.emc.com/VNXDocs/CIFS_Environment_Utilities.pdf

Working examples of emcopy commands below, with differences highlighted.

emcopy_v4.14.exe “S:\My Source Folder” D:\My_Dest_Folder *.*  /s /secfix /o /a /i /d /c /th 32 /r:5 /w:3

emcopy_1

 

 

 

 

 

emcopy_v4.14.exe “S:\My Source Folder” D:\My_Dest_Folder *.* /s /nosec /o /a /i /d /c /th 256 /r:5 /w:3

emcopy_2

 

 

 

 

 

 

A more elaborate example showing how to exclude multiple directories specified by full path or just as a name

emcopy_v4.14.exe “S:\My Source Folder” D:\My_Dest_Folder *.* /xd “System Volume Information” homedirs profiles wtsprofiles /s /secfix /o /a /i /d /c /th 32 /r:5 /w:3

I personally prefer to deal with each top level folder as a separate emcopy run, using a for loop as follows…

I have found issues using for loops with robocopy that may similarly affect emcopy whereby top level folders containing spaces still prove problematic despite the use of “delims=” in the for loop.

for /f “delims=” %%f in (‘dir S:\ /ad /b’) DO emcopy.exe “s:\%%f” “d:\%%f” *.* /nosec /a /o /s /i /de /c /th 16 /r:1 /w:1

You may want to create the top level of folders first before adopting this technique to perform the subsequent folder structure creation and or file copy…

for /f “delims=” %%f in (‘dir S:\ /ad /b) DO emcopy.exe “s:\” “d:\root\” /s /xf * /create /r:1 /w:1

Note I omitted the use of /secfix as in practice I found not using /secfix has less problems than using it.  It appears to force re-inheritance from parents which can create as many problems as it promises to fix.

emcopy is just one of a handful of adjacent tools in the CIFS_Tools.zip pack, lgcopy can be used to migrate local groups from Windows to a CIFS Server so that emcopy’s /secfix switch can do it’s thing and there are other useful tools such as sharedup for copying CIFS shares from one NAS to another.  More detailed information is available in the emc documentation.

It’s worth reading this post too…

http://www.cyberfella.co.uk/2014/05/22/data-migration-using-robocopy/

…especially if you want to exclude more than one folder, or are experiencing issues with timestamps when migrating to a NAS.

Copying an empty directory structure using emcopy

If you need to copy just the folder structure, but leave the old security behind, then this command works a treat.

emcopy s: d: /s /xf * /nosec /create

If you want to create a duplicate folder structure including ACL’s for subsequent re-ACL’ing using SetACL, leave out /nosec but don’t use /secfix.

emcopy s: d: /s /xf * /create /r:1 /w:1  

This is useful as part of a data migration to a new domain, where the security exported using SystemTools DumpACL can then be tweaked and reapplied to the new empty directory structure using icacls, prior to an emcopy of the data including files without security afterward.

icacls “C:\CYBERFELLA_LTD\PermsTestACLFolder” /grant “CYBERFELLALTD\GlobalGroup1”:(OI)RX

If you need to re-sync folder security between source and destination data, then robocopy can be used to achieve this.  See here

[paypal-donation]

Did you like this?
Tip cyberfella with Cryptocurrency

Donate Bitcoin to cyberfella

Scan to Donate Bitcoin to cyberfella
Scan the QR code or copy the address below into your wallet to send some bitcoin:

Donate Bitcoin Cash to cyberfella

Scan to Donate Bitcoin Cash to cyberfella
Scan the QR code or copy the address below into your wallet to send bitcoin:

Donate Ethereum to cyberfella

Scan to Donate Ethereum to cyberfella
Scan the QR code or copy the address below into your wallet to send some Ether:

Donate Litecoin to cyberfella

Scan to Donate Litecoin to cyberfella
Scan the QR code or copy the address below into your wallet to send some Litecoin:

Donate Monero to cyberfella

Scan to Donate Monero to cyberfella
Scan the QR code or copy the address below into your wallet to send some Monero:

Donate ZCash to cyberfella

Scan to Donate ZCash to cyberfella
Scan the QR code or copy the address below into your wallet to send some ZCash:

3 Replies to “Data Migration using emcopy”

  1. It’s worth my mentioning that recently while using emcopy to migrate specific subfolders, ignoring surrounding adjacent folders, I discovered that emcopy has a bug whereby it does not ignore all folders specified by the /xd switches – only the last one. This is a real shame, and I hope it gets fixed.

    Robocopy on the other hand does ignore all directories specified by multiple /xd directory exclusions.

    I also recently discovered that if migrating from a Windows NTFS volume to a CIFS filesystem on a Celerra / VNX, that robocopy generally thinks files are newer on the source side, even when they are not. There is a workaround already available in robocopy (even in the older versions of it) by specifying the /fft switch. This brought a 36 hour copy back down to just over an hour in practice.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.