How to Migrate From an Old NAS to a New One Overnight Using Rsync
A NAS or network attached storage device is great for storing files that you can access from any computer in the house. But when you upgrade to a new one, you get stuck copying everything manually, swapping drives and risking data loss. Here is a much more reliable method.
I recently bought a new NAS that has a lot more space than my old one. This is great, but once I set it up, I got stuck with the task of transferring terabytes of data from the new NAS. I didn’t want to physically move the disks as the old disks were smaller than the new ones. Since I was migrating from one Synology NAS to another, I could use their migration tool and you would be right, but if I were migrating from Synology to ReadyNAS or FreeNAS to Synology, I wouldn’t have that option. I need a platform independent method. After pondering over a dozen or so ways to get the job done, I settled on one of our old command line favorites: rsync .
Why rsync is the best file transfer tool
Now I hear you, “Why use the command line for this? I could use [X METHOD] to do the same, ”and you’d be right! There are many ways to do this job, but rsync has several significant advantages over just opening a couple of windows on your PC and dragging and dropping files from one to the other. Here are the biggest ones:
- rsync is probably already on your NAS . Whether your NAS is a standard enclosure filled with disks or a DIY model running your preferred operating system , it is most likely running the Linux variant. This means the command line and rsync are already installed. Of course, if you are moving from one brand of NAS to another of the same brand, built-in applications can help you, but the beauty of rsync is that it works anywhere, regardless of the brand and size of the disks or volumes.
- rsync preserves metadata . This means that things like ownership of a file, original creation date, modification dates, file permissions, and all of those things are preserved when your copy is sent to a new destination. Regardless of users and permissions on the new system, if you don’t want all of your files to get new creation dates, generate new thumbnails, or things like revision history are important to you, this is a good way to make sure this information is preserved.
- rsync removes the middleman . When you use your computer to copy files from one system to another, your computer acts as an intermediary between the two NAS. This creates another point of failure in transmission, not to mention that your host computer is overloaded. If you’ve ever performed lengthy copy operations on multiple files in Windows, you know that it can be annoying to say the least, and even the smallest thing can mess up that copy. When this happens, you get stuck trying to figure out where the copy failed, why, and where to start over. Since rsync is a copy from one device to another, it works whether your computer is turned on or not.
- rsync supports resumed transfers, diffs, and sync . If for some reason your transmission dies, such as due to an unexpected power outage, a system error, or any other issue, rsync can resume from where you left off without issue. Likewise, since rsync was designed to keep directories in sync (and keep them in sync), if you need to sync the old NAS with the new one, or you accidentally dump new files on the old NAS that should be on the new one.First, rsync can move they are for you and will only affect the changed or new files in the process.
- rsync minimizes network overhead . It might not matter to you if you do it overnight – which you probably should – but one of the best things about rsync is that it doesn’t carry the same network load as other tools. This means that your other backups or downloads won’t slow down to scan just because you are using them, and other people on your network probably won’t notice anything at all.
As we mentioned, rsync is for syncing files, not just copying them. We’ve already shown you how to mirror systems with it and how to sync iTunes to any USB device , but it’s also only useful for direct copies between machines because it’s so flexible. It can even provide you with a handy progress bar so you can make sure everything is going smoothly.
Step 1. Turn on remote access and choose a transfer method
Another nice feature of rsync is that it is just a command line. However, you can’t just fire up a terminal window and start copying – you need to do a little initial setup.
First, you need to make sure SSH access is enabled on both of your old NAS. SSH allows you to log into the NAS from the command line using a secure shell. On my Synology NAS, it was right under the dashboard labeled Terminal & SNMP. You will most likely find it in a similar location on your NAS, but usually it is under System, Remote Access, or any other synonym for “remote control”.
When you have remote access enabled on both NAS and you have administrator (or root) credentials for both systems, you are ready to go. We are now ready to log into our old NAS and move our files to the new one.
How to use rsync to transfer files
It’s time to get down to business. Use your favorite SSH tool (on Linux or OS X you can just open a terminal window, on Windows I like PuTTY for that) to log into your old NAS. On OS X or Linux, just open a terminal window and enter the following:
slogin admin@[IP ADDRESS OF YOUR OLD NAS]
You will be prompted to log in, and once you have done so, you will be taken to a new command line representing your NAS. On Windows, open PuTTY. In the hostname field, enter the IP address of your old NAS, make sure SSH is selected, and click Open. You will be prompted for your username (admin or root) and password. Once you are logged in, you will be taken to the command line of your NAS.
It’s time to start rsync. The syntax is pretty straightforward. You will need to tell rsync how to log into the remote device and where to put the files – all with a single command. Here’s how:
rsync -azP [SOURCE DIRECTORY] admin@[IP ADDRESS OF YOUR NEW NAS]:[SOURCE DIRECTORY]
So let’s say I have a folder on my old NAS named “old_movies” and a folder on my new “new_movies”. To copy everything inside “old_movies” to “new_movies”, the command would look like this:
rsync -azP old_movies/ [email protected]:new_movies
Where 192.168.1.X is the IP address of your new NAS. Once launched, you will be prompted to ensure that the SSH key for the new NAS is indeed correct and that you want to save it for future use. You may get a key warning the first time you connect, but that’s okay. Enter yes to continue. You will be prompted to log into the new NAS with an administrator password. Come on, do it too. If everything else went well, as soon as you do that, the files will sync.
Before we continue, let’s talk about these flags – ” -azP ” – that you see in the above command. They are important, and here’s why:
- The “-a” flag stands for “archive” and it is important to enable it. It makes sure the sync command is recursive (meaning that all subfolders and files inside old_movies are copied as well), and this is important to keep all those modification dates, symlinks, permissions, and other useful properties that we talked about earlier.
- The “-P” flag combines two other useful flags into one that you will want to use. It combines the flags “progress” and “partial”, and when used, you will get a progress dialog on the command line that shows which file is currently being transferred, what percentage of that transfer is complete, and how many files are left to check. As each file completes, you will see an ever-growing list of completed file transfers, which is great for ensuring that all data has been transferred successfully. It also makes it easy to resume paused or interrupted transfers. Collectively, you can see how it will show you which file was last uploaded, where it crashed, and if it failed, give you the option to resume. This is a pretty powerful combination.
- The “-z” flag is a handy rsync tool that compresses files in transit, giving you all the benefits we discussed in “low bandwidth.” If the files are already compressed, you won’t get much benefit here, but if they aren’t, it will get the job done without slowing down the rest of your home network.
You should double check your command before copying, just to make sure you are copying to the correct directory and directory structure.
When it comes time to move your entire NAS, you can do that directory by directory, or you can do it like I did, in one fell swoop, overnight. Make one small directory as a test to make sure you have the correct syntax, and when that works, go to the parent directory and copy everything at once. It will take a long time, but when you’re done, your new NAS will be configured exactly like your old one and you’re good to go.
If you are a stickler and prefer granular control, create your directories and shares on the new NAS, then simply copy the contents of each directory from the old NAS to the new one. This way, you’ve created directories and granted them permissions using your NAS’s interface, and the files inside will retain all the permissions they had.
Troubleshooting, catalog updates and more information
If you run into copy problems or other weird, bizarre problems, search Google for error messages and the OS of your old or new NAS. You will be surprised what you find and how many people have faced the same problem as you. In my case, when I migrated my Synology NAS, I ran into some bizarre “rsync service not running” errors when I tried to send or receive files, only to find out that I needed to make sure Synology “Network Backup Service” and ” Network Destination Service has been enabled on both the source and destination NAS – two checkboxes are hidden deep in the Service tab in the Information Center in Control Panel. I also needed to make sure I was logged in as root – logging in as admin did not stop me. Luckily Stefan came to my rescue with this helpful bug post .
Likewise, Justin Ellingwood ‘s step-by-step guide to rsync on the DigitalOcean Community Forums is very helpful if you want to learn the ins and outs of rsync for moving files between systems, and helped me determine the best flags to use and when to use that trailing slash and when not to. If you accidentally write data to an old NAS instead of a new one and need to run differential sync, that is, use rsync to simply copy changed or other files since your last sync, it has instructions on how to do it. … If you’re familiar with rsync, this step-by-step guide in How-to Geek will help you take those skills to the next level.
After transferring all the files, you can do whatever you want with the old NAS. Turn it into a download box , home theater system, use it for unimportant files or backups , whatever. If you plan to sell or dispose of it, make sure you wipe it down properly and clean it beforehand. If you’re lucky, you can get some money for it.