you probably want to use rsync.
lets say you want to copy all the home directories from one machine to another.
as root:
rsync -a /home
root@my.other.server.ip:/home
look at rsync --help in order to see what all the options do. -a is a mix of a number of options (archive mode)
scp is "secure copy" basically it copies files over SSH from one box to another. that would also work in this situation but rsync is the more proper tool for the job.