0

rsync is working great, with the proper exclusions so it can be run from my Raspbian systems that are on 24x7. But it thinks it is the only program on the network and tries to take over.

Of course, I use Win32DiskImager every few months to make sure I have a known, good copy of the setup for each of my 7 Raspbian sensor systems scattered about. Some of these systems operate outside in the SoCal heat. SD cards work great, until they don't.

Every night at midnight the cron scripts fire up to collect the day's data and put it all on hard disk.

To protect against forgetting the system files (like in /etc) that I have changed since the last SD card backup, I run rsync about once a week with options that make it act a lot like Windows Robocopy with the /MIR option.

However, it is really hogging the bandwidth of the PiNet subnet. Enough to block other traffic.

It also seems to stop for many minutes at kern.log while pushing more than 3-4 Mbps, for a long time.

When I look at kern.log it is usually just informational messages having to do with the WiFi drivers - useless information.

How do I solve this?

SDsolar
  • 2,378
  • 8
  • 26
  • 43

1 Answers1

0

Turns out that rsync has a built-in throttling mechanism.

Simply add --bwlimit=1000 and this will limit it to 1 Mbps. Use whatever speed works for you.

Then by using the command

du -sch /var/log

it was obvious that log files were huge. About 8 GB.

So then by using 

du -sch /var/log/* | sort

it became clear that the kern log files (and their rotations) were taking up about 3.9 GB each.

Plus, messages.log and all the others were taking up another 3.9 GB.

The solution to this is to use a script for the rsync that includes these commands:

sudo rm /var/log/kern*
sudo rm /var/log/messages*

They will get recreated when they are needed again.

This makes for a much happier network and saves space.

SDsolar
  • 2,378
  • 8
  • 26
  • 43