SSH keygen under Cygwin

You will need to install the "openssh" package using the Cygwin installer before doing this.

Notes:

• In the near future, OpenSSH is deprecating support for DSA keys.
• Minimum supported RSA key size will be 1024 bits, but you should really do 2048 or larger.

Typical steps for creating SSH keys:

$ mkdir ~/.ssh

$ chmod 700 ~/.ssh

$ cd ~/.ssh

$ ssh-keygen.exe -t rsa -b 3200 -C 'Borg backup key thomast61p July 2015' -N '' -f ~/.ssh/ssh-borg-backup-july2015

In this particular case, I am creating a RSA/3200 key with no password (-N '') and with a comment indicating that it will only be used for Borg backups.  Because the key has no password, I should only use it in conjunction with the borg backup command on a separate server-side account that has very limited permissions on the server.

Installing borg backup (fork of Attic) on Cygwin (Windows)

I've used Cygwin with rdiff-backup before to backup a Windows box to a backend Linux server over SSH, but given the success that I've had with Attic backup, I'm going to try this with the fork of Attic which is called "Borg".

Step #1 - Download and install Cygwin.  The following packages that need to be installed before you can install borg backup.

binutils (not sure)
gcc-g++
libuuid-devel (not sure)
openssl-devel
python3
python3-setuputils
wget

Step #2 is to install "pip" inside the Cygwin environment.

$ cd ~/
$ mkdir -p ~/downloads/python
$ cd ~/downloads/python
$ wget https://bootstrap.pypa.io/get-pip.py
$ python3 get-pip.py
$ pip3 --version

The "pip" command should return a version number string if it is installed.

Step #3

$ pip3 install borgbackup

Step #4

At this point, if you type "borg" at the command prompt, you should see information about Borg such as the version and list of commands.

md5sum bash script to create check file for a directory tree

Just a quick script that will run through the current directory and all descendant directories, creating a single "verify-tree.md5" file (using md5sum). If the check file already exists, then the existing one gets moved out of the way to "verify-tree.yyyymmdd-hhmmss.md5" for safekeeping.

Check files are useful for whenever you have a set of files that will not (or should not) change over time.  Such as files written off to an archive tape / disk / optical media / flash drive.  While the media and file system might also do some checking, it's good to have a second layer that is under your control and which can be queried.

(If you want file recovery features, you should look into MultiPar or par2j.)

The script's output looks like:

The file (verify-tree.md5) that is created is a standard md5sum file and can be read by just about any compatible software.  Some software cannot handle sub-folders, however, so you may have to use the md5sum program to do the verification.

----------------------------------------------------------------
#!/bin/bash

# stop script on errors
set -e

PROG=md5sum
FILENAME=verify-tree

if [[ -e "${FILENAME}.md5" ]]; then
    mv "${FILENAME}.md5" \
    "${FILENAME}.$(date --reference=${FILENAME}.md5 '+%Y%m%d-%H%M%S').md5"
fi

echo ""
echo "Output Filename: ${FILENAME}.md5"
echo "Files Found: $(find . -type f -not -name "${FILENAME}*.md5" | wc -l)"
echo "Size: $(du -chs . | grep 'total')"

time find . -type f -not -name "${FILENAME}*.md5" \
    -exec ${PROG} "{}" \; >> "${FILENAME}.md5"

echo ""
echo "Files Processed: $(wc -l ${FILENAME}.md5)"

echo ""
echo "Checking..."
time ${PROG} -c --quiet "${FILENAME}.md5"
echo ""
echo "All files verified."

----------------------------------------------------------------

Note: There are multiple ways to write the md5sum line.  It's probably better to use the xargs method, but I have not tested it out.

find . -type f -not -name "${FILENAME}*.md5" -exec ${PROG} "{}" \; >> "${FILENAME}.md5

find . -type f -not -name "${FILENAME}*.md5" -print0 | xargs -0 ${PROG} >> "${FILENAME}.md5"

find . -type f -not -name "${FILENAME}*.md5" | xargs ${PROG} >> "${FILENAME}.md5"

It's also easy to adapt this script to work with sha256sum or sha1sum.

The verification script is a trimmed down version of the original script:

----------------------------------------------------------------

#!/bin/bash

# stop script on errors
set -e

PROG=md5sum
FILENAME=verify-tree

echo ""
echo "Checking: ${FILENAME}.md5"
echo "File Hashes: $(wc -l ${FILENAME}.md5)"
time ${PROG} -c --quiet "${FILENAME}.md5"
echo ""
echo "All files verified."

----------------------------------------------------------------

I have tested this in CygWin 64-bit on Windows 7 64-bit Professional, but it should also work fine on Linux/Unix/OSX systems as long as the md5sum command is available.  The script is conservative in design with very few "tricks" so it should be portable.

rsync.conf file for Cygwin environments

You should definitely refer to the official rsync website for the real documentation on configuring the rsyncd.conf file.

Locate your /etc folder under where you installed Cygwin. Since I installed Cygwin to C:\bin\cygwin, my /etc folder is C:\bin\cygwin\etc. For a fresh install, you'll need to create the "rsyncd.conf" file in that folder (C:\bin\cygwin\etc\rsyncd.conf).

(minimal rsyncd.conf file)

use chroot = false

strict modes = false

log file = rsyncd.log



[test]

path = /cygdrive/d/rsync/test

read only = false

transfer logging = yes

Minimal Cygwin install for RSync and SSH

Source links:

How to setup the secure shell daemon on a Windows 2000 machine?
Windows Rsync Server Setup
CygwinInstallationGuide (a wiki topic about the cygwin installation)

Note: The following probably doesn't work (probably missing a package, or the fact that I have GNU's unix tools for Win32 installed is problematic), but I might come back and make it work later so I'm leaving it here for now. I ran into trouble when trying to configure SSH. Right now, I've gone back to my original plan of either hacking apart the Cygwin files and manually copying only the DLLs and EXEs that I need or using the OpenSSH for Windows project at SourceForge.

1. Run the Cygwin setup.exe file and start the instllation. I chose to install to "c:\bin\cygwin", but left the rest of the options "as-is". Pick your mirror (use the Cygwin public mirrors page to find one close to you).

2. On the "Select Packages" screen, select the "Curr" option and make sure it says "Category" next to the "View" button at the top. The installation dialog is (finally) re-sizeable, so stretch it out or maximize it so you can see all of the columns.

3. Beside the "+All" category, it will say "Install", "Uninstall", ... click on the word until all of the categories say "Uninstall". (Note: These steps assume that you're doing a new Cygwin install and that you don't already have Cygwin installed.) Now we can start picking the minimum number of packages required to setup SSH and RSync.

4a. Under the "+Admin" category, you'll need to install the "cygrunsrv" package (click once on the "Skip" indicator under the "New" column). This will turn on a few other packages that this package depends on (mostly under the "+Base", "+Libs", and "+Shells" categories).

4b. Open up the "+Net" category and select the "rsync" and "openssh" packages. You'll also end up with "openssl" which is required in order to use "openssh".

5. Click the "Next" button to start downloading and installing the packages. If the download fails, choose another mirror, double-check your package selections (my copy remembered which packages I had already selected), and try again. The base install size required around 7MB of downloads and expanded out to 24MB (34MB actual due to a 4KB cluster size).

6. Fire up the cygwin shell, you should see a command-line window open with a "$" prompt. Try out a few unix commands (pwd, ls, whoami) to see if things are working.

7. Further steps... (I'll cover these in future posts)

a) Setup your rsync.conf file (in the "etc" folder)
b) create a service account for use by the rsync service
c) create a Windows service using the "cygrunsvc" tool
d) setup OpenSSH and then re-configure rsync to use it