Linux Server Details

User Tools

Site Tools

Trace: • fork_bomb_-_dos_a_linux_server_via_command_line
duplicity_-_secure_incremental_backup

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
duplicity_-_secure_incremental_backup [2022/07/19 20:13] – external edit 127.0.0.1duplicity_-_secure_incremental_backup [2022/07/23 23:14] (current) – paramiko required for duplicity and scp admin
Line 10: Line 10:
  
 ====== Install Duplicity Encrypted Backup ====== ====== Install Duplicity Encrypted Backup ======
 +<code>
  apt-get install duplicity  apt-get install duplicity
 + apt-get install python3-paramiko
 + #If you don't install python3-paramiko, you will get the error message:
 + "BackendException: Could not initialize backend: No module named 'paramiko'" when trying to use scp.
 +</code>
 Duplicity does not run as a service, and has no default global configuration file. It is more a program/application rather than a service. Duplicity can be run completely from the command line to backup or restore files. Duplicity does not run as a service, and has no default global configuration file. It is more a program/application rather than a service. Duplicity can be run completely from the command line to backup or restore files.
  
 ====== Simple unEncrypted Backup over SCP ====== ====== Simple unEncrypted Backup over SCP ======
 +
 Setup ssh keys on the backup server allowing root to seamlessly login to the backup server. Setting up ssh keys can be found here [[Sshkeys]]. Setup ssh keys on the backup server allowing root to seamlessly login to the backup server. Setting up ssh keys can be found here [[Sshkeys]].
- <nowiki>//</nowiki>Backup of a specific Folder:+<code> 
 + #Backup of a specific Folder:
  duplicity /home/me scp://uid@other.host/some_dir  duplicity /home/me scp://uid@other.host/some_dir
    
- <nowiki>//</nowiki>Restore specific Folder:+ #Restore specific Folder:
  duplicity scp://uid@other.host/some_dir /home/me  duplicity scp://uid@other.host/some_dir /home/me
- <nowiki>//</nowiki>or if you dont want to overwrite all files:+ #or if you dont want to overwrite all files:
  duplicity scp://uid@other.host/some_dir /var/tmp/me  duplicity scp://uid@other.host/some_dir /var/tmp/me
 +</code>
  
 ====== Encrypted Backup over FTP ====== ====== Encrypted Backup over FTP ======
Line 27: Line 35:
  
 To see available gpg keys: To see available gpg keys:
 +<code>
  gpg --list-keys  gpg --list-keys
 +</code>
  
 When running Duplicity on the command line (with no config file) - both the FTP password, and the GPG passphrase need to be exported to the environment variables. When running Duplicity on the command line (with no config file) - both the FTP password, and the GPG passphrase need to be exported to the environment variables.
 +<code>
  export FTP_PASSWORD=ftppass  export FTP_PASSWORD=ftppass
  export PASSPHRASE=gpgpassphrase  export PASSPHRASE=gpgpassphrase
    
  duplicity --encrypt-key "69111111" --sign-key "69111111" --include /etc --include /home --include /root --include /var --exclude /var/tmp --exclude '**' / ftp://username@backupserver/backupfoldername  duplicity --encrypt-key "69111111" --sign-key "69111111" --include /etc --include /home --include /root --include /var --exclude /var/tmp --exclude '**' / ftp://username@backupserver/backupfoldername
- <nowiki>//</nowiki>Syntax = duplicity | gpg_encrypt_and_sign | --include files | --exclude files | MAIN_FOLDER_to_BACKUP | DESTINATION +  
- <nowiki>//</nowiki>Thus the above line includes what I want, excludes '**' and backups / (root directory).+ #Syntax = duplicity | gpg_encrypt_and_sign | --include files | --exclude files | MAIN_FOLDER_to_BACKUP | DESTINATION 
 + #Thus the above line includes what I want, excludes '**' and backups / (root directory).
  
 +</code>
 If there are errors etc. you need to check whether the environment variables (ftp and gpp passwords) are set. Whether your gpg key exists and matches the id, e.g. 69111111.  If there are errors etc. you need to check whether the environment variables (ftp and gpp passwords) are set. Whether your gpg key exists and matches the id, e.g. 69111111. 
  
Line 43: Line 55:
  
 I made the following script and called it via a nightly [[Crontab]]. I made the following script and called it via a nightly [[Crontab]].
 +<code>
  #!/bin/bash  #!/bin/bash
  export FTP_PASSWORD=ftppass  export FTP_PASSWORD=ftppass
Line 48: Line 61:
    
  duplicity --encrypt-key "69111111" --sign-key "69111111" --include /etc --include /home --include /root --include /var --exclude /var/tmp --exclude '**' / ftp://username@backupserver/backupfoldername  duplicity --encrypt-key "69111111" --sign-key "69111111" --include /etc --include /home --include /root --include /var --exclude /var/tmp --exclude '**' / ftp://username@backupserver/backupfoldername
 +
 +</code>
  
 Also - you will have to manually backup the gpg key used for backup. Otherwise if you loose the key, or if it dies with the server - your backups are encrypted and you loose. I simply tar'd up the .gnupg folder in /root/ on the server been backup'd. Also - you will have to manually backup the gpg key used for backup. Otherwise if you loose the key, or if it dies with the server - your backups are encrypted and you loose. I simply tar'd up the .gnupg folder in /root/ on the server been backup'd.
Line 53: Line 68:
 ====== Restore Encrypted Backup from FTP ====== ====== Restore Encrypted Backup from FTP ======
 On the server you want to restore the backup to: On the server you want to restore the backup to:
 +<code>
  gpg --list-keys  gpg --list-keys
- <nowiki>//</nowiki>and make sure you have the key id 69111111. Otherwise it wont work!+ #and make sure you have the key id 69111111. Otherwise it wont work!
    
  mkdir /var/tmp/backupfoldername  mkdir /var/tmp/backupfoldername
Line 60: Line 76:
  export PASSPHRASE=gpgpassphrase  export PASSPHRASE=gpgpassphrase
  duplicity --encrypt-key "69111111" --sign-key "69111111" ftp://username@backupserver/backupfoldername /var/tmp/backupfoldername  duplicity --encrypt-key "69111111" --sign-key "69111111" ftp://username@backupserver/backupfoldername /var/tmp/backupfoldername
 +</code>
  
 Thats all folks. Thats all folks.
Line 65: Line 82:
 ====== Restore a single old or deleted File from the Encrypted Backup ====== ====== Restore a single old or deleted File from the Encrypted Backup ======
 If you want to restore or recover a specific file from a Backup, and dont want to bother restoring the whole backup only to get 1 file, you can do the following: If you want to restore or recover a specific file from a Backup, and dont want to bother restoring the whole backup only to get 1 file, you can do the following:
 +<code>
  duplicity --encrypt-key "" --sign-key "" --file-to-restore home/sburke/file.txt ftp://user@ftpserver/burkesys /var/tmp/file.txt  duplicity --encrypt-key "" --sign-key "" --file-to-restore home/sburke/file.txt ftp://user@ftpserver/burkesys /var/tmp/file.txt
 +</code>
  
 If the above file was deleted 1 day ago, and there was a backup since, you need to use the "-t 1D" option. E.g.: If the above file was deleted 1 day ago, and there was a backup since, you need to use the "-t 1D" option. E.g.:
 +<code>
  duplicity --encrypt-key "" --sign-key "" -t 1D --file-to-restore home/sburke/file.txt ftp://user@ftpserver/burkesys /var/tmp/file.txt  duplicity --encrypt-key "" --sign-key "" -t 1D --file-to-restore home/sburke/file.txt ftp://user@ftpserver/burkesys /var/tmp/file.txt
  #the number of days can be set, e.g. -t #D  #the number of days can be set, e.g. -t #D
 +</code>
  
 ====== Restore a folder from the Encrypted Backup ====== ====== Restore a folder from the Encrypted Backup ======
 Actually this is the same as above using the --file-to-restore switch! Actually this is the same as above using the --file-to-restore switch!
 +<code>
  duplicity --encrypt-key "" --sign-key "" --file-to-restore home/sburke/Maildir/cur ftp://user@ftpserver/burkesys /var/tmp/cur  duplicity --encrypt-key "" --sign-key "" --file-to-restore home/sburke/Maildir/cur ftp://user@ftpserver/burkesys /var/tmp/cur
 +</code>
  
 ====== List the Current Files in an Encrypted Backup ====== ====== List the Current Files in an Encrypted Backup ======
 +<code>
  duplicity --encrypt-key "" --sign-key "" --list-current-files ftp://user@ftpserver/folderbackupname  duplicity --encrypt-key "" --sign-key "" --list-current-files ftp://user@ftpserver/folderbackupname
 Note the "-t 1D" did not seem to work with the above. This is quite annoying, as a search for the filename of the file that was deleted cannot be carried out. The user must know the exact file and exact path they are looking for. Otherwise the entire backup needs to be Decrypted, and the file picked out. Note the "-t 1D" did not seem to work with the above. This is quite annoying, as a search for the filename of the file that was deleted cannot be carried out. The user must know the exact file and exact path they are looking for. Otherwise the entire backup needs to be Decrypted, and the file picked out.
 +</code>
  
 ====== Backup and Restore Debian Packages ====== ====== Backup and Restore Debian Packages ======
 +<code>
  dpkg --get-selections > selections-$(date -I)  dpkg --get-selections > selections-$(date -I)
  dpkg --set-selections < selections-$(date -I)  dpkg --set-selections < selections-$(date -I)
 +</code>
  
 ====== More Information on Duplicity ====== ====== More Information on Duplicity ======
 +<code>
  man duplicity (must have it installed via apt-get!)  man duplicity (must have it installed via apt-get!)
 +</code>
  
-http://www.debian-administration.org/articles/209 +  * http://www.debian-administration.org/articles/209 
- +  * http://duplicity.nongnu.org/ 
-http://duplicity.nongnu.org/ +  * http://savannah.nongnu.org/bugs/?2441  -> details the 405 error message
- +
-http://savannah.nongnu.org/bugs/?2441  -> details the 405 error message+
  
 ====== Offsite Backup Restore ====== ====== Offsite Backup Restore ======
Line 96: Line 123:
  
 So I used: So I used:
- duplicity restore --encrypt-key "xxx" --sign-key "xxx" --time-separator='_' file:/<nowiki>//</nowiki>media<nowiki>//</nowiki>servername /tmp/testrecover/+<code> 
 +duplicity restore –encrypt-key “xxx” –sign-key “xxx” –time-separator='_' file:///media//servername /tmp/testrecover/ 
 +</code> 
 At first duplicity said "No backup signatures found". This didn't sound good, but uncle google led me to: https://bugs.launchpad.net/deja-dup/+bug/601243 which said to use "--time-separator='_'" Luckily this worked and all was sorted. The reason I used the LiveCD was that it had the same version of Duplicity it used. At first duplicity said "No backup signatures found". This didn't sound good, but uncle google led me to: https://bugs.launchpad.net/deja-dup/+bug/601243 which said to use "--time-separator='_'" Luckily this worked and all was sorted. The reason I used the LiveCD was that it had the same version of Duplicity it used.
  
 As I also used a livecd (http://live.debian.net/) /tmp space for duplicity was limited. As a result, I added: As I also used a livecd (http://live.debian.net/) /tmp space for duplicity was limited. As a result, I added:
 +<code>
  --tempdir /mnt/tmp  --tempdir /mnt/tmp
  #see http://duplicity.nongnu.org/duplicity.1.html  #see http://duplicity.nongnu.org/duplicity.1.html
 +</code>
  
 A couple of things on the way: I wanted to connect my external USB drive so I had somewhere to extract from with the LiveCD. I found: http://xjqian.wordpress.com/2007/11/11/readwrite-ntfs-in-debian/ which provided: A couple of things on the way: I wanted to connect my external USB drive so I had somewhere to extract from with the LiveCD. I found: http://xjqian.wordpress.com/2007/11/11/readwrite-ntfs-in-debian/ which provided:
 +<code>
  apt-get install ntfs-3g  apt-get install ntfs-3g
  mount -t ntfs-3g /dev/hda1 /media/win -o umask=0,nls=utf8  mount -t ntfs-3g /dev/hda1 /media/win -o umask=0,nls=utf8
 +</code>
 I could now happily mount and write to my ntfs external drive. Unforunately I got some errors when extracting with duplicty. They were:  I could now happily mount and write to my ntfs external drive. Unforunately I got some errors when extracting with duplicty. They were: 
- Invalid or incomplete multibyte or wide character+ 
 +** Invalid or incomplete multibyte or wide character** 
 I think this was down to the ntfs drive I had. It was a file with a very long file name that caused the issue. Luckly I just worked around this folder with the following: I think this was down to the ntfs drive I had. It was a file with a very long file name that caused the issue. Luckly I just worked around this folder with the following:
- duplicity restore --encrypt-key "xxx" --sign-key "xxx" --time-separator='_' --file-to-restore home/username file:/<nowiki>//</nowiki>media/servername /tmp/testrecover/+<code> 
 +duplicity restore –encrypt-key “xxx” –sign-key “xxx” –time-separator='_' –file-to-restore home/username file:///media/servername /tmp/testrecover/ 
 +</code> 
duplicity_-_secure_incremental_backup.1658261600.txt.gz · Last modified: 2022/07/19 20:13 by 127.0.0.1