Walter vl: /* Script to backup to local folders (database + multiple folders) */ undo mtime warning

2021-02-14T19:03:16Z

Script to backup to local folders (database + multiple folders): undo mtime warning

New page

==Goal==
Domoticz has a built in feature to automatically create hourly, daily and monthly database backups in the <code>/home/pi/domoticz/backups/</code> directory. You just need to tick the Enable Automatic Backup option in System setup. 
However in the case the SD-card crashes and is no longer accessible, a lot of work still could be lost. 
This page will provide you several backupscripts, each with other functions.

==Preparation==
Search the folder on your Network Attached Storage (NAS) where you want the backups to be stored. Check that FTP is enabled and have login credentials available. 

Make sure that if you have enabled website protection, e.g. if your Domoticz webpage is password protected, that you enable Local Networks (no username/password)in your settings. Fill in your local network IP-address (<code>192.168.1.*</code> for example), otherwise the backup will fail.

Place the script below in:<code>/home/pi/domoticz/scripts</code> and change the parameters to reflect to your settings.

==Different scripts==
===Script to backup to FTP-server (only Domoticz database)===
The script below will save a copy of your 'domoticz.db' (database) to the folder specified. 
However, you probably also use some scripts that you want to backup too, so you probably want the script further down this page.
<syntaxhighlight lang="bash">
#!/bin/bash
# LOCAL/FTP/SCP/MAIL PARAMETERS
SERVER="192.168.0.10" # IP of Network disk, used for ftp
USERNAME="root" # FTP username of Network disk used for ftp
PASSWORD="root" # FTP password of Network disk used for ftp
DESTDIR="/opt/backup" # used for temorarily storage
DOMO_IP="192.168.0.90" # Domoticz IP
DOMO_PORT="8080" # Domoticz port
### END OF USER CONFIGURABLE PARAMETERS
TIMESTAMP=`/bin/date +%Y%m%d%H%M%S`
BACKUPFILE="domoticz_$TIMESTAMP.db" # backups will be named "domoticz_YYYYMMDDHHMMSS.db.gz"
BACKUPFILEGZ="$BACKUPFILE".gz
### create backup, ZIP it and start Domoticz again
/usr/bin/curl -s http://$DOMO_IP:$DOMO_PORT/backupdatabase.php > /tmp/$BACKUPFILE
gzip -9 /tmp/$BACKUPFILE
### Send to Network disk through FTP
curl -s --disable-epsv -v -T"/tmp/$BACKUPFILEGZ" -u"$USERNAME:$PASSWORD" "ftp://$SERVER/media/hdd/Domoticz_backup/"
### Remove temp backup file
/bin/rm /tmp/$BACKUPFILEGZ
### Done!
</syntaxhighlight>

===Script to backup to FTP-server (multiple folders)===
The script below will backup the folder specified (<code>/home/pi/domoticz/aaax/</code>, change aaax to the folder you want to have backupped) to the FTP-location specified. 
This script doesn't backup the domoticz.db database, see the script above.
<syntaxhighlight lang="bash">
#!/bin/bash

## LOCAL/FTP/SCP/MAIL PARAMETERS
SERVER="192.168.5.10" # IP of Network disk, used for: ftp mail scp
USERNAME="root" # FTP username of Network disk used for: ftp mail scp
PASSWORD="root" # FTP password of Network disk used for: ftp mail scp
DESTDIR="/opt/backup" # used for: local
DOMO_IP="192.168.5.75" # Domoticz IP used for all
DOMO_PORT="8081" # Domoticz port used for all
## END OF USER CONFIGURABLE PARAMETERS

TIMESTAMP=`/bin/date +%Y%m%d%H%M%S`
BACKUPFILEDIR="domoticz_x_aaa_$TIMESTAMP.tar.gz" # Change the aaa to yours

### Create backup and ZIP it
tar -zcvf /tmp/$BACKUPFILEDIR /home/pi/domoticz/aaax/ # Change the aaa to yours # Or try /home/pi/domoticz/

### Send to Network disk through FTP
curl -s --disable-epsv -v -T"/tmp/$BACKUPFILEDIR" -u"$USERNAME:$PASSWORD" "ftp://$SERVER/media/hdd/Domoticz_backup/" # Change the ftp to yours !!!

### Remove temp backup file
/bin/rm /tmp/$BACKUPFILEDIR

### Done!

</syntaxhighlight>

===Script to backup to local folders (database + multiple folders)===
The script below can be used to create backups and store them on the same device. Storing backups on the same device is not recommended, but i have placed them in a folder that gets copied every night to an off-site NAS.<syntaxhighlight lang="bash">
#!/bin/bash
DOMO_IP="192.168.4.4" # Domoticz IP
DOMO_PORT="8084" # Domoticz port
### END OF USER CONFIGURABLE PARAMETERS
TIMESTAMP=`/bin/date +%Y%m%d%H%M%S`
BACKUPFILE="domoticzbackup_$TIMESTAMP.db" # backups will be named "domoticz_YYYYMMDDHHMMSS.db.gz"
BACKUPFILEGZ="$BACKUPFILE".gz

#Create backup and make tar archives
/usr/bin/curl -s http://$DOMO_IP:$DOMO_PORT/backupdatabase.php > /volume1/dataschijf/Domoticz_backup/database/$BACKUPFILE
tar -zcvf /volume1/dataschijf/Domoticz_backup/scripts/domoticz_scripts_$TIMESTAMP.tar.gz /volume1/@appstore/domoticz/var/scripts/
tar -zcvf /volume1/dataschijf/Domoticz_backup/www/domoticz_wwwfolder_$TIMESTAMP.tar.gz /volume1/@appstore/domoticz/www/

#Delete backups older than 31 days
/usr/bin/find /volume1/dataschijf/Domoticz_backup/database/ -name '*.db' -mtime +31 -delete
/usr/bin/find /volume1/dataschijf/Domoticz_backup/scripts/ -name '*.tar.gz' -mtime +31 -delete
/usr/bin/find /volume1/dataschijf/Domoticz_backup/www/ -name '*.tar.gz' -mtime +31 -delete
</syntaxhighlight>

===Script to backup to a Synology NAS===
The script below will backup (<code>Domoticz Database & Domoticz scripts & Telegram Scripts</code>) to the FTP-location specified. 
Don't forget to give the Synology useraccount FTP right in DSM.
<syntaxhighlight lang="bash">

#!/bin/bash
# LOCAL/FTP/SCP/MAIL PARAMETERS
SERVER="192.168.4.100" # IP of Synology NAS, used for ftp
USERNAME="User account Synology" # FTP username of Network disk used for ftp
PASSWORD="Password Synology" # FTP password of Network disk used for ftp
DESTDIR="/opt/backup" # used for temorarily storage
DESTDIRNAS="/homes/domoticz/backup/" # Path to your Synology NAS backup folder
DOMO_IP="192.168.4.4" # Domoticz IP
DOMO_PORT="8084" # Domoticz port
### END OF USER CONFIGURABLE PARAMETERS
TIMESTAMP=`/bin/date +%Y%m%d%H%M%S`
BACKUPFILE="domoticz_$TIMESTAMP.db" # backups will be named "domoticz_YYYYMMDDHHMMSS.db.gz"
BACKUPFILEGZ="$BACKUPFILE".gz
### Create backup and ZIP it
/usr/bin/curl -s http://$DOMO_IP:$DOMO_PORT/backupdatabase.php > /tmp/$BACKUPFILE
gzip -9 /tmp/$BACKUPFILE
tar -zcvf /tmp/domoticz_scripts_$TIMESTAMP.tar.gz /home/pi/domoticz/scripts/
tar -zcvf /tmp/telegram_scripts_$TIMESTAMP.tar.gz /home/pi/tg/scripts/
### Send to Network disk through FTP
curl -s --disable-epsv -v -T"/tmp/$BACKUPFILEGZ" -u"$USERNAME:$PASSWORD" "ftp://$SERVER/$DESTDIRNAS"
curl -s --disable-epsv -v -T"/tmp/domoticz_scripts_$TIMESTAMP.tar.gz" -u"$USERNAME:$PASSWORD" "ftp://$SERVER/$DESTDIRNAS"
curl -s --disable-epsv -v -T"/tmp/telegram_scripts_$TIMESTAMP.tar.gz" -u"$USERNAME:$PASSWORD" "ftp://$SERVER/$DESTDIRNAS"
### Remove temp backup file
/bin/rm /tmp/$BACKUPFILEGZ
/bin/rm /tmp/domoticz_scripts_$TIMESTAMP.tar.gz
/bin/rm /tmp/telegram_scripts_$TIMESTAMP.tar.gz
### Done!

</syntaxhighlight>

If you want to prevent your NAS from overflowing over time you can add some automated pruning. For instance by extending the script.

<syntaxhighlight lang="bash">
# get a list of files and dates from ftp and remove files older than ndays
ftpsite="sftp -b- -oPort=22 $USERNAME@$SERVER"
putdir=$DESTDIRNAS

ndays=19

# work out our cutoff date
MM=`date --date="$ndays days ago" +%b`
DD=`date --date="$ndays days ago" +%d`
TT=`date --date="$ndays days ago" +%s`

echo removing files older than $MM $DD

# get directory listing from remote source
echo "
cd $putdir
ls -l
"|$ftpsite >dirlist

# skip first three and last line, ftp command echo
listing="`tail -n+4 dirlist|head -n-1`"

lista=( $listing )

# loop over our files
for ((FNO=0; FNO<${#lista[@]}; FNO+=9));do
# month (element 5), day (element 6) and filename (element 8)
# echo Date ${lista[`expr $FNO+5`]} ${lista[`expr $FNO+6`]} File: ${lista[`expr $FNO+8`]}

fdate="${lista[`expr $FNO+5`]} ${lista[`expr $FNO+6`]} ${lista[`expr $FNO+7`]}"
sdate=`date --date="$fdate" +%s`
# check the date stamp
if [ $sdate -lt $TT ]
then
# Remove this file
echo "$MM $DD: Removing ${lista[`expr $FNO+5`]} / ${lista[`expr $FNO+6`]} / ${lista[`expr $FNO+8`]}"
$ftpsite <<EOMYF2
cd $putdir
rm ${lista[`expr $FNO+8`]}
quit
EOMYF2

fi
done
</syntaxhighlight>

==Make script executable==
use the commmand
<syntaxhighlight lang="bash">
sudo chmod +x /home/pi/domoticz/scripts/domoticz_backup.sh
</syntaxhighlight>
to make the script executable. 
Test the script by the command
<syntaxhighlight lang="bash">
sudo /home/pi/domoticz/scripts/domoticz_backup.sh
</syntaxhighlight>

==Schedule daily execution of the script==
Login as pi user and issue
crontab -e
Place this line in your crontab document:
0 1 * * * sudo ~/domoticz/scripts/domoticz_backup.sh
which says: execute script when time is 0 minutes, 1 hour any day of the month, any month of the year, any day of the week, which equals daily at 01:00.
Change the number in the time you want by changing the cron parameters
* * * * * command to be executed
- - - - -
| | | | |
| | | | +----- day of week (0 - 6) (Sunday=0)
| | | +------- month (1 - 12)
| | +--------- day of month (1 - 31)
| +----------- hour (0 - 23)
+------------- min (0 - 59)
And save the crontab

Linux will now make a daily backup of your Domoticz Database

==The forumthread==
[http://www.domoticz.com/forum/viewtopic.php?f=23&t=936 Forum]

==Script to upload to Dropbox==
While searching for backupmethods, i came by [https://github.com/andreafabrizi/Dropbox-Uploader this script] by Andrea Fabrizi. 
It is an easy script to upload files to Dropbox. See the Github page for how to use this script. 
Downside is that it can only push files to Dropbox, it doesn't work like on your PC, where you have a local folder for Dropbox. So removing old backups automatically is not possible with this uploader. 
But still a nice script for creating off-site backups of your Domoticz install.

Addition: this is a version of the above local backup script for raspi with uploading to dropbox > works after you install the dropbox_uploader script to /home/pi/domoticz/scripts/dropbox_uploader.sh (or choose another location but change the script accordingly) and after you followed the instructions on the github page to create a dropbox api and set it in the script config

<syntaxhighlight lang="bash">
#!/bin/bash
DOMO_IP="yours" # Domoticz IP
DOMO_PORT="8080" # Domoticz port
### END OF USER CONFIGURABLE PARAMETERS
TIMESTAMP=`/bin/date +%Y%m%d%H%M%S`
BACKUPFILE="domoticzbackup_$TIMESTAMP.db" # backups will be named "domoticz_YYYYMMDDHHMMSS.db.gz"
BACKUPFILEGZ="$BACKUPFILE".gz

#Create backup and make tar archives
/usr/bin/curl -s http://$DOMO_IP:$DOMO_PORT/backupdatabase.php > /home/pi/Domoticz_backup/database/$BACKUPFILE
tar -zcvf /home/pi/Domoticz_backup/scripts/domoticz_scripts_$TIMESTAMP.tar.gz /home/pi/domoticz/scripts/
tar -zcvf /home/pi/Domoticz_backup/www/domoticz_wwwfolder_$TIMESTAMP.tar.gz /home/pi/domoticz/www/

#Delete backups older than 31 days
/usr/bin/find /home/pi/Domoticz_backup/database/ -name '*.db' -mtime +31 -delete
/usr/bin/find /home/pi/Domoticz_backup/scripts/ -name '*.tar.gz' -mtime +31 -delete
/usr/bin/find /home/pi/Domoticz_backup/www/ -name '*.tar.gz' -mtime +31 -delete

/home/pi/domoticz/scripts/dropbox_uploader.sh -s upload /home/pi/Domoticz_backup/database/* /Domoticz_backup/database/
/home/pi/domoticz/scripts/dropbox_uploader.sh -s upload /home/pi/Domoticz_backup/www/* /Domoticz_backup/www/
/home/pi/domoticz/scripts/dropbox_uploader.sh -s upload /home/pi/Domoticz_backup/scripts/* /Domoticz_backup/scripts/
</syntaxhighlight>

==Credits==
This script is based on Leonardo's work published on the old Domoticz forum. Still to be found [http://pastebin.com/j4ytCMJx here].

[[Category:Bash]]

Daily backup to external hdd - Revision history

Walter vl: /* Script to backup to local folders (database + multiple folders) */ undo mtime warning