Presence detection

From Domoticz
Jump to navigation Jump to search

This script will turn on a virtual switch when a device is present on your network, and turns the virtual switch off when the device is gone. You can use this to see if a person (which always carries his/her smartphone) is at home or not, and trigger events based on this.

This Python script is available for both Linux and Windows.


Dependencies - hardware / software / operating system

Configure your router to give out static DHCP leases

This script needs the device you want to check to have a static IP. You can configure the device to use a static IP, or configure your router to grant a 'Static DHCP lease' to the device, based on the MAC-address of the device.
This last method is the most preferred, because it doesn't involve making changes on the device, which decreases the risk of errors. Also, by configuring a static IP on your device, you could encounter trouble when connecting to a different network.
Read the manual of your router on how to configure a static DHCP lease.
If you want to use it with an iPhone, please read this page first.

Domoticz Setup - creating virtual hardware dummy and virtual switches

This script needs a virtual switch for each device you want to keep an eye on.

To create a virtual switch, go to the 'Switches' tab.

Click the 'Manual Light/Switch' button in the upper left corner. Choose the dummy hardware from the drop-down. Enter a name ('John Doe smartphone'). Switch type 'On/off'. Type 'X10'. For 'House code' and 'Unit code', choose some random, doesn't really matter.

It is advisable to make the switch 'protected', so the switch can only be operated by the detection script and not by you, by accidentally clicking on it ;)

Now you need to know the ID of the switch. This can be found by going to 'Setup' --> 'Devices'. Find the switch you created in the list, and remember the IDX.
If you want multiple devices to be checked (smartphone of your wife) you will need to create a virtual switch for that device also. You will also need to find the id of that device.
Remember/write down this number, because you will need it later on.

Authentication

If you are using authentication make sure to set the type to "basic-Auth" and not "Login Page"

Linux (Raspberry Pi)

This script is available for Linux. To use it, a number of things have to be installed.

Installing Python

The Python scripts needs 'python' to be installed. You can install this by running:

sudo apt-get install python

from the terminal on the Pi.
If it says that 'python' is already installed, you can continue further with this tutorial.

Installing arping

'Ping' is a regular way to detect whether a device is present on a network or not. It is also inherently unreliable as many smartphones (independent of brand) will not respond when the phone is inactive, which leads to a lot of online / offline switching. To circumvent this, we will use 'arping' a tool that sends out an 'arp-whois', to which most device will respond.

sudo apt-get install arping

There is a small caveat with using arping, as it requires root-privileges to run. We will fix this within the script below.

Create the Python script

User Chopperrob has created a very useful script that will scan for any device you ask it to.

The script needs 4 parameters to run:

  • The IP-address of the device you want to check (In my case '192.168.4.9')
  • The ID of the virtual switch (In my case '37')
  • The interval (seconds) on which to check if a device is present or not (In my case '10', for 10 seconds)
  • The 'cool-down' period (seconds). If a device does not respond within this period, the virtual switch is turned off. (In my case '120', for 120 seconds, or 2 minutes)

Create a new file in the scripts folder inside your Domoticz folder. On a Raspberry Pi this would probably be at \home\pi\domoticz\scripts\. Name the file check_device_online.py. Copy the contents from the script on this page to the file, and save it.

#!/usr/bin/python
#   Title: check_device_online.py
#   Author: Chopper_Rob
#   Date: 25-02-2015
#   Info: Checks the presence of the given device on the network and reports back to domoticz
#   URL : https://www.chopperrob.nl/domoticz/5-report-devices-online-status-to-domoticz
#   Version : 1.6.2
 
import sys
import datetime
import time
import os
import subprocess
import urllib2
import json
import base64
 
# Settings for the domoticz server
domoticzserver="192.168.2.1:8080"
domoticzusername = "admin"
domoticzpassword = "admin"
domoticzpasscode = "Light/Switch Protection"
 
# If enabled. The script will log to the file _.log
# Logging to file only happens after the check for other instances, before that it only prints to screen.
log_to_file = False
 
# The script supports two types to check if another instance of the script is running.
# One will use the ps command, but this does not work on all machine (Synology has problems)
# The other option is to create a pid file named _.pid. The script will update the timestamp
# every interval. If a new instance of the script spawns it will check the age of the pid file.
# If the file doesn't exist or it is older then 3 * Interval it will keep running, otherwise is stops.
# Please chose the option you want to use "ps" or "pid", if this option is kept empty it will not check and just run.
check_for_instances = "pid"
 
 
 
# DO NOT CHANGE BEYOND THIS LINE
if len(sys.argv) != 5 :
  print ("Not enough parameters. Needs %Host %Switchid %Interval %Cooldownperiod.")
  sys.exit(0)
 
device=sys.argv[1]
switchid=sys.argv[2]
interval=sys.argv[3]
cooldownperiod=sys.argv[4]
previousstate=-1
lastsuccess=datetime.datetime.now()
lastreported=-1
base64string = base64.encodestring('%s:%s' % (domoticzusername, domoticzpassword)).replace('\n', '')
domoticzurl = 'http://'+domoticzserver+'/json.htm?type=devices&filter=all&used=true&order=Name'
 
if check_for_instances.lower() == "pid":
  pidfile = sys.argv[0] + '_' + sys.argv[1] + '.pid'
  if os.path.isfile( pidfile ):
    print datetime.datetime.now().strftime("%H:%M:%S") + "- pid file exists"
    if (time.time() - os.path.getmtime(pidfile)) < (float(interval) * 3):
      print datetime.datetime.now().strftime("%H:%M:%S") + "- script seems to be still running, exiting"
      print datetime.datetime.now().strftime("%H:%M:%S") + "- If this is not correct, please delete file " + pidfile
      sys.exit(0)
    else:
      print datetime.datetime.now().strftime("%H:%M:%S") + "- Seems to be an old file, ignoring."
  else:
    open(pidfile, 'w').close() 
 
if check_for_instances.lower() == "ps":
  if int(subprocess.check_output('ps x | grep \'' + sys.argv[0] + ' ' + sys.argv[1] + '\' | grep -cv grep', shell=True)) > 2 :
    print (datetime.datetime.now().strftime("%H:%M:%S") + "- script already running. exiting.")
    sys.exit(0)
 
def log(message):
  print message
  if log_to_file == True:
    logfile = open(sys.argv[0] + '_' + sys.argv[1] + '.log', "a")
    logfile.write(message + "\n")
    logfile.close()
 
def domoticzstatus ():
  json_object = json.loads(domoticzrequest(domoticzurl))
  status = 0
  switchfound = False
  if json_object["status"] == "OK":
    for i, v in enumerate(json_object["result"]):
      if json_object["result"][i]["idx"] == switchid:
        switchfound = True
        if json_object["result"][i]["Status"] == "On": 
          status = 1
        if json_object["result"][i]["Status"] == "Off": 
          status = 0
  if switchfound == False: print (datetime.datetime.now().strftime("%H:%M:%S") + "- Error. Could not find switch idx in Domoticz response. Defaulting to switch off.")
  return status
 
def domoticzrequest (url):
  request = urllib2.Request(url)
  request.add_header("Authorization", "Basic %s" % base64string)
  response = urllib2.urlopen(request)
  return response.read()
 
log (datetime.datetime.now().strftime("%H:%M:%S") + "- script started.")
 
lastreported = domoticzstatus()
if lastreported == 1 :
  log (datetime.datetime.now().strftime("%H:%M:%S") + "- according to domoticz, " + device + " is online")
if lastreported == 0 :
  log (datetime.datetime.now().strftime("%H:%M:%S") + "- according to domoticz, " + device + " is offline")
 
while 1==1:
  # currentstate = subprocess.call('ping -q -c1 -W 1 '+ device + ' > /dev/null', shell=True)
  currentstate = subprocess.call('sudo arping -q -c1 -W 1 '+ device + ' > /dev/null', shell=True)
 
  if currentstate == 0 : lastsuccess=datetime.datetime.now()
  if currentstate == 0 and currentstate != previousstate and lastreported == 1 : 
    log (datetime.datetime.now().strftime("%H:%M:%S") + "- " + device + " online, no need to tell domoticz")
  if currentstate == 0 and currentstate != previousstate and lastreported != 1 :
    if domoticzstatus() == 0 :
      log (datetime.datetime.now().strftime("%H:%M:%S") + "- " + device + " online, tell domoticz it's back")
      domoticzrequest("http://" + domoticzserver + "/json.htm?type=command&param=switchlight&idx=" + switchid + "&switchcmd=On&level=0" + "&passcode=" + domoticzpasscode)
    else:
      log (datetime.datetime.now().strftime("%H:%M:%S") + "- " + device + " online, but domoticz already knew")
    lastreported=1
 
  if currentstate == 1 and currentstate != previousstate :
    log (datetime.datetime.now().strftime("%H:%M:%S") + "- " + device + " offline, waiting for it to come back")
 
  if currentstate == 1 and (datetime.datetime.now()-lastsuccess).total_seconds() > float(cooldownperiod) and lastreported != 0 :
    if domoticzstatus() == 1 :
      log (datetime.datetime.now().strftime("%H:%M:%S") + "- " + device + " offline, tell domoticz it's gone")
      domoticzrequest("http://" + domoticzserver + "/json.htm?type=command&param=switchlight&idx=" + switchid + "&switchcmd=Off&level=0" + "&passcode=" + domoticzpasscode)
    else:
      log (datetime.datetime.now().strftime("%H:%M:%S") + "- " + device + " offline, but domoticz already knew")
    lastreported=0
 
  time.sleep (float(interval))
 
  previousstate=currentstate
  if check_for_instances.lower() == "pid": open(pidfile, 'w').close()

Source: https://www.chopperrob.nl/domoticz/5-report-devices-online-status-to-domoticz Many thanks for this good working script Chopper_Rob !

This link seems to be down

Updated version of the script that can run as a systemd service can be found here: https://github.com/jorijnsmit/onlineChecker

Cron - scheduling the script

In principle the script runs endlessly. But in case something goes wrong, it's nice to make sure it will get restarted automatically. This can be done by using 'cron', the task scheduler on Linux.

On the terminal, run crontab -e. In the screen that opens, add a line for each device you want to check.

My crontab looks like this:

*/10 * * * *  /home/pi/domoticz/scripts/check_device_online.py 192.168.4.9 37 10 120
*/10 * * * *  /home/pi/domoticz/scripts/check_device_online.py 192.168.4.10 36 10 2700

The first line is for my phone, the second line is for the iPhone of my girlfriend.

For Raspbmc, a user advised that you need to put sudo python in front of the command. So then it will look like this:
*/10 * * * * sudo python /home/pi/domoticz/scripts/check_device_online.py 192.168.4.9 37 10 120

Exit the crontab editor by pressing CTRL-O (character 'O', not zero) and hitting Enter.

Optional: make script executable

In the terminal, go to cd /home/pi/domoticz/scripts
Execute the command sudo chmod +x check_device_online.py
I am not sure if executing this command is necessary, but it doesn't harm anything ;)


Windows

The same Python script is also available for Windows.

Install Python

You can download and then install Python from the following website:
[1]

Use version 2.X as the 3.x seems to have issues with some commands in the script

Install Arping

Get arping from the following page: [2]

You can just copy and paste the arping into your c:\windows\system32\ folder to make it easier to run the script

Create the script

Create a new file in the c:\scripts folder (create if this doesn't exist. Name the file check_device_online.py. Copy the contents from the script below into the file, and save it.


#!/usr/bin/python
#   Title: check_device_online.py
#   Author: Chopper_Rob
#   Date: 25-02-2015
#   Info: Checks the presence of the given device on the network and reports back to domoticz
#   URL : https://www.chopperrob.nl/domoticz/5-report-devices-online-status-to-domoticz
#   Version : 1.6.2
 
import sys
import datetime
import time
import os
import subprocess
import urllib2
import json
import base64
 
# Settings for the domoticz server
domoticzserver="***.***.***.***:****"
domoticzusername = "*******"
domoticzpassword = "*******"
domoticzpasscode = "*******"
 
# If enabled. The script will log to the file _.log
# Logging to file only happens after the check for other instances, before that it only prints to screen.
log_to_file = False
 
# The script supports two types to check if another instance of the script is running.
# One will use the ps command, but this does not work on all machine (Synology has problems)
# The other option is to create a pid file named _.pid. The script will update the timestamp
# every interval. If a new instance of the script spawns it will check the age of the pid file.
# If the file doesn't exist or it is older then 3 * Interval it will keep running, otherwise is stops.
# Please chose the option you want to use "ps" or "pid", if this option is kept empty it will not check and just run.
check_for_instances = "pid"
 
 
 
# DO NOT CHANGE BEYOND THIS LINE
if len(sys.argv) != 5 :
  print ("Not enough parameters. Needs %Host %Switchid %Interval %Cooldownperiod.")
  sys.exit(0)
 
device=sys.argv[1]
switchid=sys.argv[2]
interval=sys.argv[3]
cooldownperiod=sys.argv[4]
previousstate=-1
lastsuccess=datetime.datetime.now()
lastreported=-1
base64string = base64.encodestring('%s:%s' % (domoticzusername, domoticzpassword)).replace('\n', '')
domoticzurl = 'http://'+domoticzserver+'/json.htm?type=devices&filter=all&used=true&order=Name'
 
if check_for_instances.lower() == "pid":
  pidfile = sys.argv[0] + '_' + sys.argv[1] + '.pid'
  if os.path.isfile( pidfile ):
    print datetime.datetime.now().strftime("%H:%M:%S") + "- pid file exists"
    if (time.time() - os.path.getmtime(pidfile)) < (float(interval) * 3):
      print datetime.datetime.now().strftime("%H:%M:%S") + "- script seems to be still running, exiting"
      print datetime.datetime.now().strftime("%H:%M:%S") + "- If this is not correct, please delete file " + pidfile
      sys.exit(0)
    else:
      print datetime.datetime.now().strftime("%H:%M:%S") + "- Seems to be an old file, ignoring."
  else:
    open(pidfile, 'w').close() 
 
if check_for_instances.lower() == "ps":
  if int(subprocess.check_output('ps x | grep \'' + sys.argv[0] + ' ' + sys.argv[1] + '\' | grep -cv grep', shell=True)) > 2 :
    print (datetime.datetime.now().strftime("%H:%M:%S") + "- script already running. exiting.")
    sys.exit(0)
 
def log(message):
  print message
  if log_to_file == True:
    logfile = open(sys.argv[0] + '_' + sys.argv[1] + '.log', "a")
    logfile.write(message + "\n")
    logfile.close()
 
def domoticzstatus ():
  json_object = json.loads(domoticzrequest(domoticzurl))
  status = 0
  switchfound = False
  if json_object["status"] == "OK":
    for i, v in enumerate(json_object["result"]):
      if json_object["result"][i]["idx"] == switchid:
        switchfound = True
        if json_object["result"][i]["Status"] == "On": 
          status = 1
        if json_object["result"][i]["Status"] == "Off": 
          status = 0
  if switchfound == False: print (datetime.datetime.now().strftime("%H:%M:%S") + "- Error. Could not find switch idx in Domoticz response. Defaulting to switch off.")
  return status
 
def domoticzrequest (url):
  request = urllib2.Request(url)
  request.add_header("Authorization", "Basic %s" % base64string)
  response = urllib2.urlopen(request)
  return response.read()
 
log (datetime.datetime.now().strftime("%H:%M:%S") + "- script started.")
 
lastreported = domoticzstatus()
if lastreported == 1 :
  log (datetime.datetime.now().strftime("%H:%M:%S") + "- according to domoticz, " + device + " is online")
if lastreported == 0 :
  log (datetime.datetime.now().strftime("%H:%M:%S") + "- according to domoticz, " + device + " is offline")
 
while 1==1:
  # currentstate = subprocess.call('ping -q -c1 -W 1 '+ device, > /dev/null shell=True)
  currentstate = subprocess.call('arping -c '+ device + ' > nul', shell=True)
 
  if currentstate == 0 : lastsuccess=datetime.datetime.now()
  if currentstate == 0 and currentstate != previousstate and lastreported == 1 : 
    log (datetime.datetime.now().strftime("%H:%M:%S") + "- " + device + " online, no need to tell domoticz")
  if currentstate == 0 and currentstate != previousstate and lastreported != 1 :
    if domoticzstatus() == 0 :
      log (datetime.datetime.now().strftime("%H:%M:%S") + "- " + device + " online, tell domoticz it's back")
      domoticzrequest("http://" + domoticzserver + "/json.htm?type=command&param=switchlight&idx=" + switchid + "&switchcmd=On&level=0" + "&passcode=" + domoticzpasscode)
    else:
      log (datetime.datetime.now().strftime("%H:%M:%S") + "- " + device + " online, but domoticz already knew")
    lastreported=1
 
  if currentstate == 1 and currentstate != previousstate :
    log (datetime.datetime.now().strftime("%H:%M:%S") + "- " + device + " offline, waiting for it to come back")
 
  if currentstate == 1 and (datetime.datetime.now()-lastsuccess).total_seconds() > float(cooldownperiod) and lastreported != 0 :
    if domoticzstatus() == 1 :
      log (datetime.datetime.now().strftime("%H:%M:%S") + "- " + device + " offline, tell domoticz it's gone")
      domoticzrequest("http://" + domoticzserver + "/json.htm?type=command&param=switchlight&idx=" + switchid + "&switchcmd=Off&level=0" + "&passcode=" + domoticzpasscode)
    else:
      log (datetime.datetime.now().strftime("%H:%M:%S") + "- " + device + " offline, but domoticz already knew")
    lastreported=0
 
  time.sleep (float(interval))
 
  previousstate=currentstate
  if check_for_instances.lower() == "pid": open(pidfile, 'w').close()

Source: https://www.chopperrob.nl/domoticz/5-report-devices-online-status-to-domoticz


== Scheduling the script in Windows==
In the task sheduler create a new task to run on computer login if you setup Domoticz as an application
click create task
Add your triggers for 12:01am and repeating every 15 minutes for actions add the following
C:\Python27\python.exe as the program\script (change it to your Python folder)

And for arguments c:\scripts\checkpresence.py 192.168.4.9 37 10 120 (change the parameters to what you need for your situation).


Working with multiple phones

I use this script to detect if my girlfriend or I are at home. My Blockly events are based on this, because i can tell to Domoticz if someone is home or not.
For each phone you want to check you need a line with a call to the script in your crontab. Of course you need to change the parameters in the different crontab lines for every phone accordingly. You also need a virtual switch for each phone.
To tell Domoticz if there is someone home i use a virtual switch 'SomeoneHome'. This virtual switch is triggered by a Blockly event. That event looks like this:

IF [SomeoneHome = Off] AND [[Phone1 = On] OR [Phone2 = On]]
DO SET [SomeoneHome = On]
ELSE IF [SomeoneHome = On] AND [[Phone1 = Off] AND [Phone2 = Off]]
DO SET [SomeoneHome = Off]


By using the virtual switch I can turn on the lights when someone comes home and it is dark. Or turn the heating off when there is nobody home.
Don't forget to make the switch 'SomeoneHome' protected to prevent accidental clicking. You only want the switch to be turned on/off by the event.



Troubleshooting

It can happen that the script is not working in your situation.

Try pinging by hand

First thing to try is, if you can ping your phone by yourself (ping IP_OF_PHONE). If you get responses ("XX bytes from XXX"), we know your phone is replying to pings. Stop the ping commmand by pressing 'CTRL-C'.

Check if the script works

Now we need to check if the script works when we activate it by hand. Go to the scripts directory (cd /home/pi/domoticz/scripts/) and then activate the script with the correct parameters:
python check_device_online.py IP_PHONE IDX INTERVAL TIMEOUT. For testing you can use a interval of 10 seconds and a timeout of 30 seconds, so you don't have to wait very long to see the switch turn on/off in Domoticz.
If everything works good, you should see some output like: Script already running, exiting or Needs more parameters. If you get other errors (unclear errors, that doesn't seem to be related to the script), maybe you don't have 'python' (not 'python3' !!) installed correctly (sudo apt-get install python), or you made a mistake when you copied the script.

Check if crontab is correct

When the script is not yet activated by the crontab (in the setup above it should activate the script every 10 minutes) and you activate it by hand, you should see some output on the screen, like: Device online, tell domoticz it's back. You now know the script is working correctly, so the error is probably in your crontab.

Check if 'cron' service is running

A user told me he had difficulties on using this script on Raspbmc. He said the 'cron' service was not running by default on his Raspbmc installation.
You can check if 'cron' is active by issueing sudo service cron status.
If it is running, it should say [ ok ] cron is running.. If it is not running, you can start it by sudo service cron start.
To view the log of cron, you can issue grep CRON /var/log/syslog. You should see some lines of text if cron is active.

To enable 'cron' on Raspbmc, have a look this tutorial: http://www.averagemanvsraspberrypi.com/2014/07/using-cron-with-raspbmc.html

iPhone trouble

For an Apple iPhone it is recommended to experiment a bit with the cooldown period. Apple devices turn their WiFi-connection off when the screen turns off. If the cool-down periode you have configured for your iPhone is too small, the script may think you are away while you are not.
I use 2700 sec. (45 minutes) for the iPhone 4S of my girlfriend. Maybe lower will work, but your mileage may vary. For my LG Nexus 5 (the first line in the crontab) i use 2 minutes, because the WiFi of my phone is always on, even with screen off (Android setting).
The downside of using a high number for the cool-down period, is that the script will wait the configured time before seeing the device as away. But this is only negative if you want to turn off something, when the phone is gone. Because the script runs every 10 secs, as soon as the phone is 'visible' for the script it turns the virtual switch on.
So turning lights on when the iPhone comes home works fine, but an event which turns the lights off when the iPhone leaves the house can have a delay of 45min.

As an alternative you could use geo-fencing apps like 'Pilot: Home Automation' for the iPhone. They work well, but are less privacy friendly, as you will now be updating your location though Google/Apple's location services.

"ps: invalid option -- 'x' " message

If you try to run the script above on a Synology NAS, you will probably get the message ps: invalid option -- 'x'.
To fix this, go to line 66 of the script, and change ps x | grep into ps | grep. (remove the 'x').

Synology arping syntax

On some Synology devices (DSM7+?) the arping command requires you to specify the network interface with the -I parameter. In most cases the interface will be eth0. So the line calling the arping should be changed like this:

currentstate = subprocess.call('sudo arping -q -c1 -w 1 -I eth0 '+ device + ' > /dev/null', shell=True)

Network limitation of Arping

Arping relies on arp requests. Those will works only for the same subnet. If you intend detect anything outside of it, arping will fails. Ping have to be used for such scenario (by example internet targets or multi-subnet networks).
A trick is to replace the arping command with a combination of both:

# currentstate = subprocess.call('ping -q -c1 -W 1 '+ device + ' > /dev/null', shell=True)
currentstate = subprocess.call('sudo arping -q -c1 -W 1 '+ device + ' > /dev/null', shell=True)

should become something like this:
currentstateping = subprocess.call('ping -q -c1 -W 1 '+ device + ' > /dev/null', shell=True)
currentstatearp = subprocess.call('sudo arping -q -c1 -W 1 '+ device + ' > /dev/null', shell=True)
currentstate = currentstateping and currentstatearp