Python - Read-out of DDS238 kWh-meter and upload to Domoticz and to PVOutput

Introduction

This wiki-section can be divided in three main parts:

a domoticz configuration part where you will be creating the appropriate virtual sensors.
a shell script and cronjob which control a python script that extracts information from the DDS238-1ZN-meter and will be posting values into Domoticz and to PVOutput.
a description of variations and their impact on the 2 earlier parts.

The interface-routines for DDS238-1ZN in part 2 are specific as well as part 3, but for part 1 and for main parts of part 2 I have gratefully borrowed useful elements from the Wiki for the Omnik Solar Inverter.

The main reasons I made this script:

my setup with Involar inverters is so small that application of Involar's eGate would be a giant overkill, not even considering that the eGate does not connect to Domoticz, PVOutput etc., but only to the 'own' portal of Involar.
still I want registration of reasonably quality as well as application of the data by Domoticz and by PVOutput.
a script for reading a kWh-meter with RS485-interface does not exist as 'open source'

The description and script in this Wiki assumes a single kWh-meter type DDS238-1ZN with default settings:

Slave-address = 01
Baudrate = 01 = 9600 Baud

For any other configuration or setting, see section 3 of this Wiki.

Pay attention that also a kWh-meter type DDS238-1 exists (without extension ZN!), which is 'lookalike' and sold much cheaper, but without RS485-interface!!!!

My current setup

As perspective for this Wiki-description, below a short description of the situation at my home.

Solar Inverter & Server

Hardware Configuration

For a small subsection of my PV-system 2* Involar-inverter type MAC250 connect to 2* PV-panel type ESE215.

The 2 inverters coupled in series feed a common 230V-grid-interface through a kWh-meter type DDS238-1ZN.

This kWh-meter has 2 types of data-interface:

S0-interface with 3200pulses/kWh, feeding a counter, of which after calculation the output is the accumulated energy (Wh), and (after backward-calculation) the actual power (W).
RS485-interface provides directly measured data for LifeEnergy (Wh), Power (W), Voltage (V), Current (A), Frequency (Hz) and several other aspects. This interface applies the ModBus RTU protocol, reading by calls with 'Modbus' Function Codes 03Hex and 04Hex, and by writing with 'Modbus' Function Code 10Hex (=21Dec).

In my configuration, both interfaces are connected to a Raspberry acting as server running Domoticz and Python:

the frontend for the S0-interface is an S0PulseCountingModule [as described elsewhere in this wiki]
the frontend for the RS485-interface is an RS485-shield by Linksprite

Because of the direct measurement at the inverter-output and due to the data-catalogue, the RS485-interface is more versatile and superior to the S0-interface, and topic of this Wiki-description.

Software

For the RS485-interface of the DDS238-1ZN-meter no ready interface exists as open-source material.

The Python-script described in this wiki-page pulls out the data from my DDS238-1ZN-meter (applying the Python-modules serial and minimalmodbus), stores it in Domoticz using JSON, and uploads info to PVOutput.

Because during night the PV-system is not generating energy, the measurement and uploads are not required after sunset till sunrise: a time-gate function using Python-module pyephem blocks at least the uploads during that dark period.

[That function to be taken out if you apply the kWh-meter to measure Consumption, but then obviously you also have to tune/adapt the upload sections to substitute Generation by Consumption ......]

For this script the front-end interfacing to the DDS238-1ZN-meter's RS485-interface (using serial and minimalmodbus) is 'own design', the idea to determine the time-gate (using pyephem) has been borrowed and converted to an adapted script, while the interface towards Domoticz and PVOutput has been borrowed & adapted from the Wiki-section for the Omnik_Solar_Inverter: ;-) re-inventing the wheel should be minimized ......

The proposed scripts are all assuming application for a DDS238-1ZN meter with default setting for Slave-address 1 and 9600Bd (straight out of the box): for a meter used before with another adress you have to adapt the script (at least) for that address.

Before you can use the script for your own purposes, at least you have to check & adapt the settings as described under Required Inputs for Settings.

Dependencies

The script described in this wiki-page requires the installation of

Python module Serial
Python Module Minimalmodbus
Python Module Pyephem

Prepare Domoticz

To create a virtual sensor, perform following actions.

Go to "Hardware"
Fill in the name field with a desired name (like "Virtual")
Choose for type the "Dummy, does nothing, use for virtual switches only"
Click on "Add"
The added hardware is now added to the hardware list. In the same row there is a "Create virtual sensor" button: click it.

Now you are ready to create virtual devices.

Go to "Hardware", and look for the section you created a moment ago
Press "Create virtual sensor".
Fill in the name-field with a desired name

To cover the most interesting elements provided at the RS485-interface of the DDS238-meter, we need ad least 3 virtual sensors:

# DDS238 Power / Energy (W / kWh)
# DDS238 AC Voltage (V)
# DDS238 AC Current (A)

Therefore repeat the above steps 3 times, after which:

Go to "Devices"
Click "Not-Used"
Search for the 3 virtual devices created a moment ago (and click on the green arrow behind it).
Give the sensors the following name:

 # "DDS238 Production1",  will contain 2 values for Current Power-production, respectively accumulated Energy in (Watt / kWh) for meter#1
 # "DDS238 Voltage1", will contain a value of inverter's output voltage (V) for meter#1
 # "DDS238 Current1", will contain a value of inverter's output current (A) for meter#1

Creating the Shell script used in cron

Now create for meter#1 a shell script DDS238_01.sh and put it in the "home/pi/domoticz/scripts/" folder.

#!/bin/bash
sudo python /home/pi/domoticz/scripts/DDS238Export01.py

Make this Shell-script executable: "sudo chmod +x /home/pi/domoticz/scripts/DDS238_01.sh"

Collect required Inputs for Settings

For settings in the script you need the following information (in the sequence in which it appears).

For setting of the interface-readout (only if the DDS238-1ZN-meter has been used before):

slave-address
baud-rate

For upload to PVOutput.org:

the api-key
the destination System ID

For the ephem-section (for getting sunrise and sunset):

home.lat(itude)
home.long(itude)

For upload to Domoticz:

the IDXes of the virtual devices
the destination IP-address, if the desination is different from (local) IP-address 127.0.0.1

Create DDS238Export01.py

#!/usr/bin/python
# -*- coding = utf-8 to enable reading by simple editors -*-
# --------------------------------------------------
# Line004 = PREPARATION & SETTINGs
# --------------------------------------------------

# Imports for script-operation

import serial                   # required for handling of the serial port
import minimalmodbus            # required for handling modbus
import datetime                 # used for timestamps & timecheck
import time                     # used for timestamps & delays

# Line014 = Provide here the settings for access to PVOutput

slave = 1             # decimal slave-address of the DDS238 kWh-meter
retry = 5             # number of retries to get power-info from DDS238
delay = 2             # time (seconds) between retries
pvout_enabled   = True
pvout_apikey    = "<YOUR api-key>"
pvout_sysid     = <YOUR SID>
pvout_cumflag   = 1   # flag is 1 if you apply lifetime-energy as reported by DDS238
                      # flag must be 0 if you upload a calculated Wh_today

# --------------------------------------------------
# Line026 = READING THE RS485-INTERFACE OF DDS238
# --------------------------------------------------
# the "print"-lines only added for local read-out e.g. during script_tuning

now = datetime.datetime.now()
print
print 'Present Date & Time = ', now
print
instrument = minimalmodbus.Instrument('/dev/ttyAMA0',slave) # port name, slave address
instrument.serial.baudrate = 9600
instrument.serial.timeout = 0.5
# instrument.debug = True

TotalEnergy1 = instrument.read_registers(0,2)
LifeEnergy_High = TotalEnergy1[0]
LifeEnergy_Low = TotalEnergy1[1]
LifeEnergy = 0.01 * ((LifeEnergy_High * 65535) + LifeEnergy_Low)
LIFENERGY = LifeEnergy * 1000
TotalEnergy2 = instrument.read_registers(8,2)
RevEnergy_High = TotalEnergy2[0]
RevEnergy_Low = TotalEnergy2[1]
RevEnergy = 0.01 * ((RevEnergy_High * 65535) + RevEnergy_Low)
REVENERGY = RevEnergy * 1000
TotalEnergy3 = instrument.read_registers(10,2)
FwdEnergy_High = TotalEnergy3[0]
FwdEnergy_Low = TotalEnergy3[1]
FwdEnergy = 0.01 * ((FwdEnergy_High * 65535) + FwdEnergy_Low)
FWDENERGY = FwdEnergy * 1000
Voltage = instrument.read_register(12,1)
VOLTAGE = Voltage
Current = instrument.read_register(13,2)
CURRENT = Current
ActivPower = instrument.read_register(14,0)
if (LifeEnergy > 0) and (ActivPower >= 0):
   Missing = False
   i = 1
else:              # try to get at least power-info
   Missing = True
   i = 1
   time.sleep(delay)   # delay till conditional repeat
   while Missing and (i < retry):   
     Voltage = instrument.read_register(12,1)
     VOLTAGE = Voltage
     ActivPower = instrument.read_register(14,0)
     Missing = (ActivPower == '')
     time.sleep(delay) # delay till next cycle    
     i = i+1

if ActivPower < 65535/2:
   PRODUCTION = ActivPower
   CONSUMPTION = 0
else:
   ActivPower = ActivPower - 65534
   PRODUCTION = 0
   CONSUMPTION = - ActivPower

#ReactPower = instrument.read_register(15,0)
#PF = instrument.read_register(16,3)
#Freq = instrument.read_register(17,2)
#Status = instrument.read_register(21,0)
#Adres = Status/256
#print 'DDS238_Status, Address  = ', Adres
#Rate = Status%256
#print 'DDS238_Status, Baudrate = ', Rate
print
print 'Run ', i
print '= RS485-Values for Upload ='
print 'Life_Net_Energy  = ', LIFENERGY, ' Wh'
print 'Forward_Energy   = ', FWDENERGY, ' Wh'
print 'Reverse_Energy   = ', REVENERGY, ' Wh'
print 'Voltage       = ', VOLTAGE, ' V'
print 'Current       = ', CURRENT, ' A'
print 'ActivePower   = ', ActivPower, ' VA'
print '=> Production   = ', PRODUCTION, ' VA'
print '=> Consumption  = ', CONSUMPTION, ' VA'

ENERGY = LIFENERGY
POWER = PRODUCTION
# --------------------------------------------------
# END READING RS485-INTERFACE OF DDS238
# --------------------------------------------------
print
# ---------------------------------------------------
# EXTRACT SUNRISE & SUNSET
# ---------------------------------------------------

# Imports for script-operation

import ephem
#import datetime    # already imported in Part1

# Presetting of parameters

sun = ephem.Sun()
home = ephem.Observer()
home.lat, home.lon = '52.2962643', '6.8033612' #your lat long
# lat/long-info must be inserted as decimal degrees
sun.compute(home)
sunrise, sunset = (home.next_rising(sun),
                   home.next_setting(sun))
print '= Sunrise & Sunset expressed in UTC! ='
print 'Sunrise = ',sunrise
print 'Sunset  = ',sunset
a = sunrise.datetime()
b = sunset.datetime()
Starthour = a.hour
if a.minute > 30:
  Starthour = Starthour - 1  # shift 1 hour before sunrise-UTC
if Starthour < 5:
  Starthour = 5
Stophour = b.hour + 1        # shift 1 hour after sunset-UTC
if b.minute < 30:
  Stophour = Stophour + 1    # shift 1 more hour
if Stophour > 21:
  Stophour = 21
print 'Upload-window opens when start-hour =',Starthour
print 'Upload-window closes after stop-hour=',Stophour
# --------------------------------------------------
# END READING SUNRISE & SUNSET
# --------------------------------------------------
print
# --------------------------------------------------
# START OF UPLOAD TO DOMOTICZ
# --------------------------------------------------
# Imports for script-operation

#import json
import urllib
from datetime import date

# Presetting of IDX for YOUR virtual sensors

domoticz_P1 = "209"    # IDX for virtual P1-meter
domoticz_V = "211"     # IDX for virtual sensor for voltage
domoticz_PE = "202"    # IDX for virtual sensor for power & energy
domoticz_C = "210"     # IDX for virtual sensor for current

# Setting & Linking of parameters

weekday = date.isoweekday(now)
hightime = (now.hour > 6) and (now.hour < 23) and (weekday < 6)
print 'Weekday = ',weekday
# (If desired) further code to be inserted to include holidays in the tariff-switching and to store at switching time the energy-values from the previous period
USAGE1 = REVENERGY
USAGE2 = 0
RETURN1 = FWDENERGY
RETURN2 = 0
CONS = CONSUMPTION
PROD = PRODUCTION
CONS = CONSUMPTION
PROD = PRODUCTION

httpresponse = urllib.urlopen("http://127.0.0.1:8080/json.htm?type=command&param=udevice&idx=" + str(domoticz_P1) + "&nvalue=0&svalue=" + str(float(USAGE1)) + ";" + str(float(USAGE2)) + ";" + str(float(RETURN1)) + ";" + str(float(RETURN2)) + ";" + str(float(CONS)) + ";" + str(float(PROD)) )
print 'Uploaded P1'

if POWER >= 0:
   httpresponse = urllib.urlopen("http://127.0.0.1:8080/json.htm?type=command&param=udevice&idx=" + str(domoticz_V) + "&nvalue=0&svalue=" + str(float(VOLTAGE)) )
   print 'Uploaded P&E'
   httpresponse = urllib.urlopen("http://127.0.0.1:8080/json.htm?type=command&param=udevice&idx=" + str(domoticz_PE) + "&nvalue=0&svalue=" + str(float(POWER)) + ";" + str(float(ENERGY)) )
   print 'Uploaded V'
   httpresponse = urllib.urlopen("http://127.0.0.1:8080/json.htm?type=command&param=udevice&idx=" + str(domoticz_C) + "&nvalue=0&svalue=" + str(float(CURRENT)) )
   print 'Uploaded C'
else:
   print 'No Power = No Upload'

# --------------------------------------------------
# END OF UPLOAD TO DOMOTICZ
# --------------------------------------------------

# --------------------------------------------------
# START OF UPLOAD TO PVOUTPUT
# --------------------------------------------------

# Imports for script-operation

import subprocess
from time import strftime
#import time       # already imported in Part1

# Presetting of parameters

SYSTEMID = pvout_sysid
APIKEY = pvout_apikey
t_date = format(strftime('%Y%m%d'))
t_time = format(strftime('%H:%M'))
pv_volts=0.0
pv_power=0.0
Wh_life=ENERGY
Wh_today=0.0

# space for addition of a calculation for current_temp

uploadtime = (now.hour >= Starthour) and (now.hour <= Stophour)
if uploadtime:
   print 'inside upload-window'
else:
   print 'outside upload-window'

if pvout_enabled and uploadtime:
   print 'uploading'
   pv_volts=VOLTAGE
   pv_power=POWER
   Wh_life=ENERGY
   cum=pvout_cumflag
   cmd=('curl -d "d=%s" -d "t=%s" -d "v1=%s" -d "v2=%s" -d "v6=%s" -d "v10=%s" -d "c1=%s" -H \
   "X-Pvoutput-Apikey: %s" -H \
   "X-Pvoutput-SystemId: %s" \
   http://pvoutput.org/service/r2/addstatus.jsp'\
   %(t_date, t_time, Wh_life, pv_power, pv_volts, Wh_life, cum, \
   APIKEY, SYSTEMID))
   ret = subprocess.call(cmd, shell=True)

# space for addition of a calculation for Wh_today

#time.sleep(20)            # delay to create space till next upload

#if Wh_today>0:            # if you also apply a calculated Wh_today
#  cmd=('curl -d "data=%s,%s,,,,,,," -H \
#  "X-Pvoutput-Apikey: %s" -H \
#  "X-Pvoutput-SystemId: %s" http://pvoutput.org/service/r2/addoutput.jsp' \
#  %(t_date, Wh_today,\
#  APIKEY,SYSTEMID))
#  ret = subprocess.call(cmd, shell=True)

print
print '= Info for upload to PVOutput ='
print 't_date %s' %t_date
print 't_time %s' %t_time
print 'pv_volts %s' %pv_volts
print 'pv_power %s' %pv_power
print 'Wh_life %s' %Wh_life
# --------------------------------------------------
# END OF UPLOAD TO PVOUTPUT
# --------------------------------------------------

Put the above python script in the "home/pi/domoticz/scripts/python" folder.

Before (test)running the script, first install the Python-modules serial, minimalmodbus and pyephem, required to operate the above Python-script.

Create the Cronjob

Now all ingredients are available to read the data from the inverter and insert them into Domoticz.

execute python DDS238Export01.py
Domoticz Inverter devices should now be filled with data.
execute "crontab -e"
add the following line "*/5 * * * * /home/pi/domoticz/scripts/DDS238_01.sh 2>&1 >> /dev/null"
exit crontab. Now everything should be working for you.

More than 1 kWh-meter type DDS238-1ZN or other kWH-meters in the configuration

Multiple DDS238-1ZN kWh-meters at one RS485-bus

If you want to apply more than 1 DDS238-1ZN in your configuration, please be aware of the following aspect:

The DDS238-1ZN meter is delivered with default Slave address 01 for it's RS485-interface and baudrate of 9600bps.
Therefore initially only one meter with that address 01 should be present at the RS485-network.
Before fitting an additional meter to the RS485-network, you must shift the first meter already connected to another address.
Change of that address is by write-command over the RS485-interface to the meter at Slave address 01. In the same command you must set which baudrate is desired under the new setting (obviously simplest to keep the 9600bps).

As a consequence, if you want to have more than 1 DDS238-1ZN meter at your RS485-network (direct out of the box), then

Additional software must first be added & run, which for that first device at Slave-address 01 shifts the adress towards another free Slave-address to make space for the second device.
After that action you can connect the next DDS238-1ZN-meter to the RS485-network, and run the same software another time to make space at Slave-address 01 for the third device, etc.
Repeat this sequence until you have fitted all DDS238-1ZN-meters to your network
To get the information to Domoticz and to PVOutput for each additional DDS238-1ZN-meter X, you now must replicate and must 'tune' the whole sequence of 'Prepare Domoticz', 'Create Shell-script', 'Create DDS238Export0X.py' and 'create the Cronjob', as described in the previous sections! Or ;-) somebody must develop a clever script which can handle multiple kWh-meters of type DDS238-1ZN ........

The required DDS238-1ZN scripts for address-check and for address-shift are shown below. On purpose they are separate scripts (with explanation of functionality in their headings):

the first script applies the Broadcast-facility of ModBus to find where the Slave-device is located at the RS485
the second script applies the found Comms Setting as starting point, and then requests/commands the Slave-device to shift the address and baudrate to desired new settings.
after the change of settings you can apply the first script to check that the change has been performed as desired.

From testing experience we know that sometimes the change of settings is not performed as expected, resulting in shift to unexpected adress and sometimes change of baudrate. Then you have to (re)find the Slave-device, which search can be done by repeated application of the first script, sometimes with different baudrates. The scripts also have no protection against accidentally wrong choices:

You as User are responsible that the correct selections are made to change the settings for the selected Slave-device!
No automatic check that de destination-address is free (due to the difficulty to make a reliable sweepsearch-function over the full range of adresses)!!!!

#!/usr/bin/python
# -*- coding = utf-8 to enable reading by simple editors -*-
# Control of DDS238-1ZN, Script for read-out of Comms Settings
# (C)2017 Toulon7559

# This script through the RS485-bus reads the broadcast address 00 of the DDS238-1ZN kWh-meter.
# The reading is aimed at register 15hex/21dec which controls communication settings.
# This script extracts the actual slave-address and baudrate of the DDS238-1ZN kWh-meter.
# Before running the script, the User has to set the expected speed of RS485-communication.
# If not sure, then it is 'try-and-error' with simple output:
# IF correct rate AND device connected, THEN you see information. Otherwise blank fields.
# Debug is foreseen as option to read more details of the communication.
# Environment & testing: 
# this script has been developed & tested on a Raspberry with Linksprite RS485-shield. 
# Application is at own risk for the User: feedback is encouraged.

#========================================================
# DEPENDENCIES
#========================================================

import serial
import minimalmodbus

#========================================================
# SETTINGS
#========================================================

speed = 9600  
# 9600(=>setting1), 4800(=>setting2), 2400(=>setting3), 1200(=>setting4)

#========================================================
# RUNTIME
#========================================================

instrument = minimalmodbus.Instrument('/dev/ttyAMA0',0) # port name, Broadcast slave address
instrument.serial.baudrate = speed
instrument.serial.timeout = 0.5
# instrument.debug = True
print instrument
print
instrument.read_register(21,0)
Status = instrument.read_register(21,0)
print 'Comms_value = ',Status
Adres = Status/256
print '=> DDS238_Status, Address  = ', Adres
Rate = Status%256
print '=> DDS238_Status, Baudrate = ', Rate
print

#!/usr/bin/python
# -*- coding = utf-8 to enable reading by simple editors -*-
# Control of DDS238-1, Script for change of Comms Settings
# (C)2017 Toulon7559

# This script through the RS485-bus modifies the slave address and baudrate of 1 ('''one!''') selected DDS238-1ZN kWh-meter.
# The writing of the modification is aimed at register 15hex/21dec which controls the communication settings.
# This script as starting values requires the actual slave-address and baudrate of the selected DDS238-1ZN kWh-meter.
# [[Before]] running the script, the User has to set in the script the applicable actual settings and the desired, new settings.
# If not sure of the actual settings, then first apply the (separate) script for read-out in Broadcast mode.
# That script is also useful for check/confirmation after modification of the Comms settings.
# Debug is foreseen as option to read more details of the communication.
# Environment & testing: 
# this script has been developed & tested on a Raspberry with Linksprite RS485-shield. 
# Application is at own risk for the User: feedback is encouraged.

#========================================================
# DEPENDENCIES
#========================================================

import serial
import minimalmodbus

#========================================================
# SETTINGS
#========================================================

present_address = 1     # decimal value of slave adress
present_rate = 9600     # 9600, 4800, 2400, 1200
new_address = 5         # decimal value of slave address
new_rate = 1            # 1=9600, 2=4800, 3=2400, 4=1200

#========================================================
# RUNTIME
#========================================================

print 'Present address = ', present_address
print 'Present rate = ', present_rate
print 'New address = ', new_address
print 'New rate = ', new_rate
newcom = new_address*256 + new_rate
print 'Command =', newcom
print

instrument = minimalmodbus.Instrument('/dev/ttyAMA0',present_address) # port name, slave address
instrument.serial.baudrate = present_rate
instrument.serial.timeout = 0.5
# instrument.debug = True
instrument.write_register(21,newcom)

Other types of kWh-meters with RS485-interface

The kWh-meter type DDS238-1ZN at it's RS485-interface applies the Modbus RTU protocol.

Although calling with Modbus' Function Codes 03H, 04H and 10H seems a common feature for kWh-meters with RS485-interface/Modbus-RTU, other types of meters most probably will have a different protocol-layout, with other default network-address, other register-addresses, different functional calls and different output information. Also different settings for Parity etc. have been seen.

Therefore for such 'other' meter with different protocollayout a specific 'own' interface-routine must be developed, replacing the DDS238 interface-routine of this Python script.

Obviously also a related, dedicated shell-script and dedicated cronjob must be made, and probably for multiple meters you have to replicate this effort.

Development status

The main script described in this Wiki-page runs for 1 DDS238-1ZN kWh-meter at Slave-address 01.

For a configuration with multiple DDS238-ZN kWh-meters the 2 additional scripts can be used to detect the location of the kWh-meters and to shift a kWh-meter to another Slave-address.

History and latest developments on this subject can be found at http://www.domoticz.com/forum/viewtopic.php?f=14&t=5808

Python - Read-out of DDS238 kWh-meter and upload to Domoticz and to PVOutput

Contents

Introduction

My current setup

Solar Inverter & Server

Hardware Configuration

Software

Dependencies

Prepare Domoticz

Creating the Shell script used in cron

Collect required Inputs for Settings

Create DDS238Export01.py

Create the Cronjob

More than 1 kWh-meter type DDS238-1ZN or other kWH-meters in the configuration

Multiple DDS238-1ZN kWh-meters at one RS485-bus

Other types of kWh-meters with RS485-interface

Development status

Navigation menu

Python - Read-out of DDS238 kWh-meter and upload to Domoticz and to PVOutput

Introduction

My current setup

Solar Inverter & Server

Hardware Configuration

Software

Dependencies

Prepare Domoticz

Creating the Shell script used in cron

Collect required Inputs for Settings

Create DDS238Export01.py

Create the Cronjob

More than 1 kWh-meter type DDS238-1ZN or other kWH-meters in the configuration

Multiple DDS238-1ZN kWh-meters at one RS485-bus

Other types of kWh-meters with RS485-interface

Development status

Navigation menu

Search