Python - Read-out of DDS238 kWh-meter and upload to Domoticz and PVOutput

Introduction

This wiki-section can be divided in three main parts:

a domoticz configuration part where you will be creating the appropriate virtual sensors.
a shell script and cronjob which control a python script that extracts information from the DDS238-meter and will be posting values into Domoticz and to PVOutput.
a description of variations and their impact on the 2 earlier parts.

The interface-routines for Energy-Meter type DDS238-1ZN in part 2 are specific as well as part 3, but for part 1 and for main parts of part 2 I have gratefully borrowed useful elements from the Wiki for the Omnik Solar Inverter.

The main reasons I made this script:

my setup with Involar inverters is so small that application of Involar's eGate would be a giant overkill, not even considering that the eGate does not connect to Domoticz, PVOutput etc., but only to the 'own' portal of Involar.
still I want registration of reasonably quality as well as application of the data by Domoticz and by PVOutput.
a script for reading a kWh-meter with RS485-interface does not exist as 'open source'

The description and script in this Wiki assumes a single kWh-meter type DDS238-1ZN with default settings:

Slave-address = 01
Baudrate = 01 = 9600 Baud

For any other configuration or setting, see section 3 of this Wiki.

My current setup

As perspective for this Wiki-description, below a short description of the situation at my home.

Solar Inverter & Server

Hardware Configuration

For a small subsection of my PV-system 2* Involar-inverter type MAC250 connect to 2* PV-panel type ESE215. The 2 inverters coupled in series feed a common 230V-grid-interface through a kWh-meter type DDS238-1ZN. This kWh-meter has 2 types of data-interface:

S0-interface with 3200pulses/kWh, feeding a counter, of which after calculation the output is the accumulated energy (Wh), and (after backward-calculation) the actual power (W).
RS485-interface provides directly measured data for LifeEnergy (Wh), Power (W), Voltage (V), Current (A), Frequency (Hz) and several other aspects. This interface applies the ModBus RTU protocol, reading by calls to 'Modbus' registers 03Hex and 04Hex, and by writing to 'Modbus' register 10Hex (=21Dec).

In my configuration, both interfaces are connected to a Raspberry acting as server running Domoticz and Python:

the frontend for the S0-interface is an S0PulseCountingModule [as described elsewhere in this wiki]
the frontend for the RS485-interface is an RS485-shield by Linksprite

Because of the direct measurement at the inverter-output and due to the data-catalogue, the RS485-interface is more versatile and superior to the S0-interface, and topic of this Wiki-description.

Software

For the RS485-interface of the DDS238-meter no ready interface exists as open-source material.

A Python-script pulls out the data from my DDS238-meter (applying the Python-module minimalmodbus), stores it in Domoticz using JSON, and uploads info to PVOutput.

For this script the front-end interfacing to the DDS238-meter's RS485-interface is 'own design', the idea to determine the time-gate (using pyephem) has been borrowed and converted to an adapted script, while the interface towards Domoticz and PVOutput has been borrowed & adapted from the Wiki-section for the Omnik_Solar_Inverter: ;-) re-inventing the wheel should be minimized ......

The proposed scripts are all assuming application for a DDS238-meter numbered 1 (straight out of the box): for meters with other number you have to adapt (at least) that number.

The script quite on purpose has been split in blocks each covering a specific aspect, with small steps in functionality, and with plenty of comments and print lines, for explanation and for checking: intending to provide help in understanding the script, but not the best example of programming! A clever programmer may significantly compact the size of the script, without any loss of functionality.

Before you can use the script, you have to adapt the settings for IDXes for your virtual sensors and for the API-key and SID applicable for upload to YOUR account at PV-Output.org

Prepare Domoticz

To create a virtual sensor, perform following actions.

Go to "Hardware"
Fill in the name field with a desired name (like "Virtual")
Choose for type the "Dummy, does nothing, use for virtual switches only"
Click on "Add"
The added hardware is now added to the hardware list. In the same row there is a "Create virtual sensor" button: click it.

Now you are ready to create virtual devices.

Go to "Hardware", and look for the section you created a moment ago
Press "Create virtual sensor".
Fill in the name-field with a desired name

To cover the most interesting elements provided at the RS485-interface of the DDS238-meter, we need ad least 3 virtual sensors:

# DDS238 Power / Energy (W / kWh)
# DDS238 AC Voltage (V)
# DDS238 AC Current (A)

Therefore repeat the above steps 3 times, after which:

Go to "Devices"
Click "Not-Used"
Search for the 3 virtual devices created a moment ago (and click on the green arrow behind it).
Give the sensors the following name:

 # "DDS238 Production1",  will contain 2 values for Current Power-production, respectively accumulated Energy in (Watt / kWh) for meter#1
 # "DDS238 Voltage1", will contain a value of inverter's output voltage (V) for meter#1
 # "DDS238 Current1", will contain a value of inverter's output current (A) for meter#1

Creating the Shell script used in cron

Now create for meter#1 a shell script DDS238_01.sh and put it in the "home/pi/domoticz/scripts/" folder.

#!/bin/bash
sudo python /home/pi/domoticz/scripts/DDS238Export01.py

Make this Shell-script executable: "sudo chmod +x /home/pi/domoticz/scripts/DDS238_01.sh"

Collect required Inputs for Settings

For the Python-script you require the following information (in the sequence in whicht they appear in the script).

For upload to PVOutput.org

your api-key
the destination SID

For setting of Ephem (to get data for sunrise and sunset)

your home.lat(itude)
your home.long(itude)

For upload to Domoticz

the IDXes of the virtual devices

Create DDS238Export01.py

#!/usr/bin/python
# -*- coding = utf-8 to enable reading by simple editors -*-
# --------------------------------------------------
# Line004 = PREPARATION & SETTINGs
# --------------------------------------------------

# Imports for script-operation

import serial                   # required for handling of the serial port
import minimalmodbus            # required for handling modbus
import datetime                 # used for timestamps & timecheck
import time                     # used for timestamps & delays

# Line014 = Provide here the settings for access to PVOutput

slave = 1             # decimal slave-address of the DDS238 kWh-meter
retry = 5             # number of retries to get power-info from DDS238
delay = 2             # time (seconds) between retries
pvout_enabled   = True
pvout_apikey    = "<YOUR api-key>"
pvout_sysid     = <YOUR SID>
pvout_cumflag   = 1   # flag is 1 if you apply lifetime-energy as reported by DDS238
                      # flag must be 0 if you upload a calculated Wh_today

# --------------------------------------------------
# Line026 = READING THE RS485-INTERFACE OF DDS238
# --------------------------------------------------
# the "print"-lines only added for local read-out e.g. during script_tuning

now = datetime.datetime.now()
print
print 'Present Date & Time = ', now
print
instrument = minimalmodbus.Instrument('/dev/ttyAMA0',slave) # port name, slave address
instrument.serial.baudrate = 9600
instrument.serial.timeout = 0.5
# instrument.debug = True

TotalEnergy1 = instrument.read_registers(0,2)
LifeEnergy_High = TotalEnergy1[0]
LifeEnergy_Low = TotalEnergy1[1]
LifeEnergy = 0.01 * ((LifeEnergy_High * 65535) + LifeEnergy_Low)
LIFENERGY = LifeEnergy * 1000
TotalEnergy2 = instrument.read_registers(8,2)
RevEnergy_High = TotalEnergy2[0]
RevEnergy_Low = TotalEnergy2[1]
RevEnergy = 0.01 * ((RevEnergy_High * 65535) + RevEnergy_Low)
REVENERGY = RevEnergy * 1000
TotalEnergy3 = instrument.read_registers(10,2)
FwdEnergy_High = TotalEnergy3[0]
FwdEnergy_Low = TotalEnergy3[1]
FwdEnergy = 0.01 * ((FwdEnergy_High * 65535) + FwdEnergy_Low)
FWDENERGY = FwdEnergy * 1000
Voltage = instrument.read_register(12,1)
VOLTAGE = Voltage
Current = instrument.read_register(13,2)
CURRENT = Current
ActivPower = instrument.read_register(14,0)
if (LifeEnergy > 0) and (ActivPower >= 0):
   Missing = False
   i = 1
else:              # try to get at least power-info
   Missing = True
   i = 1
   time.sleep(delay)   # delay till conditional repeat
   while Missing and (i < retry):   
     Voltage = instrument.read_register(12,1)
     VOLTAGE = Voltage
     ActivPower = instrument.read_register(14,0)
     Missing = (ActivPower == '')
     time.sleep(delay) # delay till next cycle    
     i = i+1

if ActivPower < 65535/2:
   PRODUCTION = ActivPower
   CONSUMPTION = 0
else:
   ActivPower = ActivPower - 65534
   PRODUCTION = 0
   CONSUMPTION = - ActivPower

#ReactPower = instrument.read_register(15,0)
#PF = instrument.read_register(16,3)
#Freq = instrument.read_register(17,2)
#Status = instrument.read_register(21,0)
#Adres = Status/256
#print 'DDS238_Status, Address  = ', Adres
#Rate = Status%256
#print 'DDS238_Status, Baudrate = ', Rate
print
print 'Run ', i
print '= RS485-Values for Upload ='
print 'Life_Net_Energy  = ', LIFENERGY, ' Wh'
print 'Forward_Energy   = ', FWDENERGY, ' Wh'
print 'Reverse_Energy   = ', REVENERGY, ' Wh'
print 'Voltage       = ', VOLTAGE, ' V'
print 'Current       = ', CURRENT, ' A'
print 'ActivePower   = ', ActivPower, ' VA'
print '=> Production   = ', PRODUCTION, ' VA'
print '=> Consumption  = ', CONSUMPTION, ' VA'

ENERGY = LIFENERGY
POWER = PRODUCTION
# --------------------------------------------------
# END READING RS485-INTERFACE OF DDS238
# --------------------------------------------------
print
# ---------------------------------------------------
# EXTRACT SUNRISE & SUNSET
# ---------------------------------------------------

# Imports for script-operation

import ephem
#import datetime    # already imported in Part1

# Presetting of parameters

sun = ephem.Sun()
home = ephem.Observer()
home.lat, home.lon = '52.2962643', '6.8033612' #your lat long
# lat/long-info must be inserted as decimal degrees
sun.compute(home)
sunrise, sunset = (home.next_rising(sun),
                   home.next_setting(sun))
print '= Sunrise & Sunset expressed in UTC! ='
print 'Sunrise = ',sunrise
print 'Sunset  = ',sunset
a = sunrise.datetime()
b = sunset.datetime()
Starthour = a.hour
if a.minute > 30:
  Starthour = Starthour - 1  # shift 1 hour before sunrise-UTC
if Starthour < 5:
  Starthour = 5
Stophour = b.hour + 1        # shift 1 hour after sunset-UTC
if b.minute < 30:
  Stophour = Stophour + 1    # shift 1 more hour
if Stophour > 21:
  Stophour = 21
print 'Upload-window opens when start-hour =',Starthour
print 'Upload-window closes after stop-hour=',Stophour
# --------------------------------------------------
# END READING SUNRISE & SUNSET
# --------------------------------------------------
print
# --------------------------------------------------
# START OF UPLOAD TO DOMOTICZ
# --------------------------------------------------
# Imports for script-operation

#import json
import urllib
from datetime import date

# Presetting of IDX for YOUR virtual sensors

domoticz_P1 = "209"    # IDX for virtual P1-meter
domoticz_V = "211"     # IDX for virtual sensor for voltage
domoticz_PE = "202"    # IDX for virtual sensor for power & energy
domoticz_C = "210"     # IDX for virtual sensor for current

# Setting & Linking of parameters

weekday = date.isoweekday(now)
hightime = (now.hour > 6) and (now.hour < 23) and (weekday < 6)
print 'Weekday = ',weekday
# (If desired) further code to be inserted to include holidays in the tariff-switching and to store at switching time the energy-values from the previous period
USAGE1 = REVENERGY
USAGE2 = 0
RETURN1 = FWDENERGY
RETURN2 = 0
CONS = CONSUMPTION
PROD = PRODUCTION
CONS = CONSUMPTION
PROD = PRODUCTION

httpresponse = urllib.urlopen("http://127.0.0.1:8080/json.htm?type=command&param=udevice&idx=" + str(domoticz_P1) + "&nvalue=0&svalue=" + str(float(USAGE1)) + ";" + str(float(USAGE2)) + ";" + str(float(RETURN1)) + ";" + str(float(RETURN2)) + ";" + str(float(CONS)) + ";" + str(float(PROD)) )
print 'Uploaded P1'

if POWER >= 0:
   httpresponse = urllib.urlopen("http://127.0.0.1:8080/json.htm?type=command&param=udevice&idx=" + str(domoticz_V) + "&nvalue=0&svalue=" + str(float(VOLTAGE)) )
   print 'Uploaded P&E'
   httpresponse = urllib.urlopen("http://127.0.0.1:8080/json.htm?type=command&param=udevice&idx=" + str(domoticz_PE) + "&nvalue=0&svalue=" + str(float(POWER)) + ";" + str(float(ENERGY)) )
   print 'Uploaded V'
   httpresponse = urllib.urlopen("http://127.0.0.1:8080/json.htm?type=command&param=udevice&idx=" + str(domoticz_C) + "&nvalue=0&svalue=" + str(float(CURRENT)) )
   print 'Uploaded C'
else:
   print 'No Power = No Upload'

# --------------------------------------------------
# END OF UPLOAD TO DOMOTICZ
# --------------------------------------------------

# --------------------------------------------------
# START OF UPLOAD TO PVOUTPUT
# --------------------------------------------------

# Imports for script-operation

import subprocess
from time import strftime
#import time       # already imported in Part1

# Presetting of parameters

SYSTEMID = pvout_sysid
APIKEY = pvout_apikey
t_date = format(strftime('%Y%m%d'))
t_time = format(strftime('%H:%M'))
pv_volts=0.0
pv_power=0.0
Wh_life=ENERGY
Wh_today=0.0

# space for addition of a calculation for current_temp

uploadtime = (now.hour >= Starthour) and (now.hour <= Stophour)
if uploadtime:
   print 'inside upload-window'
else:
   print 'outside upload-window'

if pvout_enabled and uploadtime:
   print 'uploading'
   pv_volts=VOLTAGE
   pv_power=POWER
   Wh_life=ENERGY
   cum=pvout_cumflag
   cmd=('curl -d "d=%s" -d "t=%s" -d "v1=%s" -d "v2=%s" -d "v6=%s" -d "v10=%s" -d "c1=%s" -H \
   "X-Pvoutput-Apikey: %s" -H \
   "X-Pvoutput-SystemId: %s" \
   http://pvoutput.org/service/r2/addstatus.jsp'\
   %(t_date, t_time, Wh_life, pv_power, pv_volts, Wh_life, cum, \
   APIKEY, SYSTEMID))
   ret = subprocess.call(cmd, shell=True)

# space for addition of a calculation for Wh_today

#time.sleep(20)            # delay to create space till next upload

#if Wh_today>0:            # if you also apply a calculated Wh_today
#  cmd=('curl -d "data=%s,%s,,,,,,," -H \
#  "X-Pvoutput-Apikey: %s" -H \
#  "X-Pvoutput-SystemId: %s" http://pvoutput.org/service/r2/addoutput.jsp' \
#  %(t_date, Wh_today,\
#  APIKEY,SYSTEMID))
#  ret = subprocess.call(cmd, shell=True)

print
print '= Info for upload to PVOutput ='
print 't_date %s' %t_date
print 't_time %s' %t_time
print 'pv_volts %s' %pv_volts
print 'pv_power %s' %pv_power
print 'Wh_life %s' %Wh_life
# --------------------------------------------------
# END OF UPLOAD TO PVOUTPUT
# --------------------------------------------------

Put the above python script in the "home/pi/domoticz/scripts/python" folder.

Install the Python-modules minimalmodbus and pyephem, required to operate the above Python-script.

Create the Cronjob

Now all ingredients are available to read the data from the inverter and insert them into Domoticz.

execute python DDS238Export01.py
Domoticz Inverter devices should now be filled with data.
execute "crontab -e"
add the following line "*/5 * * * * /home/pi/domoticz/scripts/DDS238_01.sh 2>&1 >> /dev/null"
exit crontab. Now everything should be working for you.

More than 1 DDS238-meter or other kWh-meters in the configuration

Multiple DDS238-meters

If you want to apply more than 1 DDS238-1ZN in your configuration, please be aware of the following aspect:

The DDS238-1ZN meter is default delivered with Slave address 01 for it's RS485-interface.

If your supplier can change that address, make use of that possibility by stating the desired address other than 00, or 01! Otherwise, change of that Slave address is by a write-command over the RS485-interface to the meter at Slave address 01.

Under default, initially only one meter with that address 01 will be present at the RS485-network.
Before fitting an additional meter to the RS485-network, therefore you must shift the meter already connected to another Slave address.

As a consequence, if you want to have at your 'normal' RS485-network more than 1 DDS238-1ZN meter, then

1) additional software must first be added & run at a mini-RS485-bus which - for the first device (direct out of the box) at Slave-address 01 shifts the adress towards another free Slave-address to make space for the second device.

2) After that action you can connect (as sole Slave-Device) the next new DDS238-meter (direct out of the box) to that mini-RS485-bus, and run the same software another time to make space at Slave-address 01 for the third device, etc.

To avoid any interference from other devices and the software in the background, the best procedure for changing the addresse for second and further Devices is:

a) first make a scheme which Slave address goes to which RS485-Device, for which Virtual Device in Domoticz and for which Subsystem in PVOutput.

b) disconnect the RS485-Master (= Raspberry+RS485-interface) from the 'normal' RS485-bus

b) connect the RS485-Device-to-be-changed to this RS485-Master at a dedicated mini-RS485-bus

c) run the software to change the address of the RS485-Device-to-be-changed

d) check that the address has been changed as desired by running the software for address-check, and take note of result

e) connect the changed device to the 'normal' RS485-bus

3) Repeat this above sequence until you have fitted all desired DDS238-meters to your network, with all a different address.

Then (only then!) reconnect the RS485-Master to the RS485-bus.

4) To get the information to Domoticz and to PVOutput for each additional DDS238-meter X at the correct places (such as IDXes of Virtual Devices in Domoticz and Subsystems at PVOutput), you now must as required, for all subject Meters and Scripts check, replicate and 'tune' the whole sequence of 'Prepare Domoticz', 'Create Shell-script', 'Create DDS238Export0X.py' and 'create the Cronjob', as described in the previous sections!

The addresses noted in the previous section are essential info for that job! Don't forget a meter connected at Slave-Address 01!

Software for address-check and for address-modification

The required software could be combined in 1 script, but the DDS238-1ZN-scripts for (address-check + address-shift) are below as 2 separate Python-scripts.

On purpose to 'force' the User to make the appropriate settings before application:

if the settings are not compatible, usually error-reports will remind you before damage is done!

As indicated above the safe way is to run this software in a mini-configuration with the RS485-Master and the Salve-device-to-be-changed as only devices on a mini-RS485-Bus [which is just the connection cable between the 2 devices].

The software-application in Summary:

1) Run the software for address-check => Slave-address from connected meter

2) Adapt the software for address-modification to include the read address and the desired new address. Take care that the speed remains unchanged, because otherwise you loose contact with the device!

3) Run the software for address-modification.

4) Run the software for address-check.

From experience, under 3) sometimes unforeseen address-shift occurs and/or unforeseen speed-change.

Unforeseen adress-shift is easy to remedy by rerun of the sequence, until successful.

Unforeseen speed-change causes some more tension & headaches, but can usually be resolved by running the software for address-check at different speed-settings, and then rerun of the sequence, until successful.

Software for address-check

#!/usr/bin/python
# Control of DDS238-1ZN, Script for read-out of Comms Settings
# (C)2017 Toulon7559

# This script through the RS485-bus reads the broadcast address 00 of the DDS238-1ZN kWh-meter.
# The reading is aimed at register 15hex/21dec which controls communication settings.
# This script extracts the actual slave-address and baudrate of the DDS238-1ZN kWh-meter.
# Before application, the User has to set the expected speed of RS485-communication.
# If not sure, then it is 'try-and-error'.
# Debug is foreseen as option to read more details of the communication.
# Environment & testing: 
# this script has been developed & tested on a Raspberry with Linksprite RS485-shield. 
# Application is at own risk for the User: feedback is encouraged.

#========================================================
# DEPENDENCIES
#========================================================

import serial
import minimalmodbus

#========================================================
# SETTINGS
#========================================================

speed = 9600  
# 9600(=>setting1), 4800(=>setting2), 2400(=>setting3), 1200(=>setting4)

#========================================================
# RUNTIME
#========================================================

instrument = minimalmodbus.Instrument('/dev/ttyAMA0',0) # port name, Broadcast slave address
instrument.serial.baudrate = speed
instrument.serial.timeout = 0.5
# instrument.debug = True
print instrument
print
instrument.read_register(21,0)
Status = instrument.read_register(21,0)
print 'Comms_value = ',Status
Adres = Status/256
print '=> DDS238_Status, Address  = ', Adres
Rate = Status%256
print '=> DDS238_Status, Baudrate = ', Rate
print

Software for address modification

#!/usr/bin/python
# Control of DDS238-1ZN, Script for change of Comms Settings
# (C)2017 Toulon7559

# This script through the RS485-bus modifies the slave address of the connected DDS238-1ZN kWh-meter.
# The reading is aimed at register 15hex/21dec which controls communication settings.
# This script starts at the actual slave-address and baudrate of the DDS238-1 kWh-meter.
# Before application, the User has to set the applicable, actual settings and the desired, new settings.
# If not sure of the actual settings, then first apply the (separate) script for read-out in Broadcast mode.
# That script is also useful for check/confirmation after modification of the Comms settings.
# Debug is foreseen as option to read more details of the communication.
# Environment & testing: 
# this script has been developed & tested on a Raspberry with Linksprite RS485-shield. 
# Application is at own risk for the User: feedback is encouraged.

#========================================================
# DEPENDENCIES
#========================================================

import serial
import minimalmodbus

#========================================================
# SETTINGS
#========================================================

present_address = 1     # decimal value of slave adress
present_rate = 9600     # 9600, 4800, 2400, 1200
new_address = 5         # decimal value of slave address
new_rate = 1            # 1=9600, 2=4800, 3=2400, 4=1200

#========================================================
# RUNTIME
#========================================================

print 'Present address = ', present_address
print 'Present rate = ', present_rate
print 'New address = ', new_address
print 'New rate = ', new_rate
newcom = new_address*256 + new_rate
print 'Command =', newcom
print

instrument = minimalmodbus.Instrument('/dev/ttyAMA0',present_address) # port name, slave address
instrument.serial.baudrate = present_rate
instrument.serial.timeout = 0.5
# instrument.debug = True
instrument.write_register(21,newcom)

Other types of kWh-meters with RS485-interface

The DDS238-1ZN meter applies a specific protocol at it's RS485-interface, usually under the Modbus-protocol.

It's family-members like DDS238-4W apply the same general protocol layout, but check for the small variations!

Although calling the Modbus' registers 03H, 04H and 10H seems a common feature for such kWh-meters, other types of meters most probably will have a different protocol-layout, with other default network-address, other register-addresses, different functional calls and different output information.

Therefore for such 'other' meter with different protocol a specific 'own' interface-routine must be developed, replacing the interface-routine of this Python script.

Obviously also a dedicated shell-script and dedicated cronjob must be made, and probably for multiple meters you have to replicate this effort.

The comparable setup for Eastron kWh-meters with RS485-interface is described in another section of this WiKi: https://www.domoticz.com/wiki/Eastron_SDM120C.

Development status

Especially for the addition of extra DDS238-devices to an RS485-network, more work has to be done for robust software avoiding in first run the 'unforeseens', as indicated in that section.

History and latest developments can be seen in this thread: http://www.domoticz.com/forum/viewtopic.php?f=14&t=5808

Python - Read-out of DDS238 kWh-meter and upload to Domoticz and PVOutput

Contents

Introduction

My current setup

Solar Inverter & Server

Hardware Configuration

Software

Prepare Domoticz

Creating the Shell script used in cron

Collect required Inputs for Settings

Create DDS238Export01.py

Create the Cronjob

More than 1 DDS238-meter or other kWh-meters in the configuration

Multiple DDS238-meters

Software for address-check and for address-modification

Software for address-check

Software for address modification

Other types of kWh-meters with RS485-interface

Development status

Navigation menu

Python - Read-out of DDS238 kWh-meter and upload to Domoticz and PVOutput

Introduction

My current setup

Solar Inverter & Server

Hardware Configuration

Software

Prepare Domoticz

Creating the Shell script used in cron

Collect required Inputs for Settings

Create DDS238Export01.py

Create the Cronjob

More than 1 DDS238-meter or other kWh-meters in the configuration

Multiple DDS238-meters

Software for address-check and for address-modification

Software for address-check

Software for address modification

Other types of kWh-meters with RS485-interface

Development status

Navigation menu

Search