Upload energy data to PVoutput

Purpose

This wiki exists out of two parts;

a Lua device script for calculating the consumption.
a Lua time script that uploads the the desired data, including the calculated consumption, to PVoutput.

The main reason I made this script is because PVoutput is calculating the consumption by calculating the energy using the power. But as the interval is 5 minutes, a lot can happen in between the two intervals so therefore the calculated energy is not really accurate and so the calculated consumption isn't either.

Note: Remember that the information below is written for the use of a Raspberry pi. But probably it is possible to use for other systems also with some minor changes.

My current setup

To give a short description of my current system, below a short description of the situation at my home.

Smartmeter

I have a smartmeter that measures the imported and exported energy. The meter is also connected to my gas meter therefore the gas usage is also available.

Hardware

My smartmeter is connected with a P1 cable to a Raspberry Pi.

Software

Currently I'm using a python script from smartmeterdashboard.nl to get the data from my smartmeter using a P1 cable. I added a few lines of code to accomplish that the script also adds the data to Domoticz. As the P1 logging is also supported by Domoticz I later have changed the code below to collect the values from the integrated P1-logger from Domoticz. For information how to do this see latest chapter.

PV-system

I have a SamilPower inverter with 9 Renesola panels, this has a maximum system size of 2295W.

Hardware

My inverter is connected to a Raspberry Pi (RPi) with a RS232 to USB cable. As the inverter is upstairs and the smart meter downstairs I use two different raspberries. The net data from the above RPi is uploaded in a second upload to PVoutput.

Software

I'm using a script from whirlpool.net.au, this pulls out the data from my SamilPower inverter and stores it in a database. As I have also Domoticz running on this RPi, I added a few lines of code so with the use of JSON the data is also added to Domoticz.

Installation instructions - Calculating the consumption using a Lua device script

For this script the following values needs already be available in Domoticz in order to make the calculation of the consumption possible.

The Power and energy values (import and export) from a smartmeter
The Power, energy and voltage values from a Solar power inverter

The consumption is calculated using the two formulas below:

EnergyConsumption = EnergyGeneration + EnergyImport - EnergyExport
PowerConsumption = PowerGeneration + PowerImport - PowerExport

After the calculation the script updates a virtual device which displays the consumed power and energy in Domoticz.

Prepare Domoticz

Create a virtual device for the consumption.

Go to "Hardware"
Fill in the name field with a desired name (like "Calculation")
Choose for type the "Dummy, does nothing, use for virtual switches only"
Click on "Add"
The added hardware is now added to the hardware list. In the same row there is a "Create virtual sensor" button shown. Click on it.
Choose for "Electricity (Current+Counter)" and click on "OK".
Go to "Devices"
Search for the above added device and click on the green arrow behind it.
Give the device a name (like "Consumption")

Creating the Lua device script

Now create a Lua script called "script_device_calculate_consumption.lua" and put it in the "home/pi/domoticz/scripts/lua/" folder. You can do this by:

Using a SSH connection (I'm using MobaXterm)

Login as user "pi"
Go to the right folder with "cd /home/pi/domoticz/scripts/lua"
Create the file using "nano script_device_calculate_consumption.lua"
Now you can copy paste the script below in the nano editor.
Save the script

Or

Using a file transfer program (I use WinSCP)

Login as user "pi"
Create a local file with the name "script_device_calculate_consumption.lua"
Copy paste the script below in it.
Save the file
upload it to "/home/pi/domoticz/scripts/lua"

The Lua device script

-- /home/pi/domoticz/scripts/lua/script_device_calculate_consumption.lua
-- This script collects the values below from Domoticz
--   * The Power and energy values (import and export) from a smartmeter 
--   * The Power and energy values from a Solar power inverter
-- It then calculates the consumed power and energy from the values above with the formula's
--   * EnergyConsumption = EnergyGeneration + EnergyImport - EnergyExport
--   * PowerConsumption = PowerGeneration + PowerImport - PowerExport
-- It then updates a virtual device which displays the consumed power and energy in Domoticz

----------------------------------------------------------------------------------------------------------
-- Domoticz IDX and names of the needed devices
----------------------------------------------------------------------------------------------------------
local GenerationDeviceName = "xxxxxx" 		-- Device name of the Generated energy
local ImportedDeviceName = "xxxxxx" 		-- Name of the energy device that shows imported energy
local ExportedDeviceName = "xxxxxx" 		-- Name of the energy device that shows exported energy
local ConsumptionIDX = xxx  			-- IDX of the energy device that shows calculated Consumption
local ConsumptionDeviceName = "xxxxxx" 		-- Name of the energy device that shows calculated Consumption

----------------------------------------------------------------------------------------------------------
-- Require parameters
----------------------------------------------------------------------------------------------------------
local http = require("socket.http")

----------------------------------------------------------------------------------------------------------
-- Script parameters
----------------------------------------------------------------------------------------------------------
EnergyGeneration = 0 	-- in Watt hours
PowerGeneration = 0 	-- in Watts
EnergyImport = 0	-- in Watt hours
PowerImport = 0		-- in Watts
EnergyConsumption = 0 	-- in Watt hours
PowerConsumption = 0 	-- in Watts
Debug = "NO" 		-- Turn debugging on ("YES") or off ("NO")

----------------------------------------------------------------------------------------------------------
-- Lua Functions
----------------------------------------------------------------------------------------------------------
function update(device, id, power, energy, index)
	commandArray[index] = {['UpdateDevice'] = id .. "|0|" .. power .. ";" .. energy}
end 

----------------------------------------------------------------------------------------------------------
-- CommandArray
----------------------------------------------------------------------------------------------------------
commandArray = {}
	-- Generated
	PowerGeneration, EnergyGeneration = otherdevices_svalues[GenerationDeviceName]:match("([^;]+);([^;]+)")
	if Debug=="YES" then
		print("  ----- PowerGeneration = " .. PowerGeneration .. " W");
		print("  ----- EnergyGeneration = " .. EnergyGeneration .. " Wh");
	end

	-- Imported
	PowerImport, EnergyImport = otherdevices_svalues[ImportedDeviceName]:match("([^;]+);([^;]+)")
	if Debug=="YES" then
		print("  ----- PowerImport = " .. PowerImport .. " W");
		print("  ----- EnergyImport = " .. EnergyImport .. " Wh");
	end

	-- Exported
	PowerExport, EnergyExport = otherdevices_svalues[ExportedDeviceName]:match("([^;]+);([^;]+)")
	if Debug=="YES" then
		print("  ----- PowerExport = " .. PowerExport .. " W");
		print("  ----- EnergyExport = " .. EnergyExport .. " Wh");
	end

	-- Calculate consumption
	PowerConsumption = PowerGeneration + PowerImport - PowerExport
	if Debug=="YES" then
		print("  ----- PowerConsumption = " .. PowerConsumption .. " W");
	end
	EnergyConsumption = EnergyGeneration + EnergyImport - EnergyExport
	if Debug=="YES" then
		print("  ----- EnergyConsumption = " .. EnergyConsumption .. " Wh");
	end

	-- Update comsumption device in Domoticz
	if devicechanged[ImportedDeviceName] then
		update(ConsumptionDeviceName, ConsumptionIDX, PowerConsumption, EnergyConsumption, 1)
	end
	
return commandArray

Edit the Lua device script

Now in the script, go to the section below and edited the xxxxx and xxx values. The xxxxx is the name of the device, the xxx is the IDX value. Both can be found in Domoticz under "Devices".

----------------------------------------------------------------------------------------------------------
-- Domoticz IDX and names of the needed devices
----------------------------------------------------------------------------------------------------------
local GenerationDeviceName = "xxxxxx" 		-- Device name of the Generated energy
local ImportedDeviceName = "xxxxxx" 		-- Name of the energy device that shows imported energy
local ExportedDeviceName = "xxxxxx" 		-- Name of the energy device that shows exported energy
local ConsumptionIDX = xxx  			-- IDX of the energy device that shows calculated Consumption
local ConsumptionDeviceName = "xxxxxx" 		-- Name of the energy device that shows calculated Consumption

Installation instructions - Uploading the data to PVoutput using a Lua time script

The second part of this wiki is the Lua time script that actually uploads the data to PVoutput. This script collects the values below from Domoticz

The Power generation, energy generation and voltage from a Solar power inverter
The temperature from a outside temperature sensor
The Power consumption and energy consumption which is calculated in another Lua script

After collecting uploads all of the values to a PVoutput account.

For more information about PVoutput, see their user documentation on http://www.pvoutput.org/help.html

Install socket library

For doing http requests we need a socket. As there is no packaged versions available, simonrg has made two gzipped tar files for the Raspberry Pi. For more infomation see this forum post.

Download and install the files with the following instructions.

"cd /usr/local/share"
"sudo mkdir lua"
"cd lua"
"sudo mkdir 5.2"
"cd 5.2"
Download the file "sudo wget http://www.domoticz.com/forum/download/file.php?id=1454"
Rename file "sudo mv file.php\?id\=1454 usrlocalsharelua5p2.tar.gz"
Extract the file with "sudo tar -xzvf usrlocalsharelua5p2.tar.gz"
"cd /usr/local/lib"
"sudo mkdir lua"
"cd lua"
"sudo mkdir 5.2"
"cd 5.2"
Download the file "sudo wget http://www.domoticz.com/forum/download/file.php?id=1453"
Rename file "sudo mv file.php\?id\=1453 usrlocalliblua5p2.tar.gz"
Extract all the files with "sudo tar -xvf usrlocalliblua5p2.tar.gz".

Creating the Lua time script

Now create a Lua script called "script_time_upload_to_PVoutput.lua" and put it in the "home/pi/domoticz/scripts/lua/" folder. You can do this by:

Using a SSH connection (I'm using MobaXterm)

Login as user "pi"
Go to the right folder with "cd /home/pi/domoticz/scripts/lua"
Create the file using "nano script_time_upload_to_PVoutput.lua"
Now you can copy paste the script below in the nano editor.
Save the script

Or

Using a file transfer program (I use WinSCP)

Login as user "pi"
Create a local file with the name "script_time_upload_to_PVoutput.lua"
Copy paste the script below in it.
Save the file
upload it to "/home/pi/domoticz/scripts/lua"

The Lua time script

-- /home/pi/domoticz/scripts/lua/script_time_upload_to_PVoutput.lua
-- This script collects the values below from Domoticz
--   * The Power generation, energy generation and voltage from a Solar power inverter
--   * The temperature from a outside temperature sensor
--   * The Power consumption and energy consumption which is calculated in another Lua script 
-- And uploads all of the values to a PVoutput account.
--
-- For more information about PVoutput, see their user documentation on http://www.pvoutput.org/help.html

----------------------------------------------------------------------------------------------------------
-- Domoticz IDX of devices
----------------------------------------------------------------------------------------------------------
local GenerationDeviceName = "xxxxx" 	-- Device name of the Generated energy
local ConsumptionDeviceName = "xxxxx" 	-- Name of the energy device that shows calculated Consumption
local VoltageDeviceName = "xxxxx"	-- Name of the voltage device that shows voltage of the inverter
local TemperatureDeviceName = "xxxxx" 	-- Name of the temperature device that shows outside temperature

----------------------------------------------------------------------------------------------------------
-- PVoutput parameters
----------------------------------------------------------------------------------------------------------
local PVoutputApi = "xxxxxxxxxxxxxxxxxxxxxxx" 				-- Your PVoutput api key
local PVoutputSystemID = "xxxxx" 					-- Your PVoutput System ID
local PVoutputPostInterval = 5 						-- The number of minutes between posts to PVoutput (normal is 5 but when in donation mode it's max 1)
local PVoutputURL = '://pvoutput.org/service/r2/addstatus.jsp?key='	-- The URL to the PVoutput Service

----------------------------------------------------------------------------------------------------------
-- Require parameters
----------------------------------------------------------------------------------------------------------
local http = require("socket.http")

----------------------------------------------------------------------------------------------------------
-- Script parameters
----------------------------------------------------------------------------------------------------------
EnergyGeneration = 0 	-- v1 in Watt hours
PowerGeneration = 0 	-- v2 in Watts
EnergyConsumption = 0 	-- v3 in Watt hours
PowerConsumption = 0 	-- v4 in Watts
CurrentTemp = 0 	-- v5 in celcius
Voltage = 0 		-- v6 in volts
c1 = 1 			-- c1 = 0 when v1 is today's energy. c1 = 1 when v1 is lifetime energy.
Debug = "NO" 		-- Turn debugging on ("YES") or off ("NO")

----------------------------------------------------------------------------------------------------------
-- Lua Functions
----------------------------------------------------------------------------------------------------------
function UploadToPVoutput(self)
	b, c, h = http.request("http" .. PVoutputURL .. PVoutputApi .. "&sid=".. PVoutputSystemID .. "&d=" .. os.date("%Y%m%d") .. "&t=" .. os.date("%H:%M") .. 

	"&v1=" .. EnergyGeneration .. "&v2=" .. PowerGeneration .. "&v3=" .. EnergyConsumption .. "&v4=" .. PowerConsumption .. "&v5=" .. CurrentTemp .. "&v6=" .. Voltage .. "&c1=" .. c1 )
	if b=="OK 200: Added Status" then
		print(" -- Current status successfully uploaded to PVoutput.")
	else
		print(" -- " ..b)
	end
	
	print(" -- Energy generation (v1) = ".. EnergyGeneration .. " Wh")
	print(" -- Power generation (v2) = " .. PowerGeneration .. " W")
	print(" -- Energy consumption (v3) = " .. EnergyConsumption .. " Wh")
	print(" -- Power consumption (v4) = " .. PowerConsumption .. " W")
	print(" -- Current temperature (v5) = " .. CurrentTemp .. " C")
	print(" -- Voltage (v6) = " .. Voltage .. "V")
	print(" -- Cumulative Flag (c1) = " .. c1 .. "")
end

function update(device, id, power, energy, index)
	commandArray[index] = {['UpdateDevice'] = id .. "|0|" .. power .. ";" .. energy}
end 

----------------------------------------------------------------------------------------------------------
-- CommandArray
----------------------------------------------------------------------------------------------------------
commandArray = {}

	time = os.date("*t")
	if PVoutputPostInterval>1 then
		TimeToGo = PVoutputPostInterval - (time.min % PVoutputPostInterval)
		print('Time to go before upload to PVoutput: ' ..TimeToGo.. " minutes")
	end
	
	if((time.min % PVoutputPostInterval)==0)then
		-- Generated
		PowerGeneration, EnergyGeneration = otherdevices_svalues[GenerationDeviceName]:match("([^;]+);([^;]+)")
		if Debug=="YES" then
			print(" ---- The total generated energy is " .. EnergyGeneration .. " Wh");
			print(" ---- The current generated power is " .. PowerGeneration .. " W");
		end

		-- Voltage
		Voltage = otherdevices_svalues[VoltageDeviceName] :match("([^;]+)")
		if Debug=="YES" then
			print(" ---- The voltage of the inverter is " .. Voltage .. " V");
		end
		
		-- Temperature
		CurrentTemp = otherdevices_svalues[TemperatureDeviceName]:match("([^;]+)")
		if Debug=="YES" then
			print(" ---- The outside temperature is " .. CurrentTemp .. " C.");
		end
		
		-- Consumption
		PowerConsumption, EnergyConsumption = otherdevices_svalues[ConsumptionDeviceName]:match("([^;]+);([^;]+)")
		if Debug=="YES" then
			print(" ---- The total consumed energy is " .. EnergyConsumption .. " Wh");
			print(" ---- The current consumed power is " .. PowerConsumption .. " W");
		end

		-- Upload data to PVoutput	
		UploadToPVoutput()
	end

return commandArray

Edit the Lua time script

Now in the script, go to the section below and edited the xxxxx values. The xxxxx is the name of the device, which can be found in Domoticz under "Devices".

----------------------------------------------------------------------------------------------------------
-- Domoticz IDX of devices
----------------------------------------------------------------------------------------------------------
local GenerationDeviceName = "xxxxx" 	-- Device name of the Generated energy
local ConsumptionDeviceName = "xxxxx" 	-- Name of the energy device that shows calculated Consumption
local VoltageDeviceName = "xxxxx"	-- Name of the voltage device that shows voltage of the inverter
local TemperatureDeviceName = "xxxxx" 	-- Name of the temperature device that shows outside temperature

Now in the script, go to the section below and edited the xxxxxxxxxxxxxxxxxxxxxxx (PVoutput API key) and xxxxx (PVoutput System ID) values. The xxxxxxxxxxxxxxxxxxxxxxx and xxxxx can be found in your PVoutput account under "Settings". If you have made a donation to PVoutput you can raise the upload interval from every 5 min to a maximum of every 1 min.

----------------------------------------------------------------------------------------------------------
-- PVoutput parameters
----------------------------------------------------------------------------------------------------------
local PVoutputApi = "xxxxxxxxxxxxxxxxxxxxxxx" 				-- Your PVoutput api key
local PVoutputSystemID = "xxxxx" 					-- Your PVoutput System ID
local PVoutputPostInterval = 5 						-- The number of minutes between posts to PVoutput (normal is 5 but when in donation mode it's max 1)
local PVoutputURL = '://pvoutput.org/service/r2/addstatus.jsp?key='	-- The URL to the PVoutput Service

When using the integrated P1 logger from Domoticz

When you want to use the Integrated P1 logger that is integrated in Domoticz, folow the instructions below. In the Lua script called "script_device_calculate_consumption.lua" Replace

local ImportedDeviceName = "xxxxxx" 		-- Name of the energy device that shows imported energy
local ExportedDeviceName = "xxxxxx" 		-- Name of the energy device that shows exported energy

For

local EnergyDeviceName = "xxxxxx" 		-- Name of the energy device that shows imported and exported energy

Add

EnergyImportLow = 0	-- in Watt hours
EnergyImportHigh = 0	-- in Watt hours
EnergyExportLow = 0	-- in Watt hours
EnergyExportHigh = 0	-- in Watt hours

To (below) script parameters.

----------------------------------------------------------------------------------------------------------
-- Script parameters
----------------------------------------------------------------------------------------------------------

Replace

	-- Imported
	PowerImport, EnergyImport = otherdevices_svalues[ImportedDeviceName]:match("([^;]+);([^;]+)")
	if Debug=="YES" then
		print("  ----- PowerImport = " .. PowerImport .. " W");
		print("  ----- EnergyImport = " .. EnergyImport .. " Wh");
	end
 
	-- Exported
	PowerExport, EnergyExport = otherdevices_svalues[ExportedDeviceName]:match("([^;]+);([^;]+)")
	if Debug=="YES" then
		print("  ----- PowerExport = " .. PowerExport .. " W");
		print("  ----- EnergyExport = " .. EnergyExport .. " Wh");
	end

For

	EnergyImportLow, EnergyImportHigh, EnergyExportLow, EnergyExportHigh, PowerImport, PowerExport = otherdevices_svalues[EnergyDeviceName]:match("([^;]+);([^;]+);([^;]+);([^;]+);([^;]+);([^;]+)")
	EnergyImport = EnergyImportLow + EnergyImportHigh
	EnergyExport = EnergyExportLow + EnergyExportHigh
	if Debug=="YES" then
		print("  ----- PowerImport = " .. PowerImport .. " W");
		print("  ----- EnergyImportLow = " .. EnergyImportLow .. " Wh");
		print("  ----- EnergyImportHigh = " .. EnergyImportHigh .. " Wh");
		print("  ----- EnergyImport = " .. EnergyImport .. " Wh");
		print("  ----- PowerExport = " .. PowerExport .. " W");
		print("  ----- EnergyExportLow = " .. EnergyExportLow .. " Wh");
		print("  ----- EnergyExportHigh = " .. EnergyExportHigh .. " Wh");
		print("  ----- EnergyExport = " .. EnergyExport .. " Wh");
	end

Replace

	-- Update comsumption device in Domoticz
	if devicechanged[ImportedDeviceName] then
		update(ConsumptionDeviceName, ConsumptionIDX, PowerConsumption, EnergyConsumption, 1)
	end

For

	-- Update comsumption device in Domoticz
	if devicechanged[EnergyDeviceName] then
		update(ConsumptionDeviceName, ConsumptionIDX, PowerConsumption, EnergyConsumption, 1)
	end

Forum

For questions and tips and feedback on the above, go to this topic

Alternative (Python) script

An alternative Python script exists, which is very easy to configure. Only the API-key, System ID from PVoutput are needed and the IDX of your counter devices of Domoticz.
It needs your electricity consumption, which can be gathered from smartmeter P1 connected to Domoticz, or by YouLess that is connected to Domoticz.
For the production data it also needs a counter, this can be a YouLess (connected to Domoticz) that monitors a separate kWh-meter, or a self built device (S0-meter for example).

The script can be found here: http://tweaken.blogspot.nl/2015/01/domoticz-uploaden-naar-pvoutputorg.html
It is made by forum member 'nickyb2' but he migrated to Zipato Zipabox, so no support anymore available.

Alternative single-step lua-script for application with S0PulseCountingModule (=S0PCM)

If you have an S0PCM in your configuration of Domoticz, an alternative lua-script can be used, which in 1 script performs the whole chain of data-collection till upload to PVOutput.
The lua-script directly reads the svalues of Domoticz as generated by the S0PCM-interface, calculates actual and cumulative values, stores the resulting values in a set of global values, generates the required upload-string, and uploads this upload string to PVOutput with a interval as set in the script.
This lua-script can be found at http://www.domoticz.com/forum/viewtopic.php?f=23&t=5006#p43815
The script has to be tuned to your configuration by setting of parameters and names in the first 35 lines and by configuring the upload-line with PVO_URL (at the end of the script).

Upload energy data to PVoutput

Contents

Purpose

My current setup

Smartmeter

Hardware

Software

PV-system

Hardware

Software

Installation instructions - Calculating the consumption using a Lua device script

Prepare Domoticz

Creating the Lua device script

The Lua device script

Edit the Lua device script

Installation instructions - Uploading the data to PVoutput using a Lua time script

Install socket library

Creating the Lua time script

The Lua time script

Edit the Lua time script

When using the integrated P1 logger from Domoticz

Forum

Alternative (Python) script

Alternative single-step lua-script for application with S0PulseCountingModule (=S0PCM)

Navigation menu

Upload energy data to PVoutput

Purpose

My current setup

Smartmeter

Hardware

Software

PV-system

Hardware

Software

Installation instructions - Calculating the consumption using a Lua device script

Prepare Domoticz

Creating the Lua device script

The Lua device script

Edit the Lua device script

Installation instructions - Uploading the data to PVoutput using a Lua time script

Install socket library

Creating the Lua time script

The Lua time script

Edit the Lua time script

When using the integrated P1 logger from Domoticz

Forum

Alternative (Python) script

Alternative single-step lua-script for application with S0PulseCountingModule (=S0PCM)

Navigation menu

Search