PRO 3EM: Balancing and display of energy data in a virtual component on the cloud dashboard

tvbshelly · August 19

With a little scripting, you can use a PRO 3EM to determine correctly balanced energy data and even display this energy data in the cloud dashboard in a second step using a virtual component.

The required script is available on GitHub:

https://github.com/sicanins/shelly-pro3EM-energycounter

First you create a script with the local WebUI of the Pro 3EM:

and copy the script there - the easiest way is to copy it directly from here: https://github.com/sicanins/shelly-pro3EM-energycounter/blob/main/energy_counter.js

As we only want to output the energy data in a virtual component, we only need the values “locally” (i.e. we do not need to process them further via MQTT in a home automation system).
The script should therefore be adapted as follows:

// Line 14
// set this to false if you DONT want to update the name
// (the updated name is necessary to read the data with the iobroker shelly plugin)
let updateName = true;

updateName = true causes the values for consumption and export to be displayed in the local WebUI instead of the previous device name.

If you wish, you can also switch off MQTT:

// Line 10
// set this to false to stop publishing on MQTT
let MQTTpublish = false;

Now save and start the script.

This running script overwrites the local device name with the current energy values. However, this is only visible in the local WebUI of the Pro 3EM.

The next step is to create a virtual component (firmware from 1.4.0) and use it to display the energy values in the cloud dashboard.

tvbshelly · August 19

Next, we create a user-defined component via the local WebUI: "create new" of type "text":

Name it "EMData", View is "Label" and "Keep current value after reboot":

Now we create a group (+). This is not absolutely necessary, but makes it easier to display the energy data in the cloud dashboard:

Edit the group, name ist "EMDataGroup" and select the already created component "EMData":

Save the group.

Now go again to the script section and edit the script.

Add the following snippet at line 124:

let virt200 = Virtual.getHandle("text:200");
virt200.setValue("Imp " + energyConsumedKWh.toFixed(2)+" kWh / Exp "+(energyReturnedKWh+energyReturnedWs / 3600000).toFixed(2)+" kWh");

Result:

After saving go to the cloud dashboad and click on the device / PRO 3EM.

The energy data is displayed in the Virtual Components tab:

By default, the created groups are displayed in this view, so I have also created a group for convenience.

You can also switch to the component view (but unfortunately you have to repeat this every time you call up the device and the Virtual Component tab):

What I don't know exactly: How often the values in the cloud are updated may depend on whether you use Cloud Premium or just the free access.

Edited August 19 by tvbshelly

Sebastian U. · August 20

You could also try this script which will create two Virtual Components on his own.

You only have to start the script. Inspired by thew same initial Script from sicanins. Without MQTT support. But his could be added.

let saveSeconds = 60;

const META = { ui: { view: "label", "unit": "kWh", "step": 0.00001, "icon": null } };

const CONFIG = {
	VCEnergy: { name: "Current Energy kWh", persisted: true, meta: META },
	VCReturnedEnergy: { name: "Current Returned Energy kWh", persisted: true, meta: META },
	VCGroup: { name: "Net Metering" }
};

let DYN = {
	VCEnergy: null,
	VCReturnedEnergy: null,
	VCGroup: null,
  currentEnergyWs: 0,
  currentReturnedEnergyWs: 0,
  currentEnergykWh: 0,
  currentReturnedEnergykWh: 0
};

let initial = true;
let loaded = false;
let progress = false;
let untilLastSave = 0;

function getValues(){	
	DYN.currentEnergykWh = Shelly.getComponentStatus('number:'+DYN.VCEnergy).value;
	print(CONFIG.VCEnergy.name + " loaded value: " + DYN.currentEnergykWh + " " + META.ui.unit);
	
	DYN.currentReturnedEnergykWh = Shelly.getComponentStatus('number:'+DYN.VCReturnedEnergy).value;
	print(CONFIG.VCReturnedEnergy.name + " loaded value: " + DYN.currentReturnedEnergykWh + " " + META.ui.unit);
	loaded = true;
}

function metering(){
	let em = Shelly.getComponentStatus("em", 0);
	if (typeof em.total_act_power !== 'undefined'){
		let power = em.total_act_power;
		if (power >= 0){
			DYN.currentEnergyWs = DYN.currentEnergyWs + power * 0.5;
		} else {
			DYN.currentReturnedEnergyWs = DYN.currentReturnedEnergyWs - power * 0.5;
		}
		

		let fullWh = Math.floor((DYN.currentEnergyWs / 3600));
		if (fullWh > 0){
			DYN.currentEnergykWh += fullWh / 1000;
			DYN.currentEnergyWs -= fullWh * 3600;
  		if (saveSeconds == 0) Shelly.call("number.set", { id: DYN.VCEnergy, value: DYN.currentEnergykWh });
			print("Changed consumed KWh: " + DYN.currentEnergykWh);
		}

		fullWh = Math.floor((DYN.currentReturnedEnergyWs / 3600));
		if (fullWh > 0){
			DYN.currentReturnedEnergykWh += fullWh / 1000;
			DYN.currentReturnedEnergyWs -= fullWh * 3600;
  		if (saveSeconds == 0) Shelly.call("number.set", { id: DYN.VCReturnedEnergy, value: DYN.currentReturnedEnergykWh });
			print("Changed returned KWh: " + DYN.currentReturnedEnergykWh);
		}


		untilLastSave = untilLastSave + 1;
		
		if (saveSeconds > 0 && untilLastSave > saveSeconds){
			untilLastSave = 0;     
  		Shelly.call("number.set", { id: DYN.VCEnergy, value: DYN.currentEnergykWh });
  		Shelly.call("number.set", { id: DYN.VCReturnedEnergy, value: DYN.currentReturnedEnergykWh });
			print("Saved consumed KWh: " + DYN.currentEnergykWh);
			print("Saved returned KWh: " + DYN.currentReturnedEnergykWh);
		}
		
	};
}

function virtualComponent(type, virtualKey){
	progress = true;
	Shelly.call("Shelly.GetComponents", { "dynamic_only": true }, function callback(result, error_code, error_message, userdata){
		if (error_code === 0){
			let _response = false;
			if (result.components > 0){
				for (let i in result.components){
					let _key = result.components[i].key;
					if (_key.slice(0, 7) === 'number:' && type == 'number'){
	    			if (result.components[i].config.name === CONFIG[virtualKey].name){
	    				_response = true;
							DYN[virtualKey] = result.components[i].config.id;
							progress = false;
							print (type+":"+DYN[virtualKey]+" loaded");
						}
					} else if (_key.slice(0, 6) === 'group:' && type == 'group') {
	    			if (result.components[i].config.name === CONFIG[virtualKey].name){
	    				_response = true;
							DYN[virtualKey] = result.components[i].config.id;
							progress = false;
							print (type+":"+DYN[virtualKey]+" loaded");
						}
					}
					
				}
			}
			if (_response === false){
				Shelly.call("Virtual.Add", { type: type, config: CONFIG[virtualKey] }, function callback(result, error_code, error_message, userdata){
					if (error_code === 0){
						if (type === "group"){
							let _value = [
							"number:"+DYN.VCEnergy,
							"number:"+DYN.VCReturnedEnergy
							];
							Shelly.call("Group.Set", {id: result.id, value: _value } );
						}
						DYN[virtualKey] = result.id;
						progress = false;
						print (type+":"+DYN[virtualKey]+" created");
					}
				});
			}
		}
	});
}

function init(){
	if (DYN.VCEnergy === null){
		print ("Start Setup.");
		virtualComponent('number', 'VCEnergy');
	} else if (DYN.VCReturnedEnergy === null){
		virtualComponent('number', 'VCReturnedEnergy');
	} else if (DYN.VCGroup == null){
		virtualComponent('group', 'VCGroup');
	} else {
		initial = false
		print ("Setup Finished.");
		print ("Start Loading...");
	}
}


function timerHandler(){
	if (initial === false && loaded === false){
		getValues();
	} else if (initial === false && loaded === true){
		metering();
	} else {
		if (progress === false){
			init();
		}
	}
}

function httpServerHandler(request, response){
	response.code = 200;
	response.headers = [["Content-Type", "application/json"]];
	
	const energyData = {
		totalEnergy: DYN.currentEnergykWh.toFixed(3) + " KWh",
		totalRetEnergy: DYN.currentReturnedEnergykWh.toFixed(3) + " KWh",
	};

	response.body = JSON.stringify(energyData);
	response.send();
	return;
}

Timer.set(500, true, timerHandler, null);
HTTPServer.registerEndpoint("energy_counter", httpServerHandler);

tvbshelly · August 20

6 hours ago, Sebastian U. said:

You could also try this script which will create two Virtual Components on his own.

Really great code to manage the virtual component and group automatically in the script - that makes it a lot easier. I haven't made the effort to do this yet.

tvbshelly · August 20

To preset or change the metering values in @Sebastian U. script put the following code snippet into the script console of the running script:

Set Current Returned Energy:

Shelly.call("number.set", { id: DYN.VCReturnedEnergy, value: 40.5 });

Then stop the script and restart it immediately.

Set Current Energy:

Shelly.call("number.set", { id: DYN.VCEnergy, value: 10.0 });

Then stop the script and restart it immediately.

Note: The values of the virtual component also survive a reboot, as configured.

Edited August 20 by tvbshelly

Sebastian U. · August 20

I have done some update to set defaultValue.

If the VirtualComponents are not present it will set the values for defaultEnergy and defaultReturnedEnergy (line 2 and 3) as default. If there is already an value set it will only overwrite value if it's greater or same.

It's only an quick edit, the previous script was running some weeks.

let saveSeconds = 60;
let defaultEnergy = 0;
let defaultReturnedEnergy = 0;

const META = { ui: { view: "label", "unit": "kWh", "step": 0.00001, "icon": null } };

const CONFIG = {
  VCEnergy: { name: "Current Energy kWh", persisted: true, default_value: defaultEnergy, meta: META },
  VCReturnedEnergy: { name: "Current Returned Energy kWh", persisted: true, default_value: defaultReturnedEnergy, meta: META },
  VCGroup: { name: "Net Metering" }
};

let DYN = {
  VCEnergy: null,
  VCReturnedEnergy: null,
  VCGroup: null,
  currentEnergyWs: 0,
  currentReturnedEnergyWs: 0,
  currentEnergykWh: 0,
  currentReturnedEnergykWh: 0
};

let initial = true;
let loaded = false;
let progress = false;
let untilLastSave = 0;

function getValues(){	
  DYN.currentEnergykWh = Shelly.getComponentStatus('number:'+DYN.VCEnergy).value;
  print(CONFIG.VCEnergy.name + " loaded value: " + DYN.currentEnergykWh + " " + META.ui.unit);
  if (defaultEnergy >= DYN.currentEnergykWh){
    DYN.currentEnergykWh = defaultEnergy;
    Shelly.call("number.set", { id: DYN.VCEnergy, value: DYN.currentEnergykWh });
    print(CONFIG.VCEnergy.name + " overwritten value: " + DYN.currentEnergykWh + " " + META.ui.unit);
  }

  DYN.currentReturnedEnergykWh = Shelly.getComponentStatus('number:'+DYN.VCReturnedEnergy).value;
  print(CONFIG.VCReturnedEnergy.name + " loaded value: " + DYN.currentReturnedEnergykWh + " " + META.ui.unit);
  if (defaultReturnedEnergy >= DYN.currentReturnedEnergykWh){
    DYN.currentReturnedEnergykWh = defaultReturnedEnergy;
    Shelly.call("number.set", { id: DYN.VCReturnedEnergy, value: DYN.currentReturnedEnergykWh });
    print(CONFIG.VCReturnedEnergy.name + " overwritten value: " + DYN.currentReturnedEnergykWh + " " + META.ui.unit);
  }
  loaded = true;
}

function metering(){
  let em = Shelly.getComponentStatus("em", 0);
  if (typeof em.total_act_power !== 'undefined'){
    let power = em.total_act_power;
    if (power >= 0){
      DYN.currentEnergyWs = DYN.currentEnergyWs + power * 0.5;
    } else {
      DYN.currentReturnedEnergyWs = DYN.currentReturnedEnergyWs - power * 0.5;
    }


    let fullWh = Math.floor((DYN.currentEnergyWs / 3600));
    if (fullWh > 0){
      DYN.currentEnergykWh += fullWh / 1000;
      DYN.currentEnergyWs -= fullWh * 3600;
      if (saveSeconds == 0) Shelly.call("number.set", { id: DYN.VCEnergy, value: DYN.currentEnergykWh });
      print("Changed consumed KWh: " + DYN.currentEnergykWh);
    }

    fullWh = Math.floor((DYN.currentReturnedEnergyWs / 3600));
    if (fullWh > 0){
      DYN.currentReturnedEnergykWh += fullWh / 1000;
      DYN.currentReturnedEnergyWs -= fullWh * 3600;
      if (saveSeconds == 0) Shelly.call("number.set", { id: DYN.VCReturnedEnergy, value: DYN.currentReturnedEnergykWh });
      print("Changed returned KWh: " + DYN.currentReturnedEnergykWh);
    }


    untilLastSave = untilLastSave + 1;

    if (saveSeconds > 0 && untilLastSave > saveSeconds){
      untilLastSave = 0;     
      Shelly.call("number.set", { id: DYN.VCEnergy, value: DYN.currentEnergykWh });
      Shelly.call("number.set", { id: DYN.VCReturnedEnergy, value: DYN.currentReturnedEnergykWh });
      print("Saved consumed KWh: " + DYN.currentEnergykWh);
      print("Saved returned KWh: " + DYN.currentReturnedEnergykWh);
    }

  };
}

function virtualComponent(type, virtualKey){
  progress = true;
  Shelly.call("Shelly.GetComponents", { "dynamic_only": true }, function callback(result, error_code, error_message, userdata){
    if (error_code === 0){
      let _response = false;
      if (result.components > 0){
        for (let i in result.components){
          let _key = result.components[i].key;
          if (_key.slice(0, 7) === 'number:' && type == 'number'){
            if (result.components[i].config.name === CONFIG[virtualKey].name){
              _response = true;
              DYN[virtualKey] = result.components[i].config.id;
              progress = false;
              print (type+":"+DYN[virtualKey]+" loaded");
            }
          } else if (_key.slice(0, 6) === 'group:' && type == 'group') {
            if (result.components[i].config.name === CONFIG[virtualKey].name){
              _response = true;
              DYN[virtualKey] = result.components[i].config.id;
              progress = false;
              print (type+":"+DYN[virtualKey]+" loaded");
            }
          }

        }
      }
      if (_response === false){
        print (JSON.stringify(CONFIG[virtualKey]));
        Shelly.call("Virtual.Add", { type: type, config: CONFIG[virtualKey] }, function callback(result, error_code, error_message, userdata){
          if (error_code === 0){
            if (type === "group"){
              let _value = [
                "number:"+DYN.VCEnergy,
                "number:"+DYN.VCReturnedEnergy
              ];
              Shelly.call("Group.Set", {id: result.id, value: _value } );
            }
            DYN[virtualKey] = result.id;
            progress = false;
            print (type+":"+DYN[virtualKey]+" created");
          }
        });
      }
    }
  });
}

function init(){
  if (DYN.VCEnergy === null){
    print ("Start Setup.");
    virtualComponent('number', 'VCEnergy');
  } else if (DYN.VCReturnedEnergy === null){
    virtualComponent('number', 'VCReturnedEnergy');
  } else if (DYN.VCGroup == null){
    virtualComponent('group', 'VCGroup');
  } else {
    initial = false
    print ("Setup Finished.");
    print ("Start Loading...");
  }
}


function timerHandler(){
  if (initial === false && loaded === false){
    getValues();
  } else if (initial === false && loaded === true){
    metering();
  } else {
    if (progress === false){
      init();
    }
  }
}

function httpServerHandler(request, response){
  response.code = 200;
  response.headers = [["Content-Type", "application/json"]];

  const energyData = {
    totalEnergy: DYN.currentEnergykWh.toFixed(3) + " KWh",
    totalRetEnergy: DYN.currentReturnedEnergykWh.toFixed(3) + " KWh",
  };

  response.body = JSON.stringify(energyData);
  response.send();
  return;
}

Timer.set(500, true, timerHandler, null);
HTTPServer.registerEndpoint("energy_counter", httpServerHandler);

tvbshelly · August 21

@Sebastian U. I have a question about the “stress” on the flash memory:
sicanins's script is optimized to not write to the KVS too often - I think to not write to the flash memory too often, since flash memory usually only has a limited number of write cycles.
For the virtual components that both you and I use, the value is set to be retained on reboot. I therefore assume that this is also implemented internally via flash memory.
The log shows very clearly that the value in the component is written very frequently. This is what people want so that they can see the most up-to-date value in the Cloud Dashboard:

What is your opinion on flash memory here? Is my reasoning correct?

Edited August 21 by tvbshelly

Sebastian U. · August 21

in the first line you can change the value for

let saveSeconds = 60;

But it's not really seconds because the script got's executed every 500ms, you have to double the value to really get seconds.

Problem is, if you restart the Shelly Pro 3EM you will loose all data collected since the last save.

I think here should everyone decide on his own.

Nosugar · August 21

@Sebastian U.

The „current Energy kWh“ has 12 digits in my Shelly App view under Virtual Components.

It is shown like 10726.2740000008. The last digit was update 8 times in around 10 hours. The first 3 digits are updated every 60 seconds. „Current returned“ has the correct 3 digits.

Für den Fall, dass du deutsch sprichst können wir auch gerne die Sprache wechseln.

Hummelchen · September 2

On 21.8.2024 at 11:02, Sebastian U. said:

Das Problem ist, dass bei einem Neustart des Shelly Pro 3EM alle Daten, die seit dem letzten Speichern gesammelt wurden, verloren gehen.

In der Fachliteratur wird von einer Auslegung von 100.000 Schreibzyklen bei einem ESP32 Chip ausgegangen.

Danach ist der Flash-Speicher definitiv unbrauchbar.

Ich setzte den Saldierungs-Code von "Sicanins" schon rund ein Jahr ein.

Das Problem der per "default" nur alle 30 Minuten Speicherung habe ich für mich Perfekt umgecodet.

Default Speicherzyklus von 30 Min auf 60 Min. nach der letzter Speicherung erhöht.
Zusätzlich speichere ich wenn zur letzten Speicherung eine gewisse [*] Energie verbraucht wurde.

[*] Diese Delta Energie wird definiert mit dem zu erwartenden täglichen kWh / 36 Speicherzyklen. (kWh/Tag div. 36)

Vorteile:

Bei höherem Verbrauch werden kurze und bei wenig Verbrauch längere Speicherzyklen aktiviert.
- Datenverlust minimiert da bei Reboot / Netzausfall maximl1/36 igstel kWh/Tag Abweichung möglich ist.
Wie im Ursprungs-Code sind 48 Speicher-Zyklen der saldierten Zählwerte pro Tag.
- Bei 100.000 Zyklen ist die zu erwartende Lebensdauer des Flash's dann rund 5 Jahre.

Bei täglichen Gesamt-Verbrauch > 50kWh, liegt die Messgenauigkeit der Saldierung(en) zum EVU Verrechnungszähler seit einem Jahr im Bereich ±0.85%.

Wolfgang1 · Tuesday at 02:15 PM

Sind denn die über das Skript generierten richtig saldierten Werte auch über die Shelly-Integration in Home Assistant nutzbar?

Sign In

PRO 3EM: Balancing and display of energy data in a virtual component on the cloud dashboard

Recommended Posts

tvbshelly

Link to comment

Share on other sites

tvbshelly

Link to comment

Share on other sites

Sebastian U.

Link to comment

Share on other sites

tvbshelly

Link to comment

Share on other sites

tvbshelly

Link to comment

Share on other sites

Sebastian U.

Link to comment

Share on other sites

tvbshelly

Link to comment

Share on other sites

Sebastian U.

Link to comment

Share on other sites

Nosugar

Link to comment

Share on other sites

Hummelchen

Link to comment

Share on other sites

Wolfgang1

Link to comment

Share on other sites

Join the conversation

Browse