Reverse engineering b516 energy statistics

[filippz]

Long story short - I've found a way to read gas/electricity consumption from my VUI 25/32 CS/1-5 (MF=Vaillant;ID=BAI00;SW=0807;HW=7603) and I need a little advice on what would be the best way to implement the support for it.

I was having an issue reading reading gas/electricity as I would get ERR: invalid position in decode when trying to read PrEnergyCountHc1/PrEnergyCountSum1 and all the rest of PrEnergyCount/PrEnergySum values. This seem to be common occurrence (for example: #437, john30/ebusd#620, john30/ebusd#1126,...) with one common "denominator": HW version of all those reports seem to be 7603

By looking at the communication between bai and ctlv2 I saw some unsupported commands/messages that pointed me to the right track with the following conclusions:

• ctlv2 sends bai 30 commands each one for a single Wh value for:
  heating/hot water (2) x
  gas/electricity/total (3) x
  previous month/current month/previous year/current year/total (5)
• commands are b516 with 100Xffff0Y0ZWVQQ as a payload where (with the kind help of @chrizzzp):
  X (period): 0=All, 1=Day, 2=Month, 3=Year
  Y (source): 1=Solar, 2=Environmental, 3=Electricity, 4=Gas, (9=unidentified heat pump-related)
  Z (usage): 0=All, 3=Heating, 4=Hot water, 5=Cooling
  W/V/QQ: - Month/Day/Year:
  • QQ denotes (half) year starting with 2000 increased by 2 for every year (increased by 1 for denoting last 5 months in a year) - ie 2000 = 00/01, 2001 = 02/03,...
    • ignored for All (X=0)
  • W denotes month increased by 2 for every month starting with 2 for first part of year (QQ is even) and 0 for last five months of the year (QQ is odd) - ie 2/4/6/8/a/c/e for months 1/2/3/4/5/6/7 (QQ is even) and 0/2/4/6/8 for months 8/9/10/11/12 (QQ is odd)
    • ignored for All (X=0) and Year (X=3)
    • for Month (X=1) increased by 1 in order to specify second part of the month (days 16-)
  • V denotes days starting with 1 for days 1-15 (1-f) when W is even, and 0 for days 16-31 (0-f) when W is odd
    • ignored for All (X=0), Year (X=3) and Month (X=2)
• replies are 11 bytes long where the last 4 bytes are Wh value in EXP format

payload examples:

1001ffff03005732 - daily sum of electrical energy consumption on Feb 23, 2025
1002ffff02038031 - monthly environmental yield used for heating in Dec of 2024
1003ffff04040032 - 2025 gas consumption used for water heating
1000ffff01000000 - total solar yield of the system (since installation)

example command:

ebusctl hex 08b516081001ffff03005732

To support this, I've added:

• an energye entry to _templates.csv:
  energye,EXP,,Wh,
• 30 entries to bai.308523.inc like:

r,,EnergySumHcPreviousYear,,,,b516,1003ffff03000130,,,IGN:7;energye,,,
r,,FuelSumHcPreviousYear,,,,b516,1003ffff04030030,,,IGN:7;energye,,,
...

Since I use those in Home Assistan, I've had to add |fuel to the filter-name value and sensor,gas,total_increasing = fuel|,Wh$ line to type_switch-number in mqtt-hassio.cfg

Before I prepare a proper PR, I would like to hear some opinions if those entries make sense or there is a more logical approach? Should HW:7603 be addressed separately and PrEnergyCount/PrEnergySum moved outside bai.308523.inc or it can be in the same file (and filtered out with [HW=7603] or similar)?

[chrizzzp]

@filippz
@burmistrzak

Wow, excellent reverse engineering (I guess)! I was investigating into decoding the b516 energy statistics, too, for some time, but got stuck (#472 (comment)). Your work finally solved it! Thanks a lot!

Good news: they also work for my heat pump (hmu, MF=Vaillant;ID=HMU00;SW=0902;HW=5103) using the same regulator (ctlv2, but will most probably work also for the related basv, basv3, ctlv3,... regulators).

Observation: for heat pumps these registers also deliver the enviromental energy statistics (energy gained from the environment) - see below!

* commands are `b516` with 100**X**ffff0**Y**0**ZW**13**Q** as a payload where:
  **X**: 0=All, 2=Month, 3=Year (1 probably means Day)

So far I had no success in getting any non-zero values with X=1 (days). See below.

  **Y**: 3=Electricity, 4=Gas

Y: 2=environmental yield

  **Z**: 0=All, 3=Heating, 4=Hot water
  **W**: 2=Previous, 4=Current (only valid for Month, ie X=2, otherwise it has a value of 2)
  **Q**: 2=Previous, 4=Current (only valid for Year, ie X=3, otherwise it has a value of 2)

I would expect there should also be a W/Q-like specifier valid for Day (X=1). I will monitor ebus traffic to investigate into daily energy statistics.

* 30 entries to bai.308523.inc like:

r,,EnergySumHcPreviousYear,,,,b516,1003ffff03032130,,,IGN:7;energye,,,
r,,FuelSumHcPreviousYear,,,,b516,1003ffff04032130,,,IGN:7;energye,,,
...

Yes, probably the way to go for the moment. I'm not sure if the new TypeScript-based definitions are more flexible in this context. Maybe @burmistrzak can comment?

Before I prepare a proper PR, I would like to hear some opinions if those entries make sense or there is a more logical approach? Should HW:7603 be addressed separately and PrEnergyCount/PrEnergySum moved outside bai.308523.inc or it can be in the same file (and filtered out with [HW=7603] or similar)?

I think it can be done using HW-specific 'conditions' in the csv.

[filippz]

@filippz @burmistrzak

Wow, excellent reverse engineering (I guess)! I was investigating into decoding the b516 energy statistics, too, for some time, but got stuck (#472 (comment)). Your work finally solved it! Thanks a lot!

Good news: they also work for my heat pump (hmu, MF=Vaillant;ID=HMU00;SW=0902;HW=5103) using the same regulator (ctlv2, but will most probably work also for the related basv, basv3, ctlv3,... regulators).

Took me a while figure it out - it kinda helped that my kid was learning about IEEE 754 format a couple of weeks ago which I wasn't thinking about for some years. Since ctlv2 refreshes those values only at midnight and startup I was pulling ctlv2 from the wall and back a few times, but I'm glad it's working out for you also :) Also, I'd tell that regulators are not important here - since we're not talking with them, but we'll have to figure out for which devices/versions should this be used.

So far I had no success in getting any non-zero values with X=1 (days). See below.

It was just a guess, since my ctlv2 doesn't display daily values but I have those numbers on the boiler itself and I haven't tested it further because these are not important for my user case.

Y: 2=environmental yield

I have 0 experience with HP so this is good for those users but we need to se if those can be used universally (ie even on devices other than heat pumps) or need to be added separately. Do you get an reply for 4=Gas which simply returns 0 or you get an error?

I would expect there should also be a W/Q-like specifier valid for Day (X=1). I will monitor ebus traffic to investigate into daily energy statistics.

Please do - from my perspective W/Q don't make sense since I would simply expect one boolean value which would indicate current/previous and be ignored for total. So those W/Q values might have some additional function/values which in my case can simply be used for current/previous. If you're up to it - you can try pulling your ctlv2 from the wall and back to force refresh and catch those messages.

Yes, probably the way to go for the moment. I'm not sure if the new TypeScript-based definitions are more flexible in this context. Maybe @burmistrzak can comment?

That's why we're chatting here :) I don't really know that much about all of this so experienced users might do it differently.

I think it can be done using HW-specific 'conditions' in the csv.

I was assuming that we're speaking about HW=7603, but since it obviously includes more devices and not only bai maybe some "includes" that would be pulled from a couple of places would be in order.

Thanks for the feedback!

[chrizzzp]

@filippz

Y: 2=environmental yield

I have 0 experience with HP so this is good for those users but we need to se if those can be used universally (ie even on devices other than heat pumps) or need to be added separately. Do you get an reply for 4=Gas which simply returns 0 or you get an error?

For 4=Gas my HP returns 0.
Indeed it would be interesting to hear from people with e.g. geothermal HP, whether they also can read the corresponding values using the Y=2 (environmental yield) or a different parameter (source yield).

I would expect there should also be a W/Q-like specifier valid for Day (X=1). I will monitor ebus traffic to investigate into daily energy statistics.

Please do - from my perspective W/Q don't make sense since I would simply expect one boolean value which would indicate current/previous and be ignored for total. So those W/Q values might have some additional function/values which in my case can simply be used for current/previous. If you're up to it - you can try pulling your ctlv2 from the wall and back to force refresh and catch those messages.

You don't actually need to restart your ctlv2 everytime to catch these messages (i.e. pulling the regulator from the wall), all EBUS messages (known and unknown) can actually be found in the ebusd logfile (loglevel 'notice') and will appear every night shortly after midnight, as you've noticed, too. So you just have to wait for them...

[filippz]

For 4=Gas my HP returns 0.

Cool - I'd say that this means that there is no need for splitting the definitions across devices as they all support all commands and simply return 0 for stuff they don't use for their operation

So you just have to wait for them

Being from Balkans waiting it out simple wasn't on the table :) Logic was to force those messages at will and compare them with whatever ctlv2 was displaying at that time and see which replies contained different payloads and what changed on the display. I even tried changing time to 23:59 and wait it out a full minute but ctlv2 would change back time to real one just before 00:00 (this might be some ebusd magic). At that time the Neanderthal part of my genome prevailed and off the wall it went...

[chrizzzp]

@filippz

I think there is more information available than only from current and previous month/year. At least on my HP it seems more complex than:

  **X**: 0=All, 2=Month, 3=Year (1 probably means Day)
  **Y**: 3=Electricity, 4=Gas
  **Z**: 0=All, 3=Heating, 4=Hot water
  **W**: 2=Previous, 4=Current (only valid for Month, ie X=2, otherwise it has a value of 2)
  **Q**: 2=Previous, 4=Current (only valid for Year, ie X=3, otherwise it has a value of 2)

To illustrate: when requesting the annual data (X=3, Y=2/3, Z=0/3/4,W=2) for my setup:

Q=0/1 result in same (previous years?) value
ebusctl hex 08b516081003ffff03002130 / 0b0301ff03002130fc0d0a4a
ebusctl hex 08b516081003ffff03002131 / 0b0301ff03002130fc0d0a4a
Q=2/3 result in (current years?) value
ebusctl hex 08b516081003ffff03002132 / 0b0300ff0300213231895d49
ebusctl hex 08b516081003ffff03002133 / 0b0300ff0300213231895d49
Q=4/5... result in 0 (see below!)

I have no idea why the same values are stored in consecutive registers (Q=0/1, 2/3).
The slight differences between them (for the current year values) reflect the current standby power consumption of the HP, as I am requesting with Z=0).

BTW: I get more data when iterating W and Q (X=3, Y=2/3, Z=0/3/4):

W=0/1 and Q=0/1: deliver same (previous years?) value as W=2/3 and Q=0/1

08b516081003ffff03000130 / 0b0301ff03002130fc0d0a4a
08b516081003ffff03000131 / 0b0301ff03002130fc0d0a4a
08b516081003ffff03001130 / 0b0301ff03002130fc0d0a4a
08b516081003ffff03001131 / 0b0301ff03002130fc0d0a4a

W=0/1 and Q=2/3: deliver (almost) same (this years?) values

08b516081003ffff03000132 / 0b0300ff03002132b3a55d49
08b516081003ffff03000133 / 0b0300ff03002132b3a55d49
08b516081003ffff03001132 / 0b0300ff03002132b5a55d49
08b516081003ffff03001133 / 0b0300ff03002132b6a55d49

W=2/3 and Q=0/1: deliver same (previous years?) value

08b516081003ffff03002130 / 0b0301ff03002130fc0d0a4a
08b516081003ffff03002131 / 0b0301ff03002130fc0d0a4a
08b516081003ffff03003130 / 0b0301ff03002130fc0d0a4a
08b516081003ffff03003131 / 0b0301ff03002130fc0d0a4a

W=2/3 and Q=2/3: deliver (almost) same (this years?) value

08b516081003ffff03002132 / 0b0300ff0300213229a75d49
08b516081003ffff03002133 / 0b0300ff030021322aa75d49
08b516081003ffff03003132 / 0b0300ff030021322ba75d49
08b516081003ffff03003133 / 0b0300ff030021322ba75d49

W=0/1 and Q=4/5... result in 0
W=2/3 and Q=4/5... result in 0
W=4/5 and Q=4/5... result in 0
etc.

Interestingly, when interating W from 0 to F (hex) with Q=2 I get the same (non-zero) values:

08b516081003ffff03000132 / 0b0300ff030021325aa75d49
08b516081003ffff03001132 / 0b0300ff030021325aa75d49
08b516081003ffff03002132 / 0b0300ff030021325aa75d49
08b516081003ffff03003132 / 0b0300ff030021325aa75d49
08b516081003ffff03004132 / 0b0300ff030021325aa75d49
08b516081003ffff03005132 / 0b0300ff030021325ba75d49
08b516081003ffff03006132 / 0b0300ff030021325ba75d49
08b516081003ffff03007132 / 0b0300ff030021325da75d49
08b516081003ffff03008132 / 0b0300ff030021325da75d49
08b516081003ffff03009132 / 0b0300ff030021325ea75d49
08b516081003ffff0300A132 / 0b0300ff030021325ea75d49
08b516081003ffff0300B132 / 0b0300ff0300213260a75d49
08b516081003ffff0300C132 / 0b0300ff0300213261a75d49
08b516081003ffff0300D132 / 0b0300ff0300213262a75d49
08b516081003ffff0300E132 / 0b0300ff0300213262a75d49
08b516081003ffff0300F132 / 0b0300ff0300213263a75d49

Similarily for Q=1:

08b516081003ffff03002131 / 0b0301ff03002130fc0d0a4a
08b516081003ffff03003131 / 0b0301ff03002130fc0d0a4a
08b516081003ffff03004131 / 0b0301ff03002130fc0d0a4a
...
08b516081003ffff0300F131 / 0b0301ff03002130fc0d0a4a

So:

1. I get valid annual values only for Q=0/1 and Q=2/3, reflecting the previous and current year.
2. I get the same annual values for W=0-F.

Now, for monthly data (X=2, Y=2/3, Z=0/3/4, Q=0) iterating W:

08b516081002ffff03000130 / 0b12ffff0300113000000000
08b516081002ffff03001130 / 0b12ffff0300113000000000
08b516081002ffff03002130 / 0b020dff03002130609fe348
08b516081002ffff03003130 / 0b020dff03002130609fe348
08b516081002ffff03004130 / 0b020cff0300413080f65f48
08b516081002ffff03005130 / 0b020cff0300413080f65f48
08b516081002ffff03006130 / 0b020bff0300613040098c48
08b516081002ffff03007130 / 0b020bff0300613040098c48
08b516081002ffff03008130 / 0b020aff03008130c08e3b48
08b516081002ffff03009130 / 0b020aff03008130c08e3b48
08b516081002ffff0300A130 / 0b0209ff0300a130006e9747
08b516081002ffff0300B130 / 0b0209ff0300a130006e9747
08b516081002ffff0300C130 / 0b0208ff0300c130009b7247
08b516081002ffff0300D130 / 0b0208ff0300c130009b7247
08b516081002ffff0300E130 / 0b0207ff0300e33000c05244
08b516081002ffff0300F130 / 0b0207ff0300e33000c05244

yield: 7 individual (non-zero) values.

...and monthly data (X=2, Y=2/3, Z=0/3/4, Q=1) iterating W:

08b516081002ffff03000131 / 0b0206ff030004310080b643
08b516081002ffff03001131 / 0b0206ff030004310080b643
08b516081002ffff03002131 / 0b0205ff0300273100277b47
08b516081002ffff03003131 / 0b0205ff0300273100277b47
08b516081002ffff03004131 / 0b0204ff03004131807bef47
08b516081002ffff03005131 / 0b0204ff03004131807bef47
08b516081002ffff03006131 / 0b0203ff0300613120799948
08b516081002ffff03007131 / 0b0203ff0300613120799948
08b516081002ffff03008131 / 0b0202ff03008131e081d948
08b516081002ffff03009131 / 0b0202ff03008131e081d948
08b516081002ffff0300A131 / 0b12ffff0300a13100000000
08b516081002ffff0300B131 / 0b12ffff0300b13100000000
08b516081002ffff0300C131 / 0b12ffff0300c13100000000
08b516081002ffff0300D131 / 0b12ffff0300d13100000000
08b516081002ffff0300E131 / 0b12ffff0300e13100000000
08b516081002ffff0300F131 / 0b12ffff0300f13100000000

yield: 5 individual (non-zero) values.
7+5=12
I think these registers contain last years monthly values, split over the two adress ranges (Q=0 and Q=1).

Q=2: (this year)

"previous month":
08b516081002ffff03002132 / 0b0201ff03002132800ccc48
08b516081002ffff03003132 / 0b0201ff03002132800ccc48
"this month":
08b516081002ffff03004132 / 0b0200ff030041327eadee48
08b516081002ffff03005132 / 0b0200ff0300413245b4ee48
W=6/7...F results in 0, also for Q=3

I think these (Q=2 and Q=3) are the monthly registers for the current year. Since we have now the second month of the year all registers W>5 are zero.

To summarize:
Q=0/1 previous year
Q=2/3 this year
.

W: 0-F (hex) monthly registers

I suspect for previous years monthly values:
Q=0
W=2/3 January
W=4/5 February
W=6/7 March
W=8/9 April
W=A/B May
W=C/D June
W=E/F July
Q=1
W=0/1 August
W=2/3 September
W=4/5 October
W=6/7 November
W=8/9 December

Similar for this years monthly values:
Q=2
W=2/3 January
W=4/5 February
etc.
Q=3
W=0/1 August
W=2/3 September
etc.

What do you think?

Could you try to confirm this on your bai unit to figure out if this is specific to the HP units?

Concerning Q: maybe more then two years are recorded, then Q could become >3 (like Q=4 with your bai unit. How many years do you have your system running?)

[filippz]

for monthly data (X=2, Y=2/3, Z=0/3/4, Q=0) iterating W

I think you're right - I tested the theory by adding all the 12 monthly values for energy last year and this matches the value returned by 1003ffff03002130. That would then make 2 x 12 entries naming the month explicitly...

Concerning Q: maybe more then two years are recorded, then Q could become >3 (like Q=4 with your bai unit. How many years do you have your system running?)

My system also reports the same for Q0/1 and Q2/3 - Since cltv2 sends Q0/2 I only considered those values.
Q4/5 returns 0 for me despite my unit being from 2023. I think system might just keep current/previous and total since 1000ffff03002132 (total electric energy) gives me more than 1003ffff03002130 + 1003ffff03002132 (previous + current year).

[chrizzzp]

Thanks for checking!

I think you're right - I tested the theory by adding all the 12 monthly values for energy last year and this matches the value returned by 1003ffff03002130. That would then make 2 x 12 entries naming the month explicitly...

Yes, and even more if you want the complete monthly data:
x3 (for Z: 0=All, 3=Heating, 4=Hot water) = 72
x2 (for Y: 3=Electricity, 4=Gas or 2=Environment) = 144

I'm not sure I want this in my CSV...

However, there is an -i (index) option in the 'ebusctl read' command (https://github.com/john30/ebusd/wiki/3.1.-TCP-client-commands#read) which could be used to reduce the number of required message definitions in the CSV. It's used for instance to read the 'errorhistory' or the timers in the regulators.

Concerning Q: maybe more then two years are recorded, then Q could become >3 (like Q=4 with your bai unit. How many years do you have your system running?)

My system also reports the same for Q0/1 and Q2/3 - Since cltv2 sends Q0/2 I only considered those values. Q4/5 returns 0 for me despite my unit being from 2023. I think system might just keep current/previous and total since 1000ffff03002132 (total electric energy) gives me more than 1003ffff03002130 + 1003ffff03002132 (previous + current year).

So how about the Q=4 readings you mentioned above? Do they work?

Q: 2=Previous, 4=Current (only valid for Year, ie X=3, otherwise it has a value of 2)

[filippz]

So how about the Q=4 readings you mentioned above? Do they work?

Sorry - my original post had a copy/paste error (fixed it now) - ctlv2 uses Q=0 for this year and Q=2 for previous one. Q=4 returns 0 for me.
For now I'm leaving W in original post as is - March starts tomorrow and we'll confirm if new queries from ctlv2 will be with W=6 as expected based on your findings.

I'm not sure I want this in my CSV...

I'm only using two total gas values (heating + hot water) for HomeAssistant as they're enough to monitor gas usage and I'm measuring electricity with Shelly Pro EM and not monitoring individual device usages. I guess that for the typical user this might be just enough.

[filippz]

We would also need to change the title of this issue as it turns out this is relevant to much more devices that I anticipated...

[filippz]

For now I'm leaving W in original post as is - March starts tomorrow and we'll confirm if new queries from ctlv2 will be with W=6 as expected based on your findings.

Confirmed by looking at the messages from ctlv2 this morning - I've updated first post accordingly.

[chrizzzp]

W+Q: - Month/Year:

* for Year (X=3):
  
  * W: has a fixed value of 2
  * Q: 0=Previous, 2=Current

As written in my post above, for annual values (X=3) W does not need to be a fixed value (W=2), any value of W will give the same result. I assume the same is true for Q and W with the total values (X=). Haven't checked this, though.

BTW: I just noticed the b516 registers can also be read from the hydraulic station (inside unit) of my HP (MF=Vaillant;ID=VWZ00;SW=0522;HW=5103) which houses the backup (immersion) heater. The energy statistics are then for the backup heater. So another device supporting this - nice!

And: Since HP can also cool, there should also be separate cooling energy statistics. I suspect there is another Z value for this (0=All, 3=Heating, 4=Hot water, ?=Cooling). Will look for this at a warmer time of the year (I normally don't use cooling).

[filippz]

As written in my post above, for annual values (X=3) W does not need to be a fixed value (W=2), any value of W will give the same result. I assume the same is true for Q and W with the total values (X=). Haven't checked this, though.

I'd prefer to use values that cltv2 is using as "safe" ones. As you noted, it might be totally irrelevant, but it might be relevant to certain device only while others ignore it, or these might be "reserved for future use" and may change over time. If we encounter any regulation device that sends different values we could investigate further if those devices do get back different values.

BTW: I just noticed the b516 registers can also be read from the hydraulic station (inside unit) of my HP

I was pondering if ebus has anything that might help to figure out if the device supports this kind of commands - ie by sending a single b516 and see if the device replies with a valid reply then load appropriate inc/csv files...

[chrizzzp]

I'd prefer to use values that cltv2 is using as "safe" ones.

OK, I see this point.

BTW: I just noticed the b516 registers can also be read from the hydraulic station (inside unit) of my HP

I was pondering if ebus has anything that might help to figure out if the device supports this kind of commands - ie by sending a single b516 and see if the device replies with a valid reply then load appropriate inc/csv files...

I don't think it is possible within ebusd right now. Only way I think is to probe product/HW/SW IDs (from scanning result) and then use conditions to include certain CSV files, like used here: https://github.com/eBUS/ebus.github.io/blob/main/next/en/vaillant/08.bai.csv

BTW: could you try the following b516 register on your bai unit which yields the current electrical power in W (on my HP system)? I'm just wondering how 'universal' the b516 registers are.

*r,,,,,,B516,,,,IGN:1,,,,,,,,,,,,,,,,,,,,,
r,,CurrentConsumedPowerWatt,,,,,14,,,EXP,,,current consumed electrical power (Watt),,,,,,,,,,,,,,,,,,

[filippz]

could you try the following b516 register on your bai

I get an error (maybe I'm doing something wrong?):

ebusctl read -f -V CurrentConsumedPowerWatt
ERR: invalid position in decode: bai CurrentConsumedPowerWatt

And the log is:

2025-03-02 11:30:33.806 [update error] unable to parse read bai CurrentConsumedPowerWatt from 3108b5160114 / 00: ERR: invalid position
2025-03-02 11:30:33.806 [bus notice] >3108b516011451<000000>00
2025-03-02 11:30:33.806 [main error] read bai CurrentConsumedPowerWatt: decode ERR: invalid position