Honda Insight Forum banner
641 - 660 of 770 Posts

·
Linsight Designer
Joined
·
2,626 Posts
Discussion Starter · #642 · (Edited)
I've spent the past several days optimizing LiBCM's firmware architecture to increase the main loop execution rate.

Last week LiBCM ran thru the main loop ~8 times per second (~8 Hz).
Tonight LiBCM runs around 714 Hz (on average)*, which is an 89x improvement...
...proof that the initial code was a complete disaster ;).

*When too many things happen at once, the worst-case single loop run time is now 5.4 ms, which works out to a worst case loop rate around 185 Hz... still a 23x improvement worst case.

There's still room for more improvement, but it's not necessary at the moment. LiBCM now has plenty of runway to start throwing other features in. The first of which is VPIN spoofing, which is what I was working on last week...
...until I realized the (old) firmware was too slow. Not anymore, so tomorrow I'll hopefully finish VPIN spoofing and then pull this branch back into main.

Our fearless beta testers are patiently waiting for the firmware architecture to stop changing so dramatically... and also for it to work reliably... that's still an issue, and will be my next focus.
 

·
Linsight Designer
Joined
·
2,626 Posts
Discussion Starter · #643 · (Edited)
Didn't work too much on VPIN spoofing today...
...LiBCM was running so fast that it was making decisions prior to reading back all cell voltages from the LTC ADCs. Specifically, the loop now runs 7x faster than LiBCM can read all QTY48 cell voltages. Note that each loop now reads QTY3 cell voltages, whereas the old, slower version read all QTY48 cells each loop.

The primary issue this 'causes'* occurs when the key first turns on:
The "actual pack voltage" the MCM expects to see on connector E - which LiBCM man-in-the-middles - isn't constant.
As LiBCM reads each QTY3 cell voltages back, the pack voltage LiBCM 'sees' increases by ~10.5 volts. Since LiBCM must know the correct pack voltage in order to correctly spoof the voltage seen on MCM'E', this causes the MCM to panic.

*Note: Speeding up the loop didn't 'cause' this issue, but rather it just made it present itself. The underlying issues always existed... LiBCM was previously just too slow for it to matter.

...

To fix this, I've added a flip-flop frame buffer:
-One frame always contains the latest complete set of cell voltage measurements retrieved from the LTC ADCs.
-The other frame is a "work in progress", while each cell voltage is read back and stored from the LTC ADCs.
-As soon as all cells are read and stored in the "WIP" frame, it becomes the "complete" frame.
-Simultaneously, the previously "complete" frame becomes the new "WIP" frame, and the cycle continues forever.

Primary Result:
When the key first turns on, LiBCM immediately starts reading each cell voltage, storage each result in the "WIP" frame. During this time, each cell in the "complete" frame remains "0.0000" volts. But the MCM doesn't care about that... it's really only the linear rise that upsets the MCM. Then when all QTY48 cell voltages in the the first "WIP" frame are stored, it becomes the "complete" frame and so the pack voltage immediately jumps from "0 volts" to "170 volts" (for example). The MCM likes this.

Other positive results:
Since we only ever compare voltages on the "complete" frame, we know for certain that all ADC cell voltage conversions happened at approximately the same time. Whereas the initial implementation could have allowed voltage comparisons using ADC data taken at vastly different times (i.e. when the oldest cell voltage ADC conversion was compared to the newest). That could be up to QTY16 runs through the superloop (48 cells / 3 cells per loop = 16 loops to retrieve all cell voltages).

...

This issue could certainly have been resolved by initializing with dummy voltage data, and adding a delay to the code... but that's sloppy and would likely cause problems later. The flip-flop frame buffer is the technically correct solution, and it's completely transparent to anything outside the file it lives in (i.e. LTC68042.c).
 

·
Administrator
Joined
·
13,067 Posts
In the 'BCM Replacer' I used a dummy average pack voltage for the very first run after flashing.

After that I saved the real pack voltage into Eeprom at switch off and used that on the next switch on until the live voltages had updated.

The 'BCM Replacer' PIC was not normally permanently powered like the LiBCM, although it did have back up power on board.
 

·
Linsight Designer
Joined
·
2,626 Posts
Discussion Starter · #645 · (Edited)
Holy smokes... I somehow convinced myself to rewrite LTC's LTC6804 library from scratch. It was just really bad. I'm more than halfway done.

Update: Finished. Works.

Details:

old code (LTC demo code)new code
number of c files14
total lines1090519
(239, 154, 95, 31)
wild use of magic numbers?yesminimal... to be designed out
enigmatic variable names?yes:
cfg, adcv, ADCV, rdcv, RDCV, wakeup_sleep,
len, data, r_config, cmd, cmd_len, rdcv_reg,
parsed_cell, adax, md_bits, dcp, ch, chg,
attemptCounter,
returnedData,
cellVoltageRegister,
chipAddress,
cellAddress,
presentState,
cellUnderTest
enigmatic function names?init_cfg (no initiation actually occurs)
set_adc (doesn't actually set adc)
adcv (same name as variable)
adax (on surface looks like it gets called... never does)
rdcv (same name as variable)
rdcv_reg...
startCellConversion,
validateAndStoreNextCVR,
serialReadCVR,
processAllCellVoltages

I didn't want to spend all day rewriting the working code... but it was just too confusing to fiddle with. Think "one line functions nested in one line functions nested in one line functions nested in functions nested in functions... all in the same file".

Splitting the single file into four individual silos makes it easier to track which parts actually interact with each other:
LTC68042cell.c (deals with everything related to reading and processing cell voltages)
LTC68042gpio.c (deals with everything related to gpio (temperature sensors, discharge FETs, LEDs, etc)
LTC68042configure.c (deals with configuring LTC parameters, low level functions, etc)
LTC68042result.c (stores latest results. This allowed me to remove the entire frame buffer that I designed yesterday)

So yesterday I designed a (working) frame buffer to workaround the sketchy old code.
And then today I designed it right back out ;).

I'm glad to finally have a place to store all the latest buffered LTC results. For example, you can now call "LTC68042result_stackVoltage_get()" wherever you want, and are guaranteed to get valid data from a complete frame.

Also super excited to have now removed all non-open-source code... LTC retains a copyright on their reference code, but does allow it to be reincorporated into any code without attribution, as long as it's substantially rewritten.
 

·
Registered
Joined
·
1,194 Posts
Holy smokes... I somehow convinced myself to rewrite LTC's LTC6804 library from scratch.
OMG that is awesome and should be its own project. I use that library for my NiMH conditioner/tester and only use it for reading voltages because I never had the bandwidth to figure out the balancing code (and I had an alternative). If you can break that out as a separate project, you are going to be a hero to many people.
 
  • Like
Reactions: mudder and Natalya

·
Linsight Designer
Joined
·
2,626 Posts
Discussion Starter · #648 ·
OMG that is awesome and should be its own project.
Maybe later. Not my top priority right now, particularly since the full feature set isn't implemented yet.

I use that library for my NiMH conditioner/tester and only use it for reading voltages because I never had the bandwidth to figure out the balancing code (and I had an alternative). If you can break that out as a separate project, you are going to be a hero to many people.
I haven't implemented cell balancing into a snazzy "dischargeCell(cellNumber)" function yet, but I just happened to have outlined exactly how to do that here (search "control cell discharge FET"). You basically set whichever DCC[n] bits you want to enable cell discharge on, and then you send the six CFGR bytes to the IC.

I figure we beta testers have a while before our packs need balancing... so I've deferred implementing that for now.
 

·
Linsight Designer
Joined
·
2,626 Posts
Discussion Starter · #649 · (Edited)
I'm hanging on for the variable VPIN hardware mod to be finalised before I reinstall my pack in the car..
My goal is to finalize that soon... I keep running into roadblocks that prevent it from working, and then end up spending hours rewriting entire sections of code. This should settle down now that the architecture is siloing itself into neat little piles.

Besides the aforementioned R48 mod (i.e. remove resistors on R48), there's a chance no other hardware mods are required... although I will certainly be making changes to the next (and hopefully last) PCB revision. If there is another mod required on the beta hardware, it'll likely be much simpler than I had initially thought: probably just adding another capacitor in parallel with the one already present on C13.

More details on this issue:
The PWM timer tied to PIN_VPIN_OUT_PWM is also used by arduino core as a time reference... so changing the PWM frequency proportionally changes all functions that have anything to do with timing (e.g. delay(), transmitting serial data, etc). In my testing over the weekend, everything still appears to work... just faster, because I'm increasing the aforementioned timer frequency from ~1kHz to ~3.9 kHz... so everything runs 3.9x faster. I need to run faster because my LPF time constant for the PWM pulse train is Tau=1uF*10kOhm = 10 ms... with a 1kHz PWM train, there's quite a bit of ripple on VPIN_OUT (which feeds into the MCM's VPIN input).

So obviously on the next revision I'll swap to a different PWM pin, and then run at 62 kHz... which will resolve the issue entirely. But for now we'll either need to change the firmware (annoying, as described above), swap the pin manually (hard), or just stuff more capacitance onto C13 (see above).

...

My end game for the architecture is to make the Superloop have only the following functions:
Code:
lcd_refresh();
LTC68042cell_nextVoltages;
key_update();
gridCharger_update();
adc_update();
battsci_update();
metsci_update();
vPackSpoof_update();
Ideally that'll be it... but that's still a long way out. I've already partially implemented the first two functions:
-All LCD updates now happen entirely within lcd_refresh(), which intelligently handles all screen updates. Previously, there were a dozen separate global functions for each screen element littered throughout the entire codebase.
-Everything related to LTC6804 cell voltage is now automatically handled by LTC68042cell_nextVoltages(). Previously there were numerous (poorly described) external functions that accomplished the same task.
 

·
Premium Member
Southern California
Joined
·
895 Posts
So obviously on the next revision I'll swap to a different PWM pin, and then run at 62 kHz... which will resolve the issue entirely. But for now we'll either need to change the firmware (annoying, as described above), swap the pin manually (hard), or just stuff more capacitance onto C13 (see above).
Should be really easy to swap the pin if you just add a jumper on the back of the Arduino from the current pin to the new one. Just leave the current pin as an input (high-impedance) and enable the new one.
I've run into the timer problem before, it's why I don't like to use Arduino on big projects. I tried to use all the timers and was unable since one of them is tied so deeply into the Arduino code.
 

·
Registered
Joined
·
1,268 Posts
Didn't work too much on VPIN spoofing today...
...LiBCM was running so fast that it was making decisions prior to reading back all cell voltages from the LTC ADCs. Specifically, the loop now runs 7x faster than LiBCM can read all QTY48 cell voltages. Note that each loop now reads QTY3 cell voltages, whereas the old, slower version read all QTY48 cells each loop.

The primary issue this 'causes'* occurs when the key first turns on:
The "actual pack voltage" the MCM expects to see on connector E - which LiBCM man-in-the-middles - isn't constant.
As LiBCM reads each QTY3 cell voltages back, the pack voltage LiBCM 'sees' increases by ~10.5 volts. Since LiBCM must know the correct pack voltage in order to correctly spoof the voltage seen on MCM'E', this causes the MCM to panic.

*Note: Speeding up the loop didn't 'cause' this issue, but rather it just made it present itself. The underlying issues always existed... LiBCM was previously just too slow for it to matter.

...

To fix this, I've added a flip-flop frame buffer:
-One frame always contains the latest complete set of cell voltage measurements retrieved from the LTC ADCs.
-The other frame is a "work in progress", while each cell voltage is read back and stored from the LTC ADCs.
-As soon as all cells are read and stored in the "WIP" frame, it becomes the "complete" frame.
-Simultaneously, the previously "complete" frame becomes the new "WIP" frame, and the cycle continues forever.

Primary Result:
When the key first turns on, LiBCM immediately starts reading each cell voltage, storage each result in the "WIP" frame. During this time, each cell in the "complete" frame remains "0.0000" volts. But the MCM doesn't care about that... it's really only the linear rise that upsets the MCM. Then when all QTY48 cell voltages in the the first "WIP" frame are stored, it becomes the "complete" frame and so the pack voltage immediately jumps from "0 volts" to "170 volts" (for example). The MCM likes this.

Other positive results:
Since we only ever compare voltages on the "complete" frame, we know for certain that all ADC cell voltage conversions happened at approximately the same time. Whereas the initial implementation could have allowed voltage comparisons using ADC data taken at vastly different times (i.e. when the oldest cell voltage ADC conversion was compared to the newest). That could be up to QTY16 runs through the superloop (48 cells / 3 cells per loop = 16 loops to retrieve all cell voltages).

...

This issue could certainly have been resolved by initializing with dummy voltage data, and adding a delay to the code... but that's sloppy and would likely cause problems later. The flip-flop frame buffer is the technically correct solution, and it's completely transparent to anything outside the file it lives in (i.e. LTC68042.c).
technically correct, the best kind of correct
 

·
Linsight Designer
Joined
·
2,626 Posts
Discussion Starter · #652 ·
Should be really easy to swap the pin if you just add a jumper on the back of the Arduino from the current pin to the new one. Just leave the current pin as an input (high-impedance) and enable the new one.
I've run into the timer problem before, it's why I don't like to use Arduino on big projects. I tried to use all the timers and was unable since one of them is tied so deeply into the Arduino code.
This is the ideal solution... however I'm using every pin I routed out, except for the uncommitted "GPIO" pin... which isn't connected to a timer. Therefore, I'd first need to move another timer-pin's signal (e.g. TEMP_SENSOR_EN) to that GPIO pin, and then I could move VPIN_OUT to that TEMP_SENSOR_EN pin. If it's required, it's probably going to require a wire cut somewhere on the PCB, and then two soldered wires across four Arduino thru hole pins.
 

·
Linsight Designer
Joined
·
2,626 Posts
Discussion Starter · #653 ·
LiBCM can now spoof the pack voltage to any logical value between 115 and 175 volts or so.
Here's an explanation of what's going on in hardware to achieve this:
(FYI: Ignore the ABS-related pcode that occurs in the video... that's not an error that will happen in the car)

Note that LiBCM can spoof a constant voltage - regardless of a varying pack voltage - as long as the actual pack voltage is ~12 volts higher than the spoofed voltage. This gives LiBCM even more power!
 

·
Premium Member
Southern California
Joined
·
895 Posts
I didn't know it was still possible to film a 360p video in 2021. :)

Great work!
 

·
Linsight Designer
Joined
·
2,626 Posts
Discussion Starter · #655 ·
I didn't know it was still possible to film a 360p video in 2021. :)
Ha, youtubez is still processing the 4k ;).

Great work!
Thanks! Take a peak at the firmware architecture... much improved. Honestly I think it's taken its final form; I no longer see any reason to add a cooperative scheduler. So now it's time to add all the bells and whistles.

Thus far I'm using:
-just over 25% CPU time required to keep the MCM happy, and;
-7% of storage space, and;
-19% of RAM...

...which means we can add every bell and whistle we want. I'm gonna start with the temperature sensors, just as soon as we get it working in all the beta vehicles (beta testers standby for instructions and a video in our private conversation).
 

·
Linsight Designer
Joined
·
2,626 Posts
Discussion Starter · #657 · (Edited)
Daydreams in broken cv shafts
R.I.P. CVT splines

...

FYI: V_spoof range is ~= 64% < Vpack_actual < 94%

Which means we can spoof 120 volts (to maximize current, and thus power) up to Vpack(actual)=188 volts!
Since the full pack voltage on a 48S system is less (187 volts), LiBCM can spoof 120 volts whenever it wants.
Therefore, a 48S LiBCM should yield 26.0 kW assist, but will probably be more like 25.0 kW. That's 33 horsepower from the IMA motor. If LiBCM can actually achieve that power (TBD), then a G1 Insight with LiBCM will break into the triple digit horsepower range (i.e. 67 engine horsepower + 33 IMA horsepower is exactly 100 horsepower)... that would be a respectable 18.7 #/hp (88 W/kg). For reference, the 2021 Accord is 16.8 #/hp.

Interestingly, a don't expect a 54S pack to achieve much more power than that:
The full pack voltage on a 54S system is 211 volts, which LiBCM can only spoof down to 135 volts.
Based on what I know about the MCM, that'll limit maximum assist to 124 A (with +40%).**
Therefore, a 54S LiBCM should yield 26.5 kW, but will probably be more like 25.5 kW.

So my expectation is that a 54S pack will only yield 0.5 kW more peak power. To be clear, this is an LiBCM-related issue (i.e. the 64% minimum achievable spoofed voltage). We'll see what happens when I get around to 54S.

**Peter knows more about the MCM's behavior with voltage spoofing, and I yield to whatever he says.

Update: Just went for a test drive. Voltage spoofing works very well, but I realized towards the end of my drive why I wasn't getting the power increase I had predicted: Right now the LiBCM firmware bases its SoC on the spoofed voltage, not the actual voltage. Imagine my surprise that when I'd set the voltage to 120 volts, that the IMA gave no assist (because LiBCM would simultaneously tell the MCM the pack was empty). Changing how LiBCM sets SoC was already on my "JTS2doNow" list... so I guess now is the time to do it.

One other odd behavior was that the SoC gauge (on the instrument cluster) never showed any bars... 0/20. I'm wondering if that's related to the above issue... obviously it worked before.

Here's a power graph of the test drive:
92188


It'll be fun to compare this to the next test run (after I fix the issue mentioned above).
 

·
Registered
Joined
·
1,194 Posts
I no longer see any reason to add a cooperative scheduler.
That's nice to know.

There is often only a couple of things in an app that need a timeout (anything that uses delay()) and it's easy to just set a future time for a function to be called, and in loop, see if it is that time.
Based on what I know about the MCM, that'll limit maximum assist to 124 A (with +40%).
I put 7600 miles on my HCH last month which included Death Valley, Baker (100+ degrees), Vegas, Utah with several climbs past 10,000 ft and many past 5000 ft, on hot days. Even saw a couple of camouflaged unreleased Kias going up the passes out of Death Valley, one towing a trailer. Would the IMA motor withstand being hot-soaked in Death Valley or Baker and then being asked to deliver 200-250% climbing up the steep passes leading out of them (perhaps trying to maintain speed if someone is behind them?)

The car may back off what it thinks is 100% - 10 kW - after a few seconds. But 5 kW still could work out to more than twice that actually. Can the car handle it in such conditions?

I think that is something that needs to be characterized to see if there is a risk of motor failure, or a temporary thermal shutdown of the iGBT (thinking Insights with no AC).

The southwest is full of places that have steep climbs of 1000+ feet in elevation, that get very hot on the summer, many above a mile in altitude where there is less air to carry away heat.
 

·
Linsight Designer
Joined
·
2,626 Posts
Discussion Starter · #659 ·
Would the IMA motor withstand being hot-soaked in Death Valley or Baker and then being asked to deliver 200-250% climbing up the steep passes leading out of them?
No idea, not a priority. I do know the following:
-The weakest link in the OEM IMA system is overwhelmingly the NiMH battery.
-No temperature sensor in motor. Not feasible for every G1 LiBCM owner to add one.
-t_motor(worst case) can be conservatively inferred by monitoring IGBT temperature while simultaneously logging motor temperature with an aftermarket sensor. The MCM monitors the IGBT temperature, and we can easily wire tap that signal for our derivation.

I leave the answer to my beta testers, or to myself at a later date. Right now I'm focused on (many) other things.

The car may back off what it thinks is 100% - 10 kW - after a few seconds. But 5 kW still could work out to more than twice that actually. Can the car handle it in such conditions?
See above.

I think that is something that needs to be characterized to see if there is a risk of motor failure, or a temporary thermal shutdown of the iGBT (thinking Insights with no AC).
I agree, but it'll be a long time before that makes my list of "exciting things to do today". I doubt the MCM's OEM drive signal will overheat the motor, but I agree someone should test it. @Bull Dog knows much more about the OEM BLAC motor's thermal capacity than I do.
FYI: Cabin air temperature makes no difference to the PDU, which is cooled via external air ducted up from below the car.
 

·
Linsight Designer
Joined
·
2,626 Posts
Discussion Starter · #660 · (Edited)
In response to Peter's feedback regarding cell reading noise during heavy regen:
I replaced the BMS' 10 nF input LPF caps with 100 nF... it made a small change, but honestly I think most of the 'noise' we're seeing is really present on the cells. During assist I don't see anything similar, so my thought is that the cell internal resistance rises during heavy regen, which cause more 'noise' (i.e. voltage fluctuation) to couple in from the noisy PDU. For example, if a cell typically has 1 mOhm resistance - but rises to 10 mOhm under heavy regen - then that would amplify the PDU noise 10x. Put another way, I believe this noise is actually on the cells, and the LTC6804 is correctly digitizing it. My primary hunch here is that decreasing LPF an entire order of magnitude didn't appreciably change the behavior.

I'm certainly going to change to 100 nF on the next PCB revision... even though it didn't change anything, that lets me get rid of the 10 nF reel altogether... which is nice because my pick and place machine only holds 27 reels. Right now I'm hand placing 8 different parts.

Update: I logged USB data for the cell voltages during heavy assist/regen... it turns out the "noise" is actually just the screen's slow refresh rate... the digits are updated slowly enough that you're seeing partial numbers (i.e. midway thru updating). My logged USB data is as noise free as I would expect it to be... not immune to noise, but not as bad as the 4x20 display makes it out to be. Supporting evidence: the summed pack voltage isn't jumping around.

You can log this data, too... the latest version of VPIN branch outputs comma-delimited voltage/current/power statistics @ 2 Hz.
 
641 - 660 of 770 Posts
Top