Hi !
I’ve recently rescued an ML350 Gen10 that still ran February 2019 firmware on everything, including iLO 5 1.40. Getting it to 2026 firmware turned into a real investigation, because every standard update method fails on old iLO 5 in confusing ways, and the storage controller was such a pain as well…
This post collects all the findings so the next person hopefully doesn’t loose a full day of work after it.
All the links I’ve used will be bellow, but please read the full guide before attempting anything.
I CANNOT BE HELD RESPONSIBLE FOR BRICKED HARDWARE !
—–The headline discovery - iLO 5 update—–
On iLO 5 1.40 every obvious update path fails :
The web UI “Update Firmware” page says “invalid signature”.
Uploading to the iLO repository and asking for an update leaves the task “Pending” for hours, while the server starts power cycling every few minutes.
ilorest firmwareupdate <file> stops instantly with no useful error.
Extracting the image from the package and uploading that fails too.
I initially thought it may be a signature check error, as the iLO version was 7 years late to the party, after all.
But none of this is a signature problem :
All downloaded packages (.fwpkg files, which are just zip archives) are intact.
The images inside are properly signed by HPE with the same key family in 2019 and 2026 builds. Even a 2019 iLO 5 1.46 package from the same era was rejected, so the “old iLO does not know the new keys” theory is dead.
The iLO Event Log shows the real reasons: invalid firmware image and file invalid.
I served the packages from a local web server and watched iLO download each file twice, then abort the transfer, then declare the file invalid.
ilorest firmwareupdate logs unable to parse input URL, because that command sends the local file name where a URL is expected. That is the silent instant failure.
ilorest flashfwpkg flashes the very same files without complaint. It pushes the image to iLO directly instead of using the broken URL path. The release notes inside every package literally say to use this command (and I had already lost a few hours before noticing this…).
So : The URL based pipeline of iLO 5 1.40 is broken. The push upload path works fine. One successful flash later (1.46, then straight to 3.22) and all normal methods work again.
—–Recipes that work—–
0. Tooling
Install HPE “iLOrest”, the RESTful Interface Tool. On any machine with Python, the easiest way is pipx install ilorest. I used version 7.4.0.0. on a separate machine running Debian 13 connected on the same network.
Log in once first: ilorest login <ilo ip> -u Administrator -p <password>. A stale saved session otherwise gives Current session has expired or is invalid.
1. iLO 5 firmware
A .fwpkg is a zip. Its payload.json names the image file inside (for example ilo5_322.bin). Extract that image into the current directory, because ilorest opens it by name from there: unzip -o ilo5_322.fwpkg ilo5_322.bin
Then: ilorest flashfwpkg ilo5_322.fwpkg
It prints Successfully checked, then Firmware has successfully been flashed and iLO reboots (give it 2 to 4 minutes).
If ilorest asks A component with the same name ... overwrite? (y/n), pipe the answer: echo y | ilorest flashfwpkg ....
Version chain notes : I went 1.40 to 1.46 to 3.22, but a direct 1.40 to 3.22 jump would have also worked. No key rotation or weird version requirement is involved, so big jumps are fine.
2. Components iLO can flash itself
The payload.json file inside the package says UpdatableBy: ["Bmc"]
Examples: the Power Management Controller (PIC) and the Smart Array storage controllers.
Same command, ilorest flashfwpkg <file>.fwpkg, flashed immediately by iLO.
Reboot afterwards if the package says ResetRequired.
3. Components only UEFI can flash
The payload.json file inside the package says UpdatableBy: ["Uefi"]
Examples: the Innovation Engine (IE) firmware and SPS (Server Platform Services, the Intel management engine piece).
ilorest flashfwpkg stores the component and creates an update task (“This firmware is set to flash on reboot”). Then reboot.
About two minutes after POST the UEFI agent picks up the task and applies it.
Critical gotcha : The update task queue processes tasks in order and stops dead at the first failed task. An “Exception” task blocks everything queued behind it, forever. This is almost certainly why a repository update stays “Pending” all night, and the stuck retry loop is what makes the server power cycle every 5 to 10 minutes. Fix : delete the failed task (see the Redfish section below) and the next task runs at the next boot.
SPS declares ResetRequired. After its flash the server power cycles itself automatically. That is normal and expected.
4. System ROM (BIOS)
The BIOS download is called “System ROM Flash Binary”. You get a file named like U41_3.70_08_19_2026.signed.flash. It is a plain signed image, not a fwpkg package, so ilorest does not know what to do with it.
The method that works: serve the file from your own PC, then tell iLO to come get it. Two steps:
In the folder where you saved the file, start a tiny web server (Python ships with most Linux systems) : python3 -m http.server 8000
In a second terminal, send one command to iLO (swap in your iLO address, your password and your PC address) : curl -k -u Administrator:<password> -X POST -H "Content-Type: application/json" -d '{"ImageURI":"http://<pc ip>:8000/U41_3.70_08_19_2026.signed.flash","TransferProtocol":"HTTP"}' https://<ilo ip>/redfish/v1/UpdateService/Actions/UpdateService.SimpleUpdate/
What happens next confused me : nothing seemed to change. The new BIOS first lands on the backup ROM chip (a safety copy the board always keeps). The firmware inventory even shows both banks, for example U41 0/0000. Reboot the server once and the new BIOS activates. The old version stays on the backup chip, which is how HPE lets you recover a bad flash.
Note : This download method needs a reasonably new iLO 5. On the ancient 1.40 it refused every file, that is the broken pipeline described at the top of this post. That’s why I strongly recommend to update the iLO first !
Fun fact : the iLO repository stores .signed.flash files natively as “System BIOS” components (the machine still had a factory 2019 entry proving it).
5. The floor version maze : Storage controller firmware
The Smart Array P408i-a SR Gen10 lives in the “HPE SR Gen10” package family. Version history: 5.32, 5.61, 6.22, 6.52, 7.11, 7.43, 7.81, 8.00, 8.23.
The 8.23 package requires the controller to already run 5.61. Mine shipped at 1.98, so it must be stepped up twice !
HPE publishes one package per controller group for some build versions. The download page for “5.61” had THREE items (unlettered, B and C) and they are completely different packages :
The unlettered one (file HPE_E208e-p_Gen11_5.61_A.fwpkg): only 3 device targets, described as “HPE Gen10 E208e-p controller for Gen11 servers”, floor 5.32(C). Useless for a P408i-a.
HPE_SR_Gen10_5.61_B.fwpkg: 13 device targets (the whole family), floor 5.32(B).
HPE_SR_Gen10_5.61_C.fwpkg: 11 device targets, floor 5.32. This is the one to use(newest revision, most permissive floor).
Always unzip the fwpkg and read payload.json before flashing: check that Devices[].Target includes your controller and that MinimumActiveVersion is not above what is installed.
The error No matching target found has two meanings : the package is for another device group, OR your installed version is below MinimumActiveVersion (the floor is enforced during target matching). Both outcomes are harmless rejections, so probing is safe.
The chain I ran : 1.98 > 5.32 > 5.61 (C) > 8.23. The 5.32 package is the host flashed type (see bellow), the rest flash through iLO.
Some old storage controller packages simply cannot be flashed by iLO or UEFI. This is the case of version 5.32. They are “Online ROM Flash Components for Linux” : a shell script plus a flasher binary that speaks to the controller through the Linux smartpqi driver. ilorest refuses them with Component Type D, Unable to flash this fwpkg file. The only way to run them is a booted Linux.
Boot a Debian live system (or any recent live Linux). The smartpqi driver is in the stock kernel and picks up the controller automatically.
Copy the smart component over (web server, USB, whatever), extract, run its setupscript as root. HPE’s own flasher then does the work and logs to /var/cpq/Component.log.
Two surprises in that flasher : it asks interactively which controllers to flash (Select which devices to flash). Over SSH with no keyboard attached it just dies with “Invalid input”. Pipe the answer in: printf "A\ny\n" | sudo ./setup (A for all devices). And when it finishes with “Exit Status: 1” that is not an error: it means “Smart Component Finished, reboot required”. The write happens during the next boot, so reboot and check the version in iLO afterwards.
This is how the step from 1.98 to 5.32 is done. After that, the normal iLO methods work again.
—–Tips & Tricks—–
Error message dictionary
invalid signature (web UI, iLO 5 1.40) : the broken update pipeline, not a signature problem.
Firmware update failed ... invalid firmware image / file invalid: same thing via the URL path.
unable to parse input URL: ilorest firmwareupdate passed a local file name as URL. Use flashfwpkg.
Current session has expired or is invalid: run ilorest login.
The file 'x.bin' could not be opened for upload: extract the image next to the fwpkg and run ilorest from that folder.
No matching target found: wrong device group or below MinimumActiveVersion.
Component Type D, Unable to flash this fwpkg file: host flashed component, needs a booted Linux.
Update Failed due to an internal error: a UEFI task failed. Delete it or the queue stays stuck.
Could not establish a connection ... URL not accessible: your web server is not running (yes, I lost 20 minutes to a dead http.server).
LicenseKeyRequired on Virtual Media: iLO Standard does not include virtual media. Use a physical USB stick instead (or pirate your way in, do as you want).
Where to get old versions when the portal says “no”
The HPE support portal only shows recent items, but older ones are not gone :
HPE Software Delivery Repository (SDR): downloads.linux.hpe.com/SDR/repo/spp-gen10/ lists every Service Pack for ProLiant back to 2020.03, and each one carries the firmware of its day as Linux smart components (RPMs named firmware-smartarray-<hash>-<version>). SPP 2022.09.1 holds the 5.32 package we need. Extract the RPM (7z works), and inside is the smart component.
The web server paths are predictable. Every file I downloaded lives under downloads.hpe.com/pub/softlib2/software1/<area>/p<id>/v<id>/<name> with areas like fwpkg-ilo, fwpkg-generic, fwpkg-power, pubfw-uefi, pubsw-linux. The item pages in the portal (URLs with MTX-... identifiers) are also searchable and often still point at live files.
Redfish endpoints that save the day
All work with basic auth and curl, no session dance needed.
/redfish/v1/Managers/1/LogServices/IEL/Entries/ : the iLO Event Log. The true reason of every failure lives here, always check it first.
/redfish/v1/UpdateService/FirmwareInventory/ : every component version on the box.
/redfish/v1/UpdateService/UpdateTaskQueue/ : queued, running and failed update tasks. DELETE failed tasks or the queue stalls.
/redfish/v1/UpdateService/ComponentRepository/ : files uploaded to the iLO repository, including the device targets iLO resolved for them.
POST /redfish/v1/Systems/1/Actions/ComputerSystem.Reset/ with {"ResetType":"ForceRestart"} to reboot.
POST /redfish/v1/UpdateService/Actions/UpdateService.SimpleUpdate/ with an ImageURIfor URL based updates (broken on 1.40, works great on 3.22).
Extra gotchas
ilorest flashfwpkg --targets <n> is NOT the device index inside the fwpkg. It indexes the iLO firmware inventory and can aim a package at the wrong device. I once pointed the storage firmware at the BIOS with it. The task failed harmlessly, but do not touch that flag.
A repository task that can never apply keeps the server in a reset loop. Mine power cycled every 7 minutes all night while a stuck task retried. Removing the task stopped it. A missing boot device can cause a similar looking loop, so park the machine in System Utilities (F9) while working on it.
On iLO 5 1.40 a rejected file produces “Firmware upgrade started” and a failure in the same second. Fast failure means verification rejection, slow failure means transfer trouble.
Firmware inventory item ids are renumbered after an iLO firmware update. Do not cache them across an iLO flash.
The upload path and the repository accept files without checking whether they can ever be flashed. “Uploaded fine” does not mean “will install”.
If your HPE download page shows several items for one version number, they are different builds for different hardware. Read payload.json, not the page.
Boot overrides via Redfish (BootSourceOverrideTarget) are refused while the machine sits in POST or System Utilities. Use the One Time Boot Menu (F11) from the front panel or remote console instead.
—–Download links—–
All links are plain file downloads, no login needed.
—–Conclusion—–
Results on my ML350 Gen10 :
iLO 5 : 1.40 (Feb 2019) to 3.22 (Sep 2026)
System ROM : U41 v2.00 (Feb 2019) to U41 v3.70 (Aug 2026)
SPS : 4.1.4.251 to 4.1.5.312
Innovation Engine : 0.2.0.11 to 0.2.3.0
Power Management Controller : 1.0.4 to 1.1.4
Smart Array P408i-a SR Gen10 : 1.98 to 8.23 (via the 5.32 and 5.61 steps)
The 2019 iLO 5 1.40 firmware was the villain here. Once updated, the machine behaves like any modern HPE server and ilorest flashfwpkg handles everything.
If you are staring at “invalid signature” on an old iLO, do not question your files, question the iLO, push the image with ilorest, and read the Event Log.
Good luck out there.