r/SteamFrame • u/Snoo25109 • 7d ago
Accessories Custom Tracker using Frame IR tracking
A little over a year ago, I started working on some custom hardware in preparation for the Steam Frame; some of you might have seen my early posts on this Reddit going over my progress on my hardware projects under the name Frame Workshop. A few days ago, I finally got my IR trackers to be tracked by the Steam Frame itself. The idea behind the Locus trackers is that they would be IMU trackers supported by IR tracking to eliminate drift. While Locus's main goal isn't to be fully tracked by IR, this shows that custom controllers or other hardware can be tracked by IR. I am beyond thrilled that the Steam Frame is turning out to be the first truly open headset, something that I have waited for so long.
The trackers themselves are pretty simple; at their heart is an NRF52833 with an LSM6DSV16X IMU both the same MCU and IMU that the Roy controllers use. On top of that, they have 3 IR LEDs in each corner to help correct IMU drift when the cameras see them. A 250mah battery powers all of this, and the device is about 43 x 43 mm and 27g. All hardware is currently open source on the GitHub linked below, with firmware and software to follow.
If anyone has any questions, feel free to leave a comment.
Frame Workshop can be found here: https://github.com/Nieko27/Frame-Workshop
16
u/silver0199 7d ago
Yes please! These kinds of trackers would open up a lot of possibilities.
I look forward to seeing your project progress!
2
u/Snoo25109 6d ago
Thank you! I'm super excited to see what other trackers people put together in the future.
7
u/TheLastOperator 7d ago
So it's similar to what the pico does ?
1
u/Snoo25109 6d ago
I wasn't aware that this was how Pico handles their trackers, but it seems you are correct.
5
u/QuajerazRevived 6d ago
This is how all SLAM tracking works. An IMU for fast, responsive, and instantaneous tracking, with the cameras, basestations, etc purely correcting for drift.
6
u/No-Selection3424 6d ago
Excellent! I hope it can achieve full-body tracking with as few as three units.
3
u/Snoo25109 6d ago
Currently, they work much like slimes, with 5 as the minimum, since IR tracking stabilizes drift. However, if they were to be always in view, you could track using only 3, as IR gives their absolute position; the only problem is that unless someone made some downward-facing cam mod for the Frame they will rarely all be in view.
2
u/NowThisNameIsTaken 4d ago
This looks really cool! I'm new to vr and a bit confused about what you mean when you say it stabilises drift. As in having them on the wall or something helps with headset tracking? And you say 5 is the minimum because the assumption is ~2 will be out of sight most of the time? Or is there another reason?
5
3
u/RickSpawn147 6d ago
Incredible work!!! I would love to use this on my project.
Will you be releasing a product?
Crowdfunding?
2
u/Snoo25109 6d ago
It would be my dream to release everything I'm working on as a fully fledged product and definitely something I would like to work towards. Still, given the costs of FCC certification and injection molds alone I would definitely need some funding from one place or another. For now, everything will be open source on GitHub with guides on how to order them for yourself from a place like JLCPCB as they get to a more finished state. As I further refine them I would like to start looking into ways to realize them as a product.
2
u/RickSpawn147 6d ago
I fully understand, in fact that is what I will be doing to my project too.
As certifications and the processing of doing the actual business is not fun.Wish you all the best. Incredible work!!
3
u/QuajerazRevived 6d ago
This is awesome! Does that mean it'll be possible to make custom, 3rd party controllers? Because I would love for alternative options to be possible again like it was with lighthouse tracking.
2
u/Snoo25109 6d ago
Absolutely, all you really need to get something tracking is a microcontroller capable of Bluetooth, a IMU and some IR LEDs. I am super excited to see what other things people track with this!
2
u/QuajerazRevived 6d ago
That is super cool! If I had any experience with electronics, and owned a frame I suppose, I would 100% try to make some goofy looking controllers.
2
u/InitialeLangmut 6d ago
Do you reckon Index's Knuckle tracking is on the table then?
2
u/Snoo25109 6d ago
The knuckles use photodiodes for tracking vs IR LEDs. If someone made a mod for the Steam Frames PCIe port that allowed for a headset-mounted solid-state Lighthouse, you could in theory track them that way. If you wanted to track them using IR you could in theory mount an IR tracker to them to track them that way. My trackers are designed to be flat against the body so I would definitely need to make one with more IR LEDs if I wanted to track something like a controller, but it definitely could be done.
2
-3
u/daelikon 7d ago
I am quite confused...
What exactly is keeping track of these trackers? I am assuming the purpose would be to make a harness or some other thing to allow full body tracking, but if the frame is the one tracking, how can it "see" them when they are attached to the body?
7
u/secretqwerty10 7d ago
if you read the body text or github, you'd know it's an IMU tracker with IR lights to compensate for drift when the cameras of the frame see the trackers
2
u/daelikon 7d ago edited 7d ago
I still have no idea of what you wrote. I looked for IMU before my first post (Inertial Measurement Units), but I still don't understand the use case.
My "only" VR experience is with my index and vive, and both do external tracking so what is this drift you mention?
edit: I have been digging a bit more, and... my main question remains the same, if this is for doing full body tracking and the frame is supposed to do the tracking (or adjustment of it), the trackers are going to be out of the vision field of the frame a lot of the time
6
u/secretqwerty10 7d ago
IMU trackers are used for full-body tracking without lighthouses used for outside-in tracking (like the Valve Index, bigscreen Beyond etc). your phone has these sensors also to detect movement and tilt. the problem is that it's not as accurate as tracking from a fixed point (like the IR trackers that use lighthouses) and thus will have mistakes in the calculation, causing "drift" where the real life position no longer matches the virtual position, making you have to recalibrate. This project uses the IR LED's on the trackers to recalibrate when the Frame's cameras see them.
3
u/daelikon 7d ago
Thanks, It's an interesting solution. I am guessing a lot of the positioning can be made from inference of the visible trackers even if not all are visible simultaneously.
3
u/Splatoonkindaguy 6d ago
You use them as SlimeVR trackers and then correct absolute pose when they are in view to correct drift





27
u/actuallylemoncurd 256 GB 7d ago
Is it tracking with the NPU?