Arcturus has released a passthrough toggle for their camera

85

u/Koolala 15d ago

Hope they can add an automatic toggle connected to when the Frame's tracking detects your room is too dark

116

u/3dpowers ARCTURUS 15d ago

We would like to see an auto-switch for very low light available at least as an option. We probably need Valve's support to do it well, but this is on the list of user feedback we are trying to address one way or another.

32

u/Jimbo0451 15d ago

It would be cool to see you work with Valve on features that integrate directly into the SteamVR dashboard, like hand tracking to manipulate windows (apple vision pro style).

48

u/3dpowers ARCTURUS 15d ago

Along those lines, we'd also like to see if hand/arm segmentation can be an option (or a default), even without full hand tracking.

5

u/Bulky_Elephant_9942 14d ago

Would that mean essentially body occlusion? If so I would love to see it!

5

u/outfoxingthefoxes Soon™ 15d ago

Any way to check the camera resolution and or to change it? The demo videos on your app look way better than anything we can manage to make

29

u/3dpowers ARCTURUS 15d ago

In the Arcturus Vision Camera steam app you can record videos with different resolutions (but there are only two choices, full res or half-res for high FPS). Have you tried outdoor yet? Our sample recorded videos on the site were mostly outdoor or in well-lit scenes, where things usually look fantastic. If you're not getting nice results in those cases it would be good to see some data. For dimmer scenes (indoor, evening, not much natural light) we need to make denoise more adaptive.

6

u/CHKCHKCHK Soon™ 14d ago

I did take a video outdoors and it looks great! It got me thinking about a warning I had on my Quest 2 saying that direct sunlight can damage the light sensors. Do you happen to know if the Steam Frame’s onboard sensors are susceptible to light damage in the same way?

1

u/Tony_Nabili Soon™ 15d ago

Is it that necessary? Cannot you just calculate average brightness of the image and if it’s below some threshold switch the camera?

15

u/3dpowers ARCTURUS 14d ago

The heuristic could be like that, but also including the current exposure time (long exposure + still dark --> night mode). The IR LEDs already turn on automatically and them turning on could be the sign as well. But to do this automatically, it'd mean reading those values from system cameras, and doing that in a proper way will mean working with the folks at Valve.

3

u/bod09 13d ago

Just going to leave this here, might be worth a check:
https://github.com/bod09/frame-autopass

1

u/OilOk9031 9d ago

I'm assuming why they mention needing to talk to valve is for the same reason the readme for frame-autopass states this:

There is no public way to switch the passthrough camera, so frame-autopass uses a private SteamVR interface. Before every switch it checks that SteamVR's code still matches what it was built against. If an update changes it, frame-autopass stops switching (passthrough keeps working normally, just without automatic switching) and autopass status says so. Run autopass update; if no fixed release exists yet, please open an issue with the output of autopass report.

I doubt the arcturus team would be interested in doing it in such a way that Valve could accidentally break it with every update lol. Not specifically for switching, since they can already do that, but if any other parts of frame-autopass also rely on reverse engineering the os and not an officially supported api,

1

u/paysen Soon™ 14d ago

Probably just by ISO - but in the end, some people prefer a grainy colorful image over the internal camera that is blue and white - so it might be hard to get it right for everyone.

7

u/Piramista 15d ago

What I find funny is how the color camera doesn't even look that bad in full darkness. It just stays dark and you can see spots light if there are any. But in dim light it is very noisy.

1

u/Javs2469 Soon™ 15d ago

Doeesn´t it loose tracking? I guess not because of the traking being done with the regular IR cameras, but it doesn´t go full black?

My Pico 4 just looses tracking when the room is too dark and goes full black, but it only has color cameras for tracking.

10

u/Piramista 15d ago

The tracking is always done by the built in cameras and the Frame has infrared lights to illuminate the room. The controllers also have built in lights in a specific pattern.

6

u/inertSpark 1 TB 15d ago

Tested last night. The Arcurus camera is still active. I could see the tiniest little LED from my laptop in the corner of the room, so it's not full black.

The IR tracking is still active regardless.

1

u/xondk Soon™ 15d ago

In theory, if it can be read, it could just check if the IR floodlights are on?

5

u/Javs2469 Soon™ 15d ago

They are always on for tracking, right? So they don´t really go off when the Frame is working.

3

u/biasio95 256 GB 15d ago edited 15d ago

I have the Frame, they are NOT always on, I can clearly see them turning on in less than a second when I switch the lights off in my room. Also I can see them on a mirror.

2

u/xondk Soon™ 15d ago

From the videos out there of people filming themselves, it seems they turn on when there isn't enough light, no need for the power drain if they aren't needed.

1

u/Javs2469 Soon™ 15d ago

Interesting, makes sense. So they indeed could make the Arcturus cameras to change out when detecting low light.

1

u/inertSpark 1 TB 15d ago

That would honestly be awesome. Does anyone know if theres a light sensor, or is it just a case that the only part active when using the Arcturus Vision is the IR sensors?

1

u/Lincolns_Revenge 14d ago

Another option would have been for Arcturus to add a couple of visible light LED emitters to the camera housing. Two smartphone strength LEDs would have been more than enough to light up an entire room. But that ship has sailed.

6

u/3dpowers ARCTURUS 14d ago

Fun fact is that we did add LEDs, and they can do R,G,B, and white, but they are way too dim for flood lights. We knew the Frame's IR emitters were coming, though, and they blow away any amount of light we'd be able to emit without increasing the weight significantly, so I hope we can get an auto-toggle going for night mode soon (when the IR emitters come on, switch to built-in cams).

18

u/ivan-ent 15d ago

I hope they add some colour/image settings for the camera to tweak the image if you want

45

u/3dpowers ARCTURUS 15d ago

We are working on this -- not everyone's tastes and preferences are the same (even if we can probably make our one-size-fits-all defaults better too.) If you want to be added to our tester list, email [support@arcturus.vision](mailto:support@arcturus.vision)

2

u/seaweeduk 14d ago

my cameras image is quite noisy, even in what i would consider good lighting, is this normal? I havent seen anyone else mentioning it, seems to affect both eyes...

15

u/TigerCow7 Soon™ 15d ago

Is it possible to do it with a command/button though? Instead of going through the menu and multiple clicks

13

u/Zetro 15d ago

I didn't see anything about that in the app. You're seeing the entirety of it in the third screenshot.

But you're free to try Komino's setup, where the command would be: hide steam dashboard -> press joystick for 1s (re-bindable).

29

u/3dpowers ARCTURUS 15d ago

We'll explore how we could do controller bindings. We just wanted to rush out an option for folks. We're also hoping this can be part of SteamVR itself in the future.

6

u/TigerCow7 Soon™ 15d ago

Thank you very much! Currently we can switch from VR to passthrough by double clicking the steam button or command button on the headset, so maybe a variation there to switch between color and B&W. Like triple click? Or 2 buttons pressed simultaneously like how it's done to take screenshots?

15

u/3dpowers ARCTURUS 15d ago

Yes, it makes sense for that to be a similar button/sequence. We'll be looking into what can be done already with bindings vs what might need SteamVR updates...

1

u/Marrond 1 TB 14d ago

Is there some limitation for... automated switching based on lighting conditions?

6

u/cbutters2000 Soon™ 15d ago

See this script demoted in the first few minutes of this video: https://youtu.be/ngZlsd2yVQQ

3

u/outfoxingthefoxes Soon™ 15d ago

Was about time...

1

u/TheBatOuttaHell 15d ago

Got mine today. It’s a shame the quality is nowhere near even Quest 3’s default. Basically impossible to read text on my phone or monitor, and seems to be performing poorly in lower light situations too. Lots of ghosting trails wherever you look, and very grainy image quality overall.

Still a big step up from the default Frame pass thru, but I expected more for $150. Solid plug and play design and crazy lightweight though.

17

u/Squishyspud 15d ago

This can't be true. They use higher resolution cameras in the vision camera.

6

u/thosemegamutts 15d ago

Switching back and forth from quest to frame you can see a pretty big difference.

Quest has had years of updates and tinkering to make those cameras shine though. I'm sure the frame will only improve and eventually overtake the quest quality.

2

u/Squishyspud 14d ago

I'm still planning on purchasing the frame and camera add on. Was just hoping to hear better. Inhale no doubt valve will continue to update and enhance the frame though.

3

u/ThePandemonium 15d ago

That may be true on the spec of the sensor, the reality of it is viewing it in the Frame itself the pass through quality is quite grainy and not at all the quality they show in the promotional videos. It’s still a necessary accessory imo but they are right that you can’t see your monitor or phone very well.

2

u/Deep_Fish_8457 15d ago

That's actually disappointing, hopefully there will be better alternatives

1

u/MisterMittens64 1 TB 15d ago

It might be a limitation of the frame so unless it's a software issue, it will probably stay how it is

2

u/Marrond 1 TB 14d ago

Recorded videos look way better than passthrough so it may be either software issue or just compute limitations for real-time processing. Recordings are quite decent, to the point where I wish there was option to stitch output from both eyes for regular ratio but "flat" video. Good for first person perspective recordings that don't involve a lot of motion (no stabilization..., every step is an earthquake)

1

u/Squishyspud 15d ago

Danny. Was hoping for a good experience bringing my real life cockpit into vr trucking.

2

u/UncleS1am 14d ago

It's fuzzy enough that I found myself trying to see if there was a resolution toggle because maybe it was on the wrong setting. There is not one.

1

u/TheBatOuttaHell 14d ago

That was my reaction to reviews I saw prior to receiving it and finally setting it up yesterday.

No clue what the technical limitations are because the specs sounded good on paper, but yeah they must have to do some trickery to get the cameras to work as real time pass thru, that would explain the ghosting effect when turning since they’re 60hz.

Maybe recording content would look way sharper than the pass thru mode.

2

u/paysen Soon™ 14d ago

I haven't really experienced it in bad light, just in daylight because I have just used it the first time. But I have a Q3 and a Steam Frame. It looks really good. And the VR Videos are actually quite good. I can perfectly read my smartphone and even my OLED monitor (oled usually sucks for photograph for example, always a very gritty look, even on qhd) I am surprised (in a positive way) that the SBS videos look that good. Going outside to do some more testing. But I am very happy with my purchase.

3

u/Squishyspud 14d ago

So its a lighting thing then. That's good, I have plenty of lighting in my office.

2

u/dark79 1 TB 14d ago edited 14d ago

Basically impossible to read text on my phone or monitor

I can read my phone fine thru the Arcturus. I can't read my monitor from normal distance with the Q3 or the Arcturus, so they seem the same to me in that regard.

Edit: The Q3 quality is definitely less grainy and better contrast ratio. Reading my phone is easier. But the Arcturus can do it, the Q3 just does it better.

1

u/TheBatOuttaHell 14d ago

Yeah I think it’s tied to contrast ratios or exposure for the cameras. My room was dimly lit so the monitor and phone was pretty blown out white in my tests. My identically lit room worked fine with the quest 3. Maybe this is something Arcturus can adjust via firmware updates.

The ghost trails were definitely annoying though, and I don’t really notice any on the quest.

1

u/Low-Golf7820 14d ago

That’s kind of disappointing. I bought one last night because I was getting tired of reading my phone through the nose gap. Looks like I’ll still be doing that.

1

u/BlueManifest 14d ago

I can read text on my phone

1

u/punchsportdrink 7d ago

I can read my phone and see most of what I need on my PC desktop.

0

u/Ecnarps Soon™ 3d ago

I can absolutely read text on the phone and monitor. As to performing poorly in low light, the Quest stops working in low light.

0

u/What1does 14d ago

You can't read text on your phone with the add on cameras?

Huh?

I have made multiple texts and looked up some info on reddit using my phone while in the frame using the cam add on...

Something wrong with your setup

1

u/TheBatOuttaHell 14d ago

It’s possible my room was just too dimly lit, but yeah even my 4k monitor was impossible to read with it on. I had to keep peeking thru the nose hole to make any adjustments quickly.

I’m guessing it has something to do with the camera’s overall exposure adjustments based on your room’s lighting.

1

u/What1does 14d ago

Might be the brightness of the overall scene as well, I use white on black for my browser, texting, ect, and I have zero issues, but I just went to test and black on white is very hard to read.

1

u/TheBatOuttaHell 14d ago

Good to know, hopefully Arcturus can make adjustments to exposure settings via firmware. It’s really not that big of a deal overall, but for $150 extra I did have some partial regrets.

I’m guessing record mode has much better quality than live pass through does based on this comparison (quest 3 definitely doesn’t look like this in pass thru)

https://youtu.be/j3Kwv2CDOSs?

1

u/JaredWyns 1 TB 15d ago

Oh heck yeah.

1

u/Marrond 1 TB 14d ago

I'm just appaled this hasn't been thought of before release. Seems like such a basic, no-brainer feature it's weird it was missing.

1

u/BlueManifest 14d ago

Is their app also where the camera receives updates? Or is that done through Steam os

1

u/Ok-Explorer1296 11d ago

Is the arcturus vision camera too in the steam library or on the Linux desktop?

1

u/Ecnarps Soon™ 3d ago

Hopefully a toggle can be added to the bottom taskbar at some point. This helps but you still have to open the app to do it.

-1

u/lumifox 15d ago

do we know anything about this company? I'm just surprised to see everyone yipeee that meta isn't watching you, then install apps from some other company (i bought one myself, and noticed that they mention an AI feature for the gaussian splat "remix" your space, and i've been conditioned by reddit to assume ai = evil, so i wanna know how safe everything is)

8

u/super9mega 15d ago

Yes we do. If I understand it correctly, they made the monochromatic cameras and are the ones that helped valve design the entire camera system in the first place. Which is why they are an official partner and had a hardware device ready day one

2

u/paysen Soon™ 14d ago

Not only the camera system, afaik also the tracking system and algorithms:
https://www.youtube.com/watch?v=OWZUXMeuuM4&t=3160s

Software Arcturus has released a passthrough toggle for their camera

You are about to leave Redlib