How to set up the ZEDm Camera and HP Windows MR Headset in Unity
Setting up the ZEDm should be pretty straight forwards. Download the latest SDK and you should be good to go, but we still ran into a few issues. Now to be fair we were working with version 2.6 of the SDK and since then it's been updated so you might not experience the same problems that we did.
The silliest problem we ran into was simply not calibrating the ZEDm. We have multiple machines, and going between them sometimes we would forget to calibrate the ZEDm. This is a simple fix, but it's also easy to forget if you're moving from one machine to another.
The first real issue we had was with NVIDIA CUDA. The ZED Camera requires CUDA to work properly, and when you install the SDK it will install CUDA alongside with it. However the CUDA installer that was included with the ZED SDK would fail, and we would have to download the installer for CUDA separately.
At the time CUDA 10 was the most recent version, which the ZED did not support yet. After we realized this we went back to CUDA 9.2, however we found that there were issues with having multiple versions of CUDA installed. When we had CUDA 10 and 9.2 installed at the same time the ZED couldn't detect either of them, and this problem was only fixed once we uninstalled the other version.
The last issue we ran into was that in builds running on our target machine the ZEDm would crash after a few minutes. I understand that this has been fixed in the latest version of the SDK, but in order to get around it we had to downgrade our graphics card drivers to an older version.
The HPMR Headset is fairly simple to set up as to my understanding everything you need to get things working is included with all versions of Windows 10. Once you plug in the headset the Mixed Reality Portal will automatically boot up and walk you through calibration and setup. At the end of it you'll find yourself in a house that works as a sort of VR desktop. Getting into the house isn't hard, it's getting out that's the tricky part.
If you look up how to make a build for Windows Mixed Reality in Unity you might find this page from the Unity documentation. It goes over making a Universal Windows Platform (UWP) build and how to run the Visual Studio Solution that it spits out and so forth. If you've only ever built .exe's then this can be a pain to wrap your head around, but thankfully it's a pain that you don't have to worry about!
As I went over in another post, Windows Mixed Reality is actually just a fancy brand name for all of Microsoft's headsets. This includes things like the HoloLens which I guess is where they got the idea to call everything Mixed Reality. Regardless the HPMR Headset is actually just a regular VR headset with all of its sensors built in.
 |
| DON'T BE FOOLED, THEY'RE JUST VR HEADSETS |
So instead of making a UWP build you can make a regular old exe. In fact, when we were working with the ZEDm it was incompatible with UWP builds, and we had to instead make .exe builds and run through SteamVR to get it to work. Using SteamVR with WMR headsets requires you to install a free tool from Steam for it to work.
One last thing is that when we were using the SteamVR plugin for Unity we had to use an older version, as something wasn't playing nicely with our setup. I forget if it was the ZEDm or the Leap Motion or some other tool, but if you run into issues with the SteamVR plugin the older versions are available for download from from Valve's github. The issue we had was specifically dealing with controller inputs, as I think one of our plugins was built with an older version of SteamVR.
Anyway, after all that everything is finally working. If you run into any other trouble with the ZEDm Setreolabs is really good with troubleshooting and support. They would often respond to our messages the same day, and they were always helpful so if something isn't working and google doesn't turn anything up so don't hesitate to send them a message.
