1) What previously existing libraries are you using (if any)?
2) Since the video capture (assuming it's from the tablet's camera) is detached from the depth capture, how do you coordinate the two to make the grid layout in the video? Structure.io seems to require attaching the depth device to the video device (so the perspectives are relatively fixed).
1) The core reconstruction technology is written from scratch (it's been a full-time project for over 18 mths.)
Some open source projects it uses: Eigen, OpenNL, OpenMesh, OpenCTM, GLM, Protobuf, Redis, Gluster, Node.
2) Video capture is actually from the RGB-D sensor, so it's registered with the depth. I believe the Structure sensor requires that the camera is fixed to the mobile device so that its IMU can be used for tracking. No such requirement here.
Jeff from the Structure Sensor team here. Structure Sensor comes with an attachment to hold it rigidly attached to an iPad (and CAD so you could 3D print attachments for other devices). Color-depth registration is handled by the SDK and is registered using vision (not IMU) -- but IMU is available if desired. Hope to see this project ported to be compatible with Structure Sensor!
Definitely possible - Forge is depth camera agnostic.
Before it could happen, though, I'd need you to ship my Structure Sensor USB cable and for you to release either an Android, Linux or Windows SDK. Hope that'll happen soon.
Comments
Great demo! A few questions:
1) What previously existing libraries are you using (if any)?
2) Since the video capture (assuming it's from the tablet's camera) is detached from the depth capture, how do you coordinate the two to make the grid layout in the video? Structure.io seems to require attaching the depth device to the video device (so the perspectives are relatively fixed).
1) The core reconstruction technology is written from scratch (it's been a full-time project for over 18 mths.)
Some open source projects it uses: Eigen, OpenNL, OpenMesh, OpenCTM, GLM, Protobuf, Redis, Gluster, Node.
2) Video capture is actually from the RGB-D sensor, so it's registered with the depth. I believe the Structure sensor requires that the camera is fixed to the mobile device so that its IMU can be used for tracking. No such requirement here.
Jeff from the Structure Sensor team here. Structure Sensor comes with an attachment to hold it rigidly attached to an iPad (and CAD so you could 3D print attachments for other devices). Color-depth registration is handled by the SDK and is registered using vision (not IMU) -- but IMU is available if desired. Hope to see this project ported to be compatible with Structure Sensor!
Definitely possible - Forge is depth camera agnostic.
Before it could happen, though, I'd need you to ship my Structure Sensor USB cable and for you to release either an Android, Linux or Windows SDK. Hope that'll happen soon.
No Ceres?
Not yet, but I've been meaning to try it out. It sounds nice.