在 cv::solvePnP 之后就有了旋转 3x3 矩阵。该矩阵,因为它是一个旋转,所以都是正交 and 归一化。因此,该矩阵的列按从左到右的顺序排列:
- 右向量(在 X 轴上);
- 向上向量(在 Y 轴上);
- 前向矢量(Z 轴上)。
OpenCV 使用右手坐标系。坐在相机上沿光轴看,X轴向右,Y轴向下,Z轴向前。
您将前向向量 F = (fx, fy, fz) 和向上向量 U = (ux, uy, uz) 传递给 Unity。这些分别是第三列和第二列。无需标准化;它们已经正常化了。
在 Unity 中,您可以像这样构建四元数:
Vector3 f; // from OpenCV
Vector3 u; // from OpenCV
// notice that Y coordinates here are inverted to pass from OpenCV right-handed coordinates system to Unity left-handed one
Quaternion rot = Quaternion.LookRotation(new Vector3(f.x, -f.y, f.z), new Vector3(u.x, -u.y, u.z));
差不多就是这样了。希望这可以帮助!
编辑职位相关评论
注意:OpenCV 中的 Z 轴位于相机的光轴上,该光轴在中心附近穿过图像,但通常不完全在中心。校准参数中有 Cx 和 Cy 参数。这些组合是图像空间中从中心到 Z 轴穿过图像的位置的 2D 偏移。必须考虑到这种转变才能在 2D 背景上准确地映射 3D 内容。
要在 Unity 中获得正确的定位:
// STEP 1 : fetch position from OpenCV + basic transformation
Vector3 pos; // from OpenCV
pos = new Vector3(pos.x, -pos.y, pos.z); // right-handed coordinates system (OpenCV) to left-handed one (Unity)
// STEP 2 : set virtual camera's frustrum (Unity) to match physical camera's parameters
Vector2 fparams; // from OpenCV (calibration parameters Fx and Fy = focal lengths in pixels)
Vector2 resolution; // image resolution from OpenCV
float vfov = 2.0f * Mathf.Atan(0.5f * resolution.y / fparams.y) * Mathf.Rad2Deg; // virtual camera (pinhole type) vertical field of view
Camera cam; // TODO get reference one way or another
cam.fieldOfView = vfov;
cam.aspect = resolution.x / resolution.y; // you could set a viewport rect with proper aspect as well... I would prefer the viewport approach
// STEP 3 : shift position to compensate for physical camera's optical axis not going exactly through image center
Vector2 cparams; // from OpenCV (calibration parameters Cx and Cy = optical center shifts from image center in pixels)
Vector3 imageCenter = new Vector3(0.5f, 0.5f, pos.z); // in viewport coordinates
Vector3 opticalCenter = new Vector3(0.5f + cparams.x / resolution.x, 0.5f + cparams.y / resolution.y, pos.z); // in viewport coordinates
pos += cam.ViewportToWorldPoint(imageCenter) - cam.ViewportToWorldPoint(opticalCenter); // position is set as if physical camera's optical axis went exactly through image center
您将从物理相机检索到的图像放在以其前轴为中心的虚拟相机正前方(缩放以适合平截头体),然后您就可以在 2D 背景上映射正确的 3D 位置!