Page 1 of 1

coordinates confusion....

PostPosted: Fri Jul 14, 2017 2:06 pm
by dma2017
Hi All,
I am trying to use head-pose estimation (http://www.learnopencv.com/head-pose-estimation-using-opencv-and-dlib/)
to automatically set the face of the makehuman avatar in the position of a given photo.
Now I do have a rotation and translation vector out of cv2.solvePnPRansac, obviously they are using cartesian coordinates.
Code: Select all
        (success, rotation_vector, translation_vector, inliers) = cv2.solvePnPRansac(model_points, image_points,
                                                                                     self.camera_matrix, self.dist_coeffs,
                                                                                     flags=cv2.SOLVEPNP_ITERATIVE)


My problem is now to apply those to the (orbital?) camera to see the avatar face as displayed in the original photo.

This what I want:
Screenshot from 2017-07-14 15-53-03.png
face of the mesh aligned to the photo of Van Gogh

This is what I have now.
frame.jpg
badly aligned view

blue - 2d facial landmarks from the original photo
red - 2d facial landmarks from the screen capture of the mesh view
green - 3d facial landmarks projected on the view witch should be very close to the blue ones, but are not because the view is not aligned correctly....

Not sure I m very clear...any help appreciated.
Cheers.

-David

Re: coordinates confusion....

PostPosted: Mon Jul 24, 2017 5:46 am
by dma2017
basically all I need is:
- what is the coordinate system used for the makehuman camera,
- how to properly set the makehuman camera to a given position.
someone ?

-David

Re: coordinates confusion....

PostPosted: Mon Jul 24, 2017 1:42 pm
by blindsaypatten
I can't help you with the camera in MakeHuman. If you don't get a reply with an answer and your goal is to get something working, my approach would be to move the makehuman model to Blender where all the coordinate systems and the camera model etc. are all known and easily controlled. I think you could get it working very quickly.

Re: coordinates confusion....

PostPosted: Tue Jul 25, 2017 8:01 am
by joepal
Sorry, I think I'm missing something here. The coordinate system is a simple XYZ space, with XYZ rotations.

If using the "scripting" tab (you might have to enable the "7_scripting" plugin) to run the follwing script:

Code: Select all
print("This is the position before modification")
MHScript.printPositionInfo()
print("\n")

print("This is the rotation before modification")
MHScript.printRotationInfo()
print("\n")

MHScript.setPositionX(2.0)
MHScript.setRotationZ(10.0)

print("This is the position after modification")
MHScript.printPositionInfo()
print("\n")

print("This is the rotation after modification")
MHScript.printRotationInfo()
print("\n")


Then you'll get the following output in the console:

Code: Select all
This is the position before modification
SCRIPT: printPositionInfo()
posX:   0.0
posY:   0.0
posZ:   0.0


This is the rotation before modification
SCRIPT: printRotationInfo()
rotX:   0.0
rotY:   0.0
rotZ:   00.0


SCRIPT: setPositionX(2.0)
SCRIPT: setRotationZ(10.0)
This is the position after modification
SCRIPT: printPositionInfo()
posX:   2.0
posY:   0.0
posZ:   0.0


This is the rotation after modification
SCRIPT: printRotationInfo()
rotX:   0.0
rotY:   0.0
rotZ:   10.0


As well as a toon that has moved two X units to the right, and is facing slightly to the right.

Note that the *toon* is moved/rotated here, not the camera.

To see what the script above does behind the scene, take a look in https://bitbucket.org/MakeHuman/makehum ... ew-default, there are functions named the same as the calls in the script.

Re: coordinates confusion....

PostPosted: Tue Jul 25, 2017 10:17 pm
by dma2017
Thanks joepal, it does show me the way...my maths are a bit long gone :)
according to OpenCV documentation, http://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html#solvepnpransac,

Code: Select all
cv2.solvePnPRansac(objectPoints, imagePoints,...) → rvec, tvec, inliers

returns rvec,tvec – Output rotation and translation vector (see Rodrigues() ) that, together with tvec , brings points from the model coordinate system to the camera coordinate system

And..if I am not mistaken, your explanations make me think the MHScript.setPosition[X,Y,Z], MHScript.setRotation[X,Y,Z] apply to the model, and therefore should be in model coordinate system....

So, I *just* need to find how to do that and give it a try I guess :-)

Thanks

-David