coordinates confusion....

MakeHuman python API, python plugins, etc

Moderator: joepal

coordinates confusion....

Postby dma2017 » Fri Jul 14, 2017 2:06 pm

Hi All,
I am trying to use head-pose estimation (http://www.learnopencv.com/head-pose-estimation-using-opencv-and-dlib/)
to automatically set the face of the makehuman avatar in the position of a given photo.
Now I do have a rotation and translation vector out of cv2.solvePnPRansac, obviously they are using cartesian coordinates.
Code: Select all
        (success, rotation_vector, translation_vector, inliers) = cv2.solvePnPRansac(model_points, image_points,
                                                                                     self.camera_matrix, self.dist_coeffs,
                                                                                     flags=cv2.SOLVEPNP_ITERATIVE)


My problem is now to apply those to the (orbital?) camera to see the avatar face as displayed in the original photo.

This what I want:
Screenshot from 2017-07-14 15-53-03.png
face of the mesh aligned to the photo of Van Gogh

This is what I have now.
frame.jpg
badly aligned view

blue - 2d facial landmarks from the original photo
red - 2d facial landmarks from the screen capture of the mesh view
green - 3d facial landmarks projected on the view witch should be very close to the blue ones, but are not because the view is not aligned correctly....

Not sure I m very clear...any help appreciated.
Cheers.

-David
dma2017
 
Posts: 10
Joined: Mon Jun 26, 2017 9:43 pm

Re: coordinates confusion....

Postby dma2017 » Mon Jul 24, 2017 5:46 am

basically all I need is:
- what is the coordinate system used for the makehuman camera,
- how to properly set the makehuman camera to a given position.
someone ?

-David
dma2017
 
Posts: 10
Joined: Mon Jun 26, 2017 9:43 pm

Re: coordinates confusion....

Postby blindsaypatten » Mon Jul 24, 2017 1:42 pm

I can't help you with the camera in MakeHuman. If you don't get a reply with an answer and your goal is to get something working, my approach would be to move the makehuman model to Blender where all the coordinate systems and the camera model etc. are all known and easily controlled. I think you could get it working very quickly.
blindsaypatten
 
Posts: 586
Joined: Tue Mar 14, 2017 11:16 pm

Re: coordinates confusion....

Postby joepal » Tue Jul 25, 2017 8:01 am

Sorry, I think I'm missing something here. The coordinate system is a simple XYZ space, with XYZ rotations.

If using the "scripting" tab (you might have to enable the "7_scripting" plugin) to run the follwing script:

Code: Select all
print("This is the position before modification")
MHScript.printPositionInfo()
print("\n")

print("This is the rotation before modification")
MHScript.printRotationInfo()
print("\n")

MHScript.setPositionX(2.0)
MHScript.setRotationZ(10.0)

print("This is the position after modification")
MHScript.printPositionInfo()
print("\n")

print("This is the rotation after modification")
MHScript.printRotationInfo()
print("\n")


Then you'll get the following output in the console:

Code: Select all
This is the position before modification
SCRIPT: printPositionInfo()
posX:   0.0
posY:   0.0
posZ:   0.0


This is the rotation before modification
SCRIPT: printRotationInfo()
rotX:   0.0
rotY:   0.0
rotZ:   00.0


SCRIPT: setPositionX(2.0)
SCRIPT: setRotationZ(10.0)
This is the position after modification
SCRIPT: printPositionInfo()
posX:   2.0
posY:   0.0
posZ:   0.0


This is the rotation after modification
SCRIPT: printRotationInfo()
rotX:   0.0
rotY:   0.0
rotZ:   10.0


As well as a toon that has moved two X units to the right, and is facing slightly to the right.

Note that the *toon* is moved/rotated here, not the camera.

To see what the script above does behind the scene, take a look in https://bitbucket.org/MakeHuman/makehum ... ew-default, there are functions named the same as the calls in the script.
Joel Palmius (LinkedIn)
MakeHuman Infrastructure Manager
http://www.palmius.com/joel
joepal
 
Posts: 3608
Joined: Wed Jun 04, 2008 11:20 am

Re: coordinates confusion....

Postby dma2017 » Tue Jul 25, 2017 10:17 pm

Thanks joepal, it does show me the way...my maths are a bit long gone :)
according to OpenCV documentation, http://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html#solvepnpransac,

Code: Select all
cv2.solvePnPRansac(objectPoints, imagePoints,...) → rvec, tvec, inliers

returns rvec,tvec – Output rotation and translation vector (see Rodrigues() ) that, together with tvec , brings points from the model coordinate system to the camera coordinate system

And..if I am not mistaken, your explanations make me think the MHScript.setPosition[X,Y,Z], MHScript.setRotation[X,Y,Z] apply to the model, and therefore should be in model coordinate system....

So, I *just* need to find how to do that and give it a try I guess :-)

Thanks

-David
dma2017
 
Posts: 10
Joined: Mon Jun 26, 2017 9:43 pm


Return to Python scripts

Who is online

Users browsing this forum: No registered users and 1 guest