Want 2 find U

Wednesday, June 6, 2012

Too hard to say Goodbye

All these troubles
will give us stories to tell each other later in life ----- Shakespeare

Time flies.
Today, is the last day in cse155 this quarter.
Thank you all for leaving me such a wonderful memory of my first quarter in America.

Hope to see you in the next 6 months in UCSD ~

Thank you!

Final Report

Want 2 Find U

---

Too Hard To Say Goodbye

0. Description

I. Milestones

II. Method

III. Tools

0. Description

Sketch-Based Face Matching

1. the User draw a picture of a front face using mouse, and choose the color of the face and eyes and hair.

/*the User note the key Points of the face, by pointing*/

2. the application gives back the most closely matched one.

I. Milestones

II. Methods

1. Using Haar Classifer to locate the Features on the Faces.

1.1. Constrain the Located Features by changing the space located by Haar classifier.

2. Using the Harris Corner Detector to get the Key Points.

2.1. Use the Harris Corner Detector in Features that are located by Haar classifier.

2.2. Get the uppermost, lowermost, leftmost, and rightmost Key Points within the Feature.

3. Getting the Color of Face, Eyes mathematic average.

3.1. Use the average Color of the nose to be the representative of the Face.

3.2. Use the Haar Classifier to locate the Eye, get the average color of the iris.

4. Compare the Faces by checking the length and distances as well as shapes.

III. tool

OpenCV 2.3 + Python 2.7

IV. Database

Front Face Pictures grabbed from google.

V. Future Plan

1. add more restrain. Such as the Moles, the eye-wrinkles and so on.

2. train better classifier for the feature detection & Face Key Point detection.

3. work out a better algorithm for the searching process.

4. Try to link this to social network, and use the Location and other information to help in finding the person.

VI. User Interface

VII. Result

(the most closely one, when i didn't put the original photo in my database)

Using Harris Corner Detector to get the KeyPoints

Using Harris Corner Detector to get the Key Points.(within the located features)
The key points I got by using Harris Corner Detector are much better than using SURF of SIFT.
Though, I have to admit that the picture becomes quite scary at the beginning.

But we can easily find that all the Features we need. the eyebrow, eyes, nose and mouth are Covered with a lot of red pots.

SO i decided to use the Harris Corner Detector and Haar Cascade Classifier together to get the key points I want.
When located in to different features, I can get the uppermost, lowermost, leftmost and right most. That is. it is enough to represent the width and length of that feature.

Draw by Mouse

Sketch the face by Mouse

The user draw a sketch of a face by Mouse.
My code of this part is written in Python 2.7 using OpenCV lib.
1. Track the movement of the mouse.
2. Record the move and print it to another page.

it is very hard for the ordinary people without painting experience to draw a face, especially draw by mouse.
So I provide a sketch picture, as the background, to make it easier for the user to draw.

When done, the picture is saved as a new .jpg

Wednesday, May 30, 2012

How r u, Want 2 find U?

after 2 weeks of changing my mind again and again and again. Want 2 find u finally can move forward.

1. Use Key Points instead of the Contour/ outlines

😄 1. Key Points, in some degree, is better than the Contours to be representative of a picture

😄 2. Key Points are clear and can speed up the Process of Comparing

😢 1. Key Points may not contain as much as useful informations as the contours,

However, I don't think the user who sketch a face can definitely confirm that ---- what he draw is what he want. 😄

2. Use Key Point inside the Feature templates

3. Use the Proportion instead of the length.

!!! The papers I follow..

1. Face Recognition: Features versus Templates

2. Feature Points Extraction from Faces

!!! What I will do this and next week:

1. get the informations such as the LIP THICKNESS (necessary when i saw this..)

the thickness of the Eyebrow if possible.

2. check the Eye-lid --- if it is double fold eyelid. (necessary)

3. check the shape of the chin.

4. get the shape/outline of the eyes.

5. Get the SKETCH, do the preprocess of it in the same way.

6. Add the color of the Eye and Skin (already done)

7. compare.

Using KeyPoints

1. Locate the Features

use Haar Cascade Classifier to locate the Eyebrows, Eyes, Ears (not useful), Nose and Mouth.

2. Get Key Points

use SURF algorithm to get the key points.

2.1 about the Key Points, (x,y)

1. Eyebrow: eyebrow_l1, eyebrow_l2, eyebrow_r1, eyebrow_r2

2. Eye: eye_l1, eye_l2, eye_r1, eye_r2

3. Iris: iris_l, iris_r

4. Nose: nose_l, nose_m, nose_r

5. Mouth: mouth_l, mouth_m, mouth_r

6. Jaw: jaw

7. Face: face_l1, face_l2, face_r1, face_r2

x=111 y=104 laplacian=1 size=16 dir=90.000000 hessian=15466.430664

x=182 y=106 laplacian=1 size=15 dir=90.000000 hessian=12413.648438

x=192 y=110 laplacian=1 size=16 dir=90.000000 hessian=13282.147461

x=79 y=132 laplacian=1 size=15 dir=90.000000 hessian=13704.891602

x=92 y=137 laplacian=-1 size=16 dir=90.000000 hessian=11122.696289

x=105 y=175 laplacian=1 size=16 dir=90.000000 hessian=11498.058594

x=76 y=216 laplacian=1 size=16 dir=90.000000 hessian=14645.329102

x=220 y=139 laplacian=-1 size=32 dir=90.000000 hessian=15401.563477

x=93 y=146 laplacian=1 size=34 dir=90.000000 hessian=16285.532227

x=215 y=153 laplacian=1 size=31 dir=90.000000 hessian=13717.638672

x=228 y=211 laplacian=-1 size=31 dir=90.000000 hessian=22596.167969

x=53 y=227 laplacian=1 size=30 dir=90.000000 hessian=13204.163086

x=270 y=240 laplacian=1 size=30 dir=90.000000 hessian=18954.589844

x=89 y=153 laplacian=1 size=57 dir=90.000000 hessian=14741.933594

x=213 y=191 laplacian=1 size=62 dir=90.000000 hessian=15877.199219

x=230 y=221 laplacian=-1 size=58 dir=90.000000 hessian=32176.843750

3. Get the Length of the features, and the Distance among the features

(to make it easier to present, just show it like below)

(actually works like [(y2-y1)^2 + (x2-x1)^2]^(1/2))

iris_dist = iris_r - iris_l ; (red points)

eye_length = eye_l2 - eye_l1 ; ( orange point, yellow point)

nose_width = nose_r - nose_l ; ( green points)

mouth_length = mouth_r - mouth_l ; (purple points)

eyebrow_length

= eyebrow_l2 - eyebrow_l1; (blue points)

eye_dist = eye_r2 - eye_l1; (orange points)

nose_mouth = nose_m - mouth_m; (green & purple)

face_length = face_r1- face_l1 ; (blue-green color points)

4. The Shape of the Face ----- Face Angle

i didn't make it to get all the key points i want on the outline of the face.

so i change it to be like

face_l1(x,y) -> find point face_l1(x,y) on FaceOutLine where x = Iris_l(x)

and also face_r1(x,y)

then find face_l2(x,y) -> find point face_l2(x,y) on FaceOutLine where x = eye_l1(x)

also face_r2(x,y)

face_angle1 = ( jaw(y) - face_l1(y) ) / ( jaw(x) - face_l1(x) )

face_angle2 = ( face_l1(y) - face_l2(y) ) / ( face_l1(x) - face_l2(x) )

face_angle = face_angle1 / face_angle2

after these four steps, I get the information of the face:

1. the length of nose, mouth, eyes, eyebrow

2. the distance of the Iris_Iris, Eye_Eye, nose_mouth...

use Proportion to compare

after getting the key points, i come across a problem
😭 Problem

Because the size of the head in different pictures are not the same, Using the Length and Distance to compare the pictures is far from accurate.

💡 Solution!

Proportion is used to help me get rid of this problem.

How?

Consider the iris_dist to be the base.

So..... I get:

eye_iris = eye_length / iris_dist ;
eyebrow_iris = eyebrow_length / iris_dist;
nose_iris = nose_width / iris_dist ;
mouth_iris = mouth_length / iris_dist ;

eye_dist_iris = eye_dist / iris_dist ;
nose_mouth_iris = nose_mouth / iris_dist ;