OpenCV & Python2.7 on Mac OS X

Finally wake up  from the NightMare....

but.....another nightmare...in app store...made me change the name of my blog...

1.Setting up OpenCV on Mac OS X (binding Python2.7) 

        http://opencv.willowgarage.com/wiki/Mac_OS_X_OpenCV_Port


      1. download and install macports( http://www.macports.org/ )
            1.1 Mac OS X has already come with Python


      2. sudo port -v install opencv +python27
(before that remember to install command line tools in Xcode. or it will 
result in the conflicks in the installing process )
(Xcode -> preference -> components -> command line tools, then install it)

2. Start with OpenCV and Python2.7

    After spend the whole weekends dealing with setting up OpenCV with Python on Mac OS X, I finally start my project.


    Since OpenCV and Python are both open source, what I can refer to is very narrow. So I start with the The OpenCV Reference Manual

3. Track the Face and Eyes

       I write some small program in python 2.7, using openCV and haarcascades classifer.
       What I am able to do now is to locate the face, sometimes eyes in a photo...
       And also track the movement of the eyes and face from the video captured by webcam.
       

followings from webcam video

(seems this is a good way to find my eyes...)

 with glasses? it's ok, but.. we have to cut the influence...

s

4.What to do next?

   i.  Locate the Pupil (I mean.. the colorful part of the eyes.).
    ( 1. try to recognize circles in the given photo)
    ( 2. try to locate the circle inside the eyes ---- that is, pupil!)
  ii. Record the color of the eyes
 iii. Record the color of the face 

OpenCV & Python

There is an app using the name of FaceMatch in appStore, so I decided to change my project name to Want2FindU.

I found this video... And....finally have some confidence in my project
http://www.youtube.com/watch?v=4zDCs5l3Sj4

Works on Want2FindU, Starts

Works on Want2FindU,starts
.....
.......with my Nightmare on App Store



Nightmare on App Store

I've just bought a Macintosh to do my project with it. However I cannot login with my Chinese Account. 

What is worse, I cannot pay for the apps with my Chinese Credit Card. So I have to wait for activating my BOA card to continue setting up my new Apple ID.

This blocked my process on FaceMatch, and cannot follow my milestones in proposal.

So I did some reading instead and re-organized my thoughts.

1. Want2FindU will be a web application

2. Want2FindU will be written in python    /* to speed up, and keep in pace with the milestones. */
      - OpenCV for Python

3.Database of Want2FindU
     As I mentioned before, I want to use the portraits in Facebook or other Social Networks. But there is a PROBLEM.

     The users of the Social Networks just don't like to use the FRONTAL FACIAL IMAGE.
     However , if I ask the user to draw a picture of someone. I possibly receive a frontal facial image.

     I have some SOLUTIONS

      1. forget this problem, because there are billions of users in Facebook. There must be millions of people who uses the frontal facial image.

      2. change the 2D input image into 3D image
            -- 3D Face Reconstruction Using A Single 2D Frontal Facial Image
             http://jccit.ccit.ndu.edu.tw/ezfiles/7/1007/img/18/40-1-17_No-415.pdf

   
      3. change the 2D side facial image in database into  2D frontal facial image
            -- I didn't know if someone had done this before. But I think it is possible, because we are so familiar with the feature of a face and most of the faces are symmetrical.
            ( I will try solution 3 this week. and if I can find out the solution, I will post it on my blog)

      4. forget the Social Network. Set up my own web application, to ask the user take a photo from the laptop camera following the rules.

      5. forget the Social networks and my own Web application, just use some existing front facial image dataset.

      solution 1&5 seems easier for me.
      solution 2 is a little bit difficult.
      solution 3 | 4 is the most possible solution that I will use.

4. My Work & existing sketch-based image searching


       I did some comparasion between Want2FindU and other software(/image searching engine) , and did some analyze.
       http://want2findu.blogspot.com/2012/04/about-mindfinder-and-facematch.html


     This Week's goal;
        1. handle the problem with Apple ID.
        2. try to work out a way to change the side facial image into front facial image.
        3. start my work on Want2FindU
     

About the MindFinder AND Want2FindU

I found a short video of MindFinder in youtube.com
http://www.youtube.com/watch?v=UAFNtG0TTTo
1) Shapes
2) Complicated pictures
3) Sketch + tag
4) Sketch + tag + Color( main colors)

So, what I am going to do is a little bit similar  to their work.
However there is something different:

1) POSITION SENSITIVE?
    in the  MindFinder, the seaching is position sensitive.
    That is "the matched object should be at a similar position" as the input sketch, for example, when you draw a triangle on the upper left, then the pictures presented will be have some triangles in the same position.

    However in the Want2FindU, I am going to use the API from face.com to locate the face in the photo. That is to say, Want2FindU doesn't have to be POSITION SENSITIVE. I will just do the comparison between the sketching and the face, located by the face.com API.

this is one example in face.com. I think you are very familiar with this kind of App

 

2) SHAPE SENSITIVE?
   Want2FindU will be shape sensitive.


3) TAGS?
     If I want to finish this project following my original idea ---- link this with Social Networks, some Tag showing the information of the person, such as Nationalities,  Address, Age will give me great help in searching.


4) COLOR?
     In MindFinder, the user can choose a main color of his searching to get the photo with similar color.
     I will going to use this technique as well, however, only the main color is not enough, because the color of eyes is also necessary when recognize a person.

5) what I will ADD?
    As I mentioned in my blog, I will ADD SOME FUNCTIONS to help the user in drawing the picture closer to what is in his mind.
    ..........

6) Method.
   Edgel Index, indexing strategy for large-scale sketch-based image search.
   ..... this may be my be my job on this weekends... @_@....

7）Difficulty 
    The differences in faces are much slighter than the differences in other kind of photos.
 
     /* in this picture, you can see how the MindFinder works.
         The outline of the pictures is very clear. That's we, the users can check if the result is closed to our imagination very easily, because other photos is totally different.
         However, when it comes to the faces......
         The shape of the eyes, nose, mouth, ears..... are similiar .
         So the Want2FindU requires high accuracy....
*/

   

About the MindFinder

About the MindFinder
When I read the paper named, 'Edgel Index for Large Scale Sketch-based Imange Search', I found this software --- MindFinder.
However, I can only find a video of it.
http://www.youtube.com/watch?v=UAFNtG0TTTo

http://www.youtube.com/watch?v=UAFNtG0TTTo

About the FaceMatch

About the Want2FindU
I've got this idea many years ago, but only by now, with the wide spreading of Social Networks,  I can finally see the  chance to turn it into reality.

What is Want2FindU?
In my imagination, it will be an application that allow the users to draw a face of someone, and then searching for the most closely matches.

Problems:
0.people are not all artists.
1.Color Photo May Bring a Lot of Obstacles.
10.The Photos in Social Network is not Regular




PROBLEMS I: people are not all artists.


However, I have to admit that the faces is not very easy to draw for the ordinary people, who are not experts in painting. Therefore, I have to offer some functions to help in the process of organizing the pictures.
( I asked my friend to try to copy my photos. Even when I knew that it might be terrible, didn't I realize that it could be horrible)
This is the original photo of ME:

This is my friend's painting:
 ( we find that the first raw of the painting, seems like Picasso's work)
 ( The following one is interesting, I thinks it seems like Michael Jackson)

POSSIBLE SOLUTIONI:
   Provide some different kinds of features for users to drag to the canvas.
  ( I don't know if you once played this game, I think in some degree, I got this idea from it. )

PROBLEMS II: Color Photo May Bring a Lot of Obstacles.


  Color photo contains more content, and rocket up the time consumption. However, I think the color concentrate more on the detail than the outline. When we see a picture, even a gray-scale picture or a black-white picture, we can figure out the person in that picture. In some degree, Color in image influence both the user and the searching process.

POSSIBLE SOLUTION II:
 Change the photo into gray-scale image: (R,G,B) -> int gray = R＊0.3 ＋ G＊0.59 ＋ B＊0.11;
or Change it into black&white image by setting the THRESHOLD 128. 




PROBLEMS III:  The Photos in Social Network is not Regular

This is one of the portrait I used
 in the "Chinese Facebook" ---
 renren.com


As you can see...
This can't be a valuable picture 
in my application









POSSIBLE SOLUTION III:
 Add a function to ask the user capture their photo with the camera in their laptop or smartphone.
 or Just add some requirements.

 this is a picture I used in the in20year.com
It's a website, that can show what you will look like in 20 years.
 However when I put my photos( right, small one), The feedback really scared me.

But I learn a lesson from that, we should ask the user to put right, regular photo.

A Romantic Story, A Sweet Reason

A Romantic Story, A Sweet Reason

Have you heard about the  http://www.nygirlofmydreams.com/ ? Do you know about the Romantic Story in the  New York subway?

from: http://gawker.com/319937/patrick-moberg-has-found-his-new-york-dream-girl

from: nygirlofmydreams.com

Patrick Moberg, the website designer in NY, used 48 hours to find his Dream girl. It is a wonderful romantic story. Maybe from the day I saw this story, I have been thinking about how to seek for a person without knowing the name. Is it possible to just draw a picture, and then find out the one you want?

Now we have the Social Networks, like facebook.com, and have millions of photos of different faces. The romantic story in the NY Subway, gives me a sweet reason to do my project, Want2FindU --- Sketch-based Face Recognition application. If my Want2FindU is born successfully, it will be possible that the next Patrick may use no more than 10 seconds to find his dream girl.