For this week we needed to use PoseNet, FaceMesh or Handpose. I chose FaceMesh, as I love working with head movements more rather than with the whole body.
Step1: First thing that I did was watching lecture video and retyping everything after Lisa because it allowed me to better understand the code. One thing I missed was changing the size of the canvas which ended up looking like that =>
Thanks to Lisa's fast response, figured that out and moved on.
Me holding the keypoints of my eyebrows
Step 2: Made the oscillator work with my eyebrows. The idea came to me on the floor when I participated in the testing for the project done by the first-year student called Bruce. He was using PoseNet and controlled oscillator with the hands. I was like "Ha, I can probably do it with my face features". So I did.
Me controlling oscillator with my eyebrows
Step 3: I wanted to use nose and mouth. Was stuck with the mouth for some time, cuz i didn't correctly update the length of array in the loop. So my mouth would have one point as it was connected to the length of nose array which one point.
Me before figuring out the loop problem on the left, and me after solving it on the right
Step 4: Then I wanted to control stop and play of oscillator with mouth opening and use distance between the lips for that. I was trying to figure out the y coordinates of lips points. Unfortunately, the idea didn't work out.
From left to right: The screenshot of the code and Y coordinates when my mouth is open and close; Screenshot of keypoints when my mouth is closed; Screenshot of keypoints when my mouth is open;
Step 5: I visualized the wave of oscillator and added some clown related styling. The nose gets bigger depending on the X coordinated and the smile depends on Y coordinates.
Me playing with the draft sketch
Step 6: Decided that the wave is too boring, so I added some fill. Also, made the smile difference more noticeable.
Me playing with the final sketch
Live Preview: https://editor.p5js.org/Kseniia/full/7jurvf3-5
Things I couldn't work out:
How to use notes with oscillator
How to us distance between 2 facemesh points
References: