Experiential
3D
Adobe CC
Code
Fabrication
Physical Computing

With This, Our Love to See by Its Own Light

Role

3D Designer, Fabrication, Physical Computing

Tools

Fusion360, Blender, Arduino, Laser cutter, 3D printer, power drill

Timeline

December 2022

Team

Self

Project Overview

With This, Our Love to See by Its Own Light features two 3D prints of me in a room to myself. A stretch sensor connects us from the back, and when pulled taut, the light between us blinks in affect. The more tense our connection, the more our light will blink away. The project is named after a song by Arthur Russell.

Process

Initial Worldbuilding

Narrative

A shapeless girl, an open house, an endless field, the anonymous world… The user plays through the walking simulator as this formless girl, though unbeknownst to them at first. There is an open-faced, colorless dollhouse in the middle of a field in nowhere, and dense fog covers the ground and shrouds whatever else may be on earth at this point in time. But why, despite the playful music and sunlight, does it feel like the beginning of the end?
Player Avatar
Player Avatar
The player is set to explore through this house that ultimately serves as the girl’s/player’s cage. We don’t know why it’s cracked open, or who set the silverware for the tables, which signifies a more mysterious and foreboding larger world– but for now, the player is stuck with their only option being to explore this house. We navigate through various rooms, all which hold a unique air and intentional, well worn objects– first the dining room, then the kitchen, the bedroom, the altar, and finally, up the attic through a keyhole-shaped entryway.
Dollhouse model in Blender
The order here is from least to most intimate, and a soft tension is hinted at throughout some rooms, with close enough observance: the worn desk and stovetop. The bedroom that resembles a dormitory. The Buddhist-style altar with a picture of… an apple? At the very top of the house is the attic, and it’s just an open room. The player is greeted by a cute and friendly pig, but aside from it, there’s nothing else but the tatami on the ground and the fiber wall. The player can only jump or descend down again. Upon jumping, the player lands on the ground gentle and intact, and may walk further out to the field to turn back and, for the first time, regard what is trapping them in its entirety.

Is it heaven or a memory? Or something else in between?

Modeling

Unity Scene

Technical Details:
  • Baking lightmaps for simulation performance
  • Post-processing (bloom & grain)
  • First Person Controller & VR Locomotion
  • Audio Trigger Zones & Spatial Audio
  • Mesh animation
  • Mesh collision physics
  • Field terrain
  • Shader graphs

Ideation

Checking Measurements

Volumetric Capture

We captured her likeness with Depthkit using volumetric capture, ensuring fine-tuning of calibration & lighting. Separate scenes for the city & the field were built in Unity, & her captures were imported as 3D objects within the scenes. Shots were recorded in Unity & exported to PremierePro for final editing. Q also provided accompanying voiceover.
Finetuning calibration. John is pictured here.

Checking Measurements

Measuring...

Materials

1/8" clear acrylic sheet
Alligator clips
Arduino Nano
LED
Nuts&bolts
Paper
PLA
Stretch sensor
Turf

3D Scanning Myself

Many, many thanks to the team at NYCAP3D for their work with 3D scanning. Without them, this project would not have been possible ❤

Printing & Prototyping

Assembling my Room

Fabrication Steps

1. Laser cut acrylic sheets
2. Adhere together with cyanoacrylate
3.  Cut out appropriate dimensions for turf & adhere with cyanoacrylate
4. Laser cut removable roof incl. holes for bolts
5. Place sky image between 2 sheets of acrylic for roof
6. Secure sheets together with nuts & bolts
7. Fit box & roof together & ensure compatibility

Modeling Renders

View my preliminary renders of the hotel room in Maya; click to enlarge.

Storyboarding

We decided to represent two alternate universes, and aim for a retro horror feeling (very fitting for a dingy motel!). The representation of time in the song was a heavily discussed point of reference for our team when considering creative direction, and we wanted to capitalize on the unique and nonlinear series of events revealed in the song. Therefore, the main character Gavin would appear seemingly normal in scenes with daylight, but off-putting and peculiar in the nighttime scenes.
Storyboarding with the lyrics

Motion Capture

Motion capture was then conducted through the Optitrack system (Thanks to Emily Hu for being our talent!) & shots recorded in Unreal Engine were exported and edited in PremierePro.

Ideation & Prototyping

Jacenia Li was responsible for the initial ideation; her initial pitch was as follows- "A constantly moving web installation that audiences can navigate through. This installation is hung from the ceiling, with its main parts around a human height. The installation will be made with chains and each corner is attached to the ceiling. The center of the web will be connected to the ceiling as well with a motor that moves it up and down. The web will automatically make sound when clashing with itself."
After several stages of iterative prototyping, the functioning prototype was decided with our installation site in mind (considering ceiling beams & weight distributions):
The interactive prototype is made of laser cut birchwood, miniature chains, & o-rings. Proposed motion is demonstrated with manual lifting of "ceiling". Following creation of BOM (Bill of Materials) and Gantt charts, we sourced materials online & from local hardware stores.

Materials & Physical Computation

Click to enlarge process pictures.
Linear actuators were used to program & mechanize the up/down movement of the chains. The actuators were controlled by code in Arduino & were positioned facing the winch remote within an open-faced wood box.

Timeline Overview

Conceptual Research

From Wolfram MathWorld: 

A strange loop is a phenomenon in which, whenever movement is made upwards or downwards through the levels of some hierarchical system, the system unexpectedly arrives back where it started. Hofstadter (1989) uses the strange loop as a paradigm in which to interpret paradoxes in logic [...] and calls a system in which a strange loop appears.

What does this mean in more interpersonal terms? I define a strange loop as a self-referential, possibly cyclical, feedback loop of affect between two subjects– in the scope of this project, my physical self and my digital self. Then, we must ask: where does the machine stand in this relationship, both as the vessel and as the perpetrator? I propose that we may begin to accept the machine as a facet of ourselves not just digitally, but also tangibly, within our so-called "real lives"...

Technical Inspiration

I was inspired by my friend and frequent collaborator John Wang's previous project "Faces", which uses similar technology to generate an AI version of himself in conjunction with photo data from This Person Does Not Exist. His project is featured here on Basta Magazine- click to learn more about his process.

Data

Why StyleGAN2 over RunwayML or StyleGAN3?

GAN2 offered more documentation than GAN3 + more stable generation results (at the time of project conception). I also had more experience in utilizing it for generation, over GAN3 or Runway.

Creating the database of my selfies

I wanted a visually consistent database of selfies, so I decided to take 1K+ selfies in one sitting. To facilitate the process, I automated the picture-taking by using p5.js, & later formatted & zipped the images with Python.

Code

Conda Env for Image Generation

p5.js & ml5.js Fakes versus Reals Detection Sketch

The trained Teachable Machine network had a 99%+ accuracy rate for telling the difference between my real selfies and my generated selfies!

Initial Generated Fakes

Context

Checking Measurements

Parents love their kids a lot- many to the point of hoarding. Clothes, old toys, books, you name it- children's clothes in particular see a rather short lifecycle given the rate at which kids grow. To keep up, parents need to constantly buy new clothes, which leads to many old clothes being thrown out or donated. Not to mention the name brands! Parents may also try to resell more expensive kids' clothes online, but without a pinpointed target demographic, their clothes don't sell well on general resell platforms. These external factors contribute to an endless purchasing cycle for parents with added pressure from social media & growing accommodations.

Understanding the User

The Challenge

Children grow quickly, which often leads to trouble for parents in keeping up with changing sizes and/or trends. The lifecycle of children's clothing is often unsustainable financially & ecologically, with many used clothes that don't resell ending up in landfills or laying around the house. To change that, we must consider...

Guiding Design Questions

How might we facilitate the process of reselling for busy parents?
How might we provide support for users who are not tech literate and/or have no previous experience selling on e-commerce websites?

The Solution

Our kids’ clothing resale app lets users resell their kids’ used clothes fuss-free which affects parents by offering to post and ship their listings for them. Effectiveness will be measured by analyzing the percentage of members listing/buying and the ratio of listings to orders. *Replace with mockups*

Research & Analysis

User Research

I conducted thorough examinations of personal accounts from a diverse group of parents, including interviews & online forums such as Reddit & so-called "mommy blogs", which are blogs written by mothers specifically tailored to the experience of motherhood & parenting. I also conducted general market research to better understand the needs & pain points of an otherwise untapped market. Did you know kids' recommerce is projected to grow 6x faster than traditional retail recommerce, hitting a 112.7% growth rate in 2020?

Though many parents (especially new moms) feel incentivized to buy into a purchasing cycle for their little tyke, there's also the incentive to save money on their clothes— with baby costs being around 10k a year on average in the US. Thus, the results of a survey with 2k+ parents are not surprising:

During the last 12 months preceding our survey, two-thirds of American parents surveyed said they purchased kids' and baby products secondhand. For those who are just starting their families, this number is even higher, with more than eight in 10 buying secondhand kids' and baby items.

And yet, the most common complaint about the process of buying online is the lack of guarantee for quality. The same goes for selling children's used clothes online, as children are notoriously messy (and it reflects onto their clothes!).

Internationally, sizing differences between also seemed to differ greatly, which caused

Competitive Audit

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec imperdiet eleifend elit in mollis. Sed pharetra eleifend augue non suscipit. Nulla gravida ut sapien et lobortis. Mauris lobortis, orci eu ornare facilisis, orci ligula posuere dui, eget sollicitudin neque odio eu lacus. Nunc id malesuada tortor. Integer tempus, lacus eu feugiat luctus, lorem nisi porta lorem, in dapibus urna elit sit amet erat. Mauris tempor elit eu pretium sollicitudin. Integer magna mi, porttitor vel erat gravida, porta ullamcorper tellus. Quisque tristique bibendum metus, in tempus diam gravida vel. Pellentesque facilisis, magna at feugiat sollicitudin, sem velit sagittis purus, eget consequat nisi mi id justo. Morbi viverra, ex ut sollicitudin porttitor, orci erat lobortis dolor, non rutrum quam velit id nisl. Proin elit purus, feugiat sed hendrerit ac, convallis a justo. Fusce luctus tincidunt consequat.

User Personas

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec imperdiet eleifend elit in mollis. Sed pharetra eleifend augue non suscipit. Nulla gravida ut sapien et lobortis. Mauris lobortis, orci eu ornare facilisis, orci ligula posuere dui, eget sollicitudin neque odio eu lacus. Nunc id malesuada tortor. Integer tempus, lacus eu feugiat luctus, lorem nisi porta lorem, in dapibus urna elit sit amet erat. Mauris tempor elit eu pretium sollicitudin. Integer magna mi, porttitor vel erat gravida, porta ullamcorper tellus. Quisque tristique bibendum metus, in tempus diam gravida vel. Pellentesque facilisis, magna at feugiat sollicitudin, sem velit sagittis purus, eget consequat nisi mi id justo. Morbi viverra, ex ut sollicitudin porttitor, orci erat lobortis dolor, non rutrum quam velit id nisl. Proin elit purus, feugiat sed hendrerit ac, convallis a justo. Fusce luctus tincidunt consequat.

User State Journey Mapping

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec imperdiet eleifend elit in mollis. Sed pharetra eleifend augue non suscipit. Nulla gravida ut sapien et lobortis. Mauris lobortis, orci eu ornare facilisis, orci ligula posuere dui, eget sollicitudin neque odio eu lacus. Nunc id malesuada tortor. Integer tempus, lacus eu feugiat luctus, lorem nisi porta lorem, in dapibus urna elit sit amet erat. Mauris tempor elit eu pretium sollicitudin. Integer magna mi, porttitor vel erat gravida, porta ullamcorper tellus. Quisque tristique bibendum metus, in tempus diam gravida vel. Pellentesque facilisis, magna at feugiat sollicitudin, sem velit sagittis purus, eget consequat nisi mi id justo. Morbi viverra, ex ut sollicitudin porttitor, orci erat lobortis dolor, non rutrum quam velit id nisl. Proin elit purus, feugiat sed hendrerit ac, convallis a justo. Fusce luctus tincidunt consequat.

Designing & Prototyping

Standout Features

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec imperdiet eleifend elit in mollis. Sed pharetra eleifend augue non suscipit. Nulla gravida ut sapien et lobortis. Mauris lobortis, orci eu ornare facilisis, orci ligula posuere dui, eget sollicitudin neque odio eu lacus. Nunc id malesuada tortor. Integer tempus, lacus eu feugiat luctus, lorem nisi porta lorem, in dapibus urna elit sit amet erat. Mauris tempor elit eu pretium sollicitudin. Integer magna mi, porttitor vel erat gravida, porta ullamcorper tellus. Quisque tristique bibendum metus, in tempus diam gravida vel. Pellentesque facilisis, magna at feugiat sollicitudin, sem velit sagittis purus, eget consequat nisi mi id justo. Morbi viverra, ex ut sollicitudin porttitor, orci erat lobortis dolor, non rutrum quam velit id nisl. Proin elit purus, feugiat sed hendrerit ac, convallis a justo. Fusce luctus tincidunt consequat.

User Flow

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec imperdiet eleifend elit in mollis. Sed pharetra eleifend augue non suscipit. Nulla gravida ut sapien et lobortis. Mauris lobortis, orci eu ornare facilisis, orci ligula posuere dui, eget sollicitudin neque odio eu lacus. Nunc id malesuada tortor. Integer tempus, lacus eu feugiat luctus, lorem nisi porta lorem, in dapibus urna elit sit amet erat. Mauris tempor elit eu pretium sollicitudin. Integer magna mi, porttitor vel erat gravida, porta ullamcorper tellus. Quisque tristique bibendum metus, in tempus diam gravida vel. Pellentesque facilisis, magna at feugiat sollicitudin, sem velit sagittis purus, eget consequat nisi mi id justo. Morbi viverra, ex ut sollicitudin porttitor, orci erat lobortis dolor, non rutrum quam velit id nisl. Proin elit purus, feugiat sed hendrerit ac, convallis a justo. Fusce luctus tincidunt consequat.

Wireframes

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec imperdiet eleifend elit in mollis. Sed pharetra eleifend augue non suscipit. Nulla gravida ut sapien et lobortis. Mauris lobortis, orci eu ornare facilisis, orci ligula posuere dui, eget sollicitudin neque odio eu lacus. Nunc id malesuada tortor. Integer tempus, lacus eu feugiat luctus, lorem nisi porta lorem, in dapibus urna elit sit amet erat. Mauris tempor elit eu pretium sollicitudin. Integer magna mi, porttitor vel erat gravida, porta ullamcorper tellus. Quisque tristique bibendum metus, in tempus diam gravida vel. Pellentesque facilisis, magna at feugiat sollicitudin, sem velit sagittis purus, eget consequat nisi mi id justo. Morbi viverra, ex ut sollicitudin porttitor, orci erat lobortis dolor, non rutrum quam velit id nisl. Proin elit purus, feugiat sed hendrerit ac, convallis a justo. Fusce luctus tincidunt consequat.

Styleguide & Identity Design

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec imperdiet eleifend elit in mollis. Sed pharetra eleifend augue non suscipit. Nulla gravida ut sapien et lobortis. Mauris lobortis, orci eu ornare facilisis, orci ligula posuere dui, eget sollicitudin neque odio eu lacus. Nunc id malesuada tortor. Integer tempus, lacus eu feugiat luctus, lorem nisi porta lorem, in dapibus urna elit sit amet erat. Mauris tempor elit eu pretium sollicitudin. Integer magna mi, porttitor vel erat gravida, porta ullamcorper tellus. Quisque tristique bibendum metus, in tempus diam gravida vel. Pellentesque facilisis, magna at feugiat sollicitudin, sem velit sagittis purus, eget consequat nisi mi id justo. Morbi viverra, ex ut sollicitudin porttitor, orci erat lobortis dolor, non rutrum quam velit id nisl. Proin elit purus, feugiat sed hendrerit ac, convallis a justo. Fusce luctus tincidunt consequat.

Finally...

View the final documentation here.

All Recent Work