Research | Evgeny Stemasov - HCI Research

Enabling Ubiquitous Personal Fabrication: Low-Effort and Expressive In-Situ Interactions Using Extended Reality

Mon, 03 Mar 2025 20:18:00 +0200

Between late 2018 and late 2024, I have been working on my dissertation at Ulm University, defending it on January 22nd, 2025. The dissertation is based on 8 publications in total: Ubiquitous Fabrication, Immersive Sampling, Mix&Match, ShapeFindAR, pARam, BrickStARt, DungeonMaker, and Ephemeral Fabrication. All projects explore futures of personal digital fabrication by developing design tools, interaction approaches, and proposing alternative lenses on personal fabrication. The document itself provides an overarching structure for all 8 papers to be situated in (expanding upon Ubiquitous Fabrication), but similarly aims to provide additional detail, context, and motivation for all projects.

pARam: Leveraging Parametric Design in Extended Reality to Support the Personalization of Artifacts for Personal Fabrication

Sun, 10 Mar 2024 09:56:15 +0100

Abstract

Extended Reality (XR) allows in-situ previewing of designs to be manufactured through Personal Fabrication (PF). These in-situ interactions exhibit advantages for PF, like incorporating the environment into the design process. However, design-for-fabrication in XR often happens through either highly complex 3D-modeling or is reduced to rudimentary adaptations of crowd-sourced models. We present pARam, a tool combining parametric designs (PDs) and XR, enabling in-situ configuration of artifacts for PF. In contrast to modeling- or search-focused approaches, pARam supports customization through embodied and practical inputs (e.g., gestures, recommendations) and evaluation (e.g., lighting estimation) without demanding complex 3D-modeling skills. We implemented pARam for HoloLens 2 and evaluated it (n=20), comparing XR and desktop conditions. Users succeeded in choosing context-related parameters and took their environment into account for their configuration using pARam. We reflect on the prospects and challenges of PDs in XR to streamline complex design methods for PF while retaining suitable expressivity.

DungeonMaker: Embedding Tangible Creation and Destruction in Hybrid Board Games through Personal Fabrication Technology

Sun, 10 Mar 2024 09:55:15 +0100

Abstract

Hybrid board games (HBGs) augment their analog origins digitally (e.g., through apps) and are an increasingly popular pastime activity. Continuous world and character development and customization, known to facilitate engagement in video games, remain rare in HBGs. If present, they happen digitally or imaginarily, often leaving physical aspects generic. We developed DungeonMaker, a fabrication-augmented HBG bridging physical and digital game elements: 1) the setup narrates a story and projects a digital game board onto a laser cutter; 2) DungeonMaker assesses player-crafted artifacts; 3) DungeonMaker’s modified laser head senses and moves player- and non-player figures, and 4) can physically damage figures. An evaluation (n=4x3) indicated that DungeonMaker provides an engaging experience, may support players’ connection to their figures, and potentially spark novices’ interest in fabrication. DungeonMaker provides a rich constellation to play HBGs by blending aspects of craft and automation to couple the physical and digital elements of an HBG tightly.

BrickStARt: Enabling In-situ Design and Tangible Exploration for Personal Fabrication using Mixed Reality

Mon, 02 Oct 2023 20:18:36 +0200

Abstract

3D-printers enable end-users to design and fabricate unique physical artifacts but maintain an increased entry barrier and friction. End users must design tangible artifacts through intangible media away from the main problem space (ex-situ) and transfer spatial requirements to an abstract software environment. To allow users to evaluate dimensions, balance, or fit early and in-situ, we developed BrickStARt, a design tool using tangible construction blocks paired with a mixed-reality headset. Users assemble a physical block model at the envisioned location of the fabricated artifact. Designs can be tested tangibly, refined, and digitally post-processed, remaining continuously in-situ. We implemented BrickStARt using a Magic Leap headset and present walkthroughs, highlighting novel interactions for 3D-design. In a user study (n=16), first-time 3D-modelers succeeded more often using BrickStARt than Tinkercad. Our results suggest that BrickStARt provides an accessible and explorative process while facilitating quick, tangible design iterations that allow users to detect physics-related issues (e.g., clearance) early on.

Immersive Sampling: Exploring Sampling for Future Creative Practices in Media-Rich, Immersive Spaces

Mon, 10 Jul 2023 20:18:36 +0200

Abstract

Creative practitioners rely on sampling to understand, explore, and construct problems; or gather resources for later use. Despite practitioners’ ability to experience immersive environments, sampling from them remains limited to primarily visual captures (e.g., screenshots, videos), which overlook the richness and variety of available media. To address these challenges, we describe “Immersive Sampling” as a new way to frame information gathering in the context of immersive environments. In the context of Immersive Sampling, practitioners engage in experiencing immersive environments while capturing, organizing, revisiting, and remixing found content. We situate this subset of tasks in literature and argue for their importance for emerging, future content creation domains. To further explore how Immersive Sampling might take place, we created VRicolage, a proof-of-concept prototype showcasing a set of interactions in Virtual Reality to sample, revisit, and remix captures. Given the democratization of immersive environments, Immersive Sampling provides practitioners with a means to collect, revisit, and remix digital materials.

ShapeFindAR: Exploring In-Situ Spatial Search for Physical Artifact Retrieval using Mixed Reality

Thu, 10 Mar 2022 09:56:15 +0100

Abstract

Personal fabrication is made more accessible through repositories like Thingiverse, as they replace modeling with retrieval. However, they require users to translate spatial requirements to keywords, which paints an incomplete picture of physical artifacts: proportions or morphology are non-trivially encoded through text only. We explore a vision of in-situ spatial search for (future) physical artifacts, and present ShapeFindAR, a mixed-reality tool to search for 3D models using in-situ sketches blended with textual queries. With ShapeFindAR, users search for geometry, and not necessarily precise labels, while coupling the search process to the physical environment (e.g., by sketching in-situ, extracting search terms from objects present, or tracing them). We developed ShapeFindAR for HoloLens 2, connected to a database of 3D-printable artifacts. We specify in-situ spatial search, describe its advantages, and present walkthroughs using ShapeFindAR, which highlight novel ways for users to articulate their wishes, without requiring complex modeling tools or profound domain knowledge.

Ephemeral Fabrication: Exploring a Ubiquitous Fabrication Scenario of Low-Effort, In-Situ Creation of Short-Lived Physical Artifacts

Mon, 13 Dec 2021 06:08:10 +0100

Abstract

Personal fabrication empowers users to create objects increasingly easier and faster. This continuous decrease in effort evokes a speculative scenario of Ephemeral Fabrication (EF), enabled and amplified by emerging paradigms of mobile, wearable, or even body-integrated fabrication. EF yields fast, temporary, in-situ solutions for everyday problems (e.g., creating a protective skin, affixing a phone). Users solely create those, since the required effort is negligible. We present and critically reflect on the EF scenario, by exploring current trends in research and building a body-worn fabrication device. EF is a plausible extrapolation of current developments, entailing both positive (e.g., accessibility) and negative implications (e.g., unsustainability). Using speculative design methodology to question the trajectory of personal fabrication, we argue that to avert the aftermath of such futures, topics like sustainability can not remain an afterthought, but rather be situated in interactions themselves: through embedded constraints, conscious material choice, and constructive embedding of ephemerality.

The Road to Ubiquitous Personal Fabrication: Modeling-Free Instead of Increasingly Simple

Tue, 05 Jan 2021 12:00:00 +0100

Abstract

The tools for personal digital fabrication (DF) are on the verge of reaching mass-adoption beyond technology enthusiasts, empowering consumers to fabricate personalized artifacts. We argue that to achieve similar outreach and impact as personal computing, personal fabrication research may have to venture beyond ever-simpler interfaces for creation, toward lowest-effort workflows for remixing. We surveyed novice-friendly DF workflows from the perspective of HCI. Through this survey, we found two distinct approaches for this challenge: 1) simplifying expert modeling tools (AutoCAD -> Tinkercad) and 2) enriching tools not involving primitive-based modeling with powerful customization (e.g., Thingiverse). Drawing parallel to content creation domains such as photography, we argue that the bulk of content is created via remixing (2). In this article, we argue that to be able to include the majority of the population in DF, research should embrace omission of workflow steps, shifting toward automation, remixing, and templates, instead of modeling from the ground up.

Mix&Match: Towards Omitting Modelling Through In-situ Remixing of Model Repository Artifacts in Mixed Reality

Wed, 22 Apr 2020 12:00:00 +0100

Abstract

The accessibility of tools to model artifacts is one of the core driving factors for the adoption of Personal Fabrication. Subsequently, model repositories like Thingiverse became important tools in (novice) makers’ processes. They allow them to shorten or even omit the design process, offloading a majority of the effort to other parties. However, steps like measurement of surrounding constraints (e.g., clearance) which exist only inside the users’ environment, can not be similarly outsourced. We propose Mix&Match a mixed-reality-based system which allows users to browse model repositories, preview the models in-situ, and adapt them to their environment in a simple and immediate fashion. Mix&Match aims to provide users with CSG operations which can be based on both virtual and real geometry. We present interaction patterns and scenarios for Mix&Match, arguing for the combination of mixed reality and model repositories. This enables almost modeling-free personal fabrication for both novices and expert makers.

Interactive Auditory Mediated Reality: Towards User-defined Personal Soundscapes

Sat, 21 Mar 2020 20:18:36 +0200

Abstract

Many people utilize audio equipment to escape from noises around them, leading to the desired isolation but also dangerously reduced awareness. Mediation of sounds through smarter headphones (e.g., hearables) could address this by providing non-uniform interaction with sounds while retaining a comfortable, yet informative soundscape. In a week-long event sampling study (n = 12), we found that users mostly desire muting or a distinct “quiet-but-audible” volume for sound sources. A follow-up study (n = 12) compared a reduced interaction granularity with a continuous one in VR. Usability and workload did not differ significantly for the two granularities but a set of four states can be considered sufficient for most scenarios, namely: ‘‘muted’’, ‘‘quieter’’, ‘’louder’’ and ‘‘unchanged’’, allowing for smoother interaction flows. We provide implications for the design of interactive auditory mediated reality systems enabling users to be safe, comfortable and less isolated from their surroundings, while re-gaining agency over their sense of hearing.

ShARe: Enabling Co-Located Asymmetric Multi-User Interaction for Augmented Reality Head-Mounted Displays

Tue, 01 Oct 2019 18:31:12 +0200

Abstract

Head-Mounted Displays (HMDs) are the dominant form of enabling Virtual Reality (VR) and Augmented Reality (AR) for personal use. One of the biggest challenges of HMDs is the exclusion of people in the vicinity, such as friends or family. While recent research on asymmetric interaction for VR HMDs has contributed to solving this problem in the VR domain, AR HMDs come with similar but also different problems, such as conflicting information in visualization through the HMD and projection. In this work, we propose ShARe, a modified AR HMD combined with a projector that can display augmented content onto planar surfaces to include the outside users (non-HMD users). To combat the challenge of conflicting visualization between augmented and projected content, ShARe visually aligns the content presented through the AR HMD with the projected content using an internal calibration procedure and a servo motor. Using marker tracking, non-HMD users are able to interact with the projected content using touch and gestures. To further explore the arising design space, we implemented three types of applications (collaborative game, competitive game, and external visualization). ShARe is a proof-of-concept system that showcases how AR HMDs can facilitate interaction with outside users to combat exclusion and instead foster rich, enjoyable social interactions.

Movelet: A Self-actuated Movable Bracelet for Positional Haptic Feedback on the User's Forearm

Fri, 05 Oct 2018 00:00:00 +0100

Abstract

We present Movelet, a self-actuated bracelet that can move along the user’s forearm to convey feedback via its movement and positioning. In contrast to other eyes-free modalities such as vibro-tactile feedback, that only works momentarily, Movelet is able to provide sustained feedback via its spatial position on the forearm, in addition to momentary feedback by movement. This allows to continuously inform the user about the changing state of information utilizing their haptic perception. In a user study using the Movelet prototype, we found that users can blindly estimate the device’s position on the forearm with an average deviation of 1.20cm to the actual position and estimate the length of a movemement with an average deviation of 1.44cm. This shows the applicability of position-based feedback using haptic perception.

Can’t You Hear Me? Investigating Personal Soundscape Curation

Wed, 26 Sep 2018 12:28:21 +0100

Abstract

Continuous advances in personal audio technology (e.g. headphones), led to efficient noise cancellation and allowed users to build and influence their personal acoustic environment. Despite the high adoption and ubiquitous character of the technology, we do not fully understand which particular factors influence and form usage patterns. As a step towards understanding the usage of personal audio technology, we conducted two focus groups (n=10) to investigate current headphone usage and users’ wishes regarding current and future personal audio technology. Based on this data, we derive a model for what we call personal soundscape curation. This model was assessed with the data of a crowdsourced survey on Amazon Mechanical Turk (n=194) on state of the art practices. Personal soundscape curation allows to describe usage strategies (curation, adaptation, renunciation) and break down influencing factors of context and environment as well as illustrate which consequences may arise from the users’ behavior.

FaceDisplay: Towards Asymmetric Multi-User Interaction for Nomadic Virtual Reality

Mon, 23 Apr 2018 00:00:00 +0100

Abstract

Mobile VR HMDs enable scenarios where they are being used in public, excluding all the people in the surrounding (Non-HMD Users) and reducing them to be sole bystanders. We present FaceDisplay, a modified VR HMD consisting of three touch sensitive displays and a depth camera attached to its back. People in the surrounding can perceive the virtual world through the displays and interact with the HMD user via touch or gestures. To further explore the design space of FaceDisplay, we implemented three applications (FruitSlicer, SpaceFace and Conductor) each presenting different sets of aspects of the asymmetric co-located interaction (e.g. gestures vs touch). We conducted an exploratory user study (n=16), observing pairs of people experiencing two of the applications and showing a high level of enjoyment and social interaction with and without an HMD. Based on the findings we derive design considerations for asymmetric co-located VR applications and argue that VR HMDs are currently designed having only the HMD user in mind but should also include Non-HMD Users.

ShareVR: Enabling Co-Located Experiences for Virtual Reality between HMD and Non-HMD Users

Mon, 01 May 2017 00:00:00 +0100

Abstract

Virtual reality (VR) head-mounted displays (HMD) allow for a highly immersive experience and are currently becoming part of the living room entertainment. Current VR systems focus mainly on increasing the immersion and enjoyment for the user wearing the HMD (HMD user), resulting in all the bystanders (Non-HMD users) being excluded from the experience. We propose ShareVR, a proof-of-concept prototype using floor projection and mobile displays in combination with positional tracking to visualize the virtual world for the Non-HMD user, enabling them to interact with the HMD user and become part of the VR experience. We designed and implemented ShareVR based on the insights of an initial online survey (n=48) with early adopters of VR HMDs. We ran a user study (n=16) comparing ShareVR to a baseline condition showing how the interaction using ShareVR led to an increase of enjoyment, presence and social interaction. In a last step we implemented several experiences for ShareVR, exploring its design space and giving insights for designers of co-located asymmetric VR experiences.