Contextual Augmentation Using Scene Graphs

Tech ID: 32037 / UC Case 2020-156-0

Patent Status

Country Type Number Dated Case
Patent Cooperation Treaty Published Application WO2021263018 01/30/2022 2020-156

Brief Description

Spatial computing experiences are constrained by the real-world surroundings of the user.  In such experiences, augmenting virtual objects to existing scenes require a contextual approach, where geometrical conflicts are avoided, and functional and plausible relationships to other objects are maintained in the target environment.  Yet, due to the complexity and diversity of user environments, automatically calculating ideal positions of virtual content that is adaptive to the context of the scene is considered a challenging task. 


UC researchers have developed a framework which augments scenes with virtual objects using an explicit generative model to learn topological relationship from priors extracted from a real-world and/or synthetic 3D datasets.  Primarily designed for spatial computing applications, SceneGen extracts features from rooms into a novel spatial representation which encapsulates positional and orientational relationships of a scene which captures pairwise topology between objects, object groups, and the room.  The AR application iteratively augments objects by sampling positions and orientations across a room to create a probabilistic heat map of where the object can be placed.  By placing objects in poses where the spatial relationships are likely, we are able to augment scenes that are realistic. 

Suggested uses


Augmented communication applications (AR conference calls, virtual calls, tele-immersion, etc.)

Virtual reality collaborative applications (games, collaborative design applications, etc.)

Tele-training and education applications.


Related Materials


Learn About UC TechAlerts - Save Searches and receive new technology matches


  • Caldas, M. Luisa G.
  • Keshavarzi, Mohammad
  • Parikh, Aakash Suketu
  • Yang, Allen Y.

Other Information


Augmented Reality, Partial Scene Completion, Scene Synthesis, Generative Modelling, Spatial Computing

Categorized As