A research team at Tsinghua University has created a promising see-through camera that can be seamlessly integrated with existing augmented reality (AR) devices. The camera, which allows light to pass through while capturing images, offers potential applications in various fields.
Conventional camera technology relies on opaque devices with multiple layers of lenses, resulting in limitations in versatility and potential applications. In contrast, the newly developed camera overcomes these constraints by enabling the passage of light. This breakthrough opens up the possibility for future visual sensing systems to seamlessly integrate into their surroundings, becoming imperceptible to users.
Efforts to create transparent cameras in the past have encountered challenges in terms of imaging quality and integration. To address these issues and achieve a compact, high-resolution see-through camera, the research team implemented a computational imaging scheme based on lightguides. Lightguides, commonly used in AR head-mounted displays, project images onto the retina while allowing external light to pass through. By reversing the projection light path, the researchers used the lightguide as an imaging relay element, reflecting object light onto the sensor while maintaining transparency. This innovative approach harnesses the properties of lightguides to enable effective image capture in a see-through camera design.
The team successfully demonstrated a prototype named LightguideCam, utilizing a commercial AR lightguide integrated into AR glasses. Although the incorporation of multiple reflective mirrors expanded the eye-box for AR displays, it introduced complex artifacts in the imaging process. To overcome this, the researchers developed an equivalent imaging model and proposed a compressive reconstruction algorithm.
The computational imaging scheme offers flexibility and compatibility with various lightguide designs, eliminating the need for highly customized solutions to achieve high-quality imaging. This enables easy integration of the prototype into existing AR devices, extending their functionality for applications such as eye-gaze tracking. Moreover, this concept can be applied to mobile phone screens, automotive windshields, and decorative elements, providing further possibilities for implementation.
This work was published on May 27, 2023, under the title " Toward a see-through camera via AR lightguide" in Optics Letters. The corresponding author of the paper is Liangcai Cao from Tsinghua University. The first author of the paper is Ph.D. student Yuchen Ma. The work was supported by the National Natural Science Foundation of China and the National Key Research and Development Program of China.
Link to the full article:
https://opg.optica.org/ol/fulltext.cfm?uri=ol-48-11-2809&id=530696
Researchers have developed a new-form camera that imaging without blocking all the light. Their LightguideCam employs the lightguide as an imaging relay element, which allows the object light to be reflected onto the sensor while maintaining an almost transparent window.
The researchers build a prototype based on a commercial lightguide. Computational methods are deployed to alleviate artifacts in the raw measurement.
Editor: Li Han
