Earlier this month the US Patent & Trademark Office published a patent application from Apple that relates to Project Titan. Apple’s patent covers systems, apparatus, and techniques for sensing defects associated with a window and for performing image processing based at least partly on the sensed defects. Apple’s patent describes a lighting and camera system on the windshield of an autonomous or semi-autonomous vehicle that could detect debris impacting the windshield. The system could warn the driver to fix the windshield or automatically clean the windshield if it’s simply dirt. In an autonomous vehicle, the system has to adjust the view of the road ahead, compensating for the debris for safety reasons.
In Apple’s patent background they note that images of objects and/or scenes may be captured for various purposes. For instance, a camera may be used to capture images to obtain information about an environment. Sometimes obstructions interfere with imaging. As an example, a dirty window may be located within a field of view of the camera and may negatively impact images captured by the camera of a scene.
Motorized vehicles which are capable of sensing their environment and navigating to destinations with little or no ongoing input from occupants, and may therefore be referred to as “autonomous” or “self-driving” vehicles, are an increasing focus of research and development. However, such vehicles typically include windows that interfere with the extent to which they are capable of sensing accurate representations of their environment.
Apple’s patent covers various embodiments that relate to sensing/detecting defects associated with a window/windshield of a vehicle (fully or semi-autonomous) and performing image processing to produce a corrected image of a scene based at least partly on data corresponding to the detected defects.
According to various embodiments, the vehicle may include an imaging system that includes one or multiple sensor devices that are configured to perform imaging of objects. For instance, the imaging system may be configured to obtain data by imaging the window and/or a scene that is exterior to the vehicle.
Apple’s patent goes into extraordinary detail about the lighting system for vehicle windows (especially the windshield) to detect obstructions and describes how sensors and computing systems onboard work around obstructions for safe operation of the vehicle.
Next, the patent states that the vehicle may include a window cleaning system configured to spot clean the window/windshield from debris, bird splat etc. For instance, the processor(s) may be configured to evaluate, based at least in part on the first data (obtained by imaging the window), one or more parameters that characterize the defects associated with the window to produce parameter evaluation data. The parameters may include, for example, a distribution of the defects with respect to the window. The processor(s) may determine, based at least in part on the parameter evaluation data, to cause the window cleaning system to spot clean one or more particular areas of the window.
Additionally, or alternatively, the processor(s) may be configured to determine to designate a repair status and/or a replace status to the window or one or more portions of the window.
In some examples, designation of the repair status and/or the replace status may be based at least in part on the first data obtained by imaging the window. Designation of the repair status may indicate a suggestion to repair at least a portion of the window. Similarly, designation of the replace status may indicate a suggestion to replace at least a portion of the window.
Apple’s patent FIG. 1 below illustrates a schematic diagram of an example system #100 for detecting defects associated with a window and/or performing image processing to produce a corrected image of a scene based at least partly on data corresponding to the detected defects, in accordance with some embodiments.
In various embodiments, the system may include a window 102, one or multiple sensor devices (e.g., first sensor device 104 and/or second sensor device 106), one or multiple lighting modules (e.g., edge lighting module 108 and/or graze lighting module 110), and/or one or multiple processors (e.g., one or more processors of image deconvolver 112).
(Click on image to Enlarge)
Apple’s patent FIG. 5 below illustrates a schematic diagram of still yet another example system for detecting defects associated with a window and/or performing image processing to produce a corrected image of a scene based at least partly on data corresponding to the detected defects, in accordance with some embodiments. The diagram of FIG. 5 includes a schematic side view of the window and sensor devices of the system, where multiple sensor devices are individually configured to image a respective portion of the window.
Apple’s patent FIG. 7 above is a block diagram illustrating an example vehicle system environment in which a control system uses multiple inputs to determine vehicle operations, in accordance with some embodiments. The inputs used by the control system to determine vehicle operations may include inputs from an imaging system and/or an image deconvolver.
Apple’s patent FIG. 8 below is a flowchart of an example method of detecting defects associated with a window and/or performing image processing to produce a corrected image of a scene based at least partly on data corresponding to the detected defects; FIG. 9 is a flowchart of an example method of modifying a state of operation of an autonomous or partially autonomous vehicle.
Apple’s patent FIG. 10 above is a flowchart of an example method of causing a window cleaning system to clean a window; FIG. 11 is a flowchart of an example method of designating a repair status and/or a replace status to a window.
For more details, review Apple’s patent application number 20220284715 that was published on September 8, 2022. The patent reveals that Apple has been working on this since 2017.
One of the Key Patents behind MagSafe for iPhone
For those interesting in diving into one of the key patents behind MagSafe for iPhone, Patently Apple discovered an Apple patent filed in India. You could review the patent filing here. The patent links to other filings in different countries that may present a clearer format, like the one filed in Australia here. You could also review the 90 pages of patent figures associated with this patent here.
