There are two types of people in this world, those who are good at drawing, and those who are not… well, if you classify yourself as part of the latter, fret not as the good people from SketchAR have created a solution to your problem. SketchAR is an app for iOS, Android, and Microsoft HoloLens that uses Augmented Reality (AR) to help improve your drawing skills. By using your phone, or HoloLens, and positioning it over a blank sheet of white printer paper, a step-by-step guide to drawing any object is displayed on your device.
In a recent blog post by the app developers, they discuss all the problems that they ran into while making this app. They first talk about how they had to find a way for the device to distinguish between the lines that you have drawn and the lines that are placed on a piece of paper by using AR. They also needed to have the device distinguish different parts of the what the camera is picking up. For example, they need to have the device know the difference between the hand, pencil, marker, and the paper you are drawing on. The solution they came up with is known as the Progressive Markers method.
The Progressive Markers method looks over 5 different areas of concern while fixing the display on the device. The first area of concern is the environment. Everything has a different color, texture and look, and the app needs to differentiate between all of it to make a comprehensible image. The next area of concern is the place you are drawing it on. The app needs to know what you are drawing on, whether it be a wall, piece of paper, etc, in order to be able to display the lines you will need to trace. The third area of concern is the drawing of the user. The developers know that people using SketchAR aren’t good at drawing, so they needed to take that into account. The next area of concern is the hands. The hands will be covering up the canvas and will make it hard to display the image on your canvas. The final area of concern was the device itself, as it will never be a stable image. After looking into all those problems, the developers knew how to “teach the app to see only what the user draws, filtering, but not ignoring the environment.”