A Dynamic Painting Tool

There are many wonderful tools that allow creators to create amazing works of digital art. These include tools like Photoshop and Krita for sketches, and Illustrator for vector art. There are also many tools that create elaborate animations and simulations, ranging from After Effects to full blown 3D editors such as Blender and Cinema4D. However, there is a spectrum of artwork that can be created that neither of these types of tools fully addresses. Picture a digital sketch, such as this wonderful work from the main menu of The Legend of Zelda: Breath of the Wild:

This is a wonderful image. But there are a lot of dynamics in the image that are left to the imagination, when they could in fact be represented as an animation! For instance, the smoke billowing out of the volcano in the distance, the clouds moving, the birds flying, the red swirls around the castle, and even Link’s hair and clothes! These are things that can be animated using any of the aforementioned animations tools. But there is a lot of complexity involved to set up the simulations for this scene.

With this in mind, I’ve been working on a new type of graphic design tool called dali. The intention is to blur the lines between a static drawing and an animated sequence, allowing both of them to be worked with simultaneously. A few initial ideas include a cartoony wave effect:

and a rain effect:

There’s a lot of work to be done before it can be considered to be in a usable state though. Apart from better quality shaders and brushes, important required additions are an easy-to-use particle system as well as a method to model “flow”. This would allow artists to simply draw in the direction an artifact of the image should be animated towards. Something like this, but the flow directions are painted by the artist.

Currently, the implementation is done using HTML5. The 2D canvas provides a way to process the user input and implement brushes, whereas the WebGL canvas permits use of shaders to create interesting effects. Combining the two by using the output of the 2D canvas as a texture to a fragment shader is what drives these animations.

This project is deeply inspired by some of Bret Victor’s talks, where he talks about major shortcomings in modern technology that fail to address the plethora of ways we can interact with a computer.