I recently gave a talk on automating the process of creating digital design using AI at the Design System Hackathon '22 organised by the Into Design Systems Conference. This article is a summary of the talk.
Context #
I have been working on 'systemising design' for many years now, but I think we can all agree that over the past 3–5 years, technological advancements have really changed the way we approach digital design, and specifically how we design 'design systems'.
Apart from design Tools like Sketch, Figma or Adobe XD, which have radically changed the way we structure design, one of the most exciting developments for me, has been the concept of Design Tokens. While Design Tokens have been used in code for quite some time now, with the help of plugins such as Figma Tokens, we now have the ability to actually use design tokens in our design process. I have been an active user and contributor of Figma tokens for almost 2 years now and the possibilities are very exciting!
This all said, building design systems and component libraries is still a laborious process with many manual & repetitive tasks. So quite often, the execution part of this work isn't all that much fun.
That's where I started wondering if I could automate the process of creating digital design using automation and AI.
In this article I will explore the idea of using AI to automate the process of creating digital products by reducing the number of manual tasks and the time it takes to create digital products.
A few general characteristics of AI that I find relevant are that it:
- generally mimics humans
- while saving us a lot of time
- by taking otherwise manual tasks and executing them much quicker than humans could generally do.
Good examples of AI in Design #
Generative Design #
One design-related industries where AI has added huge value is industrial design and manufacturing. With the help of generative design, designers have been able to focus on the problem, and have AI-driven software generate hundreds or thousands of design solutions to a particular problem. This is generally done by providing the software with a set of rules or constraints, such as dimensions, required strength, or materials after which an AI proposes many possible and often surprising solutions, that humans would have been unlikely to create.
Predictive AI #
A great example is how AI can now predict Visual Attention / Eye-tracking studies with extremely high accuracy. In this example, the AI analyses a design file and shows us an eye-track heat map which earlier was only possible through actual user testing, requiring advanced equipment while being extremely time-consuming. Now we can iteratively evaluate and optimise our designs without needing either one of those.
Image recognition #
Another inspiring example is AirBNB's Proof of concept to convert Wireframes to Code. The AI is trained to match low-fidelity wireframes to available components and to assemble those in a logical way. Here you see a short video demonstration.
As you can see on the right, the wireframe is presented to the camera and on the left, we can see the AI build components in real-time. In the same way, we could imagine an AI helping us convert wireframes to high-fidelity mock-ups and prototypes in Figma using our Design System component library as the reference.
Contextual Completion #
GitHub recently launched an AI called Copilot. Copilot uses context and comments to figure out what code you're about to write and comes up with amazingly accurate completions, but often it spits out entire functions which can save tons of time. It's still in beta, but after using it over the last couple of months it sparked an idea that a very similar approach can be applied in the Design Process as well. To put this into perspective I created a small demo:
Some thoughts on automating Design System component design #
One of the major pain points in designing Design Systems is the huge amount of repetitive work we run into when designing complex components with many variants. To illustrate a possible solution I will use the use case of a 'Button' component.
The current process #
The common process for designing a button is to create a set of variants for each of the different states of the button. For example, a button can be in a 'primary' state, a 'secondary' state, a 'disabled' state, a 'loading' state, etc.
We update the visual appearance of each variant, componentize and name each variant, combine the variants and finally design the component API.
This is great, but... Changes & new variants are still a big job!
How can we make changes more manageable? #
One approach to making changes in components easier is by implementing Design Tokens. For example, using the Figma Tokens plugin. The good thing about this is that changes become a lot more manageable, but setting up granular tokens for each component is still a lot of work.
What if we turn the process around? #
This got me thinking about a more efficient process of creating components & variants in combination with tokens. Can we automate the process of creating components & variants in combination with tokens? I think so!
What we require to do this is:
Inputs:
- A blueprint for the component
- The definition of the desired component API
- A visual hierarchy (how the component is laid out in Figma)
Automation:
- Generate a headless component with all the variants
- Generate all required design tokens and apply them granularly to each component variant.
After that, all that is left is to link the tokens to your existing option tokens.
This is exactly what we worked on during the Design System Hackathon '22. Below is a small demo of the prototype we developed.