Declarative Design and Design Runtime
The design of digital artifacts is becoming increasingly complex. Even though it long ago went far beyond the level of complexity that human beings can handle in their heads, there is still no proper solution, and designers, developers, and users are all suffering for it. A new approach is needed.
The problem: combinatorial explosion of the design space
The environments in which software runs have become extremely diverse—desktop computers, mobile phones, kiosks, car dashboards, smartwatches, XR devices, and so on—and they will become even more diverse in the future. Even for the same software, a wide range of factors like the following must be taken into account.
- Input method: Buttons on a touch screen need to be larger than buttons on a traditional GUI screen operated with a mouse. That is because fingers are thicker than a mouse cursor. And unlike a mouse cursor, fingers also block the view, which must be taken into account. In the case of handheld devices such as smartphones, it is also good to consider the areas a thumb can easily reach when designing a layout. Besides touch screens, there is also a need to consider cases where only a keyboard is used, where a stylus is used, where a remote control is used, where voice is used, where sign language is used, and so on.
- Landscape mode vs. portrait mode: Most smartphones and tablets switch between landscape and portrait mode using an accelerometer. Even within the same portrait mode, the screen ratio can vary greatly for many reasons, such as the thickness of the system status bar changing or the on-screen keyboard opening.
- Network conditions: For various reasons, network speed may become extremely slow or the network may be temporarily disconnected. In such cases, one has to think about how software can degrade gracefully rather than simply coming to a complete halt.
- Multilingual support: Russian and German words are on average longer, which can affect the shape of buttons. If the UI is designed carelessly, problems can arise where the layout breaks in some countries or labels cannot be fully read. On top of that, some countries write text from right to left or from top to bottom.
- Font size: People with low vision may prefer large text. Conversely, some users reduce the font size in order to fit as much information as possible onto a single screen.
Beyond these, there is no end to the factors to consider: high contrast (for low vision), dark mode, narrow margins vs. wide margins, e-ink panels (slow screen refresh, monochrome or poor color reproduction), thermal state (some operating systems can adjust the quality of graphic elements or the screen refresh rate depending on thermal conditions), reduced motion (some users experience motion sickness when there is too much motion), and so on.
When factors are added, the number of cases grows not by addition but by multiplication, so the number of combinations becomes enormous in an instant. For example, if there are 4 kinds of input devices, 5 kinds of form factors, 4 kinds of aspect ratios, 6 color options, and 3 text options, that already yields
The existing approach: ignore most of the problem
Since there is still no real way to solve this problem, the claim that all possible combinations should be considered can only be treated as the idle talk of someone who knows nothing about business. The currently widespread, “business-aware” realistic compromise is this:
- Do not consider a substantial number of factors: For example, simply do not consider factors that are deemed not business-critical, such as users who write text from right to left.
- Of all combinations that could be produced by the factors you have decided to consider, do not consider most of them either: For example, a user who selects the “high contrast” setting might be unable to choose “dark mode,” or English speakers might be able to choose one of five voices while Korean speakers can choose only one. In other words, even when certain factors are chosen for consideration, not all possible combinations of those factors are considered—only some of them are. It is rather like how, the moment you choose a “decaf Americano,” you can no longer choose the coffee beans.
Doing this reduces the complexity of design to a barely manageable level. But as a result, almost every digital artifact produced today comes onto the market having solved only a very small part of the problem it actually ought to solve.
As a result, those placed in marginalized situations encounter a wider variety of problems in digital environments as well. This is what always happens when the problems that need to be solved are left unattended.
The root cause: placing elements on a canvas
The existing way of designing is based on the metaphor of “placing elements on a canvas.” More specifically, it is the method of “placing static graphic elements on a canvas of fixed dimensions.” This is because existing design methods and tools are derived from design methodologies for print media.
The way we study digital design and the design software we use are all under the influence of this metaphor.
The design curricula of major universities mainly deal with theories of typography and visual design for print media. Even departments with names like “digital media design” are not fundamentally different. That is because there is still no systematic alternative theory.
The same is true of almost all design software. The largest area of the screen is occupied by the “canvas,” and the core interface is bringing something into that area with the mouse. Compared with tools used in the past (e.g. Photoshop), the tools used today (e.g. Figma) are very slowly moving away from this metaphor, but in the big picture they have still not changed.
The alternative: declaring intent
I think the way we do design and the tools we use need to change from the old “placing elements on a canvas” approach to “declaring intent.” Let us look at the difference between the two.
In the early days of web design, quite a few websites had phrases like “This site is optimized for 1024x768 resolution,” and in reality, if the user’s browser size did not match that resolution, it was difficult to use the site properly.
But what is 1024px? Suppose width: 1024px is set in the style of the HTML element containing a blog post. What does it mean for the width to be 1024px? It is a number written on the assumption that the user’s monitor width will be 1024px and that the browser will fill the screen, with the intent that, when such a user visits, the line length of the body text will fill the width of the screen.
But what if the screen is smaller than 1024px? Horizontal scrolling appears, and reading becomes deeply inconvenient. If so, then in fact width: 1024px should be changed to something like width: 100%. That way, even if the screen becomes narrow, horizontal scrolling will not appear.
But what if the screen width is not 1024px but much wider than that? Then the line length becomes too long, making the text hard to read. If so, then in fact width: 100% should probably be changed to something like max-width: 960px. That way, even if the screen gets wider, it prevents the line length from becoming too large. In this situation, max-width: 960px expresses the designer’s intent better than width: 100%.
But where did the number 960px come from? It was probably chosen in a situation where the default font size was 16px, with the hope that the line length would not exceed an appropriate level (max-width: 960px should be changed to max-width: 60rem. That is because even if the user increases or decreases the default font size, the intent (limiting line length in proportion to font size) is better reflected.
(Note: 60rem is not “60 characters.” If what you want is to directly limit the number of characters that fit on one line, you should use ch, which uses character width as the unit. Since the average character width is approximately 0.5em, 60ch corresponds not to 60rem but approximately to 30rem.)
But why is “width” important? Because it assumes that text is written horizontally, from left to right. In a culture where text is written vertically, “line length” is connected not to “width” but to “height.” If so, then in fact max-width: 60rem should probably be changed to max-inline-size: 60rem.
If width: 1024px expresses the design decision “set the width to 1024px,” then max-inline-size: 60rem expresses the design intent “taking into account the direction in which text is written, do not allow line length to exceed 60 times the default font size.” The former is closer to “placing elements on a canvas,” while the latter is closer to “declaring design intent.”
In the example above, I explained only in terms of the length of an element, but it can be applied to every aspect: the position of elements, the spacing between elements, movement, color, sound, vibration, and so on. For example, one could imagine a method where, if the designer specifies a brand color and declares an intent, colors appropriate to that intent are automatically derived from the brand color, and hue, saturation, lightness, and so on are adjusted to suit their purpose so that background colors or text colors are assigned on their own.
Some design tools such as Figma are adding features like variables and auto layout, but there is still a long way to go. CSS too is gaining more ways to express intent better, with the addition of logical properties and color-related functions, but it is still incomplete.
Static design checkers, design runtime, integrated design environment
To make the transition from “placing elements on a canvas” to “declaring intent,” I think the concept of a design runtime is necessary.
In the “placing elements on a canvas” approach, most of the design is realized while the designer is editing the design file (that is, at design time). Some parts of the design (dark mode, landscape viewing, large text, reduced motion, high contrast, and so on) are realized at the point when the user uses the artifact (that is, at runtime), and if the user selects a combination the designer failed to consider at design time (or decided not to consider because of business constraints), the design breaks. Or else such a combination cannot be selected at all.
In the “declaring intent” approach, the designer would focus on explicitly stating their intent. A static design checker would verify whether the intents explicitly stated by the designer contradict one another in terms of aesthetics or in terms of usability from the perspective of universal design, and whether they are realizable across all possible combinations. A design runtime would take as input the designer’s intent, the user’s characteristics and preferences, and the context the user is in, and realize the design in real time:
I imagine something like this:
- A declarative language, improved from today’s CSS, that reduces elements meant to express design decisions and can better capture design intent
- A static design checker far more advanced than today’s linters
- A design runtime somewhat more advanced than today’s browser rendering engines (in the sense that it decides and computes more things at runtime)
- A design theory that systematically deals with concrete ways of specifying design intent, and with the process from the declaration of intent to the implementation of design
- An integrated design environment (Integrated Design Environment?) that helps people express design intent well using this language
Will declaring intent alone solve everything?
Design is not an algorithm, so there are aspects that require human intervention. To give a few examples…
- Optical alignment: When designing things like business cards, there is a need to finely adjust the spacing of individual letterforms (kerning). There is software that automates this to some extent (e.g. InDesign), but depending on the designer, it often seems not to be satisfactory.
- Simultaneous contrast: Even if one uses perceptually corrected color spaces (CIELab and so on), the human visual system is full of all sorts of tricks (see: big world hypothesis), so it is not easy to guarantee truly(?) controlled color perception. For example, even if a color is the same, it is perceived differently depending on what colors are nearby, so delicate design requires a human touch.
- Color size effect: As the size of a color patch gets smaller, it becomes harder to distinguish differences in color.
For countless reasons like these, there is still a need for the human hand to be involved. But of course, attempts to algorithmize these characteristics of visual perception do exist.
In closing
Tools have a tremendous influence on thought. Depending on how we ordinarily work and what kinds of tools we use, even the extent to which our capabilities steadily develop in everyday life will differ.
Because we use tools of the “placing elements on a canvas” paradigm rather than “declaring intent,” it is difficult for us to receive training in consciously thinking about our own intent and explicitly describing it. Sometimes, depending on the professor, students are trained to explain the intent of their work logically, but intent described in natural language is ambiguous and full of gaps in the first place. Moreover, most of the outcomes produced in school consist of almost (or entirely) static graphic elements placed on fixed dimensions, so the organic relationship between intent and outcome disappears at the point when the “work” is finished.
I think a transition to the “declaring intent” approach (I believe that is what will gradually happen) is beneficial for designers, developers, and users alike, especially marginalized users. It is even advantageous for agent-based coding environments as well.