Premature UI Design Doomerism

I have been hearing the claim that there is nothing new left in HCI or the UI field from time to time for the past 15 years, and these days I hear it more and more often. There is also the claim that so-called “thin AI wrapper” products that merely put a slight UI layer over an LLM API have no competitiveness. It is the sort of thing VCs have been going on about over the past few years. There is also the claim that now that the age of AI has arrived, UI designers and frontend developers are no longer needed. I do not agree with these three claims, which are connected to one another. I am not trying to tell the trite story that there will forever be roles only humans can perform and AI cannot.

The claim that there is nothing new left in HCI/UI

The current GUI paradigm was established in the 1970s. More than 50 years have passed, but the interfaces of products and services still remain within this paradigm. Of course, research presented every year at conferences such as SIGCHI shows that various attempts have steadily been made, but it is hard to say that a paradigm shift in Kuhn’s sense has taken place.

PARC

In 1974, a small project group at PARC called “Applied Information Processing Psychology” organized a framework for studying and evaluating how humans handle computer interfaces. The book containing their research is The Psychology of Human-Computer Interaction, published in 1983.

A brief summary of the situation at the time is as follows.

Their research was based on cognitive psychology, which was then the latest theory in psychology. But at the time only 20 years had passed since the cognitive revolution(1950s), and cognitive psychology, still a newly emerging discipline, was heavily tilted toward the so-called “information-processing model.”
The computers they were studying at the time were extremely crude, and the “desktop,” used by placing it on a desk, was the smallest form of computer. The “laptop,” used by placing it on one’s lap, was an imagined machine in the early 1970s (for example Alan Kay’s Dynabook), and only began slowly to be commercialized in the early 1980s.

The current situation

But over the course of 50 years, research on humans has advanced, and computers have advanced as well.

In the 1970s, James Gibson’s ecological psychology began attracting attention in academia, and after the 1990s, embodied cognition began to emerge in earnest. It became self-evident that the traditional computationalist human information-processing model of “perception-information processing-execution” could not properly explain actual human behavior.
Computers also advanced astonishingly: portable computers (PDAs and smartphones), wearable computers(smart watches and so on), and virtual reality or augmented reality devices (such as Meta Quest and Apple Vision Pro) have been widely commercialized.
Around Google’s word2vec in 2013, the field of natural language processing suddenly began developing rapidly, and in 2022 ChatGPT was finally released. ChatGPT set a record by acquiring 1 million users within five days of its launch. For the first time in history, computers can now interact “like humans,” and that performance is improving extremely quickly. (Interestingly, LLMs and embodied cognition share artificial neural networks or connectionism in common.)

Our understanding of humans has advanced greatly, and both computer hardware and software have advanced greatly, so it makes no sense to say there is nothing new to be said about interfaces and interactions connecting humans and computers.

Rather, now is precisely the right time to begin new lines of thought.

The claim that products with only a UI layer have no competitiveness

People say that products with only a lightly added UI layer have no competitiveness. In the past few years, I have often heard criticism of “thin AI wrappers” that simply call the OpenAI API or take an open-weight model like Llama and merely dress it up with a UI.

Good packaging has great value

But the emphasis of the criticism should be placed not on “wrapper” but on “thin.” That is, the problem is not that it is wrapped, but that the wrapping is “too thin.”

People who have never thought seriously about interfaces often call the interface “packaging” and disparage it, but packaging matters. And a good interface is not “thin” packaging. There have been countless cases where differences in interface were the core competitive advantage of a product, and this fact does not change even if the substance of that product contains AI. (That is why I prefer the phrase “AI-infused services” to “AI-based services.”)

Sometimes I wonder whether people have become so overimmersed in AI that they have come to regard all the other components besides AI as trivial. From the perspective of making products, AI is just one ingredient among others. It is true that it is a relatively recent ingredient, and a good one that enables things that were not previously possible, but that does not mean everything other than AI becomes insignificant.

Let us also briefly think about AI in the role of helping the process of making products, rather than as an ingredient of the product. What happens if, as people predict, software starts pouring out thanks to generative AI? Humans, as always, will seek products that are easier to learn, easier to use, and more enjoyable to use. “Good UI” will remain a core competitive advantage of products and services, as always.

If we just have a chat interface, can that do everything?

Some people say that from now on, since everything can be handled just by conversing in natural language, all existing UI will become unnecessary, but this too is a mistaken claim. Because of the huge popularity of ChatGPT, when people hear “AI” they think only of “chatbots,” but natural-language-based conversation—whether text-based or voice-based—is only convenient in some contexts and has clear limitations if used for every situation.

For example, in situations that require precise instructions, a method of communication based on natural language alone becomes extremely inefficient. What would it feel like to do precise design only through speech? “Please change the icon of the third button among the buttons in the top-left toolbar to red. No, a slightly darker red. No, this is too dark.” Rather than that, it is much easier and simpler to click the icon and say “change the color,” and then have a color-selection UI appear. The greater the precision required by the task, the clearer the limits of natural language become.

If AI agents will do everything, do we not need other interfaces?

Some argue that it is fine because AI agents will take care of all the trivial tasks for us, but for some kinds of work, I have to see the “process” itself. For example, many designers do not begin design work with a “completed design” already fully formed in their heads; rather, they gradually increase the completeness of the design through the process of doing the design work. In this kind of work, omitting the “process” is impossible. That is because the value contained in the process matters. Work of this kind cannot be handed over to an AI agent. That is because at the time of giving the instruction, one does not yet know in concrete terms what the result should look like.

Is generative UI the answer?

Others present generative UI, created in real time according to the context of use at each moment, as the vision, but for this approach to succeed it will require a tremendous amount of research. For reference, experiments with “adaptive UI” that changes on the fly have been around for a long time, but most of them failed. That is because if the UI keeps changing, users become unable to form habits around that UI, which instead increases cognitive load. This is not to say that generative UI is the wrong direction, but that there are obvious problems that need to be solved. Without solving those problems, it is difficult to discuss generative UI seriously.

To sum up, good UI remains, as in the past, a core competitive advantage even in the age of AI.

The claim that UI designers and frontend developers are no longer needed

Now that coding agents can make UI for us, I also often hear the claim that UI designers or frontend developers are no longer needed. I suspect this claim is based on an image something like the following.

Software production is a kind of production line.
If you put the raw material, the “business idea,” in at the left end, finished software comes out at the right end.
On the production line, from left to right, are planners, designers, developers, and testers.
Designers and developers are bottleneck resources, so finished products do not come out quickly.
If designers and developers are replaced with AI, the bottleneck is removed and finished products come out quickly.

To some extent, that image is valid, in that design and development (especially development) have always been bottlenecks. But we need to think about what happens after this bottleneck is removed.

Looking at the current state of affairs, there are broadly two problems.

Because of the bottleneck, the world currently has far too little software. The world needs far more software than it has now.
Even the software that does exist is extremely backward and even barbaric (problems such as failing to sufficiently consider accessibility or universal design).

In the end, we need to produce much more software than we do now, and of much higher quality than we do now. And as we saw above, this is a time when we need to think about a new UI paradigm, and a time when good UI can be one of the core competitive advantages.

One of the fundamental causes of the bottleneck is that humans still have not found a good way to manage the complexity of software development (the history of software engineering is only about 60 years old). In particular, the complexity of the UI field is an enormously serious problem—so serious that the major themes of UI design are not yet even addressed from an engineering perspective. In other words, within the framework of software engineering, we are not even properly capturing a definition of what problems need to be solved.

Current software is released into the world in an extremely compromised state, and as a result the barbaric situation is reproduced and amplified in which those in marginalized situations end up suffering relatively more problems. In other words, people need to make money from software, but they do not dare try to solve all these complex problems (or they do not even recognize what the problems are), so they are just making things however they can and selling them however they sell. (See also: Declarative Design and the Design Runtime)

If one seriously thinks through the changes that will occur after the current bottleneck is removed, the idea that UI designers and frontend developers are unnecessary seems shortsighted.

Will humans be needed forever?

My argument so far stands on the premise that “if AI replaces the work humans are doing now, humans will be able to solve bigger and more complex problems.” But is that really so? What if AI can solve those “bigger and more complex problems” better too?

I believe that someday that will be the case. Perhaps that time may come sooner than we think. AI has already begun carrying out scientific research in our place, and when it comes to solving mathematical problems that require complex reasoning, AI is much better than most humans. I think AI will probably also be good at tasks such as discovering a new UI paradigm and finding ways to manage the complexity of software development. Of course, if that happens, it is obvious that similar effects will be felt not only by frontend developers and UI designers but by most developers.

That said, that is not the case right now. Right now, people are needed. I am not trying to make the romantic normative claim that “there must be something only humans can do”; I mean that, realistically, they are needed right now. I do not know what will happen next year. In this field right now, it is just too hard to predict one year ahead.

If, in the future, a world arrives in which all production takes place without human labor, and if that world is one in which distributive justice has been realized (this is a difficult and important problem), then humans could be completely liberated from alienated labor and simply live while pursuing their own values, so that too would be a good thing. Still, it does not seem very likely that merely resolving inequality of capital would automatically produce a utopia. (See also: Consciousness Capacity in the Age of AI)