# Finding the Mouse Cursor Somewhere on the Monitor > A rambling piece of writing on what design is. A rambling piece of writing on what [design](https://wiki.g15e.com/pages/Design.txt) is. ## Finding the cursor by moving your hand Because the visual field in the retinal periphery has lower resolution than , if you want to find where a small mouse cursor is, you have to dart your eyes around and search the screen. But instead of moving their eyes, people often move their hands and shake the cursor. That is because although peripheral vision has low resolution, it is excellent at detecting motion. People naturally do this sort of thing even without separately studying peripheral vision, eye movements, or how the brain's visual pathways work. So many people shake the mouse to find the cursor that even implemented a feature that enlarges the cursor when you shake the mouse quickly. This feature helps cursor-finding in two ways. - The change from a small cursor to a larger one is itself motion (motion of expanding through a change in size), so it stimulates peripheral vision. - When a large object moves, it is more easily detected in peripheral vision than a small object. ## Ecological assembly and epistemic action When people shake the cursor, what have they done? They moved their hand in order to reduce eye movement. In other words, they traded eye movement for hand movement and accomplished the task of "finding the cursor" more efficiently. In his <2008> book [Supersizing the mind](https://wiki.g15e.com/pages/Supersizing%20the%20mind.txt), [Andy Clark](https://wiki.g15e.com/pages/Andy%20Clark.txt) called this kind of act [ecological assembly](https://wiki.g15e.com/pages/The%20principle%20of%20ecological%20assembly.txt). > If a canny cognitive agent can produce an acceptable result with a minimum of effort, it tends to recruit, on the spot, whatever mix of problem-solving resources is needed. …the canny cognizer tends to recruit, on the spot, whatever mix of problem-solving resources will yield an acceptable result with a minimum of effort. In the traditional view ([the information-processing view of cognition](https://wiki.g15e.com/pages/Information%20processing%20theory.txt)), humans take in information from the world through the sensory organs, perform the appropriate computations in the head, and then move the body to carry out action. From that point of view, the movement of the hand counts as "action." But in the example above, the movement of the hand is closer to perception than to action. The reason for moving the cursor was not that one wanted to move it somewhere, but that one wanted to help bring about the cursor's discovery (that is, perception). This kind of action is called . Traditional action, in order to distinguish it from epistemic action, is called pragmatic action. In the <1980s>, as "[user-centered design](https://wiki.g15e.com/en/pages/User-centered%20design.txt)" became a major theme, the idea became widespread that "users must not be forced to perform complex actions." But I often encounter distorted versions of this idea, such as "users are incapable of complex actions" or "if users are performing complex actions, then it is not good design." Good design should not force complex action, but it should still allow, or sometimes even encourage, complex action when needed. That is because there are situations in which usability improves precisely because complex action is permitted. Users are clever. It is only unintelligent design that constrains the cleverness of users. Users are not idiots who passively use products made for them by elite designers. Good design must be able to make the user's cleverness itself part of the design. In other words, we need design that takes as much care as possible to let users make use of a variety of ecological assemblies. ## Information self-structuring and cognitive niche construction Humans transform the physical/social environment through epistemic action, and through that they increase cognitive efficiency. For example, they do things like these. - When playing poker, rearranging the cards in one's hand to make patterns easier to recognize - When doing a puzzle, starting after sorting together pieces with similar (or corners with other corners) - Lining up cooking ingredients in order - When putting Post-its on a wall, placing related contents near each other () - When playing Tetris, rotating blocks this way and that (the purpose is not really to rotate them; it is that mentally rotating them is difficult, so one moves one's hand instead) - A bartender laying out in advance, in order, the glasses corresponding to the kinds of cocktails that have been ordered (by looking at the glasses, the bartender can tell what needs to be made next and how backed up the orders are) When, as above, one performs epistemic action in order to change the form of the information spread out before one's eyes for one's own perceptual convenience, this is called [information self-structuring](https://wiki.g15e.com/pages/Information%20self-structuring.txt). In other words, sometimes the information one needs is present in the world spread out before one, but not in a form efficient for perception, or else the needed information is not present before one's eyes at all. In such cases, one uses the body to appropriately rearrange the information laid out in the world so as to improve perceptual efficiency (rotating in Tetris), or even to create the information outright (cocktail glasses). In , acts such as spiders spinning webs or beavers building dams—changing the environment they inhabit into a form favorable to survival and reproduction—are called . When humans improve their own cognitive environment through information self-structuring, that behavior is called [cognitive niche construction](https://wiki.g15e.com/pages/Cognitive%20niche%20construction.txt). Humans construct cognitive niches by transforming the spaces where they live and work in various ways. For example, they make use of the surrounding environment by sticking various Post-it notes next to a work monitor, or by stacking binders and document files in their own way (piling, not filing). And if we extend this beyond artifacts like Post-its to social relations as well—asking teammates questions or requesting help—the importance of context or environment becomes even greater. From this perspective, active interaction with the environment is an indispensable part of the overall cognitive process, so conducted by bringing users into an observation room is an extremely limited method, rather like removing a spider's web and then trying to observe the spider's ecology. That is why [contextual inquiry](https://wiki.g15e.com/pages/Contextual%20inquiry.txt) matters. Good design should help users construct cognitive niches, or harmonize well with cognitive niches they have already constructed. ## The brain embodied in the body, the body embedded in the environment, and the active role of the environment From this point of view, if we want to understand cognitive processes, we must consider together the body that contains the brain (embodied). And it also matters that the body is in turn situated within some physical/social environment (embedded, situated). That is because body and environment play important roles in cognition. Finally, the environment does not merely exist passively; it plays an active role in the overall cognitive process (). Where the traditional view saw the brain as the core cognitive subject, the body as merely a conduit connecting the world and the brain, and the environment as a passive space in which the cognitive subject solves problems, the newer view sees brain-body-environment as a single cognitive system whose parts interact closely with one another. Cognitive processes do not exist only in the brain; they are broadly extended into the body and the environment. There is a good example that shows the active role of body and environment. Honda's Asimo is one of the more advanced humanoid robots (by <2000s> standards), but in terms of energy efficiency it is 16 times worse than a human. Efficiency is measured by the amount of energy consumed to move an object of a unit weight over a unit distance; for humans it is 0.2, and for Asimo it is 3.2. There is another kind of walking artifact called a . It is not powered; it is a simple model made of a basic skeleton shaped appropriately for walking, and if you place it on a downhill slope and give it a slight push by hand, gravity causes it to walk. The efficiency of passive walkers approaches that of humans. If you provide a minimal amount of power to this kind of passive walker, you can make an extremely efficient powered walking robot. The biggest difference between Asimo and passive walkers lies in where the robot's motion-control system is located and how it works. In Asimo, a complex control program and sophisticated mechanical apparatus are intricately involved in every stage of walking. By contrast, in passive walkers, the bodily structure designed suitably for walking and the gravity-bearing environment itself take over the role of a sophisticated control program. This is called . Just as movement of the hand assists visual perception, skeletal form and material, the properties of the ground, and an environment in which gravity exists substitute for the role of the control program that corresponds to the robot's brain. In this way, when analyzing or designing an extended cognitive system made up of brain-body-environment, functional distinctions are often more useful than traditional distinctions. In other words, when trying to understand the whole system, what matters is the function or role each component performs, not how each component is implemented. The approach to understanding systems from this perspective is called [distributed functional decomposition](https://wiki.g15e.com/pages/Distributed%20functional%20decomposition.txt). These perspectives are already reflected in practical design books as well. For example, revised editions of 's data visualization textbook ([Information visualization: Perception for design](https://wiki.g15e.com/pages/Information%20visualization%20-%20Perception%20for%20design.txt)) and popular book ([Visual thinking for design](https://wiki.g15e.com/pages/Visual%20thinking%20for%20design.txt)) actively incorporate Andy Clark's [predictive brain hypothesis](https://wiki.g15e.com/pages/Predictive%20brain%20hypothesis.txt). It is especially impressive how they explain the role of interaction in and from the perspective of . , too, already introduced this perspective about 30 years ago, in 1993, in his book . To do good design, rather than thinking in terms of the brain that carries out cognitive processes, the body that executes the result of computation, and artifacts or environments that interact with humans through the body, it is more useful to think of human-artifact-environment as a single cognitive system, and to think about what function or role each element performs within it (or is to be designed to perform). ## What is design? How might the story so far be connected harmoniously with existing perspectives on design? ### Larry Tesler The first person who comes to mind is . He is famous as the person who first devised the concept of "copy-paste." In <1984>, Tesler proposed the concept of the [law of conservation of complexity](https://wiki.g15e.com/pages/Law%20of%20conservation%20of%20complexity.txt). The claim is that every system contains some minimum complexity that can no longer be removed. If so, then one of the designer's important roles could also be said to be distributing complexity to its optimal location. > Every application has an inherent amount of irreducible complexity. The only question is who will have to deal with it. The user? The application developer? The platform developer? (Tesler devised this law in order to argue that Apple, as the platform developer, should handle much of the complexity so as to reduce the burden on developers, and developers in turn should reduce the burden on users, so it does not align completely with the context in which I am quoting it here.) Lee Jaeyong, former CEO of , also once left a similar [tweet](https://x.com/arangyi/status/175214542324891648). > Every problem in UI reduces to a single fundamental problem. It is a problem of allocating limited resources. Whether it is the smartphone screen, the user's attention and time, or time, space, and ability—these are all limited resources—and the question is how to allocate them, and for whom. ### Kent Beck The second person who comes to mind is . He is a software engineer and one of the founders of the [Agile methodology](https://wiki.g15e.com/pages/Agile%20methodology.txt). I am told he drew a great deal of inspiration from . Permaculture is a word combining "permanent" and "," and refers to "sustainable farming." In permaculture, it is considered important to connect the major elements that make up a system as much as possible in various ways. For example, in duck-assisted rice farming, the duck is said to play many roles: 1) weed control, 2) pest control, 3) nutrient supply, 4) harrowing and stirring up muddy water during intertillage, 5) stimulating rice plants, and 6) controlling apple snails. Around <2009>, Kent Beck quoted permaculture and said [this](https://www.infoq.com/presentations/responsive-design/). > [Beneficially relating elements](https://wiki.g15e.com/pages/Beneficially%20relating%20elements.txt). This sentence has two meanings. First, if we read "relating" as an adjective modifying "elements," it means "design is elements related to one another beneficially," and this in turn refers to the product of design. The product of design consists of various elements, the relationships among them, and the benefits that arise from them. Second, if we read "relating" as a verb, it means "design is connecting elements to one another beneficially," and this in turn refers to the process of design. The designer introduces, removes, or adjusts elements. The designer also creates, cuts, or adjusts relationships among elements. Through this process, benefits are improved. ### Connecting with Andy Clark If we connect the views of Tesler and Beck with Andy Clark's concepts introduced in the first part of this piece—[distributed functional decomposition](https://wiki.g15e.com/pages/Distributed%20functional%20decomposition.txt), [the principle of ecological assembly](https://wiki.g15e.com/pages/The%20principle%20of%20ecological%20assembly.txt), , and [cognitive niche construction](https://wiki.g15e.com/pages/Cognitive%20niche%20construction.txt)—it comes out like this. > 모든 시스템에는 더 이상 제거할 수 없는 [내재된 복잡성이 존재](https://wiki.g15e.com/pages/Law%20of%20conservation%20of%20complexity.txt)한다. 이 복잡성을 시스템에 관여하는 여러 요소(네트워크를 이룬 컴퓨터들과 네트워크를 이룬 인간들)의 제한된 부품들(메모리, CPU, 기억, 인지, 주의, 몸, 환경, 각 엔티티 간 정보전달채널의 지연, 대역폭, 정확성 등)에 가장 효율적으로 분배하고, 각 요소들을 [서로 이롭게 연결](https://wiki.g15e.com/pages/Beneficially%20relating%20elements.txt)하는 일이 디자인이다. 이 때 디자이너는 각 요소의 물리적 속성보다는 요소가 전체 시스템 내에서 [실제로 수행하는 기능 또는 역할이 무엇인지를 기준으로 생각](https://wiki.g15e.com/pages/Distributed%20functional%20decomposition.txt)해야 한다. [#ref](https://wiki.g15e.com/en/pages/Definition%20of%20design.txt#definition) ## Example: design in the age of artificial intelligence Is there any benefit to looking at design this way? I think so. For example, I have tried thinking about design in the age of artificial intelligence. There are increasing numbers of cases where AI is used in services and products, and related discussion is active in the design field as well. When designing this kind of service or application, there is also a vague anxiety about what and how one ought to study. But if you think from the perspective proposed above, it becomes possible to think a little more clearly about the role of the designer. - We need to identify the categories into which the new technologies called artificial intelligence can be subdivided. - We need to identify, for each subdivided category, what existing thing that AI technology is "functionally" replacing, augmenting, or altering. - We need to consider what AI does better than humans, what humans still do better, and what humans must do because it is **beneficial**, even if it is not something humans do better. (Reference: [value embodied in the process](https://wiki.g15e.com/pages/Values%20in%20the%20process.txt)) - Accordingly, we need to consider how to redistribute the complexity inherent in the system, how to change the **functional roles** performed by each element, and how to connect the new technology with the existing elements so that they are beneficially connected to one another. For the time being, it is likely that there will be more fields where artificial intelligence is used in the form of augmenting some functions of a system rather than replacing the whole system outright (). Therefore, methods for designing harmonious relationships among humans, human society, and artificial intelligence will become increasingly important. I hope everyone studies diligently, applies things, receives feedback, and contributes to the work of connecting the world together more beneficially. This has turned into a strange piece that starts with the mouse cursor and ends with artificial intelligence, but I will stop here for now.