Soft Assemblies



Robotic Agency & Non Hierarchical Fabrication


May 2020
Nathaniel Banks, Yidian Liu, Emmanuel Osorno, Yinong Tao
Advisor:  Stefana Parascho
           




Introduction



This paper describes research that addresses the agency of fabrication tools in responding to disruptive agents in continuous fabrication processes. Specifically, the research explores how robotic fabricators can operate using real world sensors in order to flexibly adapt bricklaying construction processes to unforeseen real world disruptions, notably disparities between digital environments and real world topographies. It posits a non hierarchical approach to robotic fabrication that moves away from linear progression and towards ‘soft system’ operations as defined by S.Kwinter.


“A system is ‘soft’ when is flexible, adaptable, and evolving, when it’s complex and maintained by a dense network of active information or feedback loops, or, put in a more general way, when a system is able to sustain a certain quotient of sensitive, quasirandom flow” (Kwinter 1992).


Within this understanding, a resulting artifact is not a separate entity from the process that produces it: the whole workflow is envisioned as an integrated system that adapts and evolves according to design intentions, fabrication constraints, performance criteria, material behaviors and specific site conditions.



Hierarchical Fabrication


Current methods for architectural production are based on notational and geometrical representation, where only what can be drawn and measured can be built (Carpo 2011). Moreover, these methods are usually organized in a linear progression from design intention to materialization, which impedes any feedback among the different stages of the realization of a project. These processes operate with the intent of producing a singular desired outcome, often based on replicating a digitally generated global geometry in reality. The processes follow a strict hierarchy, in which the robot is initialised with an input. The robot then subsequently applies a set of processes to the input to produce a predefined target geometry, upon which the processes terminate. This hierarchical approach has several major limitations. Firstly, they cannot adapt to passive real world disparities. If a robot is set to lay a set of bricks on a flat surface, it cannot adapt to an uneven one, often causing the script to fail in real life. This same issue also applies to active disruptions. In the event that something knocks down a part of a wall, the robot will continue to lay bricks as if nothing had happened, also resulting in a script failure. The presence of a predefined target geometry as a singular goal for hierarchically organised scripts inherently hinders their flexibility in accounting for real world disruptions. Therefore, while hierarchical systems operate successfully in controlled and static environments like factory floors, their inflexibility severely inhibits the design and realisation of complex assembly systems in dynamic and volatile environments like on site construction yards.






Non-hierarchical Fabrication & Soft Systems




The shift away from top-down hierarchical approaches in favour of attributing design agency to feedback between material behaviors and robotic operations is embodied in the recent work of architects such as Adam Fure and Matias del Campo (Fig.1-1). 


Fure describes his research as moving away “from optimization and efficiency as the primary drivers of digital fabrication in pursuit of a model where materials and fabricators assume maximum agency in the fabrication process” (Fure, 2014). Similarly, Achim Menges has argued that embedding material & environmental characteristics, manufacturing constraints and assembly logics allows a design to be driven through intrinsic performative capacities rather than through hierarchical relationships that prioritize form over materialization (Menges, 2011). 


For the purpose of our research, we will be focusing purely on the role of topography, and how various unknown terrains can influence the construction processes of assembly structures. We chose to use the established robotic construction process of brick assembly for our research, as the brick module offers minimal material variability. This allows the research to concisely focus on the effects of environmental disruptions and rapidly develop a ‘soft’ assembly system rather than simultaneously account for material behavioural variables. Our structure will be built upon an unknown topography. In order to adapt our construction to this unknown topography, we will be utilising a distance sensor to harvest point data at each layer of construction. The aim of this exercise is to use this live data as part of a feedback system that influences how the fabricator chooses to place each brick. Unlike a hierarchical system, this will allow the robot to influence how the final assembly may appear based on construction processes responding to real world stimuli, prioritising flexible materialisation over the production of a specific form.



(Fig.1-1.  Material Worlds : Wire Dyepitch, Adam Fure, 2014)









Context

Environmentally reactive robotic strategies, within which this project finds precedence, affirm the importance of observation and sensing procedures in contrast to symbolic representation (Brooks 1990). Pure reactive systems are exclusively based on sensory information with no internal representation of the world, yet they are able to generate even complex behavioral responses responding to a set of stimuli or conditions (Matarić 2007)


In recent years, a series of pioneering projects introduced robotic fabrication procedures based on sensor-actuator feedback loops into the field of robotic architectural production. In 2012, K. Dorfler and R. Rust, with the project “Interlacing,” presented a digital interface equipped with a network of sensors that allowed on-line control of the robot (Dorfler and Rust 2012). Advancing this research further in 2015, Lauren Vassey and Giulio Brugnaro presented the project “Robotic Softness,” in which they developed a soft system for weaving structures from tensioned wood. Due to the woven systems of wood continually deforming the shape of any woven structure, the constructions by Brugnaro and Vassey could not have been achieved through a predictive global geometry, and necessarily utilised “real-time physical sensing and computational analysis, material monitoring, machine learning and continual (re)construction” (Menges, 2015). This responsive method of construction can potentially significantly alter how structures can be practically constructed on site, wherein changing constructional conditions can be flexibly accounted for by robotic fabricators. This represents a significant paradigm shift away from robotic fabrication as a precise and predefined methodology, towards ‘cyber physical making’ (Menges, 2015), wherein the robotic fabricator can enact decision mechanics analogous to how human construction workers decide to adapt construction assemblies to account for unforeseen circumstances or late design changes.


Within the 2017 project “Closeness,” Roland Snooks & Gwyllim Jahn presented a reusable thermoplastic wing molding system that operated by evenly extruding thermoplastic over an uneven topography. While not designed to operate as a soft system, by using a distance sensor to constantly gauge the optimal distance to extrude thermoplastic over the mould, this system could crudely adapt to changes in topography based on real world feedback rather than prescribed paths. This makes the project highly relevant to our research, not only in using a distance sensor as an active feedback tool, but also in developing a robotic construction system that accounts for environmental unknowns.


Unlike many of the precedents mentioned, our research is not concerned with material constraints determining how a robot chooses to adapt construction processes, but rather on environmental dynamics. This we consider to be crucial in the applications of soft robotic constructional systems in built architecture, as it pertains to the various unknown and changing dynamics of building sites. These dynamics can include uneven site topographies, as well as disturbances to construction from inclement weather and accidental damages. In designing a prototype soft system that can account and adapt to these environmental disturbances, robotic fabrication may become more commonplace in active construction rather than be limited to off site prefabricated assemblies.




(Fig.2-1. Robotic Softness, Giulio Brugnaro & Lauren Vassey, 2015)



(Fig.2-2. Closeness, Roland Snooks & Gwyllim Jahn, 2017)



(Fig.1-4. Site Construction)











Methodologies

1. Overall Procedure

The interactive fabrication of bricks consists of two parts: sorting the order and placing the bricks (Fig.3-1). These two parts allow the robot to gain an “overview” of the base terrain and always start stacking bricks from the lowest topographical point. Following traditional brick construction, the fabrication proceeds in layers. However, before stacking each layer, the robot will detect the vertical distances to the terrain along a scripted path and sort out an order based on the value of the distances. Based on this sorted order, the robot will pick up a brick each time and place it on the according location of the terrain. After all bricks have been placed in a layer, the robot will move to the next layer and start the sequence again. Any bricks placed in previous layers will feed into the “new terrain” for the next layer. Therefore, the robot not only follows a predefined fabricational rule, but also examines the latest terrain condition led by previous actions.




(Fig.3-1. Pseudocode Process Diagram)





2. Equipment / Set Up

a. Construction Module: Foam Brick


The brick module was chosen due to it having both simple material characteristics and established methods for robotic aggregation from which to develop a soft system. The dimensions of the brick are 200mm by 100mm by 50mm (Fig.3-2). The material for the brick would be foam, making it lightweight, easier for the robot to place, less likely to damage the topography and cheap to purchase and cut to shape.


For our assembly, the bricks will be arranged in layers following a stretcher bond pattern (Fig.3-2) to maintain structural stability as more layers are assembled.





(Fig.3-2. Brick Module)




b. Feedback Data Provider: Distance Sensor



In order to gather data from the unknown topography, an optical laser distance sensor will be applied to the Schunk Parallel Gripper mounted to the robot’s Tool Center Point (TCP) attachment manifold (Fig.3-3). As the sensor head does not align with the brick placement within the gripper, a distance (x) will need to be subtracted from any height data gathered by the sensor so as to accurately determine brick placement. The tcp will move laterally by 12 cm following topographic depth measurements to align the center of the gripper with the distance sensor measurement location before placing a brick.

When placing a brick, the distance sensor will take 3 measurements along multiple axes (Fig 3-4). Using only one measurement will give you an approximate topographic depth, a second will give a linear gradient. A third measurement along a different axis can be used to establish a directional plane. This plane can thereby not only tell the robot about where to place a brick, but also how to approach the topography and place a brick appropriately to its multiaxial gradient.




(Fig.3-3. Distance Sensor)





(Fig.3-4. Distance Sensor Analytics)









3.  Procedure Breakdown
a. Initial Scan Process



Before stacking bricks, the robot will first move along a predetermined path, stopping at evenly spaced points to obtain a list of distances to the terrain with the assistance of a distant sensor. Based on the distance values, the list will be sorted in descending numerical order to provide a new sequence for brick placement (Fig.3-5). While stacking the bricks, the robot will always move from the lower ends of the terrain, to the higher ends.


This process allows the robot to examine the latest height variations of the terrain. In the case of a local collapse or an add-on obstacle, the robot is able to detect these instant changes instead of relying on a strict scripted path.




(Fig.3-5. Distance Sensing and Sorting Points)

b. Thresholds


In order to allow the fabrication process to self-organize and work towards an even construction  field, a simple threshold formula was introduced to the script. Upon sorting the list of distances (d), the values are compared to a predetermined threshold height (Th) that restricts the number of bricks being placed, ensuring that the deepest zones on the terrain get infilled first (Fig.3-6).  At any brick centerpoint in the path, if the measured distance d is less than Th, the robot will skip that point and not place a brick. On the other hand, if the measured distance d is greater than or equal to Th, the robot will place a brick at that location. The threshold height is calculated for each layer by subtracting the depth of the brick module (50mm) from the largest d value measured (Th = dmax - 50).






(Fig.3-6. Thresholds)




C. Defining Planes using Distance Sensor
i. Initial Course
 

For the first layer of the brick laying process, the terrain will be scanned at three points that will be used to define the x and y axes of a plane, which will serve as a guide for placing the brick (Fig.3-7). All three points are derived from the geometry of the brick module. Point 1 is the center “c” of the brick, which is on the predetermined path. The second and third points are obtained from dividing the outline of the anterior face of the brick (a 200mm by 100mm rectangle) into four quadrants. This is achieved by drawing lines from the center c to the midpoints M of the rectangle. A set of two lines, perpendicular to each other, is divided in half: the midpoint m of the long axis is Point 2; the midpoint m of the short axis is Point 3. As the robot reads the depth at these three points, the script locates them in space and generates a plane tangential to the terrain. The robot will then use this plane to place the brick. The process is repeated for the remaining points in the first layer of bricks.


(Fig.3-7. Defining Planes - Initial Course)




ii. Subsequent Courses


The brick placing process becomes challenging in subsequent layers, when a brick is being placed over one or more previously placed bricks. The mortar joint implicit in the stretcher bond creates a canyon between two bricks, which prohibits the use of the center point c and the short axis (as defined in the previous section) for obtaining accurate measurements. To solve this constraint, the center point needs to be interpolated and new logic for the third point needs to be generated (Fig.3-8). Understanding that the two midpoints m along the long axis of the brick outline are over the top surface of the placed bricks, the robot gathers the depth values at these two points. The script then uses this data to locate the points in space and draw a line between them; the midpoint of this line is the interpolated center i for the brick being placed (Point 1). Point 2 can be either of the midpoints m already used. Point 3 is defined as the midpoint m of a diagonal line that connects the center point c to a corner point of the brick outline. Similar to the process for the initial course, these three points will be used to define a plane for placing the new brick.




(Fig.3-8. Defining Planes - Subsequent Courses)






d. Brick Placement Process


Following the rules above, the bricks will be placed to accommodate topographical differences defined by the terrain. For each layer, the number of bricks is variable due to culling from the height threshold. When the gradient is very steep, only a few bricks will be placed (Fig.3-9).  Moreover, since the order of placement has been sorted, each brick will be placed in sequence from the topologically lower positions to the higher ones. This prevents bricks on steep gradients from sliding into lower positions that could obstruct the assembly process. Following this method, the bricks will gradually even out topological variations, producing progressively more even brick layers as the structure continues to be assembled.

This effect is noticeable when multiple layers are fabricated (Fig.3-10). The upper layers are formally closer to the scripted path because for each layer, the terrain is getting “flatter”.


The system is also designed to be able to cope with instant interferences (Fig.3-11). For example, when some bricks are knocked down from the previous layers, the distance sensor will detect the “new terrain”, find the lower positions and fill them with new bricks. (Similar logic also applies with the condition when a new obstacle is added to the system. The robot will avoid placing bricks where the obstacle is to reduce height differences). In other words, the system aims to be flexible and adaptable regardless of the terrain condition or instant interferences.



(Fig.3-9. Brick Placement - One Layer)


(Fig.3-10. Brick Placement - Multi-layer)


(Fig.3-11. Brick Placement - Reaction to Interference)


Simulations:









Simulations became an integral part of the design process due to their ability to test order of operations, to develop distance sensor procedures, to assist in determining how to generate planes for robotic assembly, to understand the behavior of brick modules as they are being placed in the terrain, and to iterate solutions for multiple ground topographies. In this sense, the goal of the simulations is multifaceted and intended to create a feedback loop that informs the development of the methodology: the simulation is both a representation of what the script can generate and an evaluation of itself. This allows not only to construct a brick structure that adapts to an unknown terrain, but also to discover the potential moments of "failure" that arise as side effects to the flexibility inherent in the assembly process. All these concerns raise a set of criteria for assessing which inputs and outputs are crucial to include in the simulations. For the inputs, it was imperative not only to add controls for the predetermined path with an evenly spaced set of points to guide the initial placement of the bricks, but also to mimic the operation of the laser distance sensor itself. This was achieved by projectings lines from the input curve to the terrain below, measuring the distances and compiling them into a list, and then manipulating these values to inform the placement of the brick modules. In terms of the outputs, the simulation needed to provide realistic feedback of the behavior of the bricks after being placed on the terrain. For this, the script runs each brick through a physics engine to simulate the effects of gravity and friction. Even though the script cannot fully simulate, predict, and anticipate the complexities of real life conditions, it attempts to maintain the same level of control over the process, distinguishing which parameters represent human inputs and which do not. This way, the relationship between manual influence and robotic autonomy is maintained.

Three different terrains were tested: a single slope, a double sloped valley, and an undulating surface.








On surfaces with gentle, single slopes (Gif.4-1), the system produces a consistent brick bonding pattern, with a predictable behaviour. The structure is built upwards from the deepest point; bricks on subsequent layers are fully bearing on the layers below, except for those that fall at the ends of the previous layer, which bridge diagonally between the surface and the end brick; and the number of bricks being placed in each subsequent layer increases until all bricks in a layer are placed. The consistency between layers creates solid bonds and regular bridging that help to stabilize the overall structure.


On the double sloped valley surface (Gif.4-2), the system builds on the results from the single slope surface, but more nuances arise as it attempts to bridge between the two sets bricks caused by the intersecting slopes. The steepness of the slopes and the narrowness of the valley create instability on the first two layers of bricks, which amplifies and compounds as the structure climbs on both sides of the valley. However, the system is able to respond to the steep slope and symmetry of the terrain, dissolving the instabilities in subsequent layers and producing a rather consistent brick pattern which, similar to the terrain, is fairly symmetrical along the axis of the valley.


(Gif. 4-3  Undulating Surfaces)


On the undulating surface (Gif.4-3), the issues observed in the first and second iterations exacerbate the importance of adjusting the threshold height for each subsequent layer. In this case, the system responds to the multiple intersecting slopes in the terrain by slowly building up layers, correcting the unevenness of the ground through each subsequent layer until all bricks in a layer can be placed.








Results & Reflections


The research succeeded in developing a responsive brick assembly strategy for cylindrical aggregational structures on a variety of unknown topographies. The framework for the soft system, even if tailored specifically to a cylindrical structure assembly, is potentially transferable to produce other structural forms, and operate considering non topographical environmental disruptions such as suspended obstacles and unforeseen assembly collapses.


In order to generate an automated mode of brick assembly that alters brick placement to unknown environments, a custom workflow was developed utilising a culling threshold to determine both how and if a brick should be placed. Due to the limited data used to determine the culling threshold, this produced unforeseen complications in the simulations developed. Primarily, depending on the topographic variability, the culling threshold would need to be adjusted. A high threshold limit allows for the placement of many bricks, but does little to flatten uneven brick layering and often fails entirely on extreme topographies. A low threshold limit results in very few bricks being placed per layer, which can distort the assembly geometry as more

brick layers are applied. Determining an optimal threshold for brick placement that preserves the desired geometry while adequately adapting it to the landscape has so far been manually determined, with each topography requiring a different threshold limit. An integrated methodology for determining the culling threshold using distance sensor data would likely greatly improve the real world functionality of our soft assembly system and make it faster to operate, and is thereby a component we will aim to develop in the future.


Another complication is the increased tilt irregularity that occurs as more layers of brick are applied. When bricks are placed atop incomplete layers below, they land in an uneven and often tilted position. While these irregularities are manageable in lower layers, their impact on subsequent brick placements results in a cascade of uneven brick placements, leading to a greater probability of structural collapse as more brick layers are applied. To manage these irregularities, we plan to develop a tilt threshold wherein brick placements with too steep of a tilt off the horizontal axis will be avoided. Ideally, this threshold can be determined using our 3 point plane component already integrated within our existing assembly system. In addition to the complications realised through our simulations, we will likely experience further unforeseen complications when testing our assembly system in real world environments. In order to tackle these issues, we plan to continue refining and developing our system using real world topographies and ABB robots. We believe that advancing the research beyond digital simulation will be crucial in understanding how the brick modules assemble accurately in the real world, as well as address any practical complications that we may have been unable to simulate digitally (such as friction coefficients and brick release positions).




Conclusions 


The main driver of this research was the investigation of non hierarchical fabrication processes and their potential application in the field of architecture, opposing standard linear construction methods and geometrically static notational systems of design representation and construction. This expands possible methods and fabrication systems for architectural production thanks to a novel use of already existing computational tools and robotic technologies, including distance detection and online control. Through the use of environmental data feedback and adaptation, the system got closer to operating similarly to construction teams that operate in unpredictable and messy sites and often alter their workflow to overcome environmental challenges. In this regard, one of the future directions to explore from this research would be to explore the ways in which robots can continue to adapt to the environmental challenges present on building sites, thereby allowing for a reconsideration of robotic applications in ‘on site’ construction and other dynamic environments.




References


  1. Carpo, Mario. 2011. The Alphabet And The Algorithm. Cambridge, Mass.: MIT Press.

  2. Menges, Achim. 2015. “The New Cyber-Physical Making In Architecture: Computational Construction”. Architectural Design 85 (5): 28–33

  3. Kwinter, Sanford. 1992. Soft Systems. New York: Princeton Architectural Press: 211.

  4. Matarić, Maja J. 2007. The Robotics Primer. Cambridge, Mass.: MIT Press.

  5. Brugnaro, G., Baharlou, E., Vasey, L. and Menges, A., 2016. Robotic softness: an adaptive robotic fabrication process for woven structures.

  6. Snooks, Roland and Jahn, Gwyllim, 2016, Closeness: On the Relationship of Multi-agent Algorithms and Robotic Fabrication

  7. Dörfler K., Rist F., Rust R. 2013, Interlacing. In: Brell-Çokcan S., Braumann J. (eds) Rob | Arch 2012. Springer, Vienna