Introduction to Agent-Based Models with respect to the Future of Macroeconomics

Introduction to Agent-Based Models

with respect to “The Future of Macroeconomics”

Gosper's Glider Gun (Wikipedia)

Gosper’s Glider Gun (Wikipedia)

Pre-Post: Thank You NYC BlogoTweetosphere

I’m such a spoiled kid. A few months ago I was planning a long labor day weekend in NYC, so I sent out requests to a few of my heroes in the economics, finance, and math blogosphere to meet me for a lunch and/or a dinner. It was amazing getting to meet and interact with people who have been such huge influences on things-I-like-to-think-about — having been reading some of them for over five years now. Sometimes these situations can end poorly, but I am lucky enough to only have increased admiration for everyone I met. I’d like to thank (and plug) the following heroes for showing up and teaching me a thing or two about a thing or two:

I’ll be back soon. Let’s meet up again.

Now for the good stuff…

Rumble in the Econojungle

In sequence:

For those of you just tuning in, I’ll attempt to sort out this clüsterfüken shortly. For now it’s important to know that the heart of this debate is over what an ABM is or isn’t, and what economic researchers can or can’t do with them. And here everyone gets it slightly wrong. But everyone also gets it mostly right. After a few minor corrections, I believe all differences will be reconciled. However, in order to understand these minor differences we have to be able to see things through the lens of a programmer — the person who actually understands and decides how an ABM functions. After all: no software, no ABM.

My perspective on this topic comes from my work where we have designed, coded, and implemented an ABM framework which we use (and improve, and debug) daily to both conduct research and actually trade in various markets. We’ve had a surprising amount of success modeling microstructure and getting simulations to reconcile nearly spot on with reality. It’s pretty damn cool. In my academic spare time over the past few years, I’ve spent much of it thinking through abstractions of the various ways people implement ABMs, trying to arrive at more generalized forms — and then trying manipulate the properties of those forms to discover some interesting things about ABMs themselves.[1] Perhaps on some far future day when I’ve automated myself out of this job, I’ll join the others searching for sexy, macro-truths.

In the Beginning: Life, Conway’s Game of

Most broadly, an agent-based model is a computer simulation where things do stuff in an environment. Slightly more specific, the things are called ‘agents’ and are a set of unique separate entities — e.g. cars as agents to model traffic jams, or cells as agents to model tumor growth. The environment is some bounded region, like a grid or a terrain, where the agents do their stuff. The stuff can be anything, or perhaps even nothing — an agent can exist just to take up space (sort of like my blog posts, self burn). Perhaps the most ubiquitous of all possible stuffs is “living” and “dying”. This idea was introduced in what is commonly called the original ABM: John Conway’s Game of Life.

Though not referred to as an agent-based model at the time of its creation, Conway’s has become the universal go-to for explaining what an ABM is. My first encounter with it was in my AP Computer Science class, taught by the amazing Chap Percival, during an introductory lab on simulations where we actually got to code it in. Both House and Sethi refer to Conway’s at various points to make their respective cases about ABMs — which is great because we can apply the structure of the game in several contexts to see where they are each uniquely mistaken.

For nostalgia purposes I’ll quote from my original textbook, c++ for you++ (actual name):

The Game of Life is a simulation game introduced by British mathematician John Conway. It became popular in 1970, when Martin Gardner brought it to the attention of readers of Scientific American. The simulation takes place on a rectangular grid. Each cell of the grid may be “dead” (vacant) or “alive” (occupied by an “organism”). The initial configuration of alive cells goes through a series of generations. In each successive generation, some alive cells die and some new cells are born in vacant places depending on the total number of alive neighbors of the cell.

Putting this in the context of my ABM description:

  • Environment: rectangular grid
  • Things/Agents: a single grid-cell “organism”
  • Stuff/Behavior: live, die, or get born

Every ABM must have rules which govern the behavior of the agents.

The “births” and “deaths” follow the following rules:

  1. A neighbor of a given cell has a common side or corner with that cell. Each cell inside the grid has 8 neighbors.
  2. An alive cell with two or three alive neighbors remains alive in the next generation; an alive cell with less than two alive neighbors dies (of loneliness); a cell with four or more alive neighbors also dies (of overcrowding).
  3. A vacant cell becomes alive in the next generation if it has exactly three alive neighbors.
  4. All births and deaths take place at exactly the same time, so that the change from one generation to the next is instantaneous.

Based on the starting configuration of the grid, commonly referred to as the “initial conditions”, with each iteration (of “time”) cells will either die, get born, or remain unchanged. This procedure typically takes place until either (1) the program terminates because no cells can change according to the rules, or (2) the same pattern repeats itself ad infinitum. That’s pretty much all there is to it. It’s important to note that the rule-book doesn’t specify any particular mechanism for how the cells exhibit their behaviors. Sure, it notes that they might die “of loneliness” or “of overcrowding”, but when it comes time to actually program these rules, there are many different paths which lead to implementing the same rules.

Cat Skin Triptych

“There’s more than one way to skin a cat,” was frequently heard in my early programming classes. If you can get past the literal meaning, the figurative translation is that there are many different ways to write code which achieve the same result. Despite seeming obvious, this has profound implications for how ABMs can behave. Specifically: they can do anything you want them to do. Unfortunately, the economists in the debate clearly do not understand this. Both House and Sethi make horrible assumptions unnecessarily restricting the behavior of ABMs. This is likely a result of them being restricted to using the tools they have, rather than being able to make their own. But life is easier when you get to make the rules. Let’s go through some different implementations.

Environmentally Driven

When discussing the differences between ABMs and DSGE models, Professor House writes:

Probably the most important distinguishing feature is that, in an ABM, the interactions are governed by rules of behavior that the modeler simply encodes directly into the individuals who populate the environment…

A second key difference is that the interactions are often restricted to individual (or at least limited) interactions…

Not so fast. The environment doesn’t need to be inert. It can affect and drive the agents just as much as agent-to-agent interaction. There doesn’t actually need to be any interaction at all.

Consider:

  • An alive cell with less than two alive neighbors or with four or more alive neighbors gets struck by lightning and dies.
  • The environment spontaneously creates an alive cell in a vacant cell with exactly three alive neighbors.

In this implementation the rules are not encoded into any of the individuals — i.e. they don’t do anything. They are at the whim of the environment. Overall this system is just as deterministic as any other way of coding the rules. Further, a system with different types of agents isn’t bound to any specific type of interactions; the limiting factor is the will of the designer.

(House’s third point doesn’t make any sense so I ignore it)

Individually Driven

When attempting to correct House, Professor Sethi writes:

What you cannot have in an ABM is the assumption that, from the outset, individual plans are mutually consistent. That is, you cannot simply assume that the economy is tracing out an equilibrium path.

Well sure you can. Consider:

  • An alive cell with less than two alive neighbors or with four or more alive neighbors kills itself.
  • If a vacant cell with exactly three alive neighbors exists, the three neighbors get together and have a baby in the vacant cell. (don’t ask, odd species)

None of these plans make the behaviors of the cells inconsistent with each other. Perhaps each cell gains some utility by staying alive, but it can’t change its strategy. So even if a cell knows it’s going to die in some eventual iterations by giving birth to a new cell, it does so anyway. This is fundamentally no different than an equilibrium model.

In fact, studying the macroscopic behavior generated by a set of initial conditions is one of the primary interests in the game. As I previously noted, the game usually persists until no more moves can be made, referred to as stabilization, or continues ad infinitum. ConwayLife is a whole wiki dedicated to researching these various behaviors. Fun facts: the pattern R-pentomino takes 1103 generations to terminate; the Gosper’s Glider Gun at the top of this post repeats forever

Some Combination

By now you’re probably bored of hearing about different ways to achieve the same deterministic result. Fair enough; I agree. Let’s assume we can use some combination of the previous rules and it works just fine.

More interesting behaviors arise when we relax the hard rules a bit. Suppose that the lightning only has a 50% chance of killing its target. Suppose the lonely cell has a small chance of being a happy hermit. Or suppose that during the three-way-child-rearing one of the cells gets jealous and kills another. Each of these adjustments can have tremendous long term effects on the macroscopic view of the environment.

This is where the power of ABMs is really exposed. It can show us which of our rules the overall system is sensitive to. And it has the ability to show us what a deterministic system can evolve into when we add elements of indeterminism.

This Too Is Math

Functionally, there is no difference between a deterministic agent-based model and a numeric solution. Therefore, running an ABM can be thought of as providing a morphism from an ABM specification to some underlying system of equations.

So House is correct when he notes:

In fact, the first time I read Rajiv’s post, my initial thought was that an ABM with a rational decision rule would be essentially a DSGE model. All actions in DSGE models are based on private beliefs about the system. Both the system and the beliefs can change over time.  I for one would be very interested if there were any ABMs that fit Rajiv’s description that are in use today.

Yes, an ABM can be specified to represent any DSGE model.

The unfortunate reality is that the economists working on ABMs, like Sethi, have no interest in doing that. In an email conversation this past winter with Professor Robert Axtell, whose great work on systemic risk I previously covered, he was fairly adamant about eschewing the broad concept of utility in his ABMs:

[As] you probably know, we are living through the ‘behavioral revolution’ in economics and so we try not to use explicit utility functions that are then optimized. Rather, we write down empirically-grounded rules of behavior and let the agents follow these. This implicitly specifies some kind of utility function, I suppose, but that is not the way we normally think about it.

And while I agree that in the long term this is probably the way to go, you have to crawl before you can run. If the people writing down DSGE’s can be shown how much easier it would be to parameterize them as an ABM, and then be shown that they achieve the same answers, this would be a gigantic step toward mainstream acceptance.

Not only that, as previously discussed, having full control over the assumptions and rules which go into the system allows us to alter them at will. Therefore, an ABM which replicates an existing DSGE can be modified to relax certain assumptions and essentially function as a solver of the resulting system, which would have otherwise been unsolvable if not for the original assumptions.

In the end, given that ABMs can represent all canonical economic models, and given the relative ease with which you can specify an ABM, there is no reason that the language of agent-based modeling won’t become the standard for how we understand and teach economics. The economist who shuns these models is, quite literally, using pen and paper while the rest of the world has computers.

   

Footnotes    (↵ returns to text)

  1. (Nerd Splash Zone Warning) One of the more interesting results of these thought exercises is that there doesn’t ever need to be any agent-to-agent interaction at all. All behavior can be achieved by agents interacting solely with an incredibly abstracted concept of a “market.” A market is defined as an object which sends and/or receives “information” and clears according to some function. Obviously this includes our intuitive notion of a market for goods having buyers and sellers. But more generally, a market can be an object which functions merely to relay facts to agents which connect to it — e.g. something which periodically broadcasts a temperature to any agent interested in listening to it. In lieu of agent-to-agent communication, spontaneous markets can be created which serve as an intermediary to whatever direct function would have otherwise existed. While at first this might seem cumbersome, it has two incredible benefits. First, it is far more efficient to code these abstract markets because the amount of overlap between “clearing functions” is much higher than one would initially suppose. Second, it allows the modeler to study the behavior of a single agent in isolation. By abstracting away the counterparty, rigorous testing can be done for any newly intended agent behavior. The market can therefore be thought of as a vessel for which exogenous information comes into the environment. I don’t believe I can understate the awesomeness of thinking in this manner as it is for certain a higher order abstraction than any current ABM framework in existence. Unfortunately, I won’t have any time to develop this idea for at least a few years.

3 Pingbacks/Trackbacks

  • gbasin

    Good post. Isn’t that argument kind of like saying a neural network can be used to replicate an arbitrary polynomial? Sure, it’s possible, but now you have to calibrate a larger set of parameters; giving up parsimony for what exactly? If you have some proposed model of the world that you want to play with, you can always twiddle knobs and measure sensitivities there as well

    • ZHD

      Thanks!

      Yeah it would definitely be inefficient to increase dimensionality to replicate an arbitrary polynomial. Although I’d be interested to see the work of the brave people who have spent the time to do it.

      In this case, specifying an ABM to represent a DSGE would involve the same, or fewer dimensions. Numeric solutions to those models can get pretty rigorous. An ABM would actually perform the solving part, saving loads of time.

      • Zach Winston

        then do it yourself!

        • ZHD

          Like. Share.

    • muxspace

      I think it depends a bit on the goal. If you start with the premise that you want to simulate some non-mathematical behavior, it’s easier to do that with an ABM than attempt it with mathematical formulae. Zak’s argument isn’t so different from that of the machine learning community versus the statistical community.

      In your example the part you miss is that the polynomial doesn’t just exist in the abstract but was constructed to model something in the real world. So a fair analogy would question the mathematical model as a proxy for human behavior and whether that could be captured more succinctly using an ABM.

  • rdk

    On DSGE vs basic ABM: Lucas Critique.

    I’ve often thought of the point of the utility specification in a DSGE as not so much about how people actually make decisions, as that we can think about how those decisions change if we change the economic policies. So the relation between ABM and DSGE would then become more like: ABM is a DSGE with a fixed policy.

    (This criticism is probably only relevant to basic ABM models, I’m not familiar enough with ABM literature to know. But it illustrates why it is important for much of economics to model the actual decision process giving rise to the behavioural rules. Namely, economists are interested in the effects of changing economic policies.)

    The utility specification itself seems best thought of as just a modelling device to capture peoples behaviour, rather than explain their actual motives. This is related to things like Sonnenschein-Mantel-Debreu theorem that says that for any static outcome we can ‘explain’ it as the outcome of some perfectly competitive market equilibrium. I think I recall reading in some article by John Rust that dynamic versions of this exist: that we can justify any observed individual behaviour as being the outcome of a rational expectations utility maximizer.

    • ZHD

      Thanks for the comment! Reading your relation and then re-reading House’s description of DSGE makes me think I slightly misspoke.

      Here’s House: “All actions in DSGE models are based on private beliefs about the system. Both the system and the beliefs can change over time.”

      This would certainly qualify as an ABM. The policy doesn’t need to be fixed.

      Sethi had mentioned that it’s possible to have agents whose behavior is based on their solution to dynamic programming problems. He’s right! In our framework we do have agents which work on dynamic programming problems that we have been able to actually take out of the simulation environment and put into the real world.

      Thanks for the reading material. Looks like I need to do some homework.

  • Pingback: De novo, ABM | Gabriel Rega()

  • Pingback: A week of links | EVOLVING ECONOMICS()

  • ZHD

    Thanks for the comment!

    I think I may have taken some liberties in assuming that a highly generalized ABM framework would exist which would make the “coding up” of a specification trivial. Obviously one does not exist at this time. However, solving a pen and paper model would take much more time than simply specifying the individual agents.

    If the predictions from a pen-and-paper model were “testable” enough then we would have likely stopped using them a long time ago :p

    (also I apologize for it taking so long to get your comment posted up here. I don’t know why my disqus console isn’t properly notifying me of pending comments)

  • ZHD

    Thanks again for the thoughtfulness! You bring up so many good points I wish I had you standing over my shoulder when I was writing the piece.

    From a computational theoretic perspective, there can definitely (as far as current knowledge goes) be system-wide behaviors that cannot be represented at the individual level. My usage of Conway’s in this manner is perhaps unsatisfying to you because the end result is the same — analogous to two Grammars generating the same Language. But I don’t think it’s very difficult to think of an environmental rule which both helps us reflect reality and is also inappropriate to embed in an individual agent.

    “No ABM intended to reflect reality would include an environmental factor that enforces a priori consistency, because it couldn’t pass a laugh test.” I can say for certain that this is false. Imagine a realistic model of a trading exchange (the environment) which permits asynchronous order entry and activity until some ‘closing bell’, after which orders are halted and then as a result of the macroscopic behavior of all traders the environment assesses a margin rule and applies it to all participants before the next ‘opening bell.’

    I think Sethi and the other economists working on ABMs want to reflect reality. But they are missing key steps that all production worthy software goes through before replacing the legacy code. Mainly model validation. It’s so very easy to get swept away with the amazing potential of this style of framework that they’ve completely forgotten to prove that it encompasses the existing models. As such, there’s simply no reason for us to believe that these models actually do what the creators claim they do.

    I can’t fault them too hard for this, because I’ve been there myself. The few times I’ve wanted to replace the underlying framework for how my adaptive agents behave to something more robust, I’ve hit the ground running looking at all the new cool stuff I can do with them. Inevitably a teammate demands that I prove that this new framework is AT LEAST as good as the current production code. This involves a comprehensive series of tests.

    The economists working in ABMs have simply not done this validation. We cannot accept their code until then.

    As far as me saying House was correct, he was saying (ABM base) + (some rule) = DSGE. Perhaps I should have been more specific about what else he needed. But I think from the rest of my post that I don’t assume coordination is necessary.

  • Pingback: Introduction to Agent-Based Models with respect...()