Add cell-centric discrete spaces (experimental) #1994

Corvince · 2024-01-23T20:51:29Z

Summary

This PR introduces an alteranative implementation for discrete spaces. This implementation centers on the explicit inclusion of a Cell class. Agents can occupy cells. Cells have connections, specifying their neighbors. The resulting classes offer a cell centric API where agents interact with a cell, and query the cell for its neighbors.

To capture a collection of cells, and their content (i.e., Agents), this PR adds a new CellCollection class. This is an immutable collection of cells with convenient attribute accessors to the cells, or their agents.

This PR also includes a CellAgent class which extends the default Agent class by adding a move_to method that works in conjunction with the new discrete spaces.

From a performance point of view, the current code is a bit slower in building the grid and cell data structure, but in most use cases this increase in time for model initialization will be more than offset by the faster retrieval of neighboring cells and the agents that occupy them.

Motive

The PR emerged out of various experiments aimed at improving the performance of the current discrete space code. Moreover, it turned out that a cell centric API resolved various open issues (e.g., #1900, #1903, #1953).

Implementation

The key idea is to have Cells with connections, and using this to generate neighborhoods for a given radius. So all discrete space classes are in essence a linked data structure.

The cell centric API idea is used to implement 4 key discrete space classes: OrthogonalMooreGrid, OrthogonalVonNeumannGrid (alternative for SingleGrid and MultiGrid, and moore and von Neumann neighborhood) , HexGrid (alternative for SingleHexGrid and MultiHexGrid), and Network (alternative for NetworkGrid). Cells have a capacity, so there is no longer a need for seperating Single and Multi grids. Moore and von Neumann reflect different neighborhood connections and so are now implemented as seperate classes.

Usage examples

Below are a few examples demonstrating the functionality of the cell spaces.

Creating a Grid and Adding Agents

from mesa import Model
from mesa.experimental.cell_space import OrthogonalMooreGrid, CellAgent

class MyAgent(CellAgent):
    def __init__(self, unique_id, model):
        super().__init__(unique_id, model)

# Initialize a model and grid
model = Model()
grid = OrthogonalMooreGrid([10, 10], torus=True, capacity=1, random=model.random)

# Create an agent and add it to a cell
agent = MyAgent(1, model)
cell = grid.select_random_empty_cell()
agent.move_to(cell)

Agent Movement

# Assuming `agent` and `grid` are already defined
# Move the agent to a random neighboring cell
new_cell = agent.cell.neighborhood().select_random_cell()
agent.move_to(new_cell)

Querying the Neighborhood

# Get all neighboring agents within a radius of 1
neighbors = agent.cell.neighborhood(radius=1).agents

# Iterate over neighbors and perform actions
for neighbor in neighbors:
    # Example action: print neighbor's ID
    print(neighbor.unique_id)

Agent Reproduction

# Agent reproduces, creating a new agent in the same cell
def reproduce(agent):
    offspring = MyAgent(model.next_id(), model)
    offspring.move_to(agent.cell)
    model.schedule.add(offspring)

# Call reproduce function for an agent
reproduce(agent)

Removing an Agent

# Remove an agent from the simulation
def remove_agent(agent):
    agent.cell.remove_agent(agent)  # Remove from cell
    model.schedule.remove(agent)    # Remove from the scheduler

# Call remove_agent function for an agent
remove_agent(agent)

Checking Cell Properties

# Check if a cell is empty or full before moving an agent
if cell.is_empty:
    agent.move_to(cell)
elif cell.is_full:
    print("Cell is full. Agent cannot move here.")

EwoutH · 2024-01-23T20:55:32Z

There’s a lot in here that I like!

let me know when you want any input / a review.

EwoutH · 2024-01-23T21:01:50Z

If you add matplotlib here, the current error in the benchmarks is resolved.

mesa/.github/workflows/benchmarks.yml

Line 31 in 7b0d3f1

run: pip install numpy pandas tqdm tabulate

Corvince · 2024-01-23T21:03:39Z

If you add matplotlib here, the current error in the benchmarks is resolved.

mesa/.github/workflows/benchmarks.yml

Line 31 in 7b0d3f1

run: pip install numpy pandas tqdm tabulate

Thanks, I now tried to bypass the jupyter stuff, but I am not sure if this is working. If I change the benchmark.yml won't the action stop working? Because of the pull_request_target thingy?

Corvince · 2024-01-23T21:04:13Z

There’s a lot in here that I like!

let me know when you want any input / a review.

Feedback ist welcomed at all times!

Corvince · 2024-01-23T21:05:39Z

/rerun benchmarks

EwoutH · 2024-01-23T21:08:16Z

If I change the benchmark.yml won't the action stop working?

Yeah it might, honestly I don't know.

EwoutH

Just some random thoughts from reading the code. On a high level, I think I really like it but I feel like I don't have enough comprehension of all the nuances and implications to say for sure.

Guess I would need to start make models with it to see what happens!

mesa/agent.py

mesa/experimental/cell_space.py

rht · 2024-01-23T22:00:26Z

Had 3 comments at 2e78940#comments.

rht · 2024-01-23T22:08:22Z

/rerun-benchmarks

github-actions · 2024-01-23T22:13:20Z

Performance benchmarks:

Model	Size	Init time [95% CI]	Run time [95% CI]
Schelling	small	🔵 +0.0% [-0.3%, +0.4%]	🔵 -0.1% [-0.3%, +0.1%]
Schelling	large	🔵 -0.5% [-1.1%, +0.1%]	🔵 -3.9% [-6.4%, -1.2%]
WolfSheep	small	🔵 -2.4% [-2.8%, -1.9%]	🔵 -0.3% [-0.4%, -0.1%]
WolfSheep	large	🔵 -2.2% [-3.0%, -1.5%]	🔵 -3.5% [-4.4%, -2.3%]
BoidFlockers	small	🔵 -1.7% [-2.2%, -1.1%]	🔵 +0.4% [-0.0%, +0.9%]
BoidFlockers	large	🔵 +0.2% [-0.4%, +0.9%]	🔵 -0.5% [-1.0%, -0.0%]

Corvince · 2024-01-23T22:19:57Z

@EwoutH it looks like the benchmark wasn't run for this PR after the comment trigger. You can see that from the action. Maybe you can look into that?

EwoutH · 2024-01-23T22:25:30Z

it looks like the benchmark wasn't run for this PR after the comment trigger.

You forgot the dash between rerun and benchmarks. It has to be the exact string (see rht's one).

mesa/experimental/cell_space.py

Corvince · 2024-01-24T08:09:39Z

mesa/experimental/cell_space.py

+        if radius == 0:
+            return {self: self.agents}
+        if radius == 1:
+            return CellCollection(


this currently breaks neighborhoods with radius>1 Made some errors while refactoring

I tested in on Schelling, radius 2 and seems fine?

Corvince · 2024-01-24T08:15:08Z

it looks like the benchmark wasn't run for this PR after the comment trigger.

You forgot the dash between rerun and benchmarks. It has to be the exact string (see rht's one).

Ha, I see how my comment was completely miswritten. I already realized after @rht made the correct comment. But in the action that was actually triggered this PR isn't checked out, hence no difference in performance. Performance should actually be about 20% faster.

Here is the exact line of the Action. It should check out this PR, but instead creates a new branch "main"
https://github.com/projectmesa/mesa/actions/runs/7632442972/job/20792667533#step:9:149

EwoutH · 2024-01-24T08:32:38Z

Looks like it checks it out differently when initiated from a PR opened then from a comment. Will check it out!

(current workaround it so mark PR as ready to review (and back to draft))

quaquel · 2024-01-24T15:11:25Z

Thanks for this! I will take a deeper dive at this, hopefully, tonight or otherwise over the weekend. I quickly checked the code this morning and have a few additional comments and suggestions in addition to the various points already raised

I will also move my versions of Schelling and WolfSheep into this (do I have write access)? So we can properly see the change in performance.

EwoutH · 2024-01-24T15:24:00Z

It should check out this PR, but instead creates a new branch "main"

I tried some stuff in #1998. Let's see if it works.

EwoutH · 2024-01-24T15:25:17Z

Merged, here we go: /rerun-benchmarks

for more information, see https://pre-commit.ci

quaquel · 2024-02-23T18:50:25Z

I did some more testing. I can't reproduce my initial performance differences so I guess I made a mistake when I thought there was a massive difference. I also tested a dedicated SingleAgentCell next to the normal cell and updated CellCollection.agents to take advantage of this. Again, this did not produce a massive speedup. So, I suggest we leave the code as is for now and just merge it. I have updated the PR text to better reflect the actual performance figures.

EwoutH · 2024-02-23T21:44:15Z

Agreed, we can always optimize more later if needed.

On thing I'm not sure about is already porting our main benchmarks over. Maybe we could add those models to mesa-examples, and leave the benchmarks here as is.

github-actions · 2024-02-23T21:47:28Z

Performance benchmarks:

Model	Size	Init time [95% CI]	Run time [95% CI]
Schelling	small	🔴 +240.8% [+239.2%, +242.4%]	🔴 +7.5% [+7.1%, +7.8%]
Schelling	large	🔴 +170.1% [+132.4%, +190.5%]	🔴 +26.7% [+24.4%, +28.8%]
WolfSheep	small	🔴 +72.9% [+71.1%, +74.5%]	🟢 -15.1% [-16.1%, -14.2%]
WolfSheep	large	🔴 +61.9% [+44.7%, +71.8%]	🟢 -15.8% [-17.9%, -13.6%]
BoidFlockers	small	🔵 +1.8% [+0.8%, +2.7%]	🔵 +0.1% [-0.5%, +0.7%]
BoidFlockers	large	🔵 +2.0% [+1.2%, +2.7%]	🔵 +0.9% [+0.4%, +1.4%]

quaquel · 2024-02-24T08:42:04Z

In both my datacollection and devs branch, I have added a simple examples folder within the folder with the new experimental code. I am fine with using either mesa-examples or doing what I have been doing in devs and datacollection. Let me know what you prefer and why.

EwoutH · 2024-02-24T16:16:46Z

I would suggest this PR shouldn't change the models in benchmarks. I can do those last bits however, I will try to do that and merge it tomorrow.

Out of curiosity, where any models with an NetworkGrid tested?

Corvince · 2024-02-24T16:38:45Z

I think an example folder inside the experiments sounds good and like the place to be.

But maybe it would still be nice to have one of the benchmarks use this feature, since I think we all agree that cell space is going to be the recommended space implementation eventually

quaquel · 2024-02-24T18:14:13Z

I like @Corvince suggestions. It also allows us to track how the performance of the new grid stuff is affected going forward.

EwoutH · 2024-02-25T09:13:31Z

We could have two sets of benchmarks, regular ones and experimental ones

quaquel · 2024-02-26T21:28:09Z

I am also fine with adding additional benchmarks, but how would that work out if we add additional experimental stuff? For example, if I add devs, which benchmarks would I edit to showcase their impact?

EwoutH · 2024-02-26T23:27:14Z

Thought about it a bit, let's keep it simple. Benchmarks are an internal development tool, not user examples. So let's keep them on the cutting edge.

I'm good with merging (please clean ip or squash). I would like a second aproval before doing so.

Thanks for the thremendous effort!

Corvince · 2024-02-27T05:49:37Z

@quaquel can you handle the merging, since I opened the PR? I'm swamped with work until next work and won't be able to do so until then

EwoutH · 2024-02-27T07:22:16Z

I can also do it

for more information, see https://pre-commit.ci

github-actions · 2024-02-27T12:35:49Z

Performance benchmarks:

Model	Size	Init time [95% CI]	Run time [95% CI]
Schelling	small	🔴 +248.5% [+246.4%, +250.5%]	🔴 +6.8% [+6.4%, +7.1%]
Schelling	large	🔴 +174.0% [+135.6%, +195.2%]	🔴 +21.7% [+19.5%, +23.6%]
WolfSheep	small	🔴 +75.2% [+73.4%, +77.1%]	🟢 -16.1% [-17.1%, -15.1%]
WolfSheep	large	🔴 +60.8% [+45.4%, +69.1%]	🟢 -19.7% [-21.1%, -17.9%]
BoidFlockers	small	🔵 +2.1% [+1.4%, +2.8%]	🔵 +0.7% [+0.2%, +1.2%]
BoidFlockers	large	🔵 +2.6% [+1.9%, +3.1%]	🔵 +0.4% [-0.0%, +0.9%]

## Summary This PR introduces an alteranative implementation for discrete spaces. This implementation centers on the explicit inclusion of a Cell class. Agents can occupy cells. Cells have connections, specifying their neighbors. The resulting classes offer a cell centric API where agents interact with a cell, and query the cell for its neighbors. To capture a collection of cells, and their content (_i.e._, Agents), this PR adds a new CellCollection class. This is an immutable collection of cells with convenient attribute accessors to the cells, or their agents. This PR also includes a CellAgent class which extends the default Agent class by adding a `move_to` method that works in conjunction with the new discrete spaces. From a performance point of view, the current code is a bit slower in building the grid and cell data structure, but in most use cases this increase in time for model initialization will be more than offset by the faster retrieval of neighboring cells and the agents that occupy them. ## Motive The PR emerged out of various experiments aimed at improving the performance of the current discrete space code. Moreover, it turned out that a cell centric API resolved various open issues (_e.g._, projectmesa#1900, projectmesa#1903, projectmesa#1953). ## Implementation The key idea is to have Cells with connections, and using this to generate neighborhoods for a given radius. So all discrete space classes are in essence a [linked data structure](https://en.wikipedia.org/wiki/Linked_data_structure). The cell centric API idea is used to implement 4 key discrete space classes: OrthogonalMooreGrid, OrthogonalVonNeumannGrid (alternative for SingleGrid and MultiGrid, and moore and von Neumann neighborhood) , HexGrid (alternative for SingleHexGrid and MultiHexGrid), and Network (alternative for NetworkGrid). Cells have a capacity, so there is no longer a need for seperating Single and Multi grids. Moore and von Neumann reflect different neighborhood connections and so are now implemented as seperate classes. --------- Co-authored-by: Jan Kwakkel <j.h.kwakkel@tudelft.nl>

Corvince added 4 commits January 23, 2024 21:13

Add draft implementation for CellSpace

dd54f21

add features to cell

7b58d8e

remove create_neighborhood_getter

aef4a8b

update benchmark models

3240a2b

try to import Grid directly from experimental

d3df44e

EwoutH reviewed Jan 23, 2024

View reviewed changes

mesa/experimental/cell_space.py Outdated Show resolved Hide resolved

rht mentioned this pull request Jan 24, 2024

feat: Cythonize get_cell_list_contents #1995

Closed

rht reviewed Jan 24, 2024

View reviewed changes

mesa/experimental/cell_space.py Outdated Show resolved Hide resolved

rht reviewed Jan 24, 2024

View reviewed changes

mesa/experimental/cell_space.py Outdated Show resolved Hide resolved

Corvince commented Jan 24, 2024

View reviewed changes

Corvince marked this pull request as ready for review January 24, 2024 10:44

Corvince marked this pull request as draft January 24, 2024 10:46

EwoutH mentioned this pull request Jan 24, 2024

benchmarks.yml: Fix PR branch checkout when triggered by comment #1998

Merged

quaquel and others added 2 commits February 23, 2024 19:49

minor docstring update

62c844e

[pre-commit.ci] auto fixes from pre-commit.com hooks

adff7aa

for more information, see https://pre-commit.ci

EwoutH added trigger-benchmarks Special label that triggers the benchmarking CI and removed trigger-benchmarks Special label that triggers the benchmarking CI labels Feb 23, 2024

EwoutH approved these changes Feb 26, 2024

View reviewed changes

EwoutH and others added 2 commits February 27, 2024 13:29

Merge branch 'main' into experimental/cell-space

7d5e6b0

[pre-commit.ci] auto fixes from pre-commit.com hooks

b32af9f

for more information, see https://pre-commit.ci

EwoutH added trigger-benchmarks Special label that triggers the benchmarking CI and removed trigger-benchmarks Special label that triggers the benchmarking CI labels Feb 27, 2024

EwoutH changed the title ~~Experimental/cell space~~ Add cell-centric discrete spaces (experimental) Feb 27, 2024

EwoutH merged commit ac183a8 into projectmesa:main Feb 27, 2024
13 checks passed

EwoutH mentioned this pull request Mar 1, 2024

Add Voronoi tessellation space #1895

Closed

2 tasks

EwoutH mentioned this pull request Apr 24, 2024

Use new Cell Space functionality in examples projectmesa/mesa-examples#123

Open

rht mentioned this pull request Jul 7, 2024

Refactoring mesa.space projectmesa/mesa-frames#6

Open

9 tasks

EwoutH mentioned this pull request Aug 5, 2024

Cacheable mesa #2194

Draft

EwoutH mentioned this pull request Sep 9, 2024

Move experimental cell spaces to normal #2286

Closed

9 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add cell-centric discrete spaces (experimental) #1994

Add cell-centric discrete spaces (experimental) #1994

Corvince commented Jan 23, 2024 •

edited by EwoutH

Loading

EwoutH commented Jan 23, 2024

EwoutH commented Jan 23, 2024

Corvince commented Jan 23, 2024

Corvince commented Jan 23, 2024

Corvince commented Jan 23, 2024

EwoutH commented Jan 23, 2024

EwoutH left a comment •

edited

Loading

rht commented Jan 23, 2024

rht commented Jan 23, 2024

github-actions bot commented Jan 23, 2024

Corvince commented Jan 23, 2024

EwoutH commented Jan 23, 2024

Corvince Jan 24, 2024

quaquel Jan 24, 2024

Corvince commented Jan 24, 2024

EwoutH commented Jan 24, 2024 •

edited

Loading

quaquel commented Jan 24, 2024

EwoutH commented Jan 24, 2024 •

edited

Loading

EwoutH commented Jan 24, 2024 •

edited

Loading

quaquel commented Feb 23, 2024

EwoutH commented Feb 23, 2024

github-actions bot commented Feb 23, 2024

quaquel commented Feb 24, 2024

EwoutH commented Feb 24, 2024 •

edited

Loading

Corvince commented Feb 24, 2024

quaquel commented Feb 24, 2024

EwoutH commented Feb 25, 2024

quaquel commented Feb 26, 2024

EwoutH commented Feb 26, 2024

Corvince commented Feb 27, 2024

EwoutH commented Feb 27, 2024

github-actions bot commented Feb 27, 2024

Add cell-centric discrete spaces (experimental) #1994

Add cell-centric discrete spaces (experimental) #1994

Conversation

Corvince commented Jan 23, 2024 • edited by EwoutH Loading

Summary

Motive

Implementation

Usage examples

Creating a Grid and Adding Agents

Agent Movement

Querying the Neighborhood

Agent Reproduction

Removing an Agent

Checking Cell Properties

EwoutH commented Jan 23, 2024

EwoutH commented Jan 23, 2024

Corvince commented Jan 23, 2024

Corvince commented Jan 23, 2024

Corvince commented Jan 23, 2024

EwoutH commented Jan 23, 2024

EwoutH left a comment • edited Loading

Choose a reason for hiding this comment

rht commented Jan 23, 2024

rht commented Jan 23, 2024

github-actions bot commented Jan 23, 2024

Corvince commented Jan 23, 2024

EwoutH commented Jan 23, 2024

Corvince Jan 24, 2024

Choose a reason for hiding this comment

quaquel Jan 24, 2024

Choose a reason for hiding this comment

Corvince commented Jan 24, 2024

EwoutH commented Jan 24, 2024 • edited Loading

quaquel commented Jan 24, 2024

EwoutH commented Jan 24, 2024 • edited Loading

EwoutH commented Jan 24, 2024 • edited Loading

quaquel commented Feb 23, 2024

EwoutH commented Feb 23, 2024

github-actions bot commented Feb 23, 2024

quaquel commented Feb 24, 2024

EwoutH commented Feb 24, 2024 • edited Loading

Corvince commented Feb 24, 2024

quaquel commented Feb 24, 2024

EwoutH commented Feb 25, 2024

quaquel commented Feb 26, 2024

EwoutH commented Feb 26, 2024

Corvince commented Feb 27, 2024

EwoutH commented Feb 27, 2024

github-actions bot commented Feb 27, 2024

Corvince commented Jan 23, 2024 •

edited by EwoutH

Loading

EwoutH left a comment •

edited

Loading

EwoutH commented Jan 24, 2024 •

edited

Loading

EwoutH commented Jan 24, 2024 •

edited

Loading

EwoutH commented Jan 24, 2024 •

edited

Loading

EwoutH commented Feb 24, 2024 •

edited

Loading