The public hEngine is here

We have published the code at the heart of HASH simulations: you can view hEngine, the open computational engine, in this public repository. This is an alpha-stage product under active development, made available to allow developers and interested parties to explore and provide feedback on the code.

By making our simulation engine public we are delivering on several key principles:

• flexibility: you can run simulations in your own environment, using your own editor, and feed data to and from the engine in any way you please
• transparency: the inner workings of our simulation engine are free to inspect, to debate, and to improve
• extensibility: by making the code public we enable and encourage you to extend it or modify it to meet your own needs (subject to the Server Side Public License)
• portability: when you build HASH simulations, there’s no risk of platform lock-in: you’re not reliant on us, and can self-host at any time

We will now be working towards a first formal release, shipping a number of improvements and exciting new features in the weeks to come. We’d love for you to get involved, whether through contributing, raising issues, or getting in touch to share feedback and feature requests. 

Download projects from hCore

You can start using your existing projects with the open engine straight away, by exporting a project’s files and dependencies from hCore. This allows it to be downloaded for local use with hEngine, and enables you to backup your simulation code and data offline.

To do this, open up the File menubar and hit the Export Project button.

Coming soon: We’ll be introducing the ability to upload simulation project files to HASH, enabling seamless transition between developing on your machine and running simulations in-browser (in hCore) and at scale (with hCloud).

Waypoint Navigation Library

Using the new waypoint navigation library, agents will move towards their next waypoint until they are close enough to receive their next destination. Once agents have reached the waypoint closest to their final destination, they navigate directly to the location. This allows for manual or assisted creation of realistic and efficient agent movement paths around obstacles.

Simulation School

We’re launching Simulation School to complement our existing HASH Docs. Simulation School takes a topic-based approach teaching you how to use and apply simulation models in the real-world.

Our first course delves into computational economics and introduces users to five basic agent-based models, explaining step-by-step how to use those models in HASH.

We’re releasing two new simulation templates that make it easy to use your data to generate realistic models.

Modern Warehouse

Modern Warehouse: A simulation template for generating a warehouse with realistic proportions and picking operations. Forklifts move between the racks and crates picking up orders following a series of waypoints that determine their routes. Forklifts will orient to the nearest waypoint and make sensible decisions to get to their pickup and delivery location.

Cloud Infrastructure

Cloud Infrastructure: A simulation template for creating representations of cloud infrastructure with Terraform. Using either a terraform file or manual specification, the simulation generates a model of a Kubernetes cluster with requests coming in from real request data or estimated distributions.

Run bigger, longer simulations with better memory handling in hCore

By default, hCore retains all the state data from each step in a simulation. This limits how long you can run a sim for, as the state data eventually uses all the available memory.

We’ve introduced an option to retain only the most recent data – for any number of steps you choose – allowing sims to run with a much, much lower memory footprint.

Analysis can still be computed for the entire run by watching the analysis tab as the sim is computing.

Memory efficiency has also been improved elsewhere, most significantly by plugging a memory leak in the 3D viewer, which overall leads to memory use around a third of that previous for typical simulations, even when retaining all step data.

Export CSV of your analysis to inspect elsewhere

Until now, analysis metrics could only be exported as JSON, requiring further transformation in order to use this data in traditional spreadsheet-based software like Excel or Google Sheets.

Metrics can now additionally be downloaded as a structured CSV file for direct easy import into third-party applications, allowing you to complement in-IDE analysis with your other favourite tools.

To download simulation state and analysis data, right click on a run in the activity history, and click ‘Export run data’.

Quality of life improvements

This week we’ve shipped a bunch of smaller changes that improve platform quality of life.

• You can now use markdown in your simulation release notes to improve their readability
• Simulation models using the HASH process modeling library now run far more smoothly in hCloud
• Default settings for certain built-in 3D meshes have been improved

We squashed a ton of bugs, including a couple of notable nasties:

• Fixed a rare bug that could cause optimization experiments to fail and prematurely exit under certain circumstances
• Fixed a regression that meant line numbers for errors and warnings could appear off-by-one in the hCore Console when running Python behaviors client-side
• Fixed an issue that could cause activity for projects with extremely long histories to sometimes fail to load

We’ve also moved the HASH glossary into our open-source monorepo, so you can add pages or edit definitions directly. Feel free to check out our contributor guidelines and open up a Pull Request! We’ve also migrated our docs away from GitBook and into a new unified learn HASH website.

Stopping Conditions

We’ve added a new feature to HASH where simulations can be stopped at a specific point by sending a stop message to the engine. This built-in message is useful for stopping a simulation after a given number of steps, or when a particular condition has been reached in the simulation.

state.addMessage("hash", "stop", { status: "success", reason: "completed the initial optimization" })

Additional logging data can be attached to the message to help with debugging. Read more about stop messages in the HASH docs.

More Optimization Strategies

To complement our existing optimization libraries, we’ve released two additional optimization simulations, A* Search and Monte Carlo Tree Search.

A* Search

A* search is one of the most popular search algorithms. It uses an optimistic heuristic plus breadth first search to find the best possible route through a graph to a target destination node. Use the A* search library in combination with an agent based model to have agents navigate graphs efficiently.

Monte Carlo Tree Search

Monte Carlo Tree Search (MCTS) is a modified tree search that uses heuristics to prioritize searching certain branches of a game tree based on the likelihood of finding winning moves. A Monte Carlo distribution determines the game tree moves, optimizing for the branches that have returned the highest score in previous iterations but balancing that with exploring novel choices. MCTS has had a lot of success in games, most notably serving as the underlying algorithm for AlphaGo. In the simulation, an MCTS behavior powers the search of an agent playing tic-tac-toe.

Gradient Descent Optimizations

We’re releasing two simulation behavior libraries for gradient descent based optimization:

• Stochastic Gradient Descent (SGD): A classic optimization behavior, stochastic gradient descent optimizes a set of parameters by randomly exploring the solution space and ‘moving’ potential solutions up or down a gradient to find local maxima/minima. By generating solution agents randomly in the solution space, SGD is likely (though not guaranteed) to find the global maxima/minima. SGD Behavior Library
• Simulated Annealing: Akin to SGD, simulated annealing works by exploring the solution space through hill climbing behavior. However, simulated annealing implements an explore-exploit technique by randomly moving in a direction. If the choice is fitness improving its accepted; otherwise with some probability it’s rejected. Over time the probability of accepting a random deleterious move is decreased. As the simulation runs, its more likely to preserve the best move and reach the local minima/maxima. Simulated Annealing Behavior Library.

Explore the simulations and use the behaviors in your own simulations.

Genetic Programming

We’ve released a genetic programming simulation that showcases using a genetic algorithm to evolve solutions to an optimization problem.

The simulation is made up of four key behaviors:

• fitness.py: Calculates a fitness score for a potential solution.
• evaluate.py: Compares and determines the best fitness score among the solutions.
• crossover.py: Create new solution options from the existing solutions.
• mutate.py: Randomly introduce changes in the agents.

When added to a pool of agents the behaviors will converge to the optimal solution. Read more about genetic programming in our accompanying blog post.