40. Python Scripting

Python (www.python.org) is a powerful and popular scripting language for scientific computing and engineering. The main reason behind this is the the vast number of libraries, toolkits and SDK’s available.

The AGX Python library comes as SWIG-generated modules and extensions, mirroring the C++ native API as far as possible. Each C++ namespace that is part of the AGX C++ SDK and exposed to Python makes up a Python module.

40.1. AGX Python Installation and requirements

This section will give you instructions on how to use and integrate AGX Dynamics with Python.

40.1.1. Requirements

Note

  • For Windows, AGX Dynamics comes with an integrated version of Python. This is to ensure that it runs in a controlled/verified environment. This is due to the sensitivity between different versions of Python.

  • This AGX version is built against Python version: 3.9.9

At the final stage of the installation of AGX Dynamics, a question related to installing Python requirements was asked. By default this is enabled and all the required modules where installed. If not, you need to make sure you have all the required modules installed before running some of the scripts that comes with AGX Dynamics. This is also true if you plan to use a custom Python installation instead of the one that comes with the AGX Dynamics installer.

For AGX-supported Linux x64 distributions, an installed developer package is required, which can be installed using the command:

$ sudo apt-get install python3.5-dev

40.1.2. Using a custom Python installation

This section describes how to use AGX Dynamics with an existing python installation. To make sure your existing python installation works you need to verify that:

  • It is the same version of Python that was used when building AGX Dynamics: 3.9.9. You can also check which version AGX is built against using: agxViewer --version

  • That it uses the same architechture as the current AGX Dynamics build: x86/x64

Python can be downloaded at: Python 3.9.9

If you want to use your own local Python installation, there are a few ways of achieving this:

  1. In the START menu (windows) there are two shortcuts to the AGX Dynamics Command Window

AGX Dynamics Command Window - Will create a cmd window with environment setup
for AGX to use the (default) embedded Python that comes with AGX.
AGX Dynamics Command Window (External Python) - Will create a cmd window
with environment setup for AGX to use an external Python installation.
  1. or call the file setup_env.bat with the argument: /USEEXTERNALPYTHON

c:\program files\Algoryx\AGX-2.28.0.0\setup_env.bat /USEEXTERNALPYTHON
  1. Set the environment variable AGX_USE_EXTERNAL_PYTHON before executing setup_env.bat (in the AGX folder)

set AGX_USE_EXTERNAL_PYTHON=1
cmd> c:\program files\Algoryx\AGX-2.28.0.0\setup_env.bat

You will also have to make sure you have your python environment correctly setup:

set PYTHONHOME=c:\Program Files\Python39

Tip

Tip: If you are unsure where Python is installed, type ‘where python’ in a command prompt.

40.1.3. Installing required libraries

Some tutorials might use additional packages such as plotting etc. Because of licensing, these modules cannot be bundled with AGX Dynamics but must be downloaded to the local python installation using the python -m pip command.

The file <agx-dir>/data/python/requirements.txt contains the required modules (and versions) that some script need to run. On Windows:

Use –user if you want the libraries to be installed for your current user only. If you use the pre-installed Python that comes with AGX Dynamics, you might want to skip the –user argument. This will install the libraries to: <agx-dir>python-x64libsite-libraries hence it will not affect your system installed Python version.

c:\Algoryx\AGX-2.28.0.0> python -m pip install --user -r data\python\requirements.txt

On Unix/Mac:

$ pip install --user -r data/python/requirements.txt

40.1.4. Environment

To be able to use AGX Python you need to make sure that the environment is correctly setup. By executing <agx-dir>/setup_env.bat (.bash) you ensure that the required environment variables are setup. If you do not use setup_env, then make sure you investigate what is required to use AGX Python. The two important environment variables are:

  • PYTHONHOME - Specifies where python can be found on your computer

If you use a custom version of Python you also need to make sure that PYTHONPATH is updated:

  • PYTHONPATH - List directories with python modules, including AGX Python

40.1.5. Python IDE for development

There are many tools and IDE:s for developing Python code. Below are a few listed with short information how to use them with AGX Dynamics

40.1.5.1. Visual Studio code

Visual Studio Code is a text editor with support for many file formats, including Python.

  1. Download and install Visual Studio Code

  2. Start an AGX Dynamics Command Prompt (from the START menu or by starting a cmd windows and execute the <agx-dir>/setup_env.bat to setup the execution environment (it is important so that AGX Dynamics files can be found in PATH and that the python modules can be located: PYTHONPATH).

  3. Start Visual Studio Code from the command-line with the correct environment setup.

  4. Open a .py or .agxPy file in the data/python/ directory.

  5. You might want to associate the .agxPy file format to Python script. (see documentation on the Visual Studio Code site for how to do this)

40.1.5.2. PyCharm

Another Python development environment is PyCharm Below you can find a short tutorial on how to install and setup PyCharm for editing AGX Python files under the Windows platform.

  1. First you need to download and install PyCharm

  2. Start an AGX Dynamics Command Prompt (from the START menu or by starting a cmd windows and execute the <agx-dir>/setup_env.bat to setup the execution environment (it is important so that AGX Dynamics files can be found in PATH and that the python modules can be located: PYTHONPATH).

  3. Start PyCharm. Depending on where you installed PyCharm the path might be different:

D:\Algoryx\AGX-2.28.0.0> "c:\Program Files (x86)\JetBrains\PyCharm Community Edition 2016.3.1\bin\pycharm.bat"
  1. Name and create a PyCharm project.

  2. Go to Settings->Project: <project name>->Project Structure and under Add Content Root add the <agx-dir>/bin/x64/agxPy directory in the AGX Dynamics installation.

  3. Create a file and start coding:

import agx
import agxSDK
init = agx.AutoInit()
sim = agxSDK.Simulation()
sim.stepForward()

40.2. Modules and Library structure

Each C++ namespace of AGX exposed to Python is exposed as its own Python module available for import. For example: agxSDK is encoded in _agxSDK.pyd. These modules are found together with their respective native extension shared library in:

  • In Windows, beneath the directory of your AGX installation: <agx-dir>/bin/x64/agxpy

  • In Unix the default path should be something similar to: /usr/local/lib/python3.5/dist-packages/site-packages/ This might be different on your specific Unix flavor.

For Python to be able to locate the AGX Python modules, you might need to set the PYTHONPATH environment variable. See the chapter Environment below.

You can verify the AGX Python installation by creating a Python script that contains the following:

import agx
agx.init()

If this works, then Python can locate the required libraries for AGX Python.

40.3. Python/AGX Dynamics integration

Tip

File script <agx_dir>/data/python/template.agxPy works as a starting point for a new script using the AGX Dynamics Python API.

AGX Dynamics and Python can be used together in two different ways:

  1. In the native Python scripting environment using the command: python

  2. Embedded as a scripting format for the agxViewer.exe application

Both ways are simple to support from the same Python application or library, as long it is initialized correctly. Aside from importing the required Python modules used by the program, the following bare-bone example will determine how the script is being executed; whether its embedded mode from agxViewer or in a native Python environment:

import agx
import agxSDK
import agxIO
# import of additional modules ...
import agxPython
init = agx.AutoInit(); # Important so that we initialize the AGX SDK

def buildScene1(sim, app, scene_root):
    # build your AgX scene here by adding to the agxSDK.Simulation
    # object sim, aagxOSG.ExampleApplication object app and
    # the agxOSG.Group object scene_root.

Below we have the entry point for agxViewer, or an application based on agxOSG::ExampleApplication. It will look for the function buildScene() and execute its content.

def buildScene():
  # Script is run from agxViewer. Everything is set up, so
  # all we need to do is fetching our sim, app and osg root node
  # and call the function which builds our scene
    sim = agxPython.getContext().environment.getSimulation()
    app = agxPython.getContext().environment.getApplication()
    scene_root = agxPython.getContext().environment.getSceneRoot()

    # That is all we need before we teleport...x
    buildScene1(sim, app, scene_root)

Below, we have the entry point for when the script is executed with the python executable. It will find that the agxPython.getContext() return None and call the main function.

def main(fileName):
    # Initialize the AGX ExampleApplication here (see any script in data/python for specifics)
    app = agxOSG.ExampleApplication()

    # Create a command line parser.
    argParser = agxIO.ArgumentParser()
    argParser.readArguments(arg) ## feed the parser with the arguments

    # Add a scene to the application. Whenever the key '1' is pressed, this file will be reloaded
    # and the def "buildScene" is called
    app.addScene(fileName, "buildScene1", ord("1"))

    # Call the init method of ExampleApplication
    # It will setup the viewer,windows etc.
    if app.init(argParser):
        initCamera(app)
        app.run()
    else:
        print("An error occurred while initializing ExampleApplication.")


# Test to see if we are running via the native python interpretator or in embedded mode (agxViewer)
if agxPython.getContext() == None:
  # Script is run from native Python interpreter, call main with the script file name
  # as argument
    main(sys.argv[0])

40.4. Using Python for modelling in C++

It is also possible to use python for modelling and C++ for control. The tutorial tutorial_python_excavator_controller.cpp that can be located among the C++ tutorials illustrates this.

  1. Construct the model in Python, including parameterization

  2. Load the python script from C++.

  3. Access constraints, bodies etc. from C++.

  4. Control the simulation from C++.

In this way you can get the best of both worlds:

  • The fast iteration/flexibility of programming with Python without compilation/linking.

  • Low performance overhead because no code is actually running in Python during the simulation.

So this is a good way of mixing the two worlds.

When running the tutorial_python_excavator_controller you can reload the python script by pressing the button ‘1’ just as with the agxViewer application.

40.5. AGX Python coding guide

40.5.1. Sharing AGX buffers as numpy arrays

Internally AGX entities are stored in contiguous memory buffers. Looping through these buffers using the AGX C++ API is very fast. By creating numpy arrays, that points at the same memory as the AGX buffers, it is possible to inspect and change the AGX buffers using all the usual numpy operations, with comparable performance as the C++ API.

One danger, is that every time AGX is in control it can decide to reallocate the buffer. From python the user is never notified about this. Therefore, the user must recreate the numpy array between every timestep. This process is made more convenient by using the BufferWrapper python class implemented in agxPythonModules. The wrapper finds the specified buffer and creates a numpy array with the correct type and size that wraps around it. Every time the numpy array is accessed from the wrapper by the user, the wrapper class recreates the array around the buffer.

from agxPythonModules.utils.numpy_utils import BufferWrapper

# ...

# Get the position buffer for all the particles in sim
position_buffer = BufferWrapper(sim, "Particle.position")

position_array = position_buffer.array

# Get the shape of the array
shape = position_array.shape

# Set the z-position of every particle
position_array[:,2] = np.linspace(-10, 10, shape[0])

# step agx
sim.stepForward()

# re-fetch buffer in case agx has reallocated
position_array = position_buffer.array

# Print the z position for every particle.
print(position_array[:,2])

40.5.2. Proxy classes, their instances and reference counting

All C++ classes exposed to python will be accessible through a proxy class. Each namespace of AGX will belong to a separate Python module.

For example the class agx::RigidBody will be located in the module _agxPython which is imported to Python using the import keyword. Below is an example of how to import the agx namespace and instantiate a class in that namespace:

import agx
rb = agx.RigidBody()

C++ classes derived from agx::Referenced are automatically handled by Python, so that as long as there is a C++ agx::ref_ptr and/or a Python object referring to it, it will not be deallocated.

40.5.3. Attributes

Attributes added to proxy class instances in Python does not exist outside the Python object to which the attributes belong, and can’t pass through the native C++ layer back to Python again. The following code demonstrates this:

rb_vector = agx.RigidBodyVector()
rb = agx.RigidBody()
rb.my_attr = 5.0 # An attribute added to the rb object
rb_vector.append(rb)

rb_retrieved = rb_vector[0]
print(rb.my_attr)                     # Will print 5.0
print(rb_retrieved.my_attr)   # Will throw AttributeError exception! Because rb_retrieved is
                                                              # not the same Python object as rb

The reason for this is that: Proxy class instances, or Python objects of proxy classes, are not exact representations of C++ objects and multiple Python proxy objects may reference the same C++ object.

40.5.4. Function/method overload ambiguity

In Python it is forbidden to declare anything using the same identifiers or names in any scope where they already exist. In C++ however, it is commonplace to overload functions and methods by their signatures while sharing the same names. In AGX Python, proxy classes only have one representation for all possible versions of static functions or methods found in C++.

Ambiguity resolving is done within the Python extensions before a match is found based on which arguments the Python code passed to it during a call, where no match throws a NotImplementedError exception back to the caller.

40.5.5. Inherited classes/virtual methods

Some AGX classes allow for their virtual methods to be overridden in derived classes, where EventListener family of classes being the most relevant example of this. The table below show examples of classes which can be implemented in Python:

Python Class

Virtual Methods (in Python)

agxSDK.EventListener

def addNotification(self) -> "void"
def removeNotification(self) -> "void"

agxSDK.ContactEventListener

def preCollide(
   self,
   t: "float"
) -> "void"

def pre(
   self,
   t: "float"
) -> "void"

def post(
   self,
   t: "float"
) -> "void"

def last(
   self,
   t: "float"
) -> "void"

agxSDK.GuiEventListener

 def mouseDragged(
   self,
   buttonMask: "MouseButtonMask",
   x: "float",
   y: "float"
 ) -> "bool"

 def mouseMoved(
   self,
   x: "float",
   y: "float"
 ) -> "bool"

def mouse(
   self,
   buttonMask: "MouseButtonMask",
   state: "MouseState",
   x: "float",
   y: "float"
) -> "bool"

def keyboard(
   self,
   key: "int",
   modKeyMask: "int",
   x: "float",
   y: "float",
   keyDown: "bool"
) -> "bool"

def update(
   self,
   x: "float",
   y: "float"
) -> "void"

agxSDK::ContactEventListener

def impact(
   self,
   time: "double",
   geometryContact: "agxCollide::GeometryContact"
) -> "KeepContactPolicy"
def contact(
   self,
   time: "double",
   geometryContact: "agxCollide::GeometryContact"
) -> "KeepContactPolicy"
def separation(
   self,
   time: "double",
   geometryPair: "agxCollide::GeometryPair"
) -> "void"

agxSensor::JoystickListener

def axisUpdate(self,
 state: "agxSensor::JoystickState",
 axis: "int",
) -> "bool"

def buttonChanged(
 self,
 state: "agxSensor::JoystickState",
 button: "int",
 down: "bool"
) -> "void"

def povMoved(
 self,
 state: "agxSensor::JoystickState",
 pov: "int"
) -> "void"

def sliderMoved(
 self,
 state: "agxSensor::JoystickState",
 slider: "int"
) -> "void"

def addModification(
 self
) -> "void"

def removeModification(
 self
) -> "void"

It’s important to not forget about the ‘self’ argument when you override any of these methods, and that the right type of any value they return is used. Violations to this are considered an error and will break your script.

Furthermore, it’s important to follow Python rules as well, something known as “Zen of Python”, and call the base class version of the method being overridden when you wish to return some default value. In Python, you do not wish to “override” base class functionality with polymorphic inheritance as you do in C++, instead you “enhance” with new functionality by adding instead of replacing.

40.5.6. Scope locality and garbage collection

In Python, all declarations made in some scope are also bound to that scope once evaluated. In other words, the default scope used in Python is “local”. Assigning or returning a value binds them to whatever object or scope that accepts the value, and the value will persist in memory for as long it is strongly referenced by another object or scope. If no such references exist for a value it gets garbage collected. Whether this happens right away or some time later can vary, but it is safe to assume it mustn’t be touched by any code once it becomes garbage.

Reference-counted C++ objects complicate things sometimes if care is not given to avoid premature decrement reference counts of references not owned by the dereferencer. Always keep a reference around for AGX objects you create by keeping a Python object/proxy class instance alive for as long you want to be on the safe side.

40.5.7. Ref smart pointer objects

Sometimes it’s not always a raw pointer object you deal with but their smart pointer counterparts. The smart pointer template class instances are suffixed with Ref, so for example smart pointer class for the Geometry class is called GeometryRef and for RigidBody, it’s RigidBodyRef. Normally they expose the same methods as the original class with the addition of those declared by agx::ref_ptr<T> template class, such as get(), isValid and so on.

It is not, however, possible to pass them as arguments to methods expecting objects of template argument type; e.g. if a Geometry is expected, passing GeometryRef instead is not valid. Sometimes you deal with <Class>RefVectors, and any accessors will inevitably always return <Class>Ref objects - not <Class> objects.

To dereference these smart pointers, use the get() method on them, but check if they are valid before you do, or else you risk the execution of dereferencing null pointers within the C++ layer which will crash your application. Here’s an example where we define a function which extracts all the geometries associated with a rigid body and returns a Python list with raw Geometry objects:

def rigidBodyGeometriesToList(rb):
 geometries = rb.getGeometries() # geometries is of type GeometryRefVector
 geometries_list = list()        # the list to return
 for geo_ref in geometries:      # geometries contain items of type GeometryRef
     geometries_list.append(geo_ref.get()) # geo_ref.get() gives us its Geometry
 return geometries_list            # geometries_list now contain Geometry items

40.6. C++ to Python guide

Most primitive types, classes, functions and methods of the AGX C++ API exposed to Python have wrappers which work as one would expect in Python. For example, wherever a C++ NULL or nullptr would be used, you use None in Python. There are however some things to watch out for, especially when the philosophical and conventional similarities between C++ and Python end.

Below is a number of sample cases demonstrating the syntax difference between Python and C++:

40.6.1. Construction

C++:

agx::RigidBodyRef body = new agx::RigidBody();

Python:

body = agx.RigidBody()

40.6.2. Calling a normal method

C++:

body->getMassProperties()->setMass( 10 );

Python:

body.getMassProperties().setMass( 10 )

40.6.3. Calling a static method

C++:

agx::Constraint::calculateFramesFromBody(agx::Vec3(1,2,3), agx::Vec3(1,0,0),
                                                                        body1, frame1,
                                                                        body2, frame2);

Python:

agx.Constraint.calculateFramesFromBody(agx.Vec3(1,2,3), agx.Vec3(1,0,0),
                                                                      body1, frame1,
                                                                      body2, frame2);

40.7. Documentation of agxPythonModules

This section documents AGX python modules that are located in data/python/modules/agxPythonModules/.

40.7.1. Replace particleEmitter with rigidBodyEmitter

ParticleEmitters can only emit spherical bodies that belong to a particleSystem, while a rigidBodyEmitter can emit arbitrary rigid bodies that are treated like any other rigid body in the simulation. Rigid bodies are generally more computationally expensive than particles in a particleSystem, but useful for modeling granular systems of rigid bodies. A common workflow is to make prototype simulations of the system with particleEmitters and particleSystems, to figure out the rough 3D design and control of the system, and then substitute the particleEmitters with rigidBodyEmitters. Making the modeling cycle of the system more effective. This requires a systematic way of replacing an agx::ParticleEmitter with an agx::RigidBodyEmitter.

The script replaceParticleEmitterWithRigidBodyEmitter.py in the directory data/python/modules/agxPythonModules/utils, takes a serialized AGX simulation as input and replaces all the particle emitters in the simulation with rigid body emitters and saves the resulting simulation as a new .agx file. After setup of the AGX environment variables the utility script can be called from the command line with:

python -m agxPythonModules.utils.replaceParticleEmitterWithRigidBodyEmitter <path/to/simulation.agx>

By default the resulting simulation is saved as output.agx. It is possible to specify the output name with command line argument --output <name/of/output.agx>. You can then run the simulation and record a journal with the command agxviewer output.agx --journalRecord --journalIncrementalStructure.

The default geometries used as templates are taken from the data/models/convex_stones directory. It is possible to create and use a new shape library, by using the --shapeLibrary command line argument. The shape library is a directory with one .obj file per shape. The shapes must be convex. The size of the rigid body template are estimated using the diagonalized inertia tensor, of the rigid body, assuming uniform mass distribution and a cuboid shape:

\[\begin{split}x = \sqrt{6m^{-1} (Izz + Iyy - Ixx)} \\ y = \sqrt{6m^{-1} (Izz + Ixx - Iyy)} \\ z = \sqrt{6m^{-1} (Iyy + Ixx - Izz)}\end{split}\]

Choosing the smallest of \(x,y,z\) as the size is a good approximation of how mechanical sieving measures the size of rocks. The distributionTable of the rigidBodyEmitter is created by scaling the rocks to match the size of the previously used particles.

The material of the rigid bodies are copied directly from the particles materials. This can lead to unexpected changes in the bulk granular flow, since the rigid bodies have different geometry. Recalibration of the material properties can be necessary.

The friction model for rigid bodies in AGX is by default IterativeProjectedConeFriction, with the solve type SPLIT. Since these simulations often contains a large contact system the script changes the solve type to ITERATIVE. This trades realistic friction for performance. Depending on the application, it possible to speed this up even more together with agxSDK::MergeSplitHandler - AMOR.

A step-by-step description of the workflow when modeling granular flows with spherical particles and rigid bodies:

  1. Do prototype simulations with spherical particleSystems and particleEmitters.

  2. Export the simulation as an .agx file.

  3. [Optionally] Create your own shape library. One shape per .obj file.

  4. Run the script replaceParticleEmitterWithRigidBodyEmitter.py to replace particleEmitters with rigidBodyEmitters.

  5. Run the new simulation with correct solver settings and journal configuration.

  6. Post-process the results.

  7. If needed, return to point 1 and re-design the system.