The SWARM Agent Builder is a set of tools that helps you to design autonomous agents and embue them with specific capabilities through algorithms. This software is designed to be as simple and easy to use as possible.
Scenario Selection
The first step revolves around selecting a scenario. Based on the type of account that you signed up for, different sets of scenarios will be available. You can look at the Simulation Menu (Add Link) to decide.
Once you have chosen a scenario, you will be presented with the Scenario selection tool. This tool provides information about the specific scenario, such as what the objective of the simulation will be, how your agents will be evaluated during simulation and when the simulation will end.
Current options allow you to set the runtime of the simulation. This refers to the number of seconds that the simulation is running. This parameter is used as a means of ensuring the simulation does not continue indefinitely. You can slide the slide bar to select the amount of time you wish to use. Different accounts have different abilities to specify more time for their simulations. See below.
Simulation Time (Maximum amount in seconds):
- Basic: 360
- Advanced: 720
- Enterprise: 10000
Building Your Agents
Once you have completed the Scenario section, you can save your settings and move on to building your agents. Currently, options to utilize built in obstacle avoidance and perception are available. In future iterations of the platform, you will be able to select which algorithms you wish to use for these components or define your own algorithms. If you choose to write your own algorithms, these will also be available for you to utilize for testing various solutions.
Number of Agents
If available, you can select the number of agents that you wish to simulate. This number will be used to build that specific number of agents, each of which receives a copy of the same software and will execute the same algorithmic solution that you build. Slide the slide bar to set the number of agents. This number will increase in the future and also depends upon what type of account you have.
Agent Limits (Maximum number):
- Basic: 8
- Advanced: 16
- Enterprise: 24
Obstacle Avoidance
Built in obstacle avoidance is available. There are currently two options available, which depend upon which account type you have signed up for. The two options are: Right Hand Rule and SWARM OA.
Right Hand Rule
The right hand rule algorithm is provided, with the logic listed below. An explanation of how the algorithm works is also provided.
Code
Explanation The right-hand rule utilizes a built in LiDAR system for each agent. The bounds of the LiDAR are set to be +/- 45 degrees from the midline of the vehicle and +/- 30 degrees from the horizontal midline of the vehicle. Each lidar rotation sends back a matrix of detected distances to the terrain in a ray from the sensor to the ground. For a SWARM vehicle, this LiDAR is mounted 8 inches below the centerline of the vehicle to provide ground sensing support.
Algorithmically, the system will first attempt to move forward at a fixed speed of 8 meters / second. If an object is detected within the frame of view of the camera the vehicle will stop. Then the vehicle will execute a right turn (90 degree turn to the east) and attempt to move forward, repeating the process. If the system has completed a 360 degree rotation and cannot move forward, the vehicle turns 180 degrees and moves backward.
Each time the algorithm moves, the direction the action that takes the system closest to the goal, as defined by you. For example, you wanted to go to (55, 20, -10) (remember that the z-axis points down so we need negative altitude to represent above the ground), then the vehicle would execute right turns towards that objective.
Video Coming Soon!
SWARM Obstacle Avoidance (OA)
The SWARM obstacle avoidance is a custom made avoidance algorithm that utilizes the same LiDAR sensor. However, this is a discrete state algorithm that processes and decides how to avoid obstacles so as to minimize the amount of time requried to achieve the objective. This algorithm is the default OA algorithm for Advanced users and can be purchased for basic users.
States The three states of the system are as follows:
- Slope Follow
- Edge Follow
- Forward Movement
These states represent three distinct algorithmic solutions that control the vehicle to the desired end point. Slope follow is utilized to ensure that the agent can stay close to the ground to achieve the objective. Edge Follow is activated when the agent has reached a specific altitude relative to the ground (AGL) that is a constraint for the vehicle. Forward Movement defines a trajectory that allows the vehicle to move forward while avoiding obstacles that require a maximum deviation of 20 degrees from the current heading of the vehicle.
Video Coming Soon!
Perception
The SWARM perception system utilizes “perfect” object detections through a built-in process. The video game engine that is utilized allows the agent to “see” objects by casting a ray at the perscribed distance (currently set to 50 meters) from the vehicle and returns any detected targets. The detected targets are determined by a list of objects that can currently be tracked and will provide a Detection object for the User to utilize in determining what course of action to take.
Video Coming Soon!
Detections In each state of the Behavior module that you define, there is a list object that is passed to your state function called detections. See function below.
def rendezvous(self_state_vector: list,
swarm_state_vector: list,
detections: list,
incoming_messages: list,
system_information: dict) -> list:
Detections is a list object in Python that contains the detections made by the perception system at any one epoch of simulation. This list is composed of Detection objects, which are defined as a Python Data Class. See the definition of the class below.
@dataclass
class Detection():
"""
We generate a number of detections, whether through the AirSim
API or through a computer vision algorithm.
## Inputs:
- position [PosVec3] X,Y,Z NED position in the local coordinate
frame. Initialzied as X=0, Y=0, Z=0,
Frame=Local
- heading [float] Yaw angle of drone, with inital heading being 0.0
in Degrees
- speed [float] Speed to travel in the direction of travel in meters
per second
"""
position: PosVec3 = PosVec3()
gps_position: GPSPosVec3 = GPSPosVec3()
label: str = ""
timestamp: str = "
The Detection object also contains the PosVec3 and GPSPosVec3() objects, whose definition can be found the Object Libary.
Communications
The communications menu provides a set of parameters that apply to the communication system aboard each agent. Please see the documentation on the SWARM Firmware for a fully description of each communication system.
Messages per second
You can set the rate at which messages are sent from each vehicle. This is also relative to the number of times your code is run for the behavior that you define in the algorithm section.
Max Communications Range
This is the maximum distance at which any two agents can communicate with each other and is done via a simple calculation every time a message is received. The current position of the drone that received the message is used to calculate a Manhattan distance.
Noise
To emulate noisy environments, a noise parameter is added which samples from a specific probability distribution, normally a Gaussian. This sample determines if the message is received or not.
Behavior Module (Your Algorithm)
Once the different aspects of the agent have been set, you can now define the behavior of your agent. This can be done in a number of ways, by either developing your algorithm to complete the scenario or by utilizing a number of provided algorithms. This system is designed to be as flexible as possible to enable you to writeł complex behaviors and accomplish diverse sets of scenarios.
The defined paradigm is explained briefly in the module. However, this paradigm is important to understand. The cycle is defined below in the picture and incorporates the common paradigm for autonomous systems.
Add Image
Receive
Each timestep in the simulation, agent i receives messages for all other agents. A message has the following format and structure.
{
"Header": {
"Pos": {
"X": 0.0,
"Y": 0.0,
"Z": 0.0
},
"Heading": 0.0,
"Vel": {
"X": 0.0,
"Y": 0.0,
"Z": 0.0
},
"ID": "Drone1",
"Timestamp": "",
},
"Body": {
// Your Message here //
}
}
The header of this message is filled in before each message body is sent. The message body is defined by the current state that your system is in at the timestep of the simulation. You will pass the body of the message during the execution of your algorithm and the system will send that information.
The body of these messages are aggregated and provided to you in the state function as a Python list (C++ Vector). These messages are defined as incoming_messages. The headers of the message are aggregated and provided as the list swarm_state_vector, which is a list of lists, with each list containing state data. You are also provided the current agents state data as the input self_state_data, which contains the following information.
// Vector is [Drone ID, Position X, Position Y, Position Z, Velocity X, Velocity Y, Velocity Z, Heading, Speed]
// Units are meters, meters per second and degrees (for heading)
self_state_data = ["Drone1", 1.0, 1.0, 1.0, 0.5, 0.5, 0.5, 5.2, 3.2]
Compute
Here you define the behaviors of each agent in the SWARM. A copy of the algorithm that you write will be provided to each agent and run independently.
To start, you are provided a pre-defined Initialize state, which is meant to define all of the state variables that you will use throughout the excution of the scenario. You define a state variable by creating a new key-value in the dictionary that is provided in the initialize method, called system_information.
Example
system_information["position"] = [0.0,0.0,0.0]
You can also define helper functions in this state, different classes and import libraries. A list of supported imported libraries is provided below and available during algorithm input.
Supported Python Libraries
- Numpy Version 1.22
- Scipy Version 1.8.0
- Python Control Library 0.9.2
- Toml
For your convience, you can then define a set of states. Each state is a discrete state of behavior your agent should complete. This state has the same 5 inputs as the initialize method. You can transfer state by returning the appropriate next state as a string. See the example below.
def initialize(self_state_vector: list,
swarm_state_vector: list,
detections: list,
incoming_messages: list,
system_information: dict) -> list:
"""
Initialize the system information dictionary with the variables and
data structures required for follow on algorithms. The last item in
the self_state_vector is the current state of the agent. Change this
state to the first state to execute after making this. We return the
self_state_vector to keep the drone in the same location.
Provided information:
"numb_drones" [int] the number of drones in the swarm
"search_zone" [tuple] a tuple of tuples of coordinates in the
(x,y,z), of the form ((x,y,z),(x,y,z),...)
"""
system_information["leader_id"] = "Drone1"
system_information["at_origin"] = True
system_information["last_position"] = PosVec3()
# You can access the trajectory through:
# system_information["Trajectory"], which is a list of dicts containing
# points in the form {"X": 0.0, "Y": 0.0, "Z": 0.0}
next_state = "formation_control"
# Return the next POSE command and the message to send.
return False, False, next_state
When returning the next position, you can use the provided function generate_next_position function that is always defined in the initialize function. See the function below.
def generate_next_state_vector(x: float = 0.0,
y: float = float(),
z: float = float(),
heading: float = float(),
vx: float = float(),
vy: float = float(),
vz: float = float(),
speed: float = float(),
roll: float = float(),
pitch: float = float(),
yaw: float = float(),
throttle: float = float(),
move_by: str = "position") -> list:
# x = 0.0 # meters
# y = 0.0 # meters
# z = 0.0 # meters
# heading = 0.0 # degrees
# vx = 0.0 # meters / sec
# vy = 0.0 # meters / sec
# vz = 0.0 # meters / sec
# speed = 0.0 # meters / sec / sec
# roll = 0.0 # meters / sec / sec
# pitch = 0.0 # meters / sec / sec
# yaw = 0.0 # meters / sec / sec
# throttle = 0.0 # scalar between 0.0 and 1.0
# move_by = "position" # Options are "position", "velocity", "acceleration"
return [x, y, z, heading, speed, vx, vy, vz, roll, pitch, yaw, throttle, move_by]
This function generates the next_position vector with the appropriate outputs for the SWARM system to interpret and provide to the path planner. You can return any of the fields and then set how you want to control the vehicle by passing the move_by flag. See the function above for the options.
Validation
Once you have written your algorithm, you can validate the algorithm. This validation is a simple sanity check that ensures your defined function and state will compile and run in the system. This also ensures you have the proper inputs and outputs, but will not simulate any of the behaviors. Please ensure all states are validated, as you will not be able to submit until they are.
Transmit
The message body that is provided is then incoporated into the overall message format. This message is then transmitted to every agent in the SWARM.
Details
Once you have completed the input, you can add a name to your algorithm. You then have the option to either save or submit your algorithm. If you save your algorithm, you edit and submit that algorithm later through the Simulation Feed. Once you submit the algorithm, you will receive a series of notifications and emails about the progress of your simulation.