Digital Twin Robot Concept in DASA

We build up the digital twin robot concept in DASA with the aid of Unreal Engine (UE) . UE is a 3D computer graphics game engine developed by Epic Games, and it’s widely used in video games, film and television. We utilizes powerful UE functionalities in DASA to realize digital robots with realistic appearance and also permit real-time synchronization with their counterparts in reality.

../_images/robot_view.png

Software Components

../_images/ue_robot_comp_en.png

The digital twin robot concept consists of several components as depicted in the figure above:

  • DASA infrastructure software modules: DASA software to provide robot localization, communication and messge synchronization as illustrated in the tutorial Start Localization Process in Indoor DASA System. The software is located in ~/DASA_space/catkinws_dasa.

  • UE engine: The UE 4.27 engine is installed in the DASA desktop delivered with our solution under /home/dasa/src/UnrealEngine-4.27-plus, and we create customized projects with plug-ins for our DASA robots.

  • Airsm ROS wrapper: The Airsim ROS wrapper enables interactions with digital robots in the UE engine by exposing interfaces to the ROS network.

Setup Procedure

  1. Robot end configuration

    We have to follow the tutorial Start Localization Process in Indoor DASA System to install and build the catkinws_dasa workspace, and configure the topics that are shared across the network. The following are the topics for 1 uav and 2 rovers:

    '/dasa/local_position/pose',
/uav1/dasa/local_position/pose', '/uav1/dasa/local_position/velocity',
'/rover2/dasa/local_position/pose', '/rover2/dasa/local_position/velocity',
'/rover3/dasa/local_position/pose', '/rover3/dasa/local_position/velocity'

  2. DASA computer configuration

    • We have to install and build the catkinws_dasa workspace in the same fashion with the robot end.

    • We need to configure the Airsim plugin that is located in the file ~/Documents/Airsim/settings.json. By default, users don’t need to change the file. Please remember to back it up before any modification.

  3. Build the Airsim ROS wrapper

    # install software prerequisites for the first time only
sudo apt install gcc-8 g++-8
sudo apt install ros-melodic-mavros* ros-melodic-tf2-sensor-msgs

cd ~/src/cloudkernel_proj/Airsim/ros
catkin build -DCMAKE_C_COMPILER=gcc-8 -DCMAKE_CXX_COMPILER=g++-8

How to Start

  1. Robot end

    We need to turn on the DASA_SWARM router, and then power on each robot. By default, each robot will start the node corresponding to the catkinws_dasa workspace and connect to the DASA_SWARM network. Users can also refer to the tutorial Start Localization Process in Indoor DASA System for details.

  2. DASA computer end

    • Use the ping command to check whether each robot is connected to the local area network, e.g.:

    ping uav1
ping rover2
ping rover3

# or ping their corresponding ip address directly
ping 192.168.0.101
ping 192.168.0.102
ping 192.168.0.103

    If ping returns a value, it proves that each robot has been connected to the Internet. Otherwise, you need to check and troubleshoot the network connection.

    • Start the run_gcs_visualization.sh script in the ~/DASA_space/catkinws_dasa/scripts directory, and then use the rostopic command to see the pose topics published by all robots.

    • Start the UE engine, run the city sandbox project, and click the play button.

    • Start Airsim’s ROS wrapper, then try to manually move the robot to view the UE engine display. Normally, the robot’s position in the virtual world is synchronized with the actual one.

    ~/src/cloudkernel_proj/Airsim/ros
source devel/setup.bash;
roslaunch airsim_ros_pkgs airsim_node.launch;
../_images/ros_demo.jpg