Guest article: A Story of Autonomous Logistics

From rapid robot prototyping to pre-series robot production

a guest article by Deparment of Autonomous Logisitics of StreetScooter

The robotic delivery arm of Eva was self-constructed at StreetScooter

The robotic delivery arm of Eva was self-constructed at StreetScooter

The vision of a generic toolbox to solve automated delivery challenges was born in the Department of Autonomous Logistics in 2016. ROS was chosen as the framework because it was quite popular among students in robotics and many suitable open source modules were already identified. At the same time, a ROS-based software stack for urban autonomous driving called Autoware was released open source. This was a blessing for the young robotic team since multiple components could later be adapted for Eva’s Follow Me Delivery function. The fresh robotic engineers could learn from the experiences made with this stack, without having a senior robotic expert in the team. With the test track next to the developers’ office, a short iteration cycle was introduced to gather the knowledge needed.

Eva, Adam and Alfi for Follow Me and Autonomous Parcel Delivery

The first prototype was not Adam but Eva, constructed in partnership with TAS (Institute for Autonomous Systems Technology of the Bundeswehr University) [1], PEM (Production Engineering of E-Mobility Components) of RWTH Aachen and Beckhoff Automation. Eva was constructed to demonstrate the autonomous parcel delivery.

After showing the first promising in-house developments in software and hardware design, two realistic use-cases for applying robotic technology to logistics were identified in 2017:

  1. Follow Me Delivery
  2. Autonomous Yard Logistic

Both developments were chosen based on the agile mindset to deliver benefits as early as possible to the customer. A maximum speed of 10 km/h was a promising entry point since an emergency stop was possible under all circumstances. In the meantime, the usage of open source technology for robotics prototyping increased since this showed strong acceleration in the development. The re-usage of components between both use cases and vehicle types was given by the modularity of the ROS Framework [2] Two StreetScooter Work vehicles, Adam and Alfi, have been equipped with the Follow Me Delivery system (Adam for rapid prototyping and Alfi to show the next steps in system design and focus on industrialization).

Most systems have been integrated into the roof top of Alfi. In this way the integration of the Follow Me Delivery System into a series StreetScooter Work M vehicle was possible.

Most systems have been integrated into the roof top of Alfi. In this way the integration of the Follow Me Delivery System into a series StreetScooter Work M vehicle was possible.

Adam and the Demonstration of Follow Me Delivery

Nvidia invited StreetScooter to demonstrate the Follow Me Delivery function on a test track next to the NVIDIA GTC conference at Munich in 2017 [3]. The new cooperation announcement on the conference and the immense press feedback reveal the potential of Follow Me Delivery. The software itself was a combination of multiple open source ROS modules integrated into the basic move_base navigation framework of ROS. The team showed that, by combining and adapting multiple ROS modules like depth_clustering or osm_cartography of different organizations, the development of an autonomous vehicle is possible. Safety was given by the low-level controller that supervised the controllability of the system in combination with a trained safety driver.

The ground was quite uneven. This led to false-positive obstacles detected in the ground.

The ground was quite uneven. This led to false-positive obstacles detected in the ground.

Obelix for Prototyping in Autonomous Yard Logistics

In 2018, the first prototype system for the autonomous yard logistics was equipped on an electric Wiesel truck from KAMAG. At this time, the vehicle base itself was also a prototype. The first step was to automate the steering to reduce safety risks of the heavy truck with a maximum weight of about 26 tons. That way, the acceleration of the vehicle was still in control of the safety driver. This level of automation generated lots of interest at DHL since the safety concept is much simpler and the system costs are lower in comparison to a fully autonomous system.

Many benefits like lower material wear, less noise and simpler vehicle handling were still given. This worldwide unique concept was named Assisted Maneuvering and Positioning System (AMPS). Field-tested software and hardware solutions from Alfi have been adapted to the new vehicle Obelix. Based on the experiences made from the open source packages of Follow Me Delivery, a new in-house development has been started. Some powerful packages like robot_localization or Google's carthograpther are still used in our software stacks today. Because of a planned in-field test at a DHL parcel center in cologne in 2019, much more requirements and quality management had to be introduced. LiDAR was chosen as the main sensor system because a high precision in localization and detection with an error margin smaller than 3 centimeters is demanded in changing and demanding outdoor conditions.

Obelix at his daily mission on the test track of Avantis.

Obelix at his daily mission on the test track of Avantis.

Snow tracks on Avanits in the LiDAR measurement. Even under those conditions, the system needed to adhere to its requirements.

Snow tracks on Avanits in the LiDAR measurement. Even under those conditions, the system needed to adhere to its requirements.

Asterix at the parcel centers of Cologne and Hanover

In 2019, Asterix and the AMPS system were deployed to the parcel center Eifeltor next to Cologne. The operation of the new electric Wiesel vehicle from KAMAG in combination with AMPS was possible after smaller adoptions. The container loading worked right from the start, but the loading dock approach was not precise enough under all circumstances. Goal poses of the docks were defined only by mapped GNSS measurements. Even with a high-end localization system, metal objects and walls around Asterix disturbed the radio signals of the satellites. These experiences led to the development of an active loading dock detection that is fused with the global goal pose. After 3 months of daily operation, Asterix and the AMPS system received very positive feedback from evaluation with multiple DHL test drivers. The fenced area of the parcel center was an ideal use case to gather first experiences in mixed traffic with robotic transportation system. Afterward, Asterix was also successfully tested at the freight section of DHL on a newly constructed parcel center without any adaptations of the system or environment [3]. The open source package Marv Robotics supported us in the creation of a bagfile database.

Datasets from the parcel center operation tests were crucial for further development of AMPS and higher levels of autonomy.

Datasets from the parcel center operation tests were crucial for further development of AMPS and higher levels of autonomy.

LiDAR data of multiple containers and loading docks on the parcel center. Loading docks and containers can be detected with sufficient accuracy without artificial landmarks.

LiDAR data of multiple containers and loading docks on the parcel center. Loading docks and containers can be detected with sufficient accuracy without artificial landmarks.

Simba, Asterix and Columbus for Industrialization of AMPS

Based on customer feedback, data analysis methods and advanced robotic components test benches, a pre-series version of AMPS is in development. The design is focused to increase adaptability. Therefore, multiple vehicle types can be supported. Drivers of the DHL parcel centers will evaluate the system in daily operation. They will be supported by developers on demand, when the driver activates the remote access to the AMPS system. A precise absolute and relative localization is demanded for the precise maneuvering of the vehicle. The GNSS and IMU systems that have been used in the prototyping phase were too expensive and nontransparent. That's why in-house hardware and software designs have been done based on state-of-the-art electronic components. The system is called Columbus.

Simba, the virtual vehicle, got quite popular since most developers work remotely during the COVID-19 pandemic. The continuous integration testing framework runs multiple scenarios on a virtual parcel center. Since the LiDAR sensors can be simulated in the Gazebo simulation, the complete software stack is been tested closed-loop. Most errors in the software development can be detected, therefore, in this stage before deploying to the vehicle. In that way the validation of the software components is done in an automated and reproducible way with every new release.

Vehicle, parcel center, LiDAR and Containers are simulated in detail for the test bench.

Vehicle, parcel center, LiDAR and Containers are simulated in detail for the test bench.

Idefix, our Hardware-in-the-Loop test bench, is gone be refactored with industrial graded hardware. Software and hardware integration aspects like networking can be analyzed before working directly in the car. In combination with our virtual vehicle we created a virtual driver seat to drive AMPS inside the simulation on the actual hardware. Asterix is also been used to evaluate ROS2, industrial graded middlewares and operation systems. Challenges at the integration of new software frameworks on the target hardware are identified in an early development phase.

Idefix gets new dresses. Mock-up designs in the simulation allows us in an early stage to evaluate new functionality with our customer.

Idefix gets new dresses. Mock-up designs in the simulation allows us in an early stage to evaluate new functionality with our customer.

New edition of the ROS MOOC from TUDelft for ROS beginners

New edition of the ROS MOOC from TUDelft for ROS beginners

We are pleased to announce a new edition of the ROS MOOC, Hello (Real) World with ROS. The course will open on 15 January 2020 at 13:00 CET on the edX online learning platform.

You can enrol now at the Course Webpage for a fun ROS learning journey!

This course is a part of the educational activities of the EU project ROSIN and is offered by the TU Delft Cognitive Robotics department with the support of the Online Learning School.

The target audience for the course are beginner level ROS1 users. The course will be instructor paced and of 6 weeks duration. A study/work load of about 8-12 hours per week is expected.

See you online from January 15th!

The Delft ROS MOOC team

Read More

Tech Workshop on MoveIt, security & skill oriented programming with ROS

The Fall edition of ROS-Industrial EU Tech Workshop took place at Fraunhofer IPA on October 09th and 10th, 2019.

We were glad to host two European MoveIt maintainers, namely Henning Kayser of ROS-Industrial Consortium member PickNik Robotics and Michael Görner from University of Hamburg. They gave us an insight into the latest developments of MoveIt (incorporating motion planning, manipulation, 3D perception, kinematics, control & navigation), current and planned developments for ROS2 (MoveIt2), and a hands-on on ROS(1)-based 'bare-metal to product'. First they presented an inside-view of the manipulation framework. Providing complementary academic and industrial perspectives, they shared their views and experiences on MoveIt's overall structure, practical deployment of planning-based pipelines, complex manipulation planning using the MoveIt Task Constructor, and upcoming future projects and ideas for a ROS2 migration. The workshop concluded with a practical session that guided the participants to setup a functional Pick&Place pipeline from a custom bare robot description. Slides and code examples are available at https://github.com/henningkayser/ROS-Industrial_EU_Fall19_MoveIt .

20191009_112123.jpg

The first session on day 2 of the ROS-Industrial EU Fall'19 Workshop was about security in ROS where Sebastian Taurer from JOANNEUM RESEARCH presented his work on a penetration testing tool for ROS1, called 'ROSPenTo', and gave an introduction on how to use SROS2 to secure communications in ROS2. In the first part of the session ROSPenTo was introduced to provide basic information on how it works and what a user can do with it. During the hands-on section the participants were guided through a step-by-step manual showing how to analyse, penetrate and modify a running ROS1 system using ROSPenTo. In the second part of the session ROS2's security tools (a.k.a. SROS2) were explained and used to setup and configure a security infrastructure. The provided examples demonstrated the creation of all necessary security artefacts (e.g. keys, certificates, etc.) and also the procedure to securely distribute the artefacts to different machines. All the related information as well as the workshop tutorial can be found here: https://github.com/jr-robotics/ROS-Industrial_EU_Fall19_Workshop

EGg33rKWoAAHuv5.jpg

The ScalABLE4.0 session at the ROS Industrial EU Fall'19 Workshop focused on presenting the set of technologies which are enabling flexibility in production lines in two industrial pilots of the automotive sector: PSA Peugeot Citroën and Simoldes Plásticos. Within the project, a complete digital manufacturing software stack is being developed, entitled 'Open Scalable Production System' (OSPS). The OSPS aims to be applied to efficiently and effectively visualize, virtualize, construct, control, maintain and optimize production lines through a tight integration of the enterprise information systems with transformable automation equipment paired up with the necessary open interfaces for optimized solutions on all hierarchy levels (slides).

20191010_091701.jpg

During the workshop, attendees were introduced and got a chance to test, interact and develop with the set of components that compose the OSPS, namely: (i) The Advanced Plant Model, which is responsible for virtually integrating data from the industrial shop floor in a centralized digital twin; (ii) The Production Manager, which is a cloud-based software module that issues and supervises the execution of manufacturing tasks; (iii) SkiROS and Task Manager, which are distinct ROS-based approaches to orchestrating the behaviour of robotic systems; (iv) The Skill-based Robot Programming methodology, which enables the reutilization and adaptation of ROS-based robotic applications to different purposes, platforms, and environments; (v) and, finally, the ROS-CODESYS bridge (ROBIN - https://github.com/ScalABLE40/robin), which enables horizontal integration between robots and automation equipment.

As part of the Scalable project, Bjarne Grossmann from AAU and cofounder of RiACT presented their skill-based robot control software SkiROS v2 (slides). Their technology is based on extended behavior trees that allows the definition of reactive behavior for highly flexible manufacturing environments. The framework is backed by a semantic database for inference and support of task planning to automatically generate complex tasks. In the hands-on session, Bjarne demonstrated the system with a SkiROS-implementation of the classical ROS turtlesim demo. He showed that SkiROS can be easily used to create complex behavior (and not only for turtles). The demo can be found on the git repository https://github.com/Bjarne-AAU/skiros-demo. Soon, there will be an official open source release of the software. Stay tuned on www.riact.eu!

Next European expert workshops will be organized in Spring and Fall 2020. We will keep you posted!

PS: Some links to upcoming events in this respect:

ROS-Industrial Consortium Europe is heading towards ROS2

With the growing excitement and curiosity surrounding ROS2, ROS-Industrial Consortium Europe (RIC-EU) had the pleasure to host the Spring 2019 edition of the RIC-EU Tech Workshop. It took place on May 6th and 7th at Fraunhofer IPA in Stuttgart, Germany. Some of the main drivers of DDS and ROS2 developments personally presented their insights and gave hands-on sessions during the event. For this, participants were provided with USB sticks with Ubuntu Bionic and ROS Melodic and ROS Crystal pre-installed (just as for all our ROS-Industrial trainings). The event has been free for worldwide members of any ROS-Industrial Consortium and was fully booked out with 40 people attending from all over Europe.

On Day 1, the workshop started with RIC-EU manager Thilo Zimmermann who welcomed the participants at Fraunhofer IPA and introduced the ROS-Industrial Consortium Europe and its EU project funding opportunity (next cut-off dates June 14 and September 13, 2019).

As ROS 2 supports multiple DDS/RTPS implementations, RIC-EU proudly hosted one of the most popular DDS vendors, eProsima, to explain the main concepts of DDS and present their stack at the workshop. During the five hours of presentations and hands-on workshops, Borja Outerelo Gamarra and Jaime Martin Losa covered topics like DDS Introduction, presentation of the standard and motivation of DDS & DDS Architecture, and DDS QoS. Attendees practised on a “hello world” example. ePROSIMA's slides can be found here.

20190506_142557[1].jpg

On Day 2, Ralph Lange from RIC-EU member BOSCH gave an in-depth presentation of the current status of ROS2. He included hands-on tasks using ROS2 and sow new features and also provided information on the upcoming d-turtle “Dashing Diademata” release on May 31, 2019. Ralph's presentation slides "Current Status of ROS2 - Hands-on Feature Overview" can be found here.

20190507_090637[1].jpg

The second presentation by Ingo Lütkebohle, also from BOSCH Corporate Research, introduced the micro-ROS activity. Ingo is one of the investigators of the EU funded OFERA project, which ports ROS2 to “extremely resource constrained devices” (usually, microcontrollers) with the new DDS XRCE standard. He demonstrated this by using a Cortex M4 board mounted on a first generation Turtlebot. Ingo's presentation slides can be found here.

20190507_113032[1].jpg

After a lunch break, Ludovic Delval of Fraunhofer IPA gave a hands-on workshop on how to migrate ROS1 node to ROS2. Lastly, Harsh Deshpande, also from Fraunhofer IPA, previewed the porting of the ur_modern_driver to ROS2 and presented a proposal for the action_bridge, which currently bridges between ROS1 action client and ROS2 action server.

At the end of the workshop, participants and ROS-Industrial Consortium members agreed that 2019 is promising a lot of developments in ROS2. In April at ROS-I Consortium Americas 2019 Annual Meeting, RIC members interacted and exhibited an interesting panel session titled “Is ROS2 Ready for the Factory Floor”. In June, Ludovic Delval of Fraunhofer IPA will present the latest updates at ROSCon France in Paris and Harsh Deshpande at the ROS-Industrial AP Workshop 2019 in Singapore.

The next RIC-EU Tech Workshop is foreseen for Fall 2019 (tentative dates October 09-10). The 2019 edition of the ROS-Industrial Conference is planned on December 10-12, 2019 (save the date!).

ROS Industrial Conference #RICEU2018 (Session 4)

From public funding opportunities to the latest technologies in software and system integration, the combination of robotics and IT to hardware and application highlights: ROS-Industrial Conference 2018 offered a varied and top-class programme to more than 150 attendees. For the sixth time already, Fraunhofer IPA organized a ROS event in Stuttgart to present the status of ROS in Europe and to discuss existing challenges.

This is the fourth instalment of a series of four consecutive blog posts, presenting content and discussions according to the sessions:

  1. EU ROS Updates (watch all talks in this YouTube playlist)
  2. Software and system integration (watch all talks in this YouTube playlist)
  3. Robotics meets IT (watch all but 1 talks in this YouTube playlist)
  4. Hardware and application highlights (watch all but 1 talks in this YouTube playlist)

Day 3 - Session “Hardware and Application Highlights“

Georg Heppner (FZI) and Fabian Fürst (Opel) At ROS-Industrial Conference 2018

Georg Heppner (FZI) and Fabian Fürst (Opel) At ROS-Industrial Conference 2018

In the fourth and final session of the ROS-Industrial Conference 2018, the focus was on hardware developments and applications implemented in industrial use cases. Fabian Fuerst, Opel, and Georg Heppner, FZI, delivered the session keynote. They presented their solution for flexible automotive assembly with industrial robotic co-workers. The application was developed as part of the EU EuRoC project. In this four-year competition, more than 100 participants initially worked on new robotic solutions for the manufacturing industry. In the course of several evaluation rounds, the team from FZI, Opel and MRK Systeme GmbH was able to assert itself successfully to the end.

During the course of the project, the FZI developed an automated robotic assembly for flexible polymer door sealings on car doors. The sealing is a closed ring, which has to be fixed with up to 40 plastic pins depending on the model, an ergonomically unfavourable task that could not be automated until now. The developed assembly cell is very flexible and open, so that the robot can be used without a safety fence. For this purpose, an external force control was developed that can be used easily and directly also for numerous other robots as a package of ROS-Industrial. The CAD-2-PATH software is used for the simple path creation for the robot. This enables a quick adjustment to other door models and does not require any expert knowledge. This is important because there are different door models and sealing types and the automation solution must be adaptable accordingly and quickly. It is notable that the application received positive assessment from Opel with regards to safety, typically a sensitive topic when applying novel tools such as ROS in automotive applications.

Paul Evans (Southwest Research Institute / ROS-Industrial North America) at ROS-Industrial Conference 2018

Paul Evans (Southwest Research Institute / ROS-Industrial North America) at ROS-Industrial Conference 2018

The presentation by Paul Evans, Southwest Research Institute and ROS-Industrial Consortium North Americas, provided current information on the activities of the North America Consortium such as strategic initiatives, trainings, and networking activities. These also focus on voices of members and include activities for the strategy alignment, for more robustness and flexibility and agility. There are also collaborations with OEMs who support ROS or develop their own drivers. At the ROS-I Consortium Americas Annual Meeting 2018, different applications were presented, for example an order batch picking robot from Bastian Solutions and a robotic system for agile aerospace applications like sanding, blending, drilling etc. for the U.S. Air Force. A last highlight that Evans presented was the ROS-I collaboration with BMW and Microsoft. While RIC-North Americas supported the evaluation of simulation environments that included physics engines the RIC-EU partners provided additional navigation support and training for mobile robots at the BMW plant to support assembly logistics. The solution is deployed on Microsoft Azure.

Mobile robots was also the topic of the lecture by Karsten Bohlmann, E&K Automation. He presented solutions for ROS on AGVs and perception-driven load handling and PLC interfaces.

Arun Damodaran (Denso) at ROS-Inudstrial Conference 2018

Arun Damodaran (Denso) at ROS-Inudstrial Conference 2018

Denso Robotics Europe was present at the conference with Arun Damodaran, who talked about Cobotta, the ROS-enabled collaborative robot. This is a six-axis arm with a reach of 342 mm, a repeatability of 0,05 mm and a payload of 500 g. It has an inherently safe design, meets all requirements for safety-standards corresponding to the ISO norms and is compliant thanks to safety-rated monitored function. Another advantage is its easy set-up and use. This is realized by the usage of the robot programming software drag&bot. Developed by the spin-off of the same name of Fraunhofer IPA, the software enables the programming of robots like Cobotta with the drag and drop principle. No expert knowledge is needed. The software is also based on ROS, works independently from any robot manufacturer and can be reused as well as shared via the cloud. Denso has been engaged in the development of ROS components and packages (simulation, control, path creating) for its robots since 2012 and now uses an open platform for controlling the Cobotta.

Felipe Garcia Lopez from Fraunhofer IPA focused on a networked navigation solution for mobile robots in industrial applications. This is particularly useful for changing environments in which mobile robots should independently select free routes. Fraunhofer IPA and Bär Automation, for example, have implemented a navigation solution for agile assembly in automobile production. With this, AGVs can locate themselves robustly and precisely based on sensor data, even without special infrastructure. This makes it possible to easily adapt existing paths or integrate new ones even after commissioning. Since the software's sensor fusion module can process data from almost any sensor, very customer-specific solutions can be implemented.

Another example is the networked navigation for smart transport robots at BMW. Here as well there were few static landmarks, a lot of dynamic obstacles and sparse sensor data in large-scale environments. A process reliability of more than 99% had to be fulfilled. The presented navigation as well as the vehicle control are ROS-based. At the end of the presentation, an outlook into Cloud-Navigation was given: Mobile robots and stationary sensors are then connected using a Cloud-based IT-infrastructure. The environment is cooperatively modelled and SLAM is used. This enables also solutions for “Navigation-as-a-service” meaning map updates and cooperative path planning for each robot. With Cloud-Navigation, local hardware and computational resources can be reduced and the quality and flexibility of the overall navigation system is enhanced.

Thomas Pilz (Pilz GmbH & Co. KG) at ROS-Inudstrial Conference 2018

Thomas Pilz (Pilz GmbH & Co. KG) at ROS-Inudstrial Conference 2018

ROS as an appropriate solution both inside and outside of industry – this was the starting point for Thomas Pilz, Managing Partner of family owned company Pilz. Combined with his own career and his experience with the first service robots, lightweight robots and robots outside production environments, he first described how the question of safety standards has changed in recent years. The definition and understanding of a robot is currently in the process of changing significantly. For Pilz, systems such as the Care-O-bot® from Fraunhofer IPA are the new upcoming robots. They operate outside of cages, are mobile and users can easily interact with them and program them using ROS. He sees ROS as a success factor for service robots because of its modular design, its standardization, additional flexibility through programming languages and its networked, interoperable system in line with Industry 4.0.

Robots that are to interact with humans are also changing the required safety technology at Pilz in the long term because all previous infrastructure such as fences is no longer required. This led Pilz to develop its own robot arm with appropriate safety technology. They use ROS modules developed by Pilz because they are breaking new ground with the development of the robot arm and can thus fall back on a broad programming knowledge base. They had nothing to lose with the new product. However, in order for them to meet the safety standards, the modules must no longer be changed in an uncontrolled manner. To improve this, Pilz recommends changing the safety standards so that they are also amenable to Open Source. Finally yet importantly, he believes that the term robot manufacturer will also change, because this role will increasingly be fulfilled by those who implement the application and no longer by those who produce the robot or components for it. In the lively discussion after the presentation, Pilz once again emphasized two arguments in favour of ROS. First: When it is said that ROS is tedious, one should bear in mind that the development of proprietary software is also difficult. Second: ROS is tedious, but fun. Pilz also sees ROS as a decisive factor for employee satisfaction and as an argument for staying with Pilz.

At the end of the conference, Gaël Blondel from the Eclipse Foundation presented the Eclipse Foundation and its Robotics Activities. The platform with around 280 corporate members, half of them from Europe, provides a mature, scalable, and business-friendly environment for open source software collaboration and innovation. Eclipse is vendor-neutral and offers a business-friendly ecosystem based on extensible platforms. They offer their own IP management and licensing but also accept other business-friendly licenses. Several working groups are particularly engaged in development processes for robotics. One example for a robotic project managed with Eclipse is the EU project RobMoSys that aims to coordinate the whole community’s best and consorted efforts to realise a step-change towards a European ecosystem for open and sustainable industry-grade software development.

At the end of the event, Mirko Bordignon and Thilo Zimmermann thanked the participants for another great and record breaking ROS-Industrial Conference. Presentations and videos of the event have been made available on the event website: https://rosindustrial.org/events/2018/12/11/ros-industrial-conference-2018

ROS Industrial Conference #RICEU2018 (Session 3)

From public funding opportunities to the latest technologies in software and system integration, the combination of robotics and IT to hardware and application highlights: ROS-Industrial Conference 2018 offered a varied and top-class programme to more than 150 attendees. For the sixth time already, Fraunhofer IPA organized a ROS event in Stuttgart to present the status of ROS in Europe and to discuss existing challenges.

This is the third instalment of a series of four consecutive blog posts, presenting content and discussions according to the sessions:

  1. EU ROS Updates (watch all talks in this YouTube playlist)
  2. Software and system integration (watch all talks in this YouTube playlist)
  3. Robotics meets IT (watch all but 1 talks in this YouTube playlist)
  4. Hardware and application highlights

Day 2 - Session “Robotics meets IT“

Henrik Christensen (UC San Diego) at ROS-Industrial Conference 2018

Henrik Christensen (UC San Diego) at ROS-Industrial Conference 2018

The third session testified the growing importance of ROS to support the development and deployment of robotic solutions from companies outside the traditional boundaries of this industry. Predominantly software players such as Amazon or Google now offer platforms leveraging ROS, which they described during the session.

Henrik Christensen, from UC San Diego and ROBO Global, gave a very inspiring keynote speech on why robotics is increasingly using cloud technologies and how it will benefit from them. He outlined three factors as current business drivers for this development: the increasing demand for flexibility in production, the aging world population and the associated increasing demand for service robots at home, and finally the trend that more and more people live in cities, posing great challenges for logistics. All robot solutions must be cost-efficient and robust at the same time in order to offer the required reliability. If computer performance always had to be on board, the hardware would often be inadequate (e.g. for slim service robots for private use) or the costs for suitable hardware would be too high (e.g. for autonomous cars).

Technologies from or in the cloud can be a solution for this. Christensen presented the value of these ecosystems using extensive market examples and explained how they differ in agility and size. Many successful companies, primarily in the USA and Asia, have shifted their business model from owning things or technologies to orchestrating them and offering services. For robotics, ROS 2.0 can be a decisive door opener here, offering the standardization required for platforms.

Milad Geravand (Bosch Engineering) at ROS-Industrial Conference 2018

Milad Geravand (Bosch Engineering) at ROS-Industrial Conference 2018

The next presentations in the session took up these and similar ideas and presented existing solutions. Milad Geravand from Bosch Engineering presented a modular software platform for mobile systems such as cleaning, off-road and intralogistics robots and how they can be developed more efficiently. In his experience, the difficulties in the development process are similar in many companies: The applications are usually very different, the software is becoming increasingly complex, a structured deployment and integration process is lacking. ROS is not yet ready for the products and the leap from prototype to series production is still too big. With the software platform presented, which is based on ROS, Bosch would therefore like to address precisely these challenges and enable uses cases to be developed quickly and reliably.

Eric Jensen, working for Canonical, the company well known for the Ubuntu Linux distribution, presented the advantages of Ubuntu Core especially with regard to security that is still an open issue for ROS. The mentioned advantages are: A minimal, transactional Ubuntu for appliances, safe and reliable updates with tests and rollbacks, app containment and isolation with managed access to resources, a unique development environment familiar for Linux developers and the possibility to easily create app stores for all devices needed. Furthermore, Ubuntu has one of the biggest developer communities in the world and is backed by Canonical itself, an important plus for security. Last but not least, the system offers automatic security warnings for the „snaps“, the special package format in Ubuntu, system audits through package verification and compliance management – all are important features for an improved security.

Roger Barga (Amazon AWS) at ROS-Inudstrial Conference 2018

Roger Barga (Amazon AWS) at ROS-Inudstrial Conference 2018

Only a few weeks before the ROS-Industrial-Conference, Amazon, for a long time far more than an e-commerce store, had introduced its new platform AWS RoboMaker, which caused a sensation beyond the ROS-Community. Roger Barga, General Manager at AWS Robotics & Autonomous Services, kindly presented this novel development at the conference. Amazon's commitment to robotics is based on discussions with around 100 companies, during which they were able to identify two main problems in robot development. On the one hand, this is a very high demand for automation solutions with simultaneous difficulties with ROS such as security or performance. On the other hand, the development process is usually very inefficient.

The RoboMaker platform addresses these requirements with its four main components. It offers a browser-based development environment, which in turn has integrated cloud extensions for ROS as well as a simulation environment. The cloud extensions range from machine learning tools to monitoring and analytics. Concrete capabilities for robots include speech recognition and output, video streaming, image and video analysis, as well as logging and monitoring with Amazon CloudWatch. The simulation environment allows thousands of simulations to be run in parallel. The fourth component is fleet management, so that robot applications can be deployed over the air. The presentation ended with a short introduction to the learning environment of RoboMaker, with which Amazon applies reinforcement learning to robots. The robots then learn according to the principle "trial and error". By merging all errors within a fleet in the cloud, a large knowledge base is quickly available and not every single robot has to make a specific error to learn from, but it benefits from the learning experiences of other robots in the fleet.

The topic of robotics in the cloud was also the focus of the lecture by Christian Henkel from Fraunhofer IPA. In his experience, the deployment of ROS-based applications on distributed systems such as mobile robots is still too great a challenge, which he would like to address in his work with docker containers (dockeROS). With his solution, it is possible to simply run ros nodes in docker containers on remote robots.

Martin Hägele (Fraunhofer IPA) moderates a panel discussion with Henrik Christensen (UC San Diego), Oliver Goetz (SAP), Michael Grupp (magazino), Niels Jul Jacobsen (MiR) and Damon Kohler (Google).

Martin Hägele (Fraunhofer IPA) moderates a panel discussion with Henrik Christensen (UC San Diego), Oliver Goetz (SAP), Michael Grupp (magazino), Niels Jul Jacobsen (MiR) and Damon Kohler (Google).

With Damon Kohler, Google Robotics and its recently presented cloud solution were also represented at the conference. In his introductory remarks, Kohler mentioned several challenges related to cloud robotics, including security, connectivity and latency, and distributing work, e.g. partitioning problems. In contrast, he sees advantages such as scalability, collaborative perception and behaviour and a robust change management and monitoring. He sees cloud robotics as a further development of the well-known principle "sense -> plan -> act" around the component "sense -> share -> plan -> act" and as an interplay of edge and cloud processing.

The aims of cloud robotics are an increased launch cadence, more data and more users and a better resource utilization. This shall be reached by infrastructure as a service, design for small and decoupled components as well as tools for automation and orchestration. The ROS nodes correspond to the Google micro-services: They are stateless and replicable, which means horizontally scalable. The container orchestration engine Kubernetes helps to deploy and release these micro-services. Several mature and robust logging and monitoring tools like Stackdriver help managing the system. The heart of the whole is the Cloud Robotics Core, being available from beginning of 2019 that enables to integrate Kubernetes on robots. Overall, Google’s vision is an open platform and a thriving ecosystem where integrators, developers, hardware developers and operators can collaborate with customers efficiently.

The second day of the conference ended with a panel discussion. The panellists were Henrik Christensen (UC San Diego), Oliver Goetz (SAP), Michael Grupp (magazino), Niels Jul Jacobsen (MiR) and Damon Kohler (Google). Moderated by Martin Hägele (Fraunhofer IPA), they summed up some advantages from their respective company perspectives, but also existing challenges of ROS and the role of open source software and robotics for their corporate strategy.

ROS Industrial Conference #RICEU2018 (Session 2)

From public funding opportunities to the latest technologies in software and system integration, the combination of robotics and IT to hardware and application highlights: ROS-Industrial Conference 2018 offered a varied and top-class programme to more than 150 attendees. For the sixth time already, Fraunhofer IPA organized a ROS event in Stuttgart to present the status of ROS in Europe and to discuss existing challenges.

This is the second instalment of a series of four consecutive blog posts, presenting content and discussions according to the sessions:

  1. EU ROS Updates (watch all talks in this YouTube playlist)
  2. Software and system integration (watch all talks in this YouTube playlist)
  3. Robotics meets IT
  4. Hardware and application highlights

Day 2 - Session “Software and System Integration Topics“

Dave Coleman (PickNik) at ROS-Industrial Conference 2018

Dave Coleman (PickNik) at ROS-Industrial Conference 2018

The second day of the conference started with the session "Software and System Integration Topics". Dave Coleman, founder of Picknik Consulting and lead maintainer of MoveIt!, opened the session with a very personal keynote about his commitment to open source software, from his student days to his role as an entrepreneur. He reported how he got in touch with the beginnings of ROS at Willow Garage and highlighted the unique spirit with which the project was incubated. He introduced the successful MoveIt! library, shared his lessons learned and the challenges which many open source projects face. As a proof of how Open Source and business can successfully coexist, he described the founding of PickNik and how the company is profitable without investors.

The following presentations were more technical and started with Víctor Mayoral Vilches, CEO of Acutronic Robotics. He talked about his company's solutions for system integration in modular systems, through the device H-ROS SoM (System on Module), used as example. In his opinion, ROS already addresses many programming needs, but system integration goes far beyond programming and requires extensive resources for each new project. He therefore sees modularity as an essential improvement. Combining the features of a real-time capable link layer made of RTOS and the Linux Network stack, and ROS 2.0, he presented the challenges and developed solutions to achieve easier system integration. He also gave insights into the use of AI to further reduce programming efforts and to train the robot instead, a technology that is still in its infancy. As part of a Focused Technical Project with ROSIN, the company also worked on the interoperability of modules.

Jon Tjerngren (ABB) at ROS-Industrial Conference 2018

Jon Tjerngren (ABB) at ROS-Industrial Conference 2018

Jon Tjerngren presented how ABB robots can be used with ROS. For this purpose, the company developed various ease-of-use packages with ROS that simplify and accelerate the setup of ABB robots. All of them are already freely available online: abb_librws can be used to off-load of computational heavy tasks, e.g. image processing. abb_libegm can be used for motion correction and as an StateMachine add-in for remote control.

ROS2 Embedded tailored to real-time operating systems was the topic of Ingo Lütkebohle’s presentation from Bosch Corporate Research. He emphasized the importance that ROS must also be integrated into the firmware. This would better address four challenges: hardware access, latency, power savings, and safety. To this end, he presented a solution developed in the OFERA project with which ROS2 can be used in microcontrollers.

André Santos from INESC TEC and University of Minho, focused on software quality. More and more robot systems are safety-critical systems, which places very high demands on the quality of the software. Finding errors in the code early on reduces costs and development time. Although there are various static analysis tools, none offers ROS-specific analysis. This is why the HAROS (High Assurance ROS) framework was developed, which is capable of extracting and, to some extent, reverse-engineering the computation graph. It also provides a visualization of the extracted graph and enables property-based testing for ROS.

Anders Billise Beck (UR) at ROS-Inudstrial Conference 2018

Anders Billise Beck (UR) at ROS-Inudstrial Conference 2018

Anders Billersoe Beck from Universal Robots was the last speaker in the second session. He introduced the new UR e-series (with integrated force/torque sensor, 500 Hz controller frequency and more new features) and how ROS supports it. For this, a new driver is developed in a Focused Technical Project of ROSIN together with the FZI, which will also remain open-source. The goal is to make a UR robot easy to use and enable plug-and-play with ROS. The driver should make two modes of operation possible: remote control and ROS URcap embedding. More supported features are calibration, a new safety system and easier programming. Beck concluded the presentation with some points that he believes are in need of improvement to make ROS ready for industrial applications. These are easier general use, proper handling of hard and soft real-time boundaries and supporting more control in edge devices.

ROS Industrial Conference #RICEU2018 (Session 1)

From public funding opportunities to the latest technologies in software and system integration, the combination of robotics and IT to hardware and application highlights: This year's ROS-Industrial Conference 2018 offered a varied and top-class programme to more than 150 attendees. For the sixth time already, Fraunhofer IPA organized a ROS event in Stuttgart to present the status of ROS in Europe and to discuss existing challenges.

This is the first instalment of a series of four consecutive blog posts, presenting content and discussions according to the sessions:

  1. EU ROS Updates (watch all talks in this YouTube playlist)
  2. Software and system integration
  3. Robotics meets IT
  4. Hardware and application highlights

Day 1 - Session "EU ROS Updates"

Mirko Bordignon (Fraunhofer IPA) opening ROS-Industrial Conference 2018

Mirko Bordignon (Fraunhofer IPA) opening ROS-Industrial Conference 2018

The topic of open source software for robotics was present in the media throughout the year, and announcements that companies such as Google, Amazon and Microsoft would rely on ROS made waves outside the community, too. In addition, there is a booming robotics market. Martin Hägele (Fraunhofer IPA) highlighted this in his opening talk based on current market figures and areas of application for industrial and service robotics. In this respect, it is not surprising that politics and research funding on a national and international level are becoming increasingly aware of ROS. The speakers on the first day of the conference presented the projects and activities currently underway here.

ROSIN project overview

Bringing ROS into industrial application in Europe is one of the main activities of the EU project ROSIN (ROS-Industrial Quality Assured Robot Software Components) that Carlos Hernandez Corbato of TU Delft presented. This runs from 2017 to 2020 and is mainly involved in three fields:

  1. Further development of ROS components within the framework of so-called Focused Technical Projects (FTPs)
  2. Tools for software quality assurance
  3. Education activities

Applications for FTPs can still be submitted until 2020. The next cut-off date is April 5th 2019. All information on the short application process can be found here. A decisive criterion: The project provides funding for developments for which there are concrete market requirements. For this reason, the project finances one third of the software development and the applicant takes over the other two thirds.

Carlos Hernandez Corbato (TU Delft) at ROS-Industrial Conference 2018

Carlos Hernandez Corbato (TU Delft) at ROS-Industrial Conference 2018

Successful FTP examples Pilz, Nobleo, PPM and Roboception

ROSIN granted already 20 applications for FTPs and 21 more are under review. Here are four examples of successful FTPs:

For Pilz, “Industrial Trajectory Generation for MoveIt!” was granted: Most industrial robot manipulators supported in ROS come with a MoveIt! configuration. The Motion Planning plugin for RViz allows simple and visualized planning and execution of free-space motion. Planning and obstacle avoidance work mostly out-of-the-box. This FTP addresses Cartesian motion: existing libraries for Cartesian trajectory generation lacked a user-friendly interface. The FTP implements a trajectory generator with a MoveIt!-interface for easy planning and execution of Cartesian standard-paths. In addition, the blending of multiple sequential motion commands is realized.

For Nobleo Projects, “Full Coverage Path Planning and Control“ was granted: Many robotic applications need to plan a path that passes over all points of an area or volume of interest while avoiding obstacles. As soon as a path is planned, the next challenge is to control it. As neither ROS, nor ROS Industrial are currently providing needed packages incorporating this (complete) coverage path planning or trajectory tracking functionality, this FTP proposes to develop, verify and validate these packages.

For PPM, the FTP “ROSweld” was granted: It develops an innovative ROS based framework for planning, monitoring and control of multi-pass robot applications with an intuitive, user-friendly GUI. The framework is built upon components from the project partners’ previous research and existing ROS modules. ROSWELD is demonstrated by the case study in heavy, multi-pass welding.

For Roboception, the FTP “Visard4ROS” was granted: Visard4ROS will provide a ROS interface to fully exploit the capabilities of the rc_visard sensor and to easily integrate it into robotic products or research platforms. As part of the process, Visard4ROS will also provide documentation for integration of sensors with standard industrial interfaces such as GigE Vision and GenICam, plus examples and good practices for using separate libraries to build ROS-I hardware drivers.

Education activities

Stephan Kallweit (FH Aachen) and Jonathan Hechtbauer (Fraunhofer IPA) at ROS-Industrial Conference 2018

Stephan Kallweit (FH Aachen) and Jonathan Hechtbauer (Fraunhofer IPA) at ROS-Industrial Conference 2018

The second goal of ROSIN are education activities. Stephan Kallweit (FH Aachen) and Jonathan Hechtbauer (Fraunhofer IPA) presented the two formats with which the project conveys ROS knowledge. One of them is the ROS-I School. It addresses university students and young professionals to get an entry to the ROS-Industrial eco-system. Its teaching concept consists of seminars, tutorials and workshops. In addition, ROSIN has founded the ROS-I Academy. It consists of a ROS-I certified engineer program to assess certain skills within the ROS-Industrial software engineering eco-system. The certified skills comprise basic knowledge in ROS-Industrial, skills in code review and specialised ROS-Industrial topics. Check out the website for upcoming events.

Quality Assurance Tools

The third main activity of the ROSIN project are measures and technologies to improve the quality of software. Adam Alami and Zhoulai Fu (IT University of Copenhagen) presented the ongoing steps. On the one hand, a process and supporting tools are developed for quality assurance, where the quality of packages can be measured, assigned and displayed. Furthermore, ownerships for QA practices, tools and infrastructure will be appointed. Furthermore, code review practices are going to be reinstituted and a code scanning method and tool will be implemented. A quality hub website is already online in order to create a source of knowledge for quality assurance. A source of collaboration for quality assurance offers this page.

Another quality improvement measure is the automated code testing. Traditional platforms are not effective enough to provide the reliability that ROS needs today as they run the same and very few test harness for many times. However, a ROS package is reliable, when it works as expected for all run-time scenarios. That is why ROSIN aims at developing a reliability-oriented testing framework that will be integrated to the ROS ecosystem.

Outlook: Further research activities thanks to RobMoSys and SeRoNet

The conference day ended with two contributions on other research projects that also rely on ROS. Dennis Stampfer (University of Applied Sciences Ulm) presented RobMoSys. It aims at coordinating the community’s efforts to realize an industry-grade software development European ecosystem that is open, sustainable and ensures industrial quality. This shall increase the scalability and quality of robotics software development, help to commoditize base functionality, such as motion control, navigation, software components of certifiable quality and achieve predictable system integration. It addresses user requirements like, among others, reduction of development time and costs, shorter time to market and safety via a model-driven approach.

Björn Kahl (Fraunhofer IPA) presented SeRoNet. This project intends to significantly simplify the design, development, and deployment of service robots in a variety of areas, from logistics, care, and healthcare to assembly support in manufacturing operations. Through an online platform, users, system integrators and component manufacturers of service robot solutions will be able to collaborate efficiently and jointly support solutions from requirements analysis to deployment. The SeRoNet platform (available from summer 2019) will bring together users and producers of robotic solutions and will create a market for service robot solutions, services and hardware as well as software components for application solutions. Both projects will publish Open Calls in 2019, for which companies involved in robotics can apply for funding.