Open Source Improvements from Recently Completed Robotic Blending M5

A Focused Technical Project (FTP), championed by the Steel Founders’ Society of America (SFSA), sponsored by the DLA-Troop Support, Philadelphia, PA, and the Defense Logistics Agency Information Operations, J68, Research & Development, Ft. Belvoir, VA, within the ROS-Industrial Consortium (RIC), has recently completed the Focused Technical Project (FTP) Robotic Blending Milestone 5. The team included RIC team members consisting of Yaskawa America, PushCorp and Southwest Research Institute, along with SFSA member university, Iowa State University. This work culminated in a deployed system at a SFSA member foundry site.

The project built on the prior Robotic Blending Milestone 4, which demonstrated high-mix material surface finishing and edge processing of arbitrarily shaped and contouring parts, largely targeting piece-parts to be welded. This work sought to extend that work, adding new features and incorporating additional SFSA funded work to realize human in the loop high-mix casting finishing for foundry operations.

During the development process, we have contributed the following improvements to the foundational Scan-N-Plan framework that served as a starting point for the FTP. This framework is maintained as a workshop repository within the RIC GitHub repository and is used in whole or in parts for developer instruction in ROS 2 for industrial robotics.

Scan-N-Plan Updates

ROS 2 Control:

We have added ros2_control code to the Scan-N-Plan example implementations to preview the robot’s motion during a simulated motion execution.

Constant TCP Velocity Time Parameterization:

We have added a time parameterization algorithm for maintaining constant TCP speed during motion planning. The approach works by first calculating forward kinematics at each trajectory waypoint to get the pose of the TCP. Then, we create a line path between adjacent TCP poses and parameterize the line with a trapezoidal velocity profile. The joint velocity and acceleration at each waypoint are computed given the TCP Cartesian velocity and acceleration. These obtained velocities and accelerations are then validated to be within the limits of the robot.

Docker Images:

To make the deployment and development of Scan-N-Plan more efficient and standardized, we have created Docker images for ROS 2 Foxy, Humble, and Rolling that can be found here.

Selectable Representation for Collision Object:

In addition to the default behavior of converting the scan mesh into a convex hull, the collision object can now be represented as a detailed mesh or as an octree.

Default behavior: Converts the scan mesh into a convex hull. This generally results in the fastest motion plans, especially with TrajOpt, but can be too conservative and may cause motion planning failures if the scan object is not actually convex or nearly convex.

Convex Hull Representation of Collision Object

Mesh: Represents the collision object as the exact “detailed” mesh represented by the mesh file. With the contact test type CLOSEST for TrajOpt, this representation results in a somewhat slower planning time than convex_mesh, but not significantly longer.

Mesh Representation of Collision Object

Octree: Represents the collision object as an octree comprised of spheres with a diameter specified by the octree_resolution parameter. With the contact test type CLOSEST for TrajOpt, this representation results in slightly faster planning times than mesh but slower than convex_mesh.

Octree Representation of Collision Object

Simplified Raster Planner

Simplified Raster Planner Widget

The parameters exposed in the raster planner GUI widget were modified to only show commonly tuned parameters during raster planning such as rotation offset (degrees), point spacing, line spacing, minimum segment length. This declutters the raster planner to improve usability.

Python Scanning and Execution Nodes

For ease of development and debugging during deployment efforts, we have converted the mesh reconstruction and motion execution simulator nodes from C++ to Python.

Service for Generating Scan Motion Plans

A separate service for generating scan motion plans has been made as an intermediate step to support creating dynamic scan trajectories in the future. This service will be called to create scan trajectory patterns that originate from a specified starting location.

Behavior Tree and Reactive GUI

The custom application back-end logic has been replaced with a behavior tree to improve workflow modularity and customizability. The behavior tree can change the GUI appearance based on the current behavior executed. Widgets are exposed and hidden during the workflow to guide the user’s focus to actions and settings relevant to the process at hand. A progress bar and preview motion bar have been made more accessible to inform the user where they are in the process.

Noether Updates

Tool path and Mesh Visualization Tool

Mesh/Tool path Viewer Widget

When developing and debugging tool path and mesh modifiers, it is helpful to visualize the original tool path, the modified tool path, the original mesh, and the modified mesh. Previously, there was no way to view the unmodified sub-mesh after applying a mesh modifier.

We have added a widget in the Noether application to toggle the view of the mesh and tool path VTK objects. We replaced the concatenated mesh viewer using a vtkAssembly and using vtkSmartPointers instead of using raw pointers for internal objects.

From Left to Right: Modified Tool path and Modified Mesh, Modified Tool path and Original Mesh, Original Tool path and Original Mesh, Tool path Hidden and Modified Mesh

Approach and Departure Tool Paths

To better stay within selected regions during processing, we added linear approach and departure tool paths (see below for the linear approach tool path). A modified version of these tool paths (circular approach/departure) creates an approach/departure curve of specified radius.

Linear approach tool path modifier

Plane Projection Mesh Modifier

We added a mesh modifier that fits a plane to the input mesh using random sample consensus and projects the inlier points onto the plane. This modifier prevents tool paths from being generated on an unintended inner or outer surface near the edge of a complex part, where the desired processing surface should be flat.

We hope developers and those interested in experimenting with ROS 2 for their industrial robotics application development have found this resource helpful. If you have questions, comments, or have observed an issue, please do not hesitate to either engage with the ROS-Industrial community or leave an issue over at the GitHub repository.

Thanks to Michael Ripperger, ROS-I Consortium Americas Tech Lead, SwRI Sr. Research Engineer, for his contributions to the Scan-N-Plan Workshop and the FTP program and this blog post.

Cartesian Waypoints with Tolerance in TrajOpt

When translating a robot trajectory from a motion plan in a ROS system to an actual executed motion, there is an inherent loss of precision. In general, trajectories are sent to a controller that adheres to the exact position and timing constraints to the best of its ability, but compromises must be made to execute the trajectory. Additionally, robots cannot achieve infinite precision in positional accuracy because the kinematics of the system cannot be known with infinite precision. We have participated in work to maximize position and velocity accuracy, but not every process requires this level of precision. The process planning component should match the level of precision required by the application, but, traditionally, motion planners work assuming infinite precision. That is why we have now introduced tolerance into our motion planning pipeline when using Cartesian waypoints in TrajOpt.

TrajOpt is an optimization-based motion planner that uses a seed trajectory along with costs and constraints to refine towards a better trajectory. This process typically involves avoiding collisions and smoothing the motion. Previously, whenever we specified a Cartesian waypoint, the tool frame needed to adhere to the desired waypoint exactly. The total error across all six degrees of freedom (6DOF) could deviate by 1e-4 meters/radians, or it would be considered a constraint violation. In applications where we allowed free rotation or motion about an axis, the coefficient associated with that constraint could be set to zero, but users were not able to set bounds on the motion. For example, a common ROS-I application would allow free rotation about the Z-axis, so we would set the last coefficient in our waypoint coefficient vector to zero. An example of this process can be seen in the Scan N Plan Workshop.

Both the extremely high precision requirements and all-or-nothing approach to 6DOF Cartesian freedom of motion do not fit most applications. Additionally, these tight constraints can often cause unnecessary motion planning failures when a small freedom of motion would enable success. With the new tolerance parameters available in TrajOpt, we can more closely match the requirements of the system while reducing the number of motion planning failures. Now available in the trajopt_default_plan_profile are various settings to tailor the various waypoint requirements to the system’s needs.

Below are two example gifs. The first shows a failed plan when no tolerance was allowed (except for free rotation about the z-axis), and the second example illustrates a successful plan when tolerance was enabled: 15 mm in X and Y, 1.5 mm in Z, and 0.01 radians in rotation around X and Y (again, free rotation about Z was allowed). Visually, these two motions look almost identical, and infinite precision would not be required for this buffing process. Using tolerance in this application will enable the robot systems to align with the process requirements and improve overall success at incorporating robotics into various processes.

failed plan when no tolerance was allowed (except for free rotation about the z-axis)

successful plan when tolerance was enabled: 15 mm in X and Y, 1.5 mm in Z, and 0.01 radians in rotation around X and Y (again, free rotation about Z was allowed)

Shaping the Future of Robotics: Highlights from the ROS-Industrial Consortium Asia Pacific Summit 2023

The ROS-Industrial Consortium Asia Pacific Summit 2023 was held on the 29-30 November 2023. The event marked a milestone in ROS journey in Asia Pacific where we have seen the growth from the time it started in 2016 to where it is today.

Throughout the 2-day event, ROS-Industrial brought together experts and industry leaders to discuss topics that have shaped and will continue to shape the future of robotics. The Annual Summit, themed “The Everyday Robot”, signifies how robots will be an integral part of our daily lives, at work, at play and at social life. The presentations and demonstrations provided a glimpse of how robotics applications have unknowingly integrated into various aspects of our lives, from industry, service, education, among others.

The event also witnessed the signing of two MOUs. Firstly, an MOU with Southwest Research Institute (SwRI) was signed to reinforce our shared commitment to collaborate as part of the ROS-Industrial global consortia.

MOU Signing with Southwest Research Institute

Secondly, an MOU was signed between ARTC and seven esteemed IHLs in Singapore, namely Nanyang Technological University Singapore, National University of Singapore, Ngee Ann Polytechnic, Republic Polytechnic, Singapore Institute of Technology, Singapore Polytechnic and Singapore University of Technology and Design. This collaboration aims to harmonize the curriculum for ROS-based courses to respond to industry demand, nurturing talents for the future workforce.

MOU Signing with IHLs

Dr. Ettikan Kandasamy Karuppiah, Chief Technologist of Enterprise Business at the Asia Pacific South Region of NVIDIA, took the stage to deliver his keynote speech on "The Next Wave of Robotics and Edge AI." He discussed how NVIDIA ISAAC open platform leverages cutting-edge GPU technology to make robots more capable of handling complex tasks in dynamic environments, enabling them to operate autonomously and efficiently in a wide range of applications. Dr Ettikan also touches on the exciting world of GenAI and various use cases where GenAI can be leveraged on.

Dr. Ettikan Kandasamy Karuppiah, Chief Technologist of Enterprise Business at the Asia Pacific South Region of NVIDIA

The three ROS-Industrial consortia also provided an update relating to the latest ROS2 developments and community outreach efforts within each of the regions.

Maria Vergo, Consortium Manager for ROS-Industrial Consortium Asia Pacific shared about the ongoing effort within Singapore to extend the use of Robotics Middleware Framework (RMF) to applications outside of healthcare, such as manufacturing and logistics.

Maria Vergo, Consortium Manager for ROS-Industrial Consortium Asia Pacific

Michael Ripperger, Technical Lead from ROS-Industrial Consortium Americas shared their work in robotics and shared ROS 10 years journey. ROS-Industrial Consortium Americas also shared about the Focus Technical Project (FTP) that was done in collaboration with their members on robotic blending with reconfigurable workflow and characterization-based path planning for quality assurance. Furthermore, Michael shared about the growth of ROS 2 driver support from leading robotics makers, such as Yaskawa, Kawasaki, ABB and UR. He also shared about SWORD, a platform that utilizes ROS-I tools in CAD-based environment among many other initiatives that ROS-Industrial Consortium Americas are pursuing.

Michael Ripperger, Technical Lead from ROS-Industrial Consortium Americas

Christoph Hellman Santos, Consortium Manager of ROS-Industrial Consortium Europe shared information about their software engineering and system integration, as well as their ongoing efforts in Europe. He also covered the different state funded research programs and projects throughout the years.

Christoph Hellman Santos, Consortium Manager of ROS-Industrial Consortium Europe

The afternoon session continued with engaging presentations by industry leaders.

Luca Della Vedova, Software Engineering Manager at Intrinsic, delved into "Unlocking the Potential of Robotics Through Automated Negotiation.” He introduced Open-RMF, an open-source framework for peer-to-peer negotiation to overcome deadlock issues in heterogeneous fleets of mobile robots, as well as discussed, the need for mobile robots to cooperate and dynamically share space in unstructured environments.

Luca Della Vedova, Software Engineering Manager at Intrinsic

Swaminathan Vangal Ramamurthy, General Manager at OMRON Automation Centre, shared insights into "Happy Factory - Where Men and Machines Operate in Harmony." In his presentation, he discusses the evolving relationship between humans and machines, highlighting the shift from collaboration to harmony in the journey towards an Autonomous Society.

Swaminathan Vangal Ramamurthy, General Manager at OMRON Automation Centre

Dr. Zhang Jingbing, R&D Director for Smart Robotics Automation (SRA) at ARTC, rounded off the series with a presentation on "RMF 2.0 – Enabling Large Scale Implementation of RMF for Cross-Sectorial Applications." In his presentation, he dived into the NRPO-funded program, uncovering the advancements in RMF2.0 technologies designed to overcome challenges in deploying mobile robots, as well as the R&D efforts in proliferating the use of RMF for manufacturing and logistics applications.

Dr. Zhang Jingbing, R&D Director for Smart Robotics Automation (SRA) at ARTC

Day 1 concluded with rotational workshops led by NVIDIA and ARTC where Dr. Ettikan of NVIDIA brought a topic on “The Future of 3D Collaboration with NVIDIA Omniverse and ISAAC Sim.” In this workshop, participants were treated to an immersive experience in leveraging NVIDIA ISAAC’s revolutionary capabilities and gained insights into utilizing Omniverse for scalable robotics simulation application development and synthetic data generation.

The second session was led by Dr Zhang Jingbing on "RMF – Market Readiness and Technology Requirements in Manufacturing, Logistic, and Facility Management”. During this workshop, facilitators shared about the existing effort to adapt RMF for cross-sectorial applications and prompted questions to guide the audience to share their vision of the future of RMF in these spaces.

The second day of the ROS-Industrial Consortium Asia Pacific Summit 2023 continued with another series of presentations and masterclasses.

Bobbie Couhbor, Senior Solutions Architect – Digital Innovation and Emerging Technologies at Amazon Web Services, started the day off with his keynote presentation titled “From Development to Production: Accelerating Robotic Development with Cloud Computing.” Participants gained insights on how several robotic vendors and consumers have used cloud computing to accelerate the development of their robotic capabilities and get their products to market rapidly without compromising on quality.

Bobbie Couhbor, Senior Solutions Architect – Digital Innovation and Emerging Technologies at Amazon Web Services

Next, we had Michael Sayre, Chief Executive Officer at KABAM Robotics, who shared insights on “Robots Among Us: Transforming Public Spaces and Resolving the Labor Availability Crisis”. He explored the transformative impact of technological advancements on revolutionizing facilities management and public safety globally, leading to enhanced efficiency and productivity through the deployment of Kabam’s Robots.

Michael Sayre, Chief Executive Officer at KABAM Robotics

Daniel Nia, Chief Executive Officer at HOPE Technik, presented “Journey of HOPE’s Robotics”, where he covers the journey of the company’s robotics ventures, spanning UAVs, drones, AV, AMR, UGVs, mobile manipulators, robotic cells, and exoskeletons.

Daniel Nia, Chief Executive Officer at HOPE Technik

Zhi Kai Lim, Senior Field Application Engineer at Real-Time Innovations (RTI), delved into his presentation “DDS in Robotic Systems: Advancements in Technologies and Ecosystem Collaboration”. He covers the pivotal role of RTI's DDS technology in advancing distributed systems, emphasizing its significance in achieving real-time advanced data management, ensuring data continuity, and shaping the future of technology integration and interoperability.

Zhi Kai Lim, Senior Field Application Engineer at Real-Time Innovations (RTI)

After the lunch break, the afternoon sessions commenced with Reeve Chong, Co-Founder, CTO and Managing Director at Hand Plus Robotics (Malaysia Office), presenting on “Designing, Assembling, and Deploying Reliable Robotic Packing Solutions for Large Scale Operations”. He showcased the successful deployment of a 6-robot packing system enabled by leveraging ROS to its fullest potential, achieving efficiency, reliability, and significant manpower savings.

Reeve Chong, Co-Founder, CTO and Managing Director at Hand Plus Robotics (Malaysia Office

Lastly, Professor Mohan Rajesh Elara, Provost’s Chair Professor at SUTD, concluded the series of presentations sharing his perspectives in a presentation titled “The Rise of Reconfigurable Robots: SUTD Initiatives.” He explored the challenges and advancements in the development of reconfigurable robots, focusing on deployment use cases in the maintenance domain and fostering a local ecosystem of robotic startups that were mostly spined off from many local talents from SUTD.

Professor Mohan Rajesh Elara, Provost’s Chair Professor at SUTD

The remaining time in the afternoon comprised of a series of masterclasses specializing in different focus tracks led by various industry experts.

Santosh Balaji and Adriel Ho, from ROS-Industrial Consortium Asia Pacific, delved into the fundamentals of behavior trees and their applications in robotics in his masterclass titled “Introduction to Behavior Trees and its Applications in Robotics.” Participants gained a profound understanding of behavior tree fundamentals, acquired practical skills for real-world task modeling, and explored the application of behavior trees in navigation tasks. His masterclass also featured a special guest virtually – Davide Faconti, the maintainer of BehaviourTree.CPP.

Christoph Hellmann Santos, from ROS-Industrial Consortium Europe, guided participants through the creation of a robot arm in ROS 2 from the ground up in his masterclass titled “ROS 2 CANopen Unleashed: Creating a Robot Arm from the Ground Up.” Utilizing ros2_canopen, attendees learned and gained a short hands-on experience on the process of building a robot arm in ROS 2.

Michael Ripperger, from ROS-Industrial Consortium Americas, conducted a masterclass titled “Tuning Motion Planning Parameters for Manipulators.” This session explored the practical aspects of fine-tuning motion planning parameters, incorporating algorithms such as industrial motion, OMPL, Descartes, and TrajOpt.

Vinu Sivanantham, from SUTD, led a masterclass titled “Enabling Self-Reconfigurable Robots using ROS.” Participants had the unique opportunity to utilize the Smorphi robot, unlocking the potential of versatile solutions for complex tasks in dynamic environments.

There were also booths stationed during the summit that consisted of prominent industry members in Singapore, namely Hope Technik, Weston Robot, Flexiv, Augmentus, Pepperl+Fuchs, KABAM Robotics, and ROS-Industrial, offered hands-on experiences with state-of-the-art solutions.

We would like to extend our deepest gratitude to all participants who graced us with their presence, contributing to the vibrant discussions, engaging presentations throughout the 2-day event.

A special acknowledgment goes out to our partners, AutomationSG, JTC Corporation, National Robotics Programme (NRP), and Workforce Singapore (WSG), who have made this event possible. We would also like to extend our sincere gratitude to our booth partners, Flexiv, Pepperl+Fuchs, Kabam Robotics, Hope Technik, Augmentus and Weston Robot.

We anticipate 2024’s edition of the annual summit, and look forward to more collaborations, and the continuous evolution of robotics, as well as the community’s contributions to open-sourced. See you in our next edition of the event!

RICA Community Meeting Puts a Bow on 2023 Developments

The ROS-Industrial Consortium Americas hosted a fourth quarter end of year Community Meeting that highlighted recent develoments and upcoming events for 2024.

Matt Robinson led off the meeting with programmatic updates and some of the findings from the recent roadmap workshops and highlighted that a roadmap revisit at the ROS-Industrial open source project level would be coming for 2024. Furthermore, updates on training, with a proposed schedule was shared along with the announcment of the ROS-Industrial Consortium 2024 Annual Meeting.

The 2024 edition of the RICA Annual Meeting will take place March 27-28 and include demonstrations and lab tours. There is also the oppportunity to consider some hands on workshop time and surveying of the membership will take place early in 2024 to nail down specific events and Annual Meeting content. The event listing will continued to be updated as details are firmed up.

New feature enabling submesh visualization in noether

For the technical portion of the program Michael Ripperger shared recent workshops and technical updates to various ROS-Industrial tools and resources. The first noted was REACH hihgligthed by the recent ROSCon 2023 workshop that took place in New Orleans. There are now two working demos included in the workshop repository, which will eventually be moved to the ROS-Industrial GitHub organization. This workshop may be offered as a stand alone workshop or as part of an existing ROS-I training event.

Next, Michael shared a recent enhancement to noether, the tool path planning repository within ROS-Industrial. For some time, noether has supported region selection and application of tool paths within a selected region. However, it was not able to provide a visualization of that sub-mesh. Visualizations of sub-meshes has recently been merged and this enables both an improved visualization experience for an end user, but also the means to more thoroughly understand how an automated segmentation function is performing during the application development process. This is exactly the use case for the ROS-I Consortium Focused Technical Project Robotic Blending Milestone 5, where there is interest in applying tool paths in human drawn boundaries.

Tool path tolerancing to enable improve trajopt performance

Tyler Marr then dove into recent developments within Tesseract and in particular TrajOpt. The first update was the ability to use custom tasks in Tesseract motion planning pipelines. Tyler provided a couple examples in the presentation and provided a URL reference in how this is used in the Scan-N-Plan workshop.

From here Tyler covered the ability to apply Cartesian tolerance waypoints for Trajopt. Historically, Trajopt would simply attempt to find a solution for a proivded tool path or set of way points wihtout understanding what additional flexibility may be available. By enabling the application of tolerances in x,y,z, roll, pitch, and possibly free rotation around z, trajopt has greater liklihood of finding a solution. In the recording there are comparison screen casts of before and after leveraging this feature. It is anticipated this will enable a greater array of more complex tool path presentations to be solved via Trajopt. This is currently a pull request, but it is anticipated to be merged before the end of '23.

Overview of sword - ROS-I planning tools in a CAD environment

Michael wrapped up the technical session with an update on SWORD, the ROS-I planning tool that operates in a CAD environment. An alpha release is planned in early January 2024, with a website and the ability to experiment with a trial version. If you have interest in experimenting with SWORD, feel free to contact any of the ROS-I Americas team.

Thanks to all those that made this meeting. You can find the presentation material here and over at the event listing. The recording is linked directly and can be found at the ROS-I YouTube Channel.

As always reach out if there are questions or if you are interested in contributing or being involved wiht the ROS-Industrial open source project or the Consortium. Have a great end of year and start to 2024 and as always thank you for your interest and support in open source for industry.

Shaping Future Roboticists: Highlights from the ROS-Industrial Bootcamp 2023

The ROS-Industrial Bootcamp 2023 recently concluded, marking another successful chapter in our ongoing commitment to nurturing the next generation of robotic innovators. This 4-day highly anticipated event, held from October 2nd to 5th, brought together 24 aspiring Roboticists from three Institutes of Higher Learning: Ngee Ann Polytechnic (NP), Singapore Polytechnic (SP), and Singapore University of Technology and Design (SUTD).

Since its inception in 2017, the ROS-Industrial Bootcamp has been a cornerstone of our mission to equip students with essential knowledge in Robot Operating System (ROS) before embarking on their final year of studies or internships. The 4-day intensive program is designed to bridge the gap between theory and practical application, ensuring that participants are well-prepared for the challenges that lie ahead.

The bootcamp commenced with a foundational session on the fundamental principles of ROS, expertly led by Vinu from SUTD. During this session, participants delved deep into the core concepts of ROS, gaining profound insights into its pivotal role within the realm of robotics.

On the second day, participants were introduced to the practical side of robotics. Collaborating with Weefa Robotics, a local startup specializing in educational robots, students had the unique opportunity to configure a robot, providing them with hands-on experience that is invaluable for their future endeavours.

The intensive 4-days program was packed with important ROS concepts. On day 1, students were provided with the basic knowledge of ROS. Manu, from SUTD provided a lecture on ROS

On the second day, the students were given the opportunity to configure a robot. This year, we engaged Weefa Robotics,a local startup specialising in education robot.

On the third day, the focus of the bootcamp shifted towards the exploration of advanced robotics applications that enable navigation for robots. Specifically, the day was centred around two crucial components: augmented reality and autonomous tracking. These in-depth sessions broadened the students' horizons while fostering an environment conducive to innovative thinking and problem-solving.

The highlight of the bootcamp on the final day was when students participated in a mini competition. Working in teams of two to three, they faced the challenge of guiding their robots through a maze in the fastest time. The competition brought together all the knowledge and skills acquired throughout the bootcamp, providing a platform for students to showcase their abilities.

We extend our warmest and most heartfelt congratulations to the exemplary champions who emerged victorious in the maze navigation challenge. Their remarkable achievements serve as a resounding testament to the unwavering dedication and extraordinary talent exhibited by our future robotic engineers.

At ROS-Industrial, we believe that events like the ROS-Industrial Bootcamp 2023 inspire young engineers to innovate and push the boundaries of robotics applications. As we look forward to the future, we remain committed to fostering the next generation of robotics pioneers who will undoubtedly shape the landscape of robotics and automation.

We would like to express our sincere appreciation to our esteemed trainers: Adriel, Shalman, Vinu (SUTD), Sheila, Fairuz, and our amazing interns - Yu Cheng, Gabriel, and Ryan. Your dedication and support have played a pivotal role in making this event a resounding success.

Author - Seok Yu Cheng

ROS-Industrial Consortium Americas September '23 Community Meeting

On September 21 ROS-I Consorium Americas hosted a community meeting where a number of topics were covered. The presentations have been made available and the recording may be found over at the ROS-I YouTube channel.

To kick things off I reviewed some Consortium initiatives. Most recently we have been addressing concerns around industrial_core. There have been comments on a ROS 2 support issue about porting of industrial_core. However, as pointed out on the GitHub repository we will not be porting industrial_core due to the desire to take advantange of advances in OEM provided external motion interfaces and new approaches in driver development such as ros2_control based drivers and the recent Yaskawa micro-ROS based driver.

The idea is that OEMs will see opportunity in engaging the ROS community by offering a driver they are comfortable supporting, therefore there are not issues relative to performance as OEM changes occur within their software stack. We look forward to following up with an updated industrial driver specification, and will keep the community engaged on progress on that front.

From there we covered a couple focused projects. You can catch all the details in the slides, but the Robotic Blending Milestone 5 Focused Technical Project was covered, as well as work and progress on SWORD.

The FTP has already resulted in numerous enhancements to scan_n_plan_workshop and will also realize improved capabilities in a few other supporting repos.

SWORD came about as manufacturing engineering stakeholders continued to request ease of use tools, that also enabled greater ties to the CAD environmetn. SWORD seeks to provide, via a plug-in to FreeCAD, a simple UI for end-users to set up work cells and do various types of motion planning and motion tuning operations. We look forward to showing more, beyond what is in this deck or this deck, of SWORD in the coming months.

For the guest speaker at the meeting we were excited to have Pablo Garrido Sanchez from eProsima on board to share the latest around micro-ROS and supporting advanced ROS 2 deployments on microcontrollers and similar resource constrained devices. Check out his deck also included here and on the event page, as well as his full talk on the ROS-I YouTube channel.

Thanks to all those that attended and we look forward to hosting upcoming Developer Meetings and the Workshop at ROSCon. As always check out ros-i.org/events for the latest in chances to learn and collaborate with others excited about open source for industry.

10th Meeting of the ROS-Industrial Consortium Americas highlights reaffirming of the mission

On May 25 in Detroit the ROS-Industrial Consortium Americas (RICA) held the 10th iteration of the annual meeting, where Consortium members can get together, learn from each other, assess the state of the ROS-Industrial open-source project and provide feedback and engage in guided discussions, all with the intent of providing a two way interaction to understand the state and happenings relative to ROS-I and to provide their needs and areas for improvement.

A few wrinkles were included in this year’s edition. First, during the opening talk around the state of the Americas Consortium Shaun Edwards offered insights and thoughts relative to the beginnings and what we should be possibly considering looking forward.

The morning program continued with regional updates, including recent work by ROS-I Consortium EU seeking to instantiate a ROS 2 reference work cell, and highlighted recent contributions and plans to realize their vision. ROS-I Consortium AP shared their recent efforts to realize system level capabilities stemming from their extensive work around the interoperability framework Open RMF.

Kat Scott of Intrinsic shared the latest features included in the recent Iron Irwini ROS 2 release which happened only days before. A number of features highlighted include inclusion of REP-2012 Service Introspection, which is sort of a ‘ROS debug mode’ for understanding service calls and their results. Also, pre and post parameter callback, to enable parameter lists on the fly, and external logger configuration to enable only one node in verbose mode if you choose, as well as improvements in launch, rclcpp, rcl, and CLI.

Dave Coleman of PickNik shared the latest developments around MoveIt Studio and their recent efforts in support of SpaceROS with some key highlights that drive relevancy for industry, such as code quality and reliability.

To wrap up the morning session David Poweleit of the Steel Founders’ Society of America (SFSA) shared their recent initiative to sponsor a collaboration project across a diverse team including Yaskawa Motoman, PushCorp, Iowa State University, Southwest Research Institute, EWI, and ARIS Technology. The intent is to build on prior Scan-N-Plan collaborative efforts, referred to as Robotic Blending which had been through 4 prior milestones, to realize a prototype job shop system to enable high mix low-lot foundry finishing. This served as a lead-in for the broader goal and mission around the SFSA and their members relative to enabling funded research to get into foundries more efficiently.

From here Keynote speaker Shane Whitaker, Applications Engineer at Lockheed Martin, shared how and why their team has chosen to leverage ROS and open source in their pursuit of advanced capability. The value proposition for the nature of the operations is around the realization of operational agility for their environments and the nature of the tasks they need to get done to produce the products in their factories. The number of processes of interest and developed, leveraging open source, were broad, and it was inspiring to understand that these developments are making a difference on the floors of Lockheed Martin factories.

Shane Whitaker from Lockheed Martin shares their Agile Automation journey and how ROS has made an impact for their operations

New for this iteration of the annual meeting was offering externally guided workshop time. Per past annual meetings we spend time discussing what is within ROS-I, seek to drive conversations to understand what those that are looking for capability need, both with regards to capabilities and resources to enable their success. A lot of feedback was gathered, and some really interesting insights were shared. A more formal report on feedback will be coming soon.

The additional workshops were led by David Poweleit of SFSA and Adam Norton of the Nerve Center at University of Massachusetts Lowell. David led a dialog, largely centered around the SFSA members that were in attendance, but also solicited great responses from other general industry members that have similar challenges as the foundry industry. Technology developers also engaged, asking about requirements, the desired types of capabilities, and means to operate.

Adam Norton along with colleague Berk Calli of Worcester Polytechnic University led a conversation around their National Science Foundation funded work on Pathways to Enable Open-Source Ecosystems starting with Open-Source Robotic Manipulation and Benchmarking: Current Gaps and Future Solutions. The scope was broad as it covered open-source assets, such as software, designs of physical robotic components and systems, datasets that support AI/ML, and instructional elements that enable benchmarking of various components. The goal is to drive a discussion on what is missing and how can we better assess and continue to grow leverageable open-source assets and compare them to sustain a future where open-source plays a key role in innovative development. You can learn more about this initiative via this handout and the presentation shared at the workshop.

Diving into the afternoon Bharath Rao, Senior Research Lead at Spirit AeroSystems, covered their vision for sustained deployment of advanced systems and solutions for entire value stream monitor and control across many assets, some human driven, others if not autonomous a future where they are. Bharath laid out a compelling vision for how a manufacturing entity may leverage open-source to realize a vision over incremental capability introduction that grows with testing, validation and improvement.

Dr. Eugen Solowjow, Head of Research at Siemens Berkeley, shared the recent work within Siemens to enable AI for robotics and how they are delivering products that enable AI solutions to run on and interoperate with traditional industrial hardware, such as PLCs.

The latter part of the afternoon included the latest in improved motion profile accuracy at high speeds for complex 3D curves presented by Dr. John Wen of Rensselaer Polytechnic Institute. This work has been really exciting for those that want to leverage autonomous motion planning but struggle with how the actual manipulator turns the sent trajectories into real motion. The two-step process includes a redundancy resolution algorithm that filters and adjusts the input target tool path and then leverages a reinforcement learning approach to further tune the manipulator that may be done on hardware via a handful of dry runs or in the OEM simulation engine if it accurately represents the physics. These tools have been provided in the RPI open-source repositories and have been developed in the course of an ARM Institute project.

Wrapping up the day, Joseph Giampappa of the ARM Institute provided an update on the latest initiatives within the ARM Institute, and the day concluded with Ted Miller of Yaskawa Motoman announcing the official release of MotoROS2. This is the latest in ROS 2 support in the industrial manipulator domain and the first where the ROS 2 node actually runs on the robot controller.

Looking back, it was a great week in Detroit and a great day spent with all the organizations, collaborators and individual developers seeking to make open-source for industry a leverageable resource for innovation in industrial settings. The continued development and latest work taking place both in the open-source and within the organizations that are leveraging open-source and making contributions, are building on the legacy and enabling new players to deliver novel new solutions. This brick-by-brick approach enables broader opportunities for innovation while enabling educational resources for future roboticists.

ROS-I Consortium americas Annual Meeting attendees gather for a group shot to wrap up the event

Thanks to all those that have made ROS-Industrial a successful collaboration, and thanks to all of those that have supported the ROS-Industrial Consortium around the globe. Through collaboration and a framework that enables continuous progress, open-source for industry can continue to make an impact for the advanced manufacturing needs of tomorrow.

All the talks described above are in a playlist at the ROS-Industrial YouTube channel. If you are interested in getting involved in the ROS-Industrial open-source project, you can learn more here. If you are interested in the benefits of the ROS-Industrial Consortium, you can check out the membership details here and feel free to reach out!

Americas Puts on Updated ROS 2 Training with Revamped Motion Panning Special Topic

This July, Southwest Research Institute (SwRI) hosted the most recent ROS Industrial training workshop. A large group came in from around the Americas region to learn a mix of ROS 2 fundamentals and advanced topics, as well as explore the city of San Antonio.

ROS-I trainers and students gathered to enjoy dinner out in San Antonio

To start the workshop, beginner students worked through the basics of ROS 2, while those already familiar with ROS were walked through our freshly updated exercise on performing motion planning with Tesseract. All students then came together to learn about setting up their robot URDF, do motion planning with MoveIt, and process pointclouds using PCL. Notably, these students were the first to experience the ROS-I training fully in ROS 2 Humble. No more spending time during training wrapping your mind around ROSbouncing between ROS 1 and ROS 2 1 just to get MoveIt up and running.

The students were also given a tour of Southwest Research Institute’s robotics facilities, where they got to see what they were learning in action. SwRI demonstrated a scan-n-plan application that utilized the ideas of motion planning and perception taught during training.

Students observing the demonstration of a scan-n-plan application on a physical robot

All training materials can be found on the ROS Industrial Training wiki. From there, you can download preconfigured VMs and jump right into exercises, or, if you want to stay on your own machine, our GitHub repository hosts a script that will install everything you need automatically.

If you are interested in hostingattending or organizing a future training, please reach out to a team member! We are always excited to bring our knowledge and expertise to you.

ROS-Industrial celebrates 10 years of uniting researchers and industry to solve industrial automation challenges with open-source software

Industrial robotics has changed dramatically from a decade ago, when a small group of researchers held the first annual meeting of the ROS-Industrial Consortium. Back then, much of the robotics research happening in university and corporate labs would languish in papers because it was difficult to replicate custom implementations using proprietary software.

Things started to change when the Robot Operating System (ROS) – an open-source software platform founded in 2007 by Willow Garage, the Stanford AI Laboratory, and Open Robotics – started reappearing in research papers. ROS helped open the door to R&D scalability across industry and academia, enabling a means to abstract robotics hardware, while also bringing a modularity and notion of abstraction and reproducibility to the robotic research space.

ROS caught the attention of a group of leading robotics companies and research institutions, including Yaskawa Motoman and Southwest Research Institute (SwRI). They teamed up with Willow Garage and founded the ROS-Industrial Consortium (ROS-I), which held its first annual meeting in 2013. On May 25 of 2023, the ROS-Industrial Consortium celebrates its 10th Anniversary.

In 2013, when ROS-Industrial held its first meeting at SwRI in San Antonio, there were 14 member organizations. Today, membership has expanded to over 85 organizations representing a broad range of industries and academic institutions.

The First ROS-I Consortium Gathering March 2013

How ROS-I Harnessed the Potential of Open-Source for Industry

The ROS-Industrial founding organizations saw the potential of open-source robotics technologies to revolutionize the manufacturing industry, and they came together to create a platform that would enable the development and deployment of these technologies in a cost-effective and efficient manner. ROS-I sought to bring more advanced computer science tools and standard path planning/motion planning libraries that were not sole propriety of the robot manufacturer. This confluence enabled investigation of a new approach to industrial robotics and a chance to set a path towards new capability that was not either only in a research lab or only available via one robot manufacturer.

Shortly after the founding of ROS-I, that first commit by SwRI’s Shaun Edwards and Paul Hvass—with the support of SwRI’s Paul Evans and Clay Flannigan—launched the ROS-Industrial Consortium. It wasn’t long before Fraunhofer IPA in Germany and the Advanced Remanufacturing Technology Centre in Singapore launched collaborative ROS-Industrial Consortia for the European Union and Asia-Pacific regions. Their energy and ingenuity have made Consortium growth and management such a fruitful experience.

Since that time the Consortia, via collaboration and engagement with industry, have made significant progress towards the goal of accelerating the development and adoption of open-source robotics technologies for industrial applications. Some of the consortium's key accomplishments include:

• Developing a comprehensive set of software tools and libraries that enable the use of ROS for industrial applications.
• Creating a vibrant community of ROS users and developers in the industrial robotics space.
• Conducting several successful Focused Technical Projects (FTPs) that have developed new capabilities for ROS-I.
• Working with leading robotics companies and research institutions to promote the use of ROS for industrial applications.

Growing ROS-I by Improving Standardization & Expansion

Shaun and Paul have since benefitted since joining and leading Plus One Robotics while also continuing to be stewards and champions for open-source collaborative development.

Shaun provided his take on what he has seen from the start of the ROS-I Consortium and what it has enabled: “In the very beginning of ROS-I, we found ourselves defending the robustness and reliability of open-source software. Only time would demonstrate that ROS could be as robust and reliable as commercial software as it was adopted across industrial and commercial robotics. Those companies with existing robot software stacks were not able to wholesale switch to ROS. Instead, they created integrations with ROS, while mimicking features of ROS that users found compelling (e.g. rich visualizations, physics simulation, and advanced computer vision). End users with complex and dynamic robot processes found ROS was necessary in order to build advanced solution. One such example was the ROS-Industrial Scan-N-Plan project(s). Scan-N-Plan leads to deployments at end users and was later followed by several startups who productized similar technologies for welding (Path Robotics) and sanding (GreyMatter). In addition to those examples, many startups without existing software stacks, were free to start with ROS as the core of their product offering. Plus One Robotics, Fetch Robotics, Renu Robotics, and Amp Robotics are just of few of hundreds. These companies have gone on to solve valuable problems and become respectable robot companies.”

Though a lot of progress has been made, there is still a role for a consortium that fosters collaboration between the open-source community, industry, and academia. Recently it has become very clear that standardization of various interfaces, communication standards, and modeling is lacking, manifesting in numerous translation efforts and frustration when multiple protocols are required within one application. Numerous working groups are in progress to investigate these, and the ROS-I Consortium has played a role in providing input to drive an industry perspective.

Shaun took the time to add his take on what the Consortium can contribute moving forward: “The consortium speaks with a louder voice today than ever before. It should use this new standing to expand its footprint. There remain many challenges in robotics today that could be addressed by cooperative research in the vein of Scan-N-Plan and other consortium projects.”

Shaun added that, “Expansion beyond robotics should also be considered. There are more open standards and open software in industrial automation today than ever before. An advanced robot cell contains robots, PLCs, and HMIs, as well as links to the cloud for analytics and planning. All these aspects have a software component that already have or could have has open-source alternatives. Just as ROS is now firmly a part of industrial robotics, the ROS-Industrial Consortium could usher in the age of open source for the entire factory.”

The Next Decade: Building Community & Integrating AI

ROS-I is well-positioned to continue to accelerate the pace of advanced robotics for industry in the years to come. The consortium has a strong track record of success, and it has a vibrant community of users and developers who are committed to the advancement of open-source robotics technologies. In addition, ROS-I is working with leading robotics companies and research institutions to promote the use of ROS for industrial applications. ROS-Industrial seeks to enable efficient incorporation of Artificial Intelligence in its various forms, and supporting commercial entities that seek to streamline the ease of use for robotics and AI together.

The ROS-I Consortium will continue to work towards providing value not only to its members, but also to the open-source robotics community. This includes example applications, helping to make open-source robotics technologies more accessible and affordable, and it is enabling the development of new and innovative industrial capability that can improve productivity, safety, and quality.

We look forward to collaborating with the robotics community more broadly to enable more end-users to solve their complex challenges in an efficient way and provide the means to entrepreneurs to have bankable new tools for which to address the needs of their target market. At this, the 10-year anniversary of the Americas Consortium meeting where members gather to share the latest in open source for industry, we will seek pathways to continue this mission. The ROS-I Consortium has enjoyed providing the foundation and we look forward to continuing to enable the industrial robotics solutions for tomorrow.

For remote attendees live streaming of talks has been a tradition and was made available and recordings of the sessions will be available in the future.

To learn more about the Annual Meeting, visit https://rosindustrial.org/events/2023/5/ros-industrial-consortium-americas-2023-annual-meeting.

To learn more about the ROS-Industrial Consortium, visit https://rosindustrial.org.

The basics of getting up and running with ROS 2 in Windows

Development in ROS used to only be able to be done in Linux, with Ubuntu being the only fully supported operating system. With ROS 2, more options have opened up to support ROS development. In this blog post, we’ll discuss how you can get ROS 2 up and running on the Windows.

Why use ROS 2 in Windows?

There are a number of reasons you may wish to pursue ROS development in Windows.

• You or a client are limited in available operating systems
• You want to utilize applications/libraries that are only supported in Windows
• You are much more comfortable with Windows development over Linux development

Why not use ROS 2 in Windows?

While there are good reasons to use ROS in Windows, the reasons against it are substantial:

• ROS 2 packages are still typically developed with Linux in mind, and often don’t build without
• modification
• ROS in Windows is generally less supported and complete than it is in Linux.
• You will have more trouble finding solutions for any problems that come up
• More unexpected issues are likely to pop up than in Linux
• ROS behaviors are not entirely consistent between operating systems

RVIz launching in windows

What are my options?

The simplest option to using ROS 2 in Windows is to avoid using Windows as much as possible by using Windows Subsystem for Linux (WSL). WSL operates a Linux kernel on top of the already running Windows kernel, allowing it to execute programs meant for Linux while within Windows. This is much more lightweight than a full virtual machine and allows easy access to files located within your Windows system. Head here (https://learn.microsoft.com/en-us/windows/wsl/install) to get started with WSL.

>A second option is to use a fully pre-built package from the chocolatey package manager. This will streamline the installation process to a few copy-paste commands and greatly reduces the likelihood of errors occurring in the installation process (which can be non-trivial to debug). aka.ms/ros hosts a package for ROS 2 foxy, with instructions available here (https://ms-iot.github.io/ROSOnWindows/GettingStarted/SetupRos2.html)

A final option if you really need a version that is not installable as a full package via chocolatey is to follow the installation instructions for that distro from the ROS 2 docs (https://docs.ros.org/en/humble/Installation/Windows-Install-Binary.html). There are a large number of steps that are far more prone to error, and since Windows is not as well supported, these instructions are more likely to be out of date themselves.

Working with ROS in Windows:

If you choose to use WSL, then congratulations! You don’t have anything new to learn. You have access to a bash shell with all the usual ROS 2 commands you know and love. Simply start your WSL instance and you’re good to go.

If you are truly only using Windows, you will be working within a Powershell or Command Prompt window, which will work with their own commands for you to learn instead of bash. Make sure to call the appropriate setup.ps1 or setup.bat for the terminal you are using, and expect to modify the Cmake for any ROS projects that you are building from source. A few other considerations are listed here (https://docs.ros.org/en/foxy/The-ROS2-Project/Contributing/Windows-Tips-and-Tricks.html)

Conclusion

While it is recommended to stick with Linux for your ROS development, you have options to continue your applications in Windows. There are some extra limitations and difficulties, but once you get past the initial pains, the development process is largely the same.

ROS-Industrial Focused Technical Project Robotic Blending Milestone 5 Launches

The ROS-Industrial Consortium, championed by the Steel Founders’ Society of America (SFSA), has launched the Focused Technical Project (FTP) Robotic Blending Milestone 5. This builds on the prior Robotic Blending Milestone 4, which demonstrated high-mix material surface finishing and edge processing of arbitrarily shaped and contouring parts, largely targeting piece-parts to be welded.

In this new milestone, launched in March, the key goal is to place a working system at an SFSA member foundry site, and enable that end-user to leverage the CAD free, entirely perception-driven system to remove unwanted surface features due to casting pour process and finish the total surface area for subsequent inspection and ship to the client.

To make this FTP a reality, a diverse team provided various forms of contribution to enable successful application development, core capability implementation, testing, and a pathway to replication for additional foundries. This team consists of champion SFSA, Southwest Research Institute (SwRI), Yaskawa America, PushCorp, Iowa State University (ISU), deployment foundry Fisher Cast Steel, and in-kind team members ARIS Technology and Edison Welding Institute.

As is the case in prior FTP efforts, there will be multiple integration events that enable cross-team collaboration and will leave functioning capability at partner sites. In this case with partner university ISU, which will integrate recently developed capability, host a collaborative integration event, while learning the blending framework. SFSA has a long history of working with ISU and a key milestone will be implementing into the Scan-N-Plan framework recently developed riser removal perception driven finishing that has been demonstrated in the ISU lab. The goals of this integration of the ISU capability are to leave a functioning Scan-N-Plan system in the ISU lab to enable them to implement and test capabilities developed with SFSA and enable more efficient transfer to end-user sites that may adopt the Robotic Blending capability in the future.

The project completes with an integration event at the SFSA member, Fisher Cast Steel, and demonstration of capability on production cast parts. The removal surface irregularities due to riser removal will be evaluated across the parts as well as overall surface finishing to support surface inspection processes where required. A summary of core repositories that were improved will be detailed and a highlight video and supporting documentation will be provided. In-kind partners will have the opportunity to support additional end-users to adopt the capability and make further refinements on top of the core developed capability.

The ROS-Industrial Consortium, in collaboration with SFSA and the development team look forward to furthering advanced robotics capability for industry and to provide a pathway for foundries to adopt intelligent automation for high mix casting environments.

Acknowledgement: This research is sponsored by the DLA-Troop Support, Philadelphia, PA and the Defense Logistics Agency Information Operations, J68, Research & Development, Ft. Belvoir, VA.

Disclaimer: The publication of this material does not constitute approval by the government of the findings or conclusion herein. Wide distribution or announcement of this material shall not be made without specific approval by the sponsoring government activity.

What Went Down in the April ROS-I Developers' Meeting

On April 11 we held the ROS-I Developers’ Meeting. For those that haven’t checked one of these out, this is a gathering of developer-centric folks that are interested in hearing what is in development within the ROS-I ecosystem, what new resources may be available, and have some dialog on issues and/or areas for potential collaboration. This edition was hosted by the Americas ROS-I team, and we were fortunate to have three updates after a brief introduction.

• Doug Smith shared work around OPC-UA and how to take advantage of this standard to enable PLC to ROS-based system interaction. Doug provided a few resources and shared a couple reference repos.

  1. https://github.com/smith-doug/pushcorp_opcua
  2. https://github.com/smith-doug/pushcorp_opcua

• Tyler Marr shared the latest Tesseract updates around ROS 2. Tesseract ROS 2 is now fully working and has been tested in a few sample applications. Tyler also demonstrated some of the new Qt widgets for data visualization and collision checking.

  1. https://github.com/tesseract-robotics/tesseract_ros2

• Lily Baye-Wallace shared the recent updates to the focused training topics that are built from the Scan-N-Plan Workshop which has evolved since the original offering as part of ROS World 2021. The new training content was piloted at the recent March 2023 ROS-Industrial Consortium training event.

1. https://industrial-training-master.readthedocs.io/en/foxy/_source/session8/Motion-Planning-with-Tesseract.html
2. https://github.com/ros-industrial-consortium/scan_n_plan_workshop
3. https://github.com/ros-industrial-consortium/scan_n_plan_workshop/tree/master/snp_automate_2022

The meeting wrapped up around some great discussion around the FreeCAD ROS plug-in that has been in Beta informed by feedback from the ROS-Industrial end-user and developer community. Updates are coming and we appreciate the feedback and interest from the community to date. You can check out the full discussion in the video below and feel free to review the slides here. ROS-I Developer meetings are held monthly on a regional rotation basis. You can see the full schedule and register for each region here.

ROS 2 Training Event Hosted by RPI Featuring New Advanced Topic

At the start of March, Tiffany Cappellari, my colleague, and I made our way to Rensselaer Polytechnic Institute in Troy NY, to deliver ROS2 Training to members of our ROS-Industrial community. Rensselaer Polytechnic, in addition to being a prestigious academic institution, has been a proud partner in project work with many other ROS-Industrial Consortium members, collaborators, and those in need of more advanced capability. RPI kindly hosted our first in-person training of calendar year 2023.

Tiffany Cappellari introduced the experienced students to a new motion planning pipeline exercise utilizing the Tesseract toolbox while I introduced the fundamentals of ROS 2 to engineers and developers of a wide range of experience levels. All our attendees were able to explore a variety of capabilities and packages for motion planning, perception, and image processing, either native to ROS 2 or hosted by ROS-Industrial's many open-source toolbox repositories.

Workshop attendees hard at work on motion planning exercises in an RPI Low Center for Innovation Lecture Hall

Workshop attendees included graduate students, mechanical engineers, software developers, and project managers that together were enabled to discuss, in context, how the training tools can be applied to their systems and processes. For participants familiar with ROS 1, our in-person discussions included porting best-practices and lessons learned about the framework & operational differences between the two. Our exercise-based fundamentals course begins the development of a basic ROS workspace and setting up basic ROS packages for incorporation into other projects. From there, all trainees construct URDFs, basic motion planning packages using MoveIt, and Python script launch files to generate a robot workcell, plan and enact movements with that robot, and bring it all up through one, simple script.

Trainees also tried ‘Building a Perception Pipeline’, a special topic exercise that introduced several tools from Point Cloud Library (PCL) to filter and segment camera data for use in manipulation tasks. Our latest special topic, ‘Motion Planning with Tesseract’ highlighted the motion planning pipeline capabilities in Tesseract to create smooth, optimized trajectories.

RPI’s precocious graduate students kindly took us on a tour of their workspaces, including spaces where the Tesseract toolbox utilized for the special topic on motion planning is implemented on hardware.

All training materials can be found on the ROS-Industrial Training wiki, and they are updated regularly for new capabilities. Check out the github for a setup script used for a VM or Docker to get a head-start on system settings if you’re operating on an OS other than Ubuntu 20.04. We hope to see you at our next training event in July 2023 in San Antonio, TX!

Note: If you would like to learn more about hosting a ROS-Industrial Consortium training event, please reach out to a team member! We are excited to bring our expertise and knowledge to you.

Teaching an Old Robot New Tricks

Robotics is increasingly present in our daily lives in one way or another. Although many hear the word 'Robotics' think of humanoid-type robots or even robotic arms used in industry, the reality is that robotics has many forms and applications, from autonomous mobile robots (AMR) to standard industrial robots. Robots range in size from as small of the palm of your hand, to robots capable of reaching the top of an airplane.

Xyrec laser coating removal robot (https://www.swri.org/industry/industrial-robotics-automation/blog/laser-coating-removal-robot-earns-2020-rd-100-award)

GAPTER (http://wiki.gaitech.hk/gapter/index.html)

Robots, and in particular industrial robots, are programmed to perform certain functions. The Robot Operating System (ROS) is a very popular framework that facilitates the asynchronous coordination between a robot and other drives and/or devices, and has been a go-to means to enable the development of advanced capability across the robotics sector.

how ros-i extends ROS and ROS 2 to industrial relevant hardware and applications

Southwest Research Institute (SwRI) and the ROS-I community more broadly often develop applications in ROS 2 , the successor to ROS 1. In many cases, particularly where legacy application code is utilized bridging back to ROS 1 is still very common, and one of the challenges in supporting adoption of ROS for industry. This post does not aim to explain ROS, or any of the journey to migrating to ROS 2 in detail, but if interested as a reference, I invite you to read the following blogs by my colleagues, and our partners at Open Robotics/Open Source Robotics Foundation.

• Robot Operating System 2: Design, architecture, and use in the wild link in this blog post: https://www.openrobotics.org/blog/2022/5/12/science-robotics-paper
• ROS-I and managing a world of ROS 1 to ROS 2 transition in industry:
• https://www.swri.org/industry/industrial-robotics-automation/blog/the-ros-1-vs-ros-2-transition
• Curious about the ROS-Industrial open source project as a whole?: https://ifr.org/post/from-academia-to-industry-and-beyond

Giving an Old Robot a New Purpose

Robots have been manufactured since the 1950’s, and, logically, over time there are newer versions with better properties and performance than their ancestors, and this is where the question comes in, how can you give capability to those older but still functional robots? This is becoming a more important question as the circular economy has gained momentum and understanding of the carbon footprint impact of the manufacture of robots that can be offset by reusing a functional robot. Each robot has its own capabilities and limitations and those must be taken into account. However across the ROS-I community this question of can I bring new life to this old robot always comes up, and this exact use case came up recently here at SwRI.

In the lab an older Fanuc robot was acquired it seemed a good candidate to set up a system that could demonstrate basic Scan-N-Plan capabilities in an easy to digest way with this robot that would be constantly available for testing and demonstrations. The particular system was a demo unite from a former integration company and included an inverted Fanuc Robot manufactured in 2008.

The demo envisioned for this system would be a basic Scan-N-Plan implementation that would locate and execute the cleaning of a mobile phone screen. Along the way, we encountered several obstacles which are described below.

Let's talk first about the drivers, 'a driver is a software component that lets the operating system and a device communicate with each other'. Each robot has its own drivers to properly communicate with whatever is going to instruct it on how to move. So when speaking of drivers, the handling of that is different from a computer's driver to a robot's driver, and this is because a computer’s driver can be updated faster and easier than that of a robot. When device manufacturers identify errors, they create a driver update that will correct them. In computers you will be notified if a new update is available, you can accept the update and the computer will start updating, but in the world of industrial robots, including the Fanuc in the lab here, you need to manually upload the driver, and the supporting software options to the robot controller. Once the driver software and options are installed a fair amount of testing is needed to understand what the changes you made to the robot have impacted elsewhere in the system. In certain situations may receive your robot with the required options needed to facilitate external system communication, however it is always advised to check and confirm functionality.

With the passing of time, the robot will not communicate as fast as newer versions of the same model, so to obtain the best results you will want to try to update your communication drivers, if available. The Fanuc robot comes with a controller that lets you operate it manually, via a teach pendant that is in the users hand at all times, or set it to automatically and it will do what it has instructed via a simple cycle start, but all safety systems need to be functional and in the proper state for the system to operate.

The rapid position report of the robot state is very important for the computer’s software (in this case our ROS application) to know where the robot is and if it is performing the instructions correctly. This position is commonly known as the robot pose. For robotic arms the information can be separated by joint states, and your laptop will probably have an issue with the old robot due to reporting these joint states at a slower speed while in auto mode that the ROS-based software on the computer expects. One way to solve this slow reporting is to update the drivers or by adding the correct configurations for your program to your robot’s controller, but that is not always possible or feasible.

Updated location of the RGB-d camera in the fanuc cell

Another way to make the robot move as expected is to calibrate the robot with an RGB-D camera. In order to accomplish this, you must place the robot in a strategic position so that most of the robot is visible by the camera. Then view the projection of the camera and compare it to the URDF, which is a file that represents the model of the robot in simulation. Having both representations, in Rviz for example, you can change the origin of the camera_link, until you see that the projection is aligned with the URDF.

For the Scan n’ Plan application, the RGB-D camera was originally mounted on part of the robot's end-effector, but when we encountered this joint state delay, the camera was changed to a strategic position on the roof of the robot’s enclosure where it could view the base and the Fanuc robot for calibration to the simulation model as can be seen in the photos below. In addition, we set the robot to manual mode, where the user needed to hold the controller and tell the robot to start with the set of instructions given by the developed ROS-based Scan-N-Plan generated program.

CONFIRMING VIEWS OF CAMERA TO ROBOT CALIBRATION

Where we landed and what I learned

We were able to successfully give this old robot a new purpose and shared how to do it and how to solve some issues you could encounter. While note as easy as a project on “This Old House” you can teach an old robot new tricks. It is very important to know the control platform of your robot, it may be that a problem is not with your code but with the robot itself, so it is always good to make sure that the robot and the associated controller and software work well and then seek alternatives to enable that new functionality within the constraints of your available hardware. Though not always efficient in getting to the solution, older robots can deliver value when you systematically design the approach and work within the constraints of your hardware taking advantage of the tools available, in particular those in the ROS ecosystem.

Summary of ROS-Industrial Conference 2022

The 10th edition of the ROS-Industrial Conference took place on December 15-16, 2022 in Stuttgart, Germany and remotely. During the conference, 55 participants present in Stuttgart and an online audience of more than 200 people attended 17 talks in six sessions. The goal of the conference was to show and discuss what currently is possible in the ROS2 ecosystem when it comes to industrial applications.

Creating Manipulation cells in ROS2 has now stronger tool support

In the first session with the title “Manipulation Workshop”, Ragesh Ramachandran (Fraunhofer IPA) explained how you can currently setup a robotic manipulation cell that is based on ROS2 and controlled via moveit2. The workshop showed that you can export a model of your cell from CAD and create a fully functional manipulation cell using off the shelf components such as the universal robot driver for ROS2. In the second talk of the session, Michal Milkowski (Universal Robots) presented the integration of UR Script with ROS2.

Bosch uses ROS in product development

The second session of conference focused on boader topics. Dr.-Ing. Werner Kraus (Fraunhofer IPA) reported the newest robot sales statisctics and pointed out that robot solutions are a key factor for Europe staying economically relevant even with growing demographic issues. Dr. Ralph Lange (Bosch) explained in his talk, how ROS and ROS2 are used at Bosch, a company that has been active in the ROS community since the beginning. ROS and ROS2 are used in many research and development departments at Bosch. ROS and ROS2 are also integrated into a small number of products. Maria Vergo (ARTC) explained about the current progress of spreading ROS in the Asia Pacific region as well as recent and planned developments of ROS2 components by the ROS-Industrial Consortium Asia & Pacific. She highlighted the recently published packages Easy Perception Deployment and Easy Manipulation Deployment that simplify the deployment of perception and manipulation in ROS2. Michael A. Ripperger (SwRI) talked about developments that are currently ongoing in the Americas region of the consortium. A major development ongoing there is the industry friendly motion planner tesseract as well as a plugin for FreeCAD that will simplify the integration of ROS and CAD models significantly.

OpenRMF enables hardware-agnostic fleet management with building infrastructure integration

The next session was title “Navigation and ROS2”. Marcel Schnizler (Pilz) explained in his talk, how the PSENScan laserscanner can be used to easily build safe mobile robot systems using ROS or ROS2. The second presenter Aaron Chong (Open Robotics) introduced OpenRMF to the European audience. OpenRMF is a manufacturer-agnostic fleet management system for robotics that is currently developed by Open Robotics and others. Next to the fleet management capabilities, it also features integration with building infrastructure such as doors and elevators. The ecosystem currently supports more than 10 different robots for different kind of purposes. The system is deployed in multiple hospitals in Singapore at this time. Victor Mayoral-Vilches revealed information about his start-up Acceleration Robotics that aims at integrating hardware acceleration into the ROS2 ecosystem. The start-up is heavily involved in the development of new ROS2 features that simplify the usage of hardware acceleration such as GPUs and FPGAs. It has also developed a Robot Processing Unit that can be used by robot developers to leverage the newly developed hardware acceleration features.

Universal Robots and Yaskawa Motoman integrate ROS2 interfaces into their robot controllers

In the last session of the first day “Manipulation and ROS2”, Rune Søe-Knudsen (Universal Robots) and Felix Exner (FZI) presented the latest updates to the Universal Robots ROS2 robot diver. Liana Bertoni and Davide Torelli (IIT) explained the capabilities of the ROS2 End-effector framework that aims to facilitate integration, planning and control of heterogeneous robotic end-effectors. Ted Miller (Yaskawa Motoman) introduced MotoROS2 and ongoing driver development for ROS2. The interesting thing about the driver is that it runs directly on the robot controller using MicroROS. He announced that the driver will soon be published for public beta testing.

ROS2 fieldbus interfaces are becoming available

Day 2 of the conference started of with the session “ROS2 and Hardware Interfaces”. In the first talk of the session, Dr.-Ing. Denis Stogl (Stogl Robotics) explained the different features of ros2_control. For robot drivers and other hardware interfaces, ros2_control provides the tools to build hardware-agnostic interfaces as well as an infrastructure for controllers and a collection of ready-to-use controllers for different purposes. Maciej Bednarczyk from ICUBE at Université de Straßbourg presented their EtherCAT stack, which leverages ros2_control to enable simple integration into ROS2. The stack is available as v1.0.0 release. Christoph Hellmann Santos from Fraunhofer IPA presented ros2_canopen, a stack to integrate ROS2 with CANopen. The stack provides different interfaces to control CANopen Devices from ROS2, one of them being ros2_control.

ROS2 is starting to be embedded in to industrial devices such as PLCs

In the embedded platforms session, Andrei Kholodnyi from Wind River presented the integration of ROS2 with the Yocto framework. Yocto is a toolchain, which enables companies to create tailor-made Linux distributions for their high performance industrial computing products. The integration of ROS2 into the Yocto framework is achieved by the meta-layer “meta-ros”. The second talk of the session stressed the importance of Yocto for industrial equipment. Hermann Spies and Özkan Öztürk from Phoenix Contact presented work on integrating ROS2 with their PLCNext device series. The PLCNext devices use Yocto for their operating system. The ROS2 Bridge is integrated into the PLCNext devices using Docker containers that run alongside the IEC 61131 execution environment directly on the PLC hardware.

Intrinsic to strongly support the ROS community

On the first day of the conference, it became known that Intrinsic, a company that plans to democratize robotics by providing simpler and hardware-agnostic robot software with AI integration has bought parts of Open Robotics, the foundation that governs the ROS ecosystem. Interesting is, that Intrinsic is financed by Alphabet. In a short statement, Torsten Kröger, CTO at Intrinsic, and Brian Gerky, CEO of Open Robotics explained their plans and what the acquisition means for the ROS community. From their statement, it seems that the ROS ecosystem has gained a financially strong supporter in Intrinsic and will see a lot of work towards more industrial deployment in the next years.

The conference has shown that the ROS-Industrial Community has been quite active during the pandemic years and we are a looking into a bright future for open source robotics.

ROSCon 2022 Rewind

ROSCon 2022 Group Photo

This October I was fortunate enough to attend ROSCon with fellow colleagues Jerry Tower and Michael Ripperger in beautiful Kyoto, Japan. By luck, it just so happened that the month-long trip I booked to Japan one year ago lined up with Japan's borders opening and the conference's location and dates. Now that I'm back in America and have my work and personal business back in order, I'd like to share with you my ROSCon 2022 experience.

With an attendance of approximately 800 ROS developers ranging from absolute beginners to seasoned industry and academia experts, there was something for everyone at ROSCon. The panels were particularly useful to better understand the current state of ROS, ROS2, future plans, and concerns of the community. I found the presentations about integrating CANopen with ROS 2 in addition to the development work on a ROS 2 simulator with the Unreal Engine 4 interesting as well. The full ROSCon program with videos of the presentations can be found here:  https://roscon.ros.org/2022/.

Formant Robotics' robot dog being controlled via a Steam Deck roaming around the conference grounds during lunch.

The conference's exhibitors covered a healthy spectrum of different roles necessary to bring robots to the forefront of industry. This spectrum included companies that specialized in cutting edge hardware necessary for perception, navigation, and path planning, to companies that regularly deploy hundreds of robots to factory floors with cloud services. I also took this opportunity to visit with our ROS-I Asia Pacific counterparts in-person at their booth which featured an impressive demonstration of two robots working together to pick and place items into manipulated bins.

Overall, ROSCon was a great success and Open Robotics did a fantastic job putting everything together. We will most certainly make it to New Orleans for ROSCon 2023!

It's a wrap for ROS-Industrial Asia Pacific Workshop 2022!

After two years of running the ROS-Industrial Asia Pacific workshop digitally, this year’s edition has returned to physical and was held from 9 to 10 Nov 2022. The ROS-AP team has seen an overwhelming response, with tickets sold out happening early in the registration. Over 100 attendees from the ROS community attended this 2-day event, with 15 esteemed speakers who are leaders in business, industry, research, and education sectors. Prof Quek Boon Tong, CE of the National Robotics Programme Singapore, has graced the event as a Guest-of-honour, and among the keynote speakers are Prof Selina Seah, Assistant CEO of Changi General Hospital and Prof Giorgio Metta, Scientific Director of IIT.

The entire speaker lineup can be found here.

This workshop has provided many insights into ROS’s global development and adoption. Particularly on how ROS has become an essential component for large organisations to achieve their developmental goal in robotics and how ROS has supported the early-stage journey growth of aspiring start-ups such as Lionsbot and Augmentus. Matthew Festo, the General Manager of Open Robotics Singapore, has shared that there are more than $6 billion in known acquisitions of ROS-based companies and 2.2 million ROS-based installations and updates per month. With these staggering stats, ROS is making a huge global impact! Similarly, Mr Su Lian Jye, the Research Director of ABI Research, shared that ROS had greatly accelerated the development of robotics solutions, creating a rippling effect in driving the higher demand for automated solutions by numerous industries. Lastly, there were multiple sharing on Open-RMF, a ROS-based interoperability framework that enables different fleets of robots to interoperate with the infrastructure assets and building management systems. Prof Selina Seah and Dr Pongsak, General Manager for Pansonic R&D centre, have shared different use cases deployed within a hospital environment. The demand for robotics in the service industry has dramatically increased, and the healthcare sector remains a forerunner in using robotics to enhance services and efficiencies.

The end of this workshop marks another successful year of the ROS-Industrial Asia Pacific team proliferating ROS and cultivating a robust, inclusive ROS ecosystem in Asia. As the effort intensifies, the ROS-Industrial Asia Pacific team will continue to grow and see a new leadership pairing of Maria Vergo and Tan Chang Tat taking the team forward into the next phase. Watch this space!

Darryl Lee, Consortium Manager

ROS-Industrial Asia Pacific

The approved slides and presentations are made available through the links below.

Taking ROS-Industrial and ROS 2 Training to Boston

In mid-October, my colleague, Lily Baye-Wallace, and I made trip out to Analog Devices in Boston, Massachusetts to provide ROS 2 training to members of our ROS-Industrial community. Analog Devices, a full ROS-Industrial Consortium member took advantage of the ability to host a ROS-I training event, bringing ROS-I training to the East Coast.

From ROS 2 basics and fundamentals to a new image processing pipeline exercise, our attendees were able to explore the many different aspects and capabilities of both ROS 2 and ROS-Industrial as well as explore the beautiful historic city of Boston.

The Analog Devices “Analog Garage”

We had many attendees fly in from across the country this week (and even a few international ones!) to hear us talk about ROS and its importance to the robotic community around the world. Beginning with the basics of how to set up a ROS workspace, we led the students through various exercises designed to guide them through the process of how to set up a ROS environment and incorporate it into their various projects. Other content included using MoveIt! for motion planning and how to create a URDF. Our recently updated advance topic, Building a Perception Pipeline, taught the students how to use several tools within the Point Cloud Library (PCL) to filter and segment camera data to be used with their robots.

Overall, this was a great way to engage with members of the robotics community and share some of our knowledge and expertise with them. All of our training material can be found on our ROS-Industrial Training wiki. We hope to see you at our next training event in Febuary 2023 in San Antonio, TX!

Note: If you are interested in learning more about hosting a ROS-Industrial Consortium training event, reach out to a team member!

ROSCon 2022 - ROS-Industrial Consortium Americas Look Back

Last week, my colleagues, Jerry Towler and Fernando Martinez, and I made the grueling 14-hour trans-Pacific flight and 3-hour train ride to attend ROSCon in Kyoto Japan. All told, we probably would have endured a lot worse for the opportunity to travel to Japan and talk about robots.

Visiting with the ROS-I Asia-Pacific Team

On the first day, we attended the ros2_control workshop. The focus of the workshop was a deep dive into the revised architecture of ros2_control followed by a tutorial for creating a custom hardware interface. The coding exercise helped solidify some of the new ros2_control interface concepts and provided the opportunity to see the components in action via simulation using Gazebo. The Construct also provided access to a robot in their lab in Spain for hardware interface testing. Unfortunately, we did not finish in time to use it, but the Construct showed a live* demo on the hardware using a ros2_control interface.

On to the ROSCon program! As always, the keynote presentations highlighted some interesting applications of ROS, specifically in augmented reality for assistance in surgical applications and in GPS-denied UAV autonomy. A majority of the other presentations focused on software interoperability (OpenRMF), RMW layer development and challenges, AMR/UAV autonomy, and, of course, porting from ROS1 to ROS2. The program this year was light on manipulator-specific content, touching primarily on application deployment using ROS2 and ros2_control. The presentations that stood out the most to us covered changes to the ROS2 navigation stack enabling improved motion planning and support for different types of mobility (e.g., Ackermann, legged, and non-circular diff-drive robots); an improved file format for ROS2 bag files (MCAP); an IDE with visualization for defining kinematic chains in URDF; and updates to the BehaviorTree framework. Our final key takeaway from the presentations: all robotics companies everywhere are hiring. The full ROSCon program is posted online; links to the video presentations and slides are expected to be published within the next few weeks.

ROS-Industrial exhibition booth with Easy manipulation demo

In addition to the presentation program, ROSCon also hosted an exhibit hall for robotics vendors displaying robots, sensor technology and software/simulation applications. The ROS-I AP consortium also staffed a booth with an impressive demo showcasing an application where two robots pick and place objects into a bin and jointly manipulate the bin based on 3D perception feedback.

Overall, the first in-person ROSCon since 2019 was a success with roughly 800 attendees from more than 30 countries. It was a great opportunity to engage with the open-source community and learn more about what other roboticists around the world are working on. Looking forward to another ROSCon next year in New Orleans!

*live on hardware in Spain, viewing concurrently and remotely in Kyoto…neat!

A turn in the welding robotics community

In early October in Denver, Colorado, the American Welding Society (AWS) held the first Automated Welding & Sensors Conference. This is a follow up to the prior National Robotic Arc Welding Conference, which last took place in 2019, but was not an official AWS conference. Since then, a lot has changed. One change is a pivot to acknowledging there is a lot going on in robotics, sensors, human robot interaction, and emerging trends in workforce education and sustainment, that it makes sense to have a conference on automation in welding.

The venue and a full house for the first official American Welding Society Automated Welding & Sensors Conference

Full disclosure, I cut my teeth and bare many scars by implementing robotic welding, on large structures, where quality was paramount, and input variation was prevalent. I also spent many hours on precision sheet metal assemblies, that also had high quality requirements, and interesting material combinations. Robotics enabled repeatability, and a means to ensure quality when we had our processes under control, but the robotic systems of the early to mid-2000s had their limitations. My journey to further robotic welding capability and performance during my time in industry working for Caterpillar led working with tools such as ROS, and subsequently getting involved in the ROS-Industrial open-source project and consortium.

Fast forward, and there are still challenges. There are also a lot of new tools, innovations, new voices with new ideas, and plenty of examples of applying technology in a sustainable way to realize gains for small and medium enterprises. First and foremost was the prevalence of collaborative robots, in particular Universal Robots power and force limited manipulators. The conference was organized by Vectis Automation founder and CEO Doug Rhoda and his team, in partnership with Jeff Noruk of ServoRobot.

Some of the key takeaways from my perspective is there is a hunger for intelligent yet easy to use solutions. There is an inherent high mix, and, at times, harsh environment. It is now at the point where collaborative robots – power and force limited manipulators – are now appearing in several job shops, and large manufacturers around the world. Caterpillar shared their experience in taking advantage of leveraging collaborative hardware-based systems to realize flexible and agile welding capability.

Caterpillar’s Don Stickel sharing industry technological needs

Furthermore, there have been advances in welding data management, which has been a growth area for those passionate about welding quality and the wealth of “big data,” that can be harvested from welding operations. In parallel, there has been significant advances in the ability to monitor with camera systems to richly visualize the welding process. The arc, molten pool, the filler metal/electrode and perform real-time vision analysis to close the loop, beyond historically simply the electrical signals previously leveraged.

Advances in leveraging mobility were shared, and digital technologies, making their way into the welding equipment, are giving the welding industry a lot to pull into their operations. However, there is still further opportunity to support the welding domain through implementation and scaling of AI-based capabilities. This could be for low-level real-time control, such as what Novarc shared during their talk, or the ability for higher level weldment process planning to be more robust in AI generalization, to scale across classes of fabrications more efficiently.

The growth in robotics has been well documented in many domains such as warehouse and logistics, on-road autonomy, drone-based inspection, and monitoring through legged robots performing various tasks in dynamic environments.

There is an opportunity for all the contributors in the robotics domain to learn about the needs, challenges, and opportunities within the materials joining community. While often labeled as dirty, dull, and dangerous, there is both a need via the well-documented labor shortage, and a technical challenge to establish advanced approaches to make the products of tomorrow right here where they will be consumed.

It was great to see the progress in the welding community, relative to adopting technology and aligning tech capability with the needs of this end-user community. Let’s keep supporting these communities, and continue further for the coating industry, forging and casting as well, again to support the ability to make products efficiently where they are consumed. Not just to advance technology for the sake of technology but to build the ecosystem for a more resilient economy and supply chain for the next generation. The ROS-I open-source project aims to provide utilities that serve as building blocks for these sorts of initiatives, and we look forward to seeing how they get onto the shop floor.