From automation to true autonomy

What is the current state of autonomy in mining? Which technologies do we need, and which have yet to be developed?

While fascinating and exciting, the world of automation and autonomous mining can be confusing and, to be truthful, a little bewildering for non-engineers. I needed an expert guide who could help me better understand the current state-of-play. And who better to ask than Professor Elisabeth Clausen, Director of the Institute for Advanced Mining Technologies (AMT) at RWTH Aachen University in Germany?

Professor Clausen leads an interdisciplinary team of researchers whose work is, quite literally, at the cutting edge in this field, and she was happy to answer my questions, and more…

Carly Leonida: Autonomous mining is one of the AMT’s key areas of expertise. Why is it such an important part of your research and teaching programs?

Elisabeth Clausen: I firmly believe that the future of mining will be the green and autonomous mine, where people are largely removed from hazardous and rough environments and where we will see connected mines with intelligent equipment and integrated processes. Autonomous technologies, therefore, are key to advancing safety, efficiency and productivity and will quite substantially change the way minerals will be mined in the not-too-distant future.

With our research at the AMT, we are intending to contribute to green, autonomous, and sustainable mining through our activities in research, teaching and transfer. In fact, we have contributed to technological developments in mining since the founding of our institute decades ago.

In the past, we focused on studying the fundamentals of mechanical systems in mining, while today we focus on the development of integrated technology solutions to automate and digitalise mining machines and processes by ‘teaching’ them how to ‘see, hear, and feel’.

We do that in different research areas, such as asset monitoring and diagnosis, mine design and energy management, mine automation and control, communication, material characterisation, localisation, navigation and mapping, as well as environmental perception and robotised inspection.

With our research and development (R&D) in these key areas, our key objective is to contribute to the progression towards higher levels of automation until full autonomy becomes possible.

It seems important to mention here that the terms ‘automation’ and ‘autonomous mining’ are often used interchangeably. However, getting from automating single pieces of equipment to true autonomy is quite a jump. That is why I think it’s important to understand that there is an automation continuum and what it means technologically to get to true autonomy.

True autonomy means that a machine can adjust to dynamic environments, change its planned actions dynamically to avoid suboptimal behaviour, adapt to variations in conditions and communicate relevant information.

This requires machines to have situational awareness of their own state and their surroundings, including other pieces of equipment. It also requires machines to communicate with a central control system, but also with other machines. An autonomous machine or system at this stage is more or less self-sufficient, meaning it requires no or very little human intervention.

This is the next frontier in mining, and you can imagine that it is particularly challenging to reach autonomy in underground, GPS deprived, rough environments where the parameters and the environment are highly dynamic and changing all the time. To achieve this, advanced technology development is required, especially in underground environments.

In addition to technology development, however, we mustn’t forget the importance of education and training. We need to educate the engineers of tomorrow who bring the right competences and skillset and who are able to deal with uncertainty and complexity, based on a deep technical understanding.

At the AMT, we are committed to a future-oriented competence-based teaching and learning, which means that in addition to teaching engineering fundamentals we focus on project-based learning, industry challenges, and on creating an environment that fosters entrepreneurial thinking and start-up activity.

Considering our transfer activities, we are part of the Excellence Start.Up Center, which is currently established at RWTH Aachen University. As part of this initiative, we are in the process of building a raw material focused start-up incubator, which is a novel approach to fostering start-ups in mining and raw materials.

In addition, we have built up a real-world laboratory in an active sand quarry near Aachen, where we can do applied research in an operating quarry. However, this would lead into a whole new area of discussion, which we may better keep for another interview…

What do you think is currently holding back the widespread deployment of autonomy in mining?

Let me answer with a few examples to illustrate it. On the technology side of things, higher levels of automation towards autonomy require extensive connectivity, and sometimes the required bandwidth and network technology already present challenges. For example, we need high-quality (real-time) data and high-resolution video with stable networks in order to operate automated machines safely and to supervise an autonomous fleet continuously. The network needs to be highly performant and reliable at all times.

Furthermore, sensor technologies being able to acquire these high-quality data must exist and be adapted to the challenging mining environment.

In addition, especially in underground environments, machines need to self-navigate and perceive their position and environment without an external reference system, such as GPS sensors. In this area, we still need further R&D in order to achieve full autonomy underground.

The high degree of variability in the environmental conditions make it technologically challenging to achieve full automation for mobile machines, let alone entire processes.

Another factor is the challenge of implementing machine-to-machine communication and creating compatible interfaces between machines which are independent from the manufacturer.

We are very pleased that we were able to, together with VDMA Mining and member companies of VDMA, that we published the Companion Specification Mining for the OPC-UA – as a start into standardised interfaces and information exchange – this year and launch it at the bauma event in Munich this October. This catalogue of standardised communication protocols and interfaces provides a basis for a unified ‘language’ for machine-to-machine communication.

Especially when it comes to mixed traffic, meaning the interaction between humans and machines in a particular environment, safety is a huge challenge. Many sensors and connected perception pipelines are not or can hardly be certified yet. We need extremely high levels of accuracy and reliability if we have environments where humans and autonomous equipment are to co-exist.

These are quite significant technological challenges we still need to tackle. In addition, these challenges get exacerbated by the unique conditions of mining, which are permanently changing as the mine ‘moves’ with the advancements of extraction and imply that information is always uncertain and imperfect.

Combined with the fact that each operation is unique and has unique requirements and conditions, this implies that it is almost impossible to develop off-the-shelf solutions. Rather, it means that solutions will need to be adapted to the specifics of each mine site. This can slow down the implementation of autonomous solutions. 

However, as I have touched on above, in many areas the technologies we need to move from automation to autonomy have not yet been developed or, if they have been, there are very few references resulting in a reluctance by companies to invest in these technologies. This also means that we need first movers on prototypical and test deployments to enable the advancement of technology development in general.

To conclude, I would say that at this stage, selecting the required technologies and data that are best suited for a specific operation is a challenging task for many companies and there is still reluctance to invest in new technology without a clear cost / benefit rationale or business case behind them.

What we need are companies that ‘blaze a trail’ together with research institutions and technology providers. Companies require a holistic strategy and a clear roadmap for moving from automation of single actions towards autonomous systems.

We need to continue fostering trust in cooperation in order to enable collaboration and overcome the challenges I outlined.

Could you tell us about some of the automation projects that the AMT has underway and what they aim to achieve?

All of our projects are application-oriented and conducted collaboratively with industry, mostly with small-medium enterprises (SMEs). In the area of automation, we currently have four main projects that address different aspects of automation, from assistance systems to fully autonomous navigation and machine-to-machine communication.

In our PAM 4.0 project, the AMT, together with TML Technik GmbH, is developing an intelligent de-slagging machine for hot operating conditions. The project aims to develop the sensor and actuator systems for the first automated, newly designed and optimised de-slagging machine on the market, which will carry out the de-slagging process at 1,500°C without human intervention.

In our ScaleSense project, together with Hermann Paus Maschinentechnik GmbH, we aim to develop a sensor-based system for the detection of loose rock within the scaling process and to integrate the system into a scaler to assist the operator in identifying loose rock and making the scaling process safer and more efficient. This is an assistance system for the operator.

Further up the automation continuum, we have the Horizon 2020 NEXGEN SIMS (Next Generation Carbon Neutral Pilots for Smart Intelligent Mining Systems) project, which aims at the further development and advanced demonstration of technologies for future-oriented sustainable mining in Europe.

Within the project, we are involved in several work packages tackling autonomous loading, robotised inspection (using sensor equipped drones), communication networks in areas with no infrastructure (such as after blasting), and the future workplaces of the digital miner. Our responsibility is to develop the respective sensor technologies for environmental perception, automated muck pile detection, and a communication network in areas without any infrastructure.

And lastly there is ARTUS, which is short for ‘autonomous robust transport system for hybrid (underground / surface) sustainable raw material extraction based on articulated haulers’ [translated title] – which already describes quite well what we are aiming to achieve in this project. Here, together with a large consortium of excellent partners from the Aachen universities in combination with technology providers, we are looking more specifically at advanced automation for mobile machines.

At what stage is the ARTUS project and what are the next steps?

ARTUS has been an ambitious and complex, multi-layered project with many partners developing and demonstrating technologies in different areas, all aiming at advancing true autonomy for different types of mobile machines for hybrid, i.e., surface and underground mining environments.

This means machines will dynamically plan routes themselves based on recognising their environment and making dynamic decisions. This is really exciting because, so far, autonomous navigation is limited to predefined fixed routes, whereas in ARTUS we are demonstrating autonomous navigation by vehicles driving on flexible routes.

Core components of the project include environmental perception, localisation and mapping, enabling dynamic navigation, fleet management and machine interaction as well as machine-to-machine and process communication. Within ARTUS, a team at AMT developed a decentralised, machines-based mesh communication that enables automated machine interaction without infrastructure. This eliminates the need to build a fixed infrastructure anywhere, which is a big cost driver when implementing automated systems.

The overall objective of this project is to demonstrate advanced autonomy with respect to an autonomous driving fleet of vehicles in surface and underground environment based on radar technology, where the mobile machines are communicating and interacting with each other.

So far, the technologies for continuous mapping, decentralised communication and machine interaction, as well as intelligent self-learning control algorithms, which enable faster and safer travel speeds in autonomous driving on loose uneven ground, have been demonstrated by the research partners of RWTH Aachen University and the University of Applied Sciences in Aachen. Mapping, sensing, control and communication are prerequisites for autonomous and flexible navigation.

Thus, in a next phase of the project, these technologies will be demonstrated in the field in different areas (dump trucks, trucks, loaders) together with industry partners.

How do you see autonomous technologies fitting into the future of mining?

The potential of full autonomy in mining is immense, since it will change the role of people and their tasks as well as unleashing potential in safety and productivity. Autonomy will allow people to be moved from dangerous environments, especially underground, to more comfortable working conditions in remote operation centres or home office locations.

Rather than supervise and remote control the machines from surface or underground, people will eventually take the role of managing the actions and interactions of machine fleets and systems, taking operating decisions and giving high level instructions, instead of closely supervising and controlling single actions of machines. This means work will be more productive too, as one person can supervise more than one machine.

Lastly, autonomous machines also mean improved machine, process, and resource efficiency as maintenance, energy and waste can be reduced and machines deployed continuously and with maximum efficiency.

However, it’s important for me to stress that there must still be and will always be employees actively involved in the production processes on the ground. Not all processes can be fully automated or autonomised and I don’t want to make it appear as if mining can soon only be realised from above ground or by joystick.

This transition will be a challenge and a task in the coming years that we need actively to address and manage.

Important work has been done within the SIMS and NEXGEN SIMS projects to work out what the environment will be like for the future ‘Digital Miner’, and what we need to think about and be attentive to now in order to reap the benefits of automation while keeping a human-centred perspective.