Vehicle Control Enhancements of ADAS

ADAS, or Advanced Driver-Assistance Systems, are developing rapidly and in several stages. ADAS is not autonomy; by definition, it is Advanced Driver-Assistance Systems while autonomous vehicles have systems that are designed to completely control a vehicle in most or all situations.

Current ADAS systems perform functions ranging from warning the operator of a potential threat to taking full control of the vehicle under certain circumstances. Referred to as ‘Active Driver Assist’, these control functions typically include steering, throttle, transmission, and brake control. The intent of Active Driver Assist features is to avoid collisions and thus improve safe vehicle operations.

ADAS typically consist of multiple sensors, processing units, and associated software. The sensors normally include vision systems and Radars, and in some instances, Lidar (Light Detection and Ranging) to understand the environment surrounding the vehicles. Depending on the level of automation, these ADAS sensors may be mounted on the front, sides, or the rear of the vehicle, or all around. The majority of automotive ADAS systems implemented have sensors on the front, sides, and back of the vehicle to offer lane change assist, lane departure warning, rear cross-traffic alert, and adaptive cruise control. All of these sensors communicate with one or more Electronic Control Units (ECU) to make the proper decisions and cause the vehicle to respond.

What is the History of ADAS?

Historically, the development of ADAS can be categorized within ‘five eras of safety:’

  • 1950-2000 safety and convenience features were introduced, such as cruise control, seat belts, and antilock brakes;
  • 2000-2010 advanced safety features were added, such as electronic stability control, blind spot detection, forward collision warning, and lane departure warning;
  • 2010-2016 more advanced driver assistance safety features, including rearview video, automatic braking for emergencies, pedestrians and backing, rear cross-traffic alert, and lane centering assist;
  • 2016-2025 partially automated safety features with lane-keeping assist, adaptive cruise control, traffic jam assist, and self-parking;
  • 2025+ ADAS systems assist the vehicle’s autonomous systems rather than the driver – highway autopilot, where all human occupants are considered ‘passengers.’

ADAS is a preemptive approach to mitigating collisions and other crashes, in which they aid in avoiding the incident all together and not just decreasing the effects of a crash. In eliminating errors while operating moving machinery, ADAS is revolutionizing the mining, automotive, and heavy-duty industries through active braking, blind spot monitoring, collision warnings, lane change assistance, and warning for Vulnerable Road Users (VRU). Along with even further integration, ADAS, with cameras, navigation systems, sensors, and monitors, continues to develop the safety system of the future.

When you talk about the evolution of automation, the mining industry provides an excellent example in that they have had fully autonomous mining machines for many years. This accelerated adoption is possible mainly because the mining industry operates within a very controlled environment, and with its production levels and a large number of high-cost assets, expensive automation and ADAS have made business sense for mining operations for many years.

We are starting to see automation (or ADAS) in mining operations including SAE levels 0, 1, 2 and even 4 and 5, in less controlled environments, we see production vehicles and equipment implementing levels 0, 1, and 2:

  • SAE Level 0: ADAS systems are limited to providing the driver warnings and momentary assistance only
  • SAE Level 1: the driver has complete control of all functions, but either steering or acceleration/braking can be done automatically by the system in certain circumstances;
  • SAE Level 2: the driver has complete control of all functions, but steering and acceleration/braking are both automated in certain situations;
  • SAE Level 3: The driver is not involved in the driving task in most situations, but the driver must take over the complete driving task immediately upon request from the ADAS/autonomy system. This level is being avoided by most for obvious safety reasons;
  • SAE Level 4: The driver is not involved in the driving task in most situations;
  • SAE Level 5: The driver is not involved in the driving task in any situation.

What is the experience of an operator in a truck equipped with ADAS? How does it improve the work environment and routine?

A conventional ADAS system can detect objects, do primary classification, alert the driver of hazardous road conditions, and sometimes move, slow down, or even stop vehicles. If done well, this form of assistance provides a welcomed sense of convenience and safety allowing the operator to feel more confident.

To operators, there are obvious benefits and situational negatives to ADAS. The understanding is that ADAS is integral to the future of mining, the automobiles industry, and the operation of heavy-duty machinery. For there is always a margin of human error in which these systems would be able to counteract and avoid costly and dangerous collisions. When considering the number of new operators and drivers the industry needs, these ADAS features may also increase safety for those with less experience. While ADAS continues to develop, work environments are improved with less margin of error, fewer accidents, and increased reassurance for the operator.

Today, adaptive speed control, night vision, and blind spot monitoring functions are more widely accepted. But, on the other hand, the ADAS features that take over operation of the vehicle to avoid accidents are still quite controversial. Many drivers are under the impression that autonomous and ADAS technologies are safety features designed to protect them from other driver’s mistakes, and not their own.

Lower-level autonomous vehicles use ADAS features that are designed to take over if the driver does not make the correct evasive maneuvers. For instance, if an incorrect or unsafe lane departure is initiated, the system would correct the mistake and steer the machine back on track. Automatically applying brakes is another instance where automated avoidance maneuvers can override an operator.

The Differences between Operator Awareness Systems (SAE Level 0) and Operator Assist Systems (SAE Level 1 and 2)

In the general SAE model, ADAS transforms into Automated Driving System (ADS) at Level 3. The transition begins when the vehicle is capable of performing all aspects of driving without interference or aid from a human in certain conditions. If you search driver levels on mining machines, there is another quite similar standard, in which Level 0 has no automation with only driver warning systems, and Level 4 is considered ‘full automation.’

In many complex operating environments both on-road and off-road, operators and fleets are choosing advanced Level 0 systems to improve the operator’s situational awareness. These systems evaluate the environment surrounding the vehicle or equipment and provide information to the operator to allow them to make better decisions. In many cases, the same sensors used for these advanced Level 0 systems are also used in Level 1+ systems. The primary difference is who (or what) the sensors are providing information to. In Level 0, the information goes directly to the operator. In Level 1+ systems, the information goes to an ECU or autonomy processor where decisions are made and actions are taken.

What seems to be occurring, is rather than having equipment with no automation or equipment that is fully autonomous, vehicles and mobile equipment are emerging more often with Level 0, 1 and 2 automation. This is a trend most heavy mobile equipment is following because the most relevant safety feature to the industry is automated braking. Depending on the specific industry, this feature may be used mainly for backing, or primarily to avoid rear-ending another vehicle on a highway – two problems most heavy-duty machine operators look to solve across all sectors.

Based on this observation, Level 2 automation may be steering and braking override, whereas Level 1 would be automatic braking in most industries. Level 2 is often a combination used as an evasive maneuver, made by a computer in which information has been entered and processed to decide whether to override an operator in dangerous situations or not.

Another capability that is discussed in a lot of the literature, and is a part of the South Africa requirements, is vehicle-to-vehicle/infrastructure/etc. (V2X) communication. Many of these new systems are requiring some form of V2X communication to mitigate risk. For example, if a system has sensors, and senses a vehicle coming too close to another, it would tell the machine via V2X that there is an immediate risk. Some mines have already deployed this technology, and many are talking about deployment in the future.

Active Braking Capabilities of ADAS

Active braking, also known sometimes as advanced emergency braking or autonomous emergency braking, uses sensors to monitor the presence of surrounding objects or vehicles. However, not all active braking systems are created equal. Specific systems are designed to mitigate collisions, whereas others are engineered to avoid said collisions ultimately—the auto-braking mitigation systems are designed to slow the vehicles to lessen impact and damage. In contrast, the avoidance systems attempt to prevent a crash altogether by entirely stopping a moving machine before hitting an object. Staying informed and educated on the featured braking system is imperative when deciding which to integrate.

The benefits of active braking capabilities have not been lost on decision-makers across the globe. In fact, heavy trucks throughout Western Europe have been required to include forward looking AEB systems for years and DMR regulations in Africa will soon require all equipment in mines to have active braking in 2020. Requiring this auto braking in mines means technologies, such as radar, are going to be used to report backing, forward, or side risks to an automation controller or brain, if you will, which will make decisions about braking the machine and/or overriding the operator when necessary. It’s similar to cars, and quite a few OEM manufacturers are working on automatic braking support.

Note: Advanced Driver Assistant Systems are used to create autonomous vehicles. AEB can be considered an ADAS system. Depending on the AEB system, there may or may not be control software, tracking, communication between each vehicle, etc. Sentry is used in AEB backing systems where the Sentry provides an electrical signal causing the braking system to engage when an object is detected within a certain distance. On the other end of complexity are the forward AEB systems implemented by WABCO and Bendix for on-road trucks that combine cameras / image processing with radar for sensing and the anti-lock braking and stability control systems on the truck for action.

How does the PRECO solution work? (i.e., one solution puts sensors on a shovel to prevent haul trucks from coming too close)

PRECO’s PreView Sentry® was designed from the ground up to be rugged and flexible technology capable of withstanding the harshest environments. Initially intended for mining and construction, the radar(s) can be placed on each side of the machine to bring the operator a near 360-degree cocoon of safety. The Sentry radar was designed with a wide-viewing capability, making it optimal for those with backing or side blind spot monitoring needs. The sensor detects both moving and stationary objects in a pre-defined coverage area and can report the distance of the closest object via visual range indicators (display) and an audible signal (display or buzzer) to the equipment operator or to other ECUs or processors for autonomous functions.

The way this works is the sensor transmits and receives low power 24 GHz radar signals. It then processes the returned signals to determine if an object has reflected any energy to the sensor and reports this to the operator display or other processor. The sensor is designed to process and report detections within 240 milliseconds (ms), allowing the operator (or machine) to respond to any object within the detection zone quickly.

When placed on the shovel, Sentry alerts the operator if the swing of the shovel will hit an object the system has detected. This information is provided to the shovel operator and other systems that control the shovel’s movement. If V2X were added between the shovel and haul truck, the functionality would be no different other than the same information would be provided to the autonomous hauler which would be able to re-position itself.

Over time we have collected a number of case studies with clients in the mining industry successfully implementing systems using the PreView Sentry®, where a global mining company cuts costly vehicle collisions in half in two years, and another where radar gives mining loader operators just the right amount of warning.

Other Success Stories of Mines Deploying ADAS:

PRECO Electronics® has done a lot of ADAS with operational machinery in the Sishen mine located in central South Africa in Kathu, Northern Cape. Sishen mine is one of the largest iron ore reserves in South Africa, and as a way to improve safety, active braking systems for Kumba’s haul trucks were put in place as an additional innovative strategy the company was looking to make.

With our many international dealers, such as APS and Trysome in South Africa, PRECO is able to integrate our systems with machines already in different stages of automation. We have also partnered with BEML in India, where we work our systems into their haul trucks. In the mines in India they require collision avoidance systems on haul trucks, so they have a high level of interest in ADAS and automation, all of which use systems such as ours for additional improvement.

In Conclusion…

As people are recognizing active warning systems save lives, integration and implementation of this technology is becoming commonplace. This technology is driving towards higher levels of automation, and PRECO’s systems play an immeasurable role in ADAS and the future of automation.