Artificial Intelligence for Vehicles

Artificial Intelligence for Vehicles – In this article, we will discuss in detail about AI in vehicles. Therefore, please read in order to understand the whole context from understanding AI to its application.

Artificial Intelligence for Vehicles

In modern times like today, the progress of the automotive sector cannot be separated from other fields. Let’s take an example in the field of electronics. Advances in electronics and network technology, for example, greatly affect today’s modern vehicles. Starting from the shift in the use of carburetor to injection, the use of electronic control units or ECU, ECM, PCM, BCM, and so on. Even today many are developing AI for inclusion in vehicles as we will discuss today.

What is AI?

The term “Artificial Intelligence” was coined by John McCarthy defined it as “the science and engineering of making intelligent machines.” It is worth noting that this definition does not presume the types of methods employed. It is the functional definition of a goal. But this definition mentions the word “intelligent” in the characterization of “artificial intelligence”. It is therefore not entirely satisfactory.

A second definition was proposed by Marvin Minsky: “the science of making machines do things that would require intelligence if done by men.” This definition is a little more accurate. Like the first, it is the functional definition of a goal. The term “intelligence” is included in the definition, which again is still not very satisfactory.

What is AI?

Other definitions are commonly used, such as, for example, “a branch of computer science dealing with the simulation of intelligent behavior in computers” and “the capability of a machine to imitate intelligent human behavior”, or “the theory and development of computer systems able to perform tasks normally requiring human intelligence, such as visual perception, speech recognition, decision-making, and translation between languages.” This last definition offers a little more information about the tasks included within the sphere of intelligence.

Note: “thinking”, “spirit”, “understanding” and so on are not part of this definition of AI.

Nowadays, we consider that the characteristics that make it possible to say whether a technical solution involves AI or not are:

  • perception;
  • learning;
  • reasoning;
  • problem-solving;
  • using natural language.

Here we have a more detailed definition including five verifiable characteristics. If at least one of these features is verified by a system, then – according to this definition – we can say that AI is involved. We observe that learning is only one aspect of what is considered to be AI. This is important because some people systematically associate AI with the ability to learn from data, which leads them to say that the prerequisite for using AI is building large databases. The most commonly accepted definition of AI does not presuppose this, and we may talk about AI systems that do not necessarily learn from data but integrate knowledge extracted from humans (knowledge-based systems). For years, these systems represented more than 90% of what was called AI, and their weight in current works is increasing, after having almost completely disappeared in favor of learning techniques in recent years. For example, it is now common to again read expressions such as “knowledge representation” in scientific programs.

We can also notice that as technology advances, what was once considered an AI problem is now found in the list of ordinary computer problems, such as for example playing chess: modern computers beat chess champions, not due to any intelligence whatsoever, but because they have the sufficient calculation capacity to play every game sequence combination in parallel virtually and always retain the best sequence. However, for years, this problem was a hot topic for AI researchers.

This remark would tend to show that the notion of intelligence contains an implicit idea of transcendence: as soon as we easily understand how a system works, we no longer consider it to be AI. This leads to three apparently divergent attitudes:

  • those who consider that “artificial intelligence does not exist”;
  • those considering AI to be a goal which, like the horizon, recedes as we advance;
  • those who consider that we will have really created AI when – even perfectly knowing all of its components and parameters – it will not be possible to predict its operation. This is not a far fetched definition insofar as this notion exists in what is called “deterministic chaos theory”.

In fact, these three attitudes are convergent after all: for those three ways of thinking, the very notion of intelligence must remain transcendent to any explanation reducing AI to a “simple automatism”.

The main methods of AI

This section does not claim to be exhaustive or to present the techniques in detail. It is rather a summary for raising awareness about the existence of the main approaches, in order to understand their specific guidelines and uses.

Deep Learning

Deep Learning refers to making a network of “formal neurons” automatically learn a function. Most often, these neurons are made up of a scalar product stage: the input vector is weighted by coefficients called “synaptic weights”. Note that when dealing with data vectors, it is customary to use the standard scalar product. But if we process signals (temporal speech signals, images, videos, etc.), we use the scalar product of the signals’ Vector Space, which is the intercorrelation function. Since this function is written as a convolution operation, we then speak of “Convolution Neural Networks” (CNNs). This scalar product stage with adjustable coefficients is followed by what is called an “activation function”.

Artificial Intelligence for Vehicles

There are many possible activation functions, the most common being an “S” curve often encoded by a hyperbolic tangent. There are also variants which replace the scalar product with a distance and use non-monotonic activation functions, such as “radial basis functions neural networks”.

In industrial applications, neurons are generally positioned in layers following a “feedforward” pattern. There are two families of layers of neurons: on the one hand, the input layer and the output layer which are in direct contact with the outside world, and on the other hand, the intermediate layer(s), called the hidden or deep layer (hence the term Deep Learning).

In industrial applications, neurons are generally positioned in layers following a “feedforward” pattern. There are two families of layers of neurons: on the one hand, the input layer and the output layer which are in direct contact with the outside world, and on the other hand, the intermediate layer(s), called the hidden or deep layer (hence the term Deep Learning).

The learning of such layer networks involves iteratively adjusting the variable parameters of the scalar products, so as to minimize the error made by the neural network on what is called a “resolved” database (or learning base). We speak of supervised learning, because during the learning process, for each input of the neural network, we know its desired output. This iterative search for the possible minimum of the error made by the network uses classical concepts of operational research, such as random search, and sophisticated features like simulated annealing, or the notion of the error slope with gradient descent leading to what was called the “backpropagation” learning rule. There are many other types of neural networks, but multi-layer feedforward neural networks are the most commonly used. 

Machine Learning

In general, a distinction is made between Machine Learning and Deep Learning. Deep Learning is the expression of Machine Learning in the particular case of multi-layered feedforward neural networks.

The framework of Machine Learning is broader: whatever the constituent parts of a system, we can represent them as being a calculation system or by a logical reasoning element whose parameters can be automatically adjusted. The iterative search for parameters to minimize an error on a database is called Machine Learning and employs the same concepts of operational research.

Artificial Intelligence for Vehicles

Machine Learning systems quite often include;

  • Bayesian networks: they manipulate “Bayes” conditional probability calculation cells;
  • transfer learning systems: this involves reapplying knowledge and skills initially used to solve one task, in order to solve another one. The difficulty lies in measuring task proximity and in knowing which characteristics of the task are related to what kind of knowledge;
  • fuzzy decision tables: these implement the classic notion of the decision table by replacing the classic “yes/no” logic by fuzzy logic which admits all the degrees between “yes” and “no” via an operator membership function to a fuzzy set, which is a configurable continuous function. Machine Learning involves iteratively finding the adjustment parameters of fuzzy membership operators in order to minimize decision errors on a database. Note that fuzzy sets can be used to represent both uncertain knowledge and imprecise data and often use “possibility theory” to compute a decision;
  • decision trees: the decision space is iteratively split into groups of cases, and each one is split into smaller groups, etc. The iteration of this segmentation process builds a decision tree. Each tree node has adjustable parameters which make it possible to classify a case into one group of cases or into another. The iterative search for node parameters minimizing decision errors on a database also falls under the scope of Machine Learning.


One of the questions for understanding a large volume of data is the following: is it possible to divide the database into so-called homogeneous groups? Groups are said to be homogeneous if the intra-group variance is smaller than the inter-group variance. This question, also called “vector quantization” has led to the development of many methods, among which we can mention;

Artificial Intelligence for Vehicles

  • Moving average or K-means clustering; this algorithm is simplistic. It involves initializing “seeds” (random vectors in the space of the data vectors), and assigning to each seed the vectors which are closer to one seed than to the other seeds. Thus, we obtain a group of vectors per seed. New seeds are calculated as being the barycenters of each group and the operation is repeated N times. After a while, the seeds stop moving, and the algorithm is said to have converged. Note that this algorithm was developed outside the AI sector by researchers working on Data Analysis and Signal Theory, and so moving average is not customarily considered to be part of the AI techniques. But this method is still the basis used by researchers in automatic clustering neural networks, which use this simple idea but implement a stochastic version into an iterative learning rule to improve performance.
  • Moving average or K-means clustering; this algorithm is simplistic. It involves initializing “seeds” (random vectors in the space of the data vectors), and assigning to each seed the vectors which are closer to one seed than to the other seeds. Thus, we obtain a group of vectors per seed. New seeds are calculated as being the barycenters of each group and the operation is repeated N times. After a while, the seeds stop moving, and the algorithm is said to have converged. Note that this algorithm was developed outside the AI sector by researchers working on Data Analysis and Signal Theory, and so moving average is not customarily considered to be part of the AI techniques. But this method is still the basis used by researchers in automatic clustering neural networks, which use this simple idea but implement a stochastic version into an iterative learning rule to improve performance.

A moving average is thus implemented without ever calculating the barycenter, something which offers mathematical advantages. But in order for this to become a SOM, we geographically position the neurons on a (usually 2D) map and apply an interaction between neurons – called lateral inhibition – inspired by the structure of the biological neural networks of our perception. The key point is to consider that the geographical neighbors of the elected neuron also have to modify their synaptic coefficients, but following a function called a “Mexican hat”: the near geographical neurons also bring their synaptic coefficients closer to the input vector which has contributed to choosing this selected neuron, but they do so with decreasing force due to their distance from the selected neuron. On the contrary, geographically distant neurons have their synaptic coefficients far from the input example. At the end of this process, each zone of the map is sensitive to a type of input, and the classes are geographically positioned depending on the topology: two neighboring classes are represented by groups of close neurons.

  • Neural gas: these are an extension of SOMs. Instead of measuring the distance between neurons in a geographic space, we calculate this distance within the space of synaptic coefficients. In other words, we consider that the closest neuron to the elected neuron is the neuron which would have been elected had the elected neuron not been there. This establishes a distance which sorts neurons differently for each input example, as if neurons were constantly moving in relation to one another, which explains the term “neural gas”.

At the end of the clustering operation, we obtain homogeneous groups of data without ever having to say which example fell within each group. The examples are grouped according to a similarity principle (“birds of a feather flock together”). It is for this reason that we speak of unsupervised learning.

Reinforcement learning

Reinforcement learning can be compared to supervised learning, indirectly: we do not know the desired output, but we know whether the computed output is admissible or not. The idea is to discourage unacceptable solutions and to reinforce the presence of acceptable solutions. This is called a cumulative reward mechanism.

Artificial Intelligence for Vehicles

Note that we also find this kind of idea of “admissible” solutions in other fields, such as in automation, for example, using the viability theory for servocontrol problems. This involves not seeking an optimal output depending on an optimization criterion, but a solution enabling the system to be “viable”, that is to say, acceptable from the point of view of piloting (a robot, for example).

Most of the classical reinforcement learning methods suffer from a combinatorial problem and are only suitable for problems with few dimensions (5 maximum for example). For instance, let us quote the Markov Decision Process (MDP) serving as Q-Learning or dynamic programming support: we consider that the system to be solved takes N number of states modeled by a Markovian process.

In the case where we do not know the process, we can use Q-learning. This involves learning a decision (a robotics action) for each state of the system. During the learning process, we make a Q (a, b) function evolve. This represents the (observed) reward given to action “b” when the system is in the “a” state. When operating, the system applies the action which maximizes the reward in the S state in which the system is set.

If we know the MDP, we can use dynamic programming, an optimization method dating from the 1950s, stipulating that the optimal response to a problem is a combination of the optimal responses to subproblems. “Divide and conquer”, as they say.

For problems with a large number of dimensions, it is possible to use:

  • Deep Learning, as presented above, and in that case, we speak of Deep Reinforcement Learning (DRL).
  • Genetic Algorithms [HOL 84]: a set of solutions is considered as a population of individuals represented by a data vector which is called a genotype. Each genotype (potential solution) gives rise to a result called a phenotype. A cumulative reward function makes it possible to kill or not to kill individuals. Individuals can reproduce in pairs by crossing each other (crossing over), which amounts to copying and pasting the ends of genotypes to make new individuals, and individuals can mutate (random modification of an element of the genotype vector).

The mechanism is Darwinian: the best solutions survive and are the only ones to contribute to reproduction in view of creating new solutions. Unlike Deep learning, we do not iteratively modify a solution to minimize an error, but iteratively modify a population of solutions in order to maximize the acceptability of these solutions. We, therefore, try to develop many solutions at the same time by crossing them, so as to find the best ones.

It is for this reason that we speak of “implicit parallelism”. The use of this approach requires having coded the genotypes using coding for which at least one local topology exists so that the interpolations between solutions are meaningful.

Case-based reasoning

Case-based reasoning draws its inspiration from the resolution of similar problems. For this, it is necessary to have a description of the problems which accept a topology in order to be able to calculate the distance between cases: if two representations are similar, the problems should be close, and vice versa. The search for similar problems is generally performed heuristically, or quite simply by searching for the N closest neighbors.

The underlying principle is to record the reasoning which made it possible to successfully solve a case, and then, either to directly apply the reasoning behind the closest solved case, or to apply a modification (a derivative) of the reasoning behind the closest cases.

The system continues to learn by storing more and more cases with the reasoning that made it possible to solve them.

Logical reasoning

Knowledge-based systems generally seek to retrieve knowledge from human experts capable of solving complex problems through logical deduction. Before knowledge-based systems are ready for use, it is necessary to elicit knowledge from the experts, by means of interviews and the resolution of study cases. When this logical knowledge is drawn, it needs to be modeled (as knowledge representation, generally in the form of mathematical logic).

To solve a problem, logical reasoning is applied by a special program called an inference engine which can operate with several levels of complexity: it can make assumptions, validate or invalidate them, manipulate logic using all of “de Morgan’s rules”, etc. The logic used can be binary or fuzzy. In the latter case, we can use more recent mathematical methods in order to combine logical operators with the theory of possibilities [DUB 88], for example.

Multi-agent systems

Multi-agent systems are inspired by groups of animals functioning in swarms: bees, ants, etc. Each agent (a bee) has a very simple behavior, like an automaton, but it is the interactions between automatons that endow the group with what is called an emergent property making it possible to solve a complex problem.

An agent has an autonomy of action and a reaction model towards its environment, following a “Belief-Desires-Intentions” (BDI) type [RAO 95]. The key to multiagent functioning is communication between agents as well as their interaction (influence of the action of an agent on the actions of nearby agents, for instance). Learning in multi-agent systems resorts to known methods like the Markov Decision Process (MDP), among others.

PAC learning

PAC (Probably Approximately Correct) learning introduces complexity theory into the learning process. The idea is that we have pre-established classification functions, associating a class with each input vector. One of these pre-established functions is called a “concept” and is the (complex) function responsible for properly classifying new entries.

In practice, we can initialize a collection of functions. For that, we have to find the function which is the closest to the concept in polynomial time, that is to say, the function that has a higher probability than P of providing the concept, with an error smaller than E.

Modern AI challenges for the industry

Explainability: XAI (eXplainable Artificial Intelligence)

AI techniques may work very well without their output is easy to understand, even when performing an a posteriori audit. However, in certain areas involving discrimination between individuals (decision to award credits, for example), or other areas with strong safety stakes (driving an autonomous vehicle on an open road, for example), transparency requirements are demanding. In the first case, transparency or “explainability” makes it possible to justify a decision, whereas in the second case, it helps produce a formal security proof, for example by using the SIL (Security Integration Level) formalism.

Those works which make the functioning of AI systems explainable are grouped under the term XAI (eXplainable Artificial Intelligence). One should note that – for this subject – knowledge-based systems have an advantage over purely numerical computing methods, such as Deep Learning, for instance. The dilemma when retrieving an XAI is that one generally must accommodate a decrease in performance in order to gain better explainability.

The design of so-called “hybrid” AI systems

Apart from a few experts in a given technique who try to solve all the problems with that technique at all costs, the engineers who have to design AI systems have come to the conclusion that it is necessary to use a collection of several techniques, in other words, a “hybrid” AI solution.

This, therefore, leads to solutions that can jointly integrate:

  • Deep Learning;
  • knowledge-based systems;
  • stats;
  • physics;
  • applied mathematics;
  • classic algorithms (image processing, signal processing, etc.).

The design of such a solution requires having broad general knowledge, and to avoid drowning in the almost infinite combination of choices.

For this, a methodology has been created to design solutions mixing classical algorithms with neural networks. It is called “AGENDA” (Approche Générale des Etudes Neuronales pour le Développement d’Applications, General Approach to Neural Studies for the Development of Applications). This methodology considers the system to be designed as an information processing machine, specified by its variants and invariants. In practice, it makes it possible to facilitate team creativity (it is necessary to pool the skills of several experts on various techniques), so as to obtain the traceability of technical choices, improve testing procedures, and – as far as learning is concerned – qualitatively and quantitatively polish learning and validation databases.

One of the effects observed from the application of this method is that one can obtain more efficient and more robust learning with a much smaller number of examples.

What is an “intelligent” vehicle?

In the past, humankind has already had access to a mobility vehicle with its own intelligence: the horse.

But it is clear that the piloting of this vehicle can sometimes be complex, namely when the assessment of the situation made by the horse and by the rider, differs. When we speak of an intelligent vehicle, we exclude the case of vehicles having their own free will, except in specific cases where we consider that the driver is not able to properly assess the situation. This is called driving assistance, or delegation of driving, which can either be partial or complete.


Artificial Intelligence for Vehicles

The purpose of ADAS (Advanced Driver Assistance Systems) is to inform the driver of situations he/she might not have perceived, but above all to act on the vehicle automatically in order to help the driver by acting in his/her place. We can divide the field of ADAS into two parts;

  • the action while driving normally: there is no emergency, the vehicle is traveling normally in a suitable environment, and the vehicle can, for example, automatically regulate its speed (Intelligent Speed Adaptation-ISA) via a device called Automated Cruise Control (ACC). We can consider that this constitutes a delegation of the longitudinal part of driving. Likewise, when the driving lane ahead of the vehicle can be clearly detected, the vehicle can automatically control the steering wheel in order to remain at the center of this lane, using Lane Departure Control and steering assist. We can consider this function as a delegation of the lateral part of driving.
  • Emergency action: when the vehicle detects an emergency situation, it is usually too late to alert the driver (this can even be counterproductive because it generates disruptive stress at a dangerous moment). There are several phases: during the emergency, during a crash, after the crash.

The different driving phases

Normal driving – The system is expected to “perform well”, seeking to minimize the occurrence of an emergency situation. As seen above, this is the domain of ACC (Automated
Cruise Control) and “Lane Departure Control”. 

Emergency situation –  If possible, we want the system to avoid the accident by reacting to the emergency situation with a quick and appropriate reflex. In the worst scenario, if the accident is inevitable, we want the system to make it the least serious possible. In this category, we can find;

  • the vehicle’s dynamics control systems: brake assist and traction control;
  • automatic braking systems: AEB (Automatic Emergency Braking);
  • potentially, automatic avoidance systems. Since its implementation is likely to generate an accident with the vehicles in the adjacent lane – surprised by the emergency avoidance – this action is more delicate than emergency braking. 

During the accident – If the accident cannot be avoided, the vehicle offers solutions to minimize its severity: deformation zone, safety capsule, airbag, seat belt. 

After the accident – Depending on the estimated severity of the shock, an automatic telecommunications system can decide to call for rescue. 

Highly distinct notions related to the idea of danger on the road

When it comes to potential accidents in the future, all engineers instantly think about the notion of “probability of collision”. Of course, this notion is important, but, as we will see below, other notions are important too.

Potential danger (“hazard”) – For a long time, the potential danger has been signaled by road signs like, for example, “ice hazard” or “rockfalls”. The disadvantage of these signs is that it is not always clear whether they make sense in all driving conditions. Nowadays, alerts can be integrated on variable message panels, or directly in the cockpit of the vehicle in the form of sound or voice alerts. Some such alerts are accidents in 3 km, traffic jams in 4 km, people walking in the lane, etc. 

This information regarding potential danger is either integrated into the digital map, or sent to the vehicle from other vehicles, or from the road infrastructure using a telecommunications system, generally called the V2X system (vehicle to vehicle (or infrastructure) telecommunications, coining the phrase “Connected Vehicle”). This information is interesting, but in practice, the driver often does not know what to do or when to do it. The hazard will only become an effective danger if the driver’s driving behavior is unsuitable.

Driving risk – When one monitors both the driving behavior (speed, acceleration) and the driving context (complexity of the infrastructure ahead, hazard warning, traffic conditions, etc.), any mismatch between these two elements can be detected. Such a mismatch is considered as a lack of caution in “driving risk”. This lack of caution/driving risk can be estimated in real time using knowledge-based systems. Driving risk is estimated continually during normal driving. Its level determines the frequency (low or high) with which a vehicle can find itself in an emergency situation (emergency scenarios being the trigger for an accident situation). As part of the above-mentioned ISA problem, an ACC can automatically regulate the speed of the vehicle in order to prevent speeding, of course, but also to prevent emergency situations that may lead to an accident. 

Criticality – The criticality of an emergency situation is the effective level of danger during an emergency scenario. This level can vary very quickly over time. Criticality can be estimated via several indicators such as;

  • probability of collision (or probability of trajectories interception);
  • time to collision (TTC);
  • probable shock energy.

Note that common natural language makes it possible for the word “risk” to be used quite naturally in these three cases (potential danger, lack of caution/driving risk, criticality). As a result, this makes some communications incomprehensible. Here, we prefer to distinguish the concepts by using the appropriate term each time. 

Global diagram of ADAS – A modern vehicle equipped with driving assistance can be seen as an automatic device including two features: a branch dedicated to normal driving, with ACC, for example (or the Lane Departure System), and a second, faster branch, dedicated to ADAS for emergency situations, such as AEB. The normal driving branch focuses on the extended ISA (Intelligent Speed Adaptation) aspect with automatic control of speed, which seeks to reduce the vehicle’s exposure to risk (keeping the driving behavior cautious). It is in this branch that we will come across the problem of respecting speed limits and that we will first have to estimate the driving risk, so as to keep it at an acceptable value. The emergency management branch continuously assesses criticality. When criticality exceeds a severity threshold, a switch device enables switching to emergency mode, whereas if criticality is zero or very low, the switch continues in normal driving mode.

The robot (ACC for example) executes the command in all cases.

Global diagram of ADAS
Figure 1. Global diagram of ADAS. For a color version of this figure, see

Note that this architecture makes it possible to implement an extension of the ISA functionality (automatic speed control) as well as to guarantee the control of risk-taking, without modifying the architecture of the classic ACC. It only requires integrating the function for calculating the driving risk (in green in Figure 1), which also includes the speed limit (read on the map or captured by perception).

The autonomous vehicle

Artificial Intelligence for Vehicles
Credit: smartcitiesworld

Different missions

Autonomous vehicles have been highly publicized, but in the end, one can realize that this term covers different types of missions which have often been quite shallowly explained:

  • The autonomous bus: a normal bus always runs at the same times on the same route (the bus line’s route). This route and its characteristics include invariants which can be “learned by heart”. There still may be environmental variability since sunshine and other weather conditions are likely to modify the outputs of the perception sensors, for the same place. In addition, temporary, moving objects (works on the road, for example) can modify the perceived local geometry. But these variations are few in number and easy to supervise or even “learn”, for instance in the case of sunshine and weather conditions. The traveling objects of this type that are currently being deployed are minibusses, called “autonomous shuttles”.
  • The autonomous taxi: the autonomous taxi as currently envisaged makes it possible to go from one known locality to another known locality, on-demand. This represents additional complexity since the potential routes are all those which go from one locality to another. If we have N localities, we have N x (N-1) routes. We can refer back to the case of the bus and “learn by heart” all the possible routes, even if N is relatively large (for example N = 100 produces 100 x 99 = 9,900 possible routes). We can also restrict the taxi’s radius of action (example: Paris or Los Angeles), and memorize every street in both directions (in the case when both directions are allowed), in which case the taxi can go from any address to any address within this restricted perimeter, instead of connecting only a limited number of localities. In a city like Paris which has about 5,000 streets, this yields an order of magnitude of 10,000 streets to be learned by heart, in terms of the sensors’ expected outputs: for example, we can have images at “every” place on every street, with no moving elements (deserted streets), and at each moment compare the image acquired in real time with the corresponding reference image: all the differences between these two images are objected leads (except the shadows cast in sunny weather and other such disturbances).
  • The autonomous personal vehicle: one might think that the autonomous personal vehicle looks like a taxi. But a personal vehicle has no range limitation: a driver from Canada can decide to go to Brazil by car, to a specific address. In this case, we can clearly see that it would be difficult to imagine having precise references for each place of the journey. We should all the same note that digital map suppliers wish to make this type of reference with a 3D map with an accuracy close to 5 cm. But building such a reference at the global level is a very long process. This means that, for the time being, this particular autonomous vehicle must be able to drive in an unfamiliar environment (just like a human being). The level of complexity then increases considerably, which explains the delay in achievements compared to the announcements made by manufacturers a few years ago. Let us observe that a partial solution could be assimilated to the case of the taxi: in Europe, for those of working age, 56% of the distance traveled per year is between home and work. One could imagine a vehicle that learns the references of the homework journey by heart, then the children’s home–school trajectory, etc. in order to complete 90% of the journeys independently, like a shuttle. But most car manufacturers have chosen to break down journeys, not into mobility functions, but into infrastructure types: highways, roads, cities. The vehicle must therefore move properly in an unfamiliar environment, as well as cope with the ample variability of infrastructures. It is therefore easy to understand why autonomy functions start with highways, the simplest and most “standardized” type of infrastructure. 

Different levels of autonomy

Autonomy levels are divided into five levels;
– Level 1
At level 1, the driver is still in charge of most of the car’s driving functions, but with a little assistance. For example, a Level 1 type of vehicle may provide brake assist if you get too close to another vehicle, or may have an adaptive cruise control function to control inter-distance and highway speed.
– Level 2
Level 2 vehicles may have automatic systems which take charge of all the aspects of driving: steering, acceleration and braking. But the driver must be able to take control of the driving in the event of a failure of part of the system. Level 2 could therefore be seen as “with no intervention”, but the driver must in fact always keep his hands on the wheel.
– Level 3: conditional autonomy
Conditional automation allows drivers to sit back and let the car do all of the driving, with no eye involvement. The driver can focus his attention on other activities. But this functionality is only available in certain cases (for example in traffic jams or on the highway). When we get out of these conditions, the driver must regain control, he must therefore stay awake.
– Level 4
Level 4 cars are called “mind-off” vehicles because the driver is not required at all, to the point that the driver can even go to sleep. But there are some restrictions, because the autonomous driving mode can only be activated in the certain areas, or in case of traffic jams and other well-defined cases.
– Level 5
Level 5 does not require any human interaction. The vehicle can steer, accelerate, brake and monitor road conditions, enabling the driver to sit down without having to pay any attention to the car’s functions. As explained above, the complexity of the achievement is not only related to the autonomy level, but also to whether the environment is entirely known or not, or on the contrary, is completely unknown. In this sense, an autonomous shuttle at level 5 is only able to complete one trajectory. This ultimately looks easier to achieve than making a level 3 vehicle go from one place to another at any moment. 

Liability in the event of an accident

From the moment that it is no longer the driver who drives, but the vehicle itself, the question of liability for potential accidents arises. In the event of an accident, it might be interesting to have a “black box” having recorded signals clearly showing whether the vehicle had been driven carefully or not. This corresponds to the concept of driving risk (or lack of caution), as defined above. The use of such a system for monitoring caution or the lack of caution at the time of the accident could be one lead for dealing with the issues of legal liability.

General diagram of the autonomous vehicle

We can draw a diagram very similar to that of ADAS: the “emergency” part is very similar: the purpose is to detect emergency situations. The most important part in terms of complexity is the normal driving branch, since the vehicle must follow the road’s rules under all conditions, and respect the driving cautiously rules, adapted to the situation.

Artificial Intelligence for Vehicles
Figure 2. Global diagram of ADAS. For a color version of this figure, see 

Perception is almost always muti-sensor (several cameras and/or lidar, radar, etc.). Compared to predictive ACC, it is no longer just a question of defining a safe driving speed, but of managing the vehicle’s entire driving behavior (speed, acceleration, direction, etc.). The “understanding” part is generally based on the concept of “situation holding”: at a given moment, we have a representation of the world with the infrastructure, all detected objects (their nature, relative speed, direction, etc.), the detection’s reliability, the context (day, night, fog, etc.), and so on. We predict the evolution of this representation at the following instant. If the predicted situation is sufficiently close to the acquired situation, then the situation is considered to be “well understood”. For objects which have been detected and disappear without being able to see them leaving the sensors’ field, we generally continue to make them evolve as ghosts in the representation of the world, with a potential oblivion over time. For example, this would be the case of two wheels remaining at a possible blind spot. If the prediction turns out to be seriously false, this means that the representation of the world is erroneous (we then speak of “hypothesis breaking”), and we are confronted with an incident which is generally reported to the branch, and possibly triggers specific monitoring. 

The construction of the intelligent vehicle’s basic building blocks employing AI methods

In this part, we present application examples of AI techniques to the basic building blocks making it possible to build the diagrams presented above. We do not claim to be exhaustive, but rather illustrative. In point of fact, in most engineering cases, there are no unique solutions.

Artificial Intelligence for Vehicles
Credit: frontiersin

Normal driving management

The purpose of perception bricks in normal driving

In normal driving conditions, the sensors (camera, lidar, radar, etc.) are not intended to detect emergency situations in order to estimate their criticality, but rather to estimate how the driving complies with the rules of caution by extracting the risk factors from the scene. A good illustrative example is the measurement of the distance from the vehicle ahead: if two vehicles both travel at exactly the same speed of 130 km/h, and they follow each other keeping to a two meter distance (inter-distance = 2 m), then one understands that the follower (vehicle) is not being careful, in other words, is running a risk. But criticality is low, almost zero: the probability of collision is zero and the time to collision is infinite since the two vehicles are both traveling at the same speed.

Conventionally, perception sensors are cameras, radars, and lidars. These sensors provide signal streaming and real-time images. We will not focus on sensors here because the focus of this chapter is the use of AI techniques (which intervene after sensors do).

Note that perception in the normal driving mode also has the objective of feeding the “situation holding” brick, in order to check that the understanding of the scene is correct. The AI technique most used for this perception brick is Deep Learning.

The main applications are;

  • detection of other vehicles;
  • detection of vulnerable people (pedestrians, bicycles, etc.);
  • detection of weather conditions (rain, fog, snow, etc.), and in particular their impact on visibility.

In the images obtained from cameras, in particular, we find real applications of CNNs, and other applications of hybrid systems combining algorithms for retrieving image features, using multilayered neural networks.

Understanding brick

The understanding and situation holding brick as described above can use different techniques: of course, the whole “prediction” part can be carried out by logical reasoning propagating and validating or invalidating hypotheses to achieve consistency. This part is still embryonic in many vehicles which, for the moment, are limited to classifying the data of integrated sensors in one class. The class number is named by a typical life situation on the road. But it is important to check the spatial and temporal consistency of this life situation so as to really speak of “understanding”.

Decision brick

Decision-making can involve;

  • logical reasoning: the advantage is the transparency of the system’s operation. The downside is that it is difficult to find a method for representing life situations on the road which is easy to handle and sufficiently comprehensive;
  • a purely numerical calculation method such as neural networks is simpler to implement, but is not conducive to the transparency of the decision-making process. To train a (symbolic or digital) decision system, we are in the typical case of reinforcement learning: if the decision leads to a road safety situation which is acceptable, then one reinforces this decision, otherwise one penalizes it. After learning, the decision system always remains in construction for acceptable decision cases. At least, this is the goal that must be validated.

We must estimate acceptability: as we have explained, there is a brick called “driving risk assessment” which has been precisely designed for this. This function can be used during reinforcement learning, in real time as in control, or during validation tests.

Driving risk estimation

Driving risk is the degree to which driving behavior is unsuitable for the road context. We interpret this degree of mismatch as a lack of caution which coincides with the notion of the risk triangle proposed by Frank E. Bird and H.W. Heinrich. Today there is a construction of such a brick [BRU 19], which gives a new impetus to the autonomous vehicle. It is a hybrid AI, mainly based on a knowledgebased progressive logic system (running deep knowledge in road safety), which – among other things – employs the theory of possibilities, and apart from that, Deep Learning for the recognition of life situations on the road.

Potential danger alerts

The potential danger warning (or “hazard warning”) is a direct application of digital geographic maps and telecommunications (V2X). This involves informing the vehicle of potential dangers (fog, accident, etc.). We should note that these alerts can be used as an entry for the estimation of the driving risk, because they characterize the driving context. The approach by the vehicle towards a potential hazard will or will not be transformed into a driving risk, depending on the driving behavior.

Emergency management

The purpose of perception bricks in normal driving

As presented above, emergency situations are characterized by their level of criticality (probability of interception of trajectories, time to collision, etc.). An emergency is generally defined as “less than 1.4 seconds to a collision if nothing is done”. The goal is to apply a safe and automatic reflex very quickly, such as the AEB (automatic emergency braking).

The elements detected are the same as during normal driving, but the information retrieval that is made from it is different since we focus on the calculation of criticality instead of the driving risk calculation (for example, the “time to the collision” instead of the “inter-distance”).

A short chain towards decision-making: the concept of reflex

In the case of reflexes, the decision is directly connected to perception, as happens with human beings (example: rapid closing of eyes in the case of rapid movement in the vicinity). There is usually no elaborate step of understanding because the emergency requires immediate action. Most systems are based on a ranking of perception data that tells what to do. This is the preferred field of neural networks and Deep Learning. 

2 thoughts on “Artificial Intelligence for Vehicles

  1. Pingback: 5 Signs of Bad Oxygen Sensor | mech4cars

  2. Pingback: There Will Be Humanoid Robots in Every Home by 2050 | Hi-tech for Future

Leave a Reply

Your email address will not be published.