"Yandex" told how her robotic vehicles coped with the harsh conditions of the Moscow winter

“Yandex” told how her robotic vehicles coped with the harsh conditions of the Moscow winter


Since its first appearance on the roads in 2017, Yandex’s unmanned vehicles have traveled more than 10 million kilometers, mainly in Moscow. The company noted that the conditions in the capital are among the most difficult: these are the busy roads of the metropolis, and the bizarre network of streets and lanes, and the varied weather. The company shared its experience gained in the winter of 2020/21.

According to Yandex, this year winter in the capital was characterized by heavy snowfalls, and on February 13, a record for precipitation was set in the entire history of observations. Despite this, robotic vehicles continued to travel the streets in all weather conditions. In winter, automation has a number of additional difficulties that must be overcome.

First of all, this is poor visibility during a snowfall. When it snows, some of the leader’s rays (more than a million per second) are reflected from the snowflakes and create noise. The company’s specialists have developed a special neural network capable of leveling this side information. It works quite efficiently (before and after filtering):

Also, at subzero temperatures, problems for automation can be created by steam coming from the exhaust pipes of cars, ventilation shafts and rain showers. It can be so thick that it can look like a physical obstacle on a lidar cloud. Neural networks must also confidently distinguish the vapor curtain from real objects. At the expense of several million kilometers of robo-car mileage in winter, the AI ​​receives the knowledge necessary for training in order to assess the real situation on the road from the camera picture. Here’s what a lidar cloud looks like before and after filtering:

It is worth noting that due to the lidars, the system was able to recognize a pedestrian hidden behind the steam, despite the dense steam. Even a human driver would find it difficult to cope with this task in such conditions:

In addition, the automation must recognize the parameters of the road surface: the snow can be fresh and loose or hard and packed. Sometimes it looks harmless, but in fact, ice is hidden under it, and a variety of options can meet during one trip. Robomobiles need to be able to adapt to different surfaces, and therefore the Yandex autopilot is trained to determine the coefficient of friction and take it into account when planning actions on the road. Here is just one of the trips around Moscow in heavy snow:

It should be said that Yandex’s robotic vehicles move along the streets, relying not only on the data of cameras and lidars, but also on pre-loaded three-dimensional maps of the terrain, which are constantly compared with the data of the lidar and allow determining their location with an accuracy of a centimeter. Heavy snowfall can change the city beyond recognition: road markings can be hidden, the boundaries of the roadway are invisible, and during snow removal, snowdrifts as high as a two-story house can grow.

The autopilot retains the ability to navigate in space even in such conditions. The system uses data from both the lidar cloud, and readings of inertial meters (IMU), and odometry. By comparing the information, the automation determines with high precision where the car is located even in difficult situations – for example, when the wheels slip and the ABS is activated. Also, the robotic car is able to automatically update the 3D map. If a huge snowdrift appears on an empty spot, the system will “see” it, and the snowdrift will become one of the map objects. This is how the same street looks like at normal times and after heavy snowfalls (the color shows the height, red – above 3 m):

These are just some of the difficulties that robotic vehicles face in winter. For example, in the cold season there are more road vehicles in the city; because of the snowdrifts, the roadway becomes narrower; there are fewer parking spaces, and many park in the second row; traffic jams occur more often, and the most impatient road users try to go around them in the oncoming lane; pedestrians are in a hurry to get into a warm house or office and take a shortcut.

If you notice an error, select it with the mouse and press CTRL + ENTER.

Leave a Comment