Seeing

Supports bridge operators in operating bridges safely

Rijkswaterstaat (the Dutch Ministry of Infrastructure and Water Management) appointed Pegamento as the party with the best solution for increasing the safety of bridge operation through detection. Rijkswaterstaat had issued a challenge to industry players to detect and help prevent dangerous situations in bridge operation through innovative applications. By using cameras with image recognition and artificial intelligence, the operator gets to see via a dashboard what is happening on the bridge deck.

Rijkswaterstaat Detection Challenge

By means of detection, Rijkswaterstaat can increase the safety of bridge operation. To this end, they had issued a challenge to industry players to use innovative applications to detect and help prevent dangerous situations when operating bridges. During the award ceremony, the results of the practical test were presented, with Pegamento's solution being judged the best!

The BCR system can use multiple cameras to track road users using Artificial Intelligence and enables the operator to see what is happening on the bridge deck via a dashboard:

• Custom software detects cars, bicycles, and pedestrians on bridges and roads using AI
• The situation model informs and alerts operators in a timely manner via an online dashboard
• Selected as winner by Rijkswaterstaat in the Detection Challenge

Safety through image recognition

BCR is an intelligent system designed to support bridge operators. In combination with cameras, BCR detects and tracks objects on a bridge using Artificial Intelligence (AI). The operator receives a real-time overview of the situation on the bridge via the dashboard. There is also a message of 'bridge safe', 'bridge not safe,' or 'bridge possibly unsafe' to enable quick assessment of the situation. The operator can review what happened in potentially unsafe situations from any angle to make an informed decision.

The next logical step appears to be testing the BCR system on a real bridge. In addition, we foresee many more possibilities where we can use connected cameras to map a traffic situation on an online dashboard. Examples include situations in waterways, locks, traffic intersections, public areas, and logistics centres. Various combinations with other systems are also possible, allowing statistics to be collected on traffic volume, weight, and wear of the road surface in general.

Training an AI network

After winning the Rijkswaterstaat Detection Challenge, we are now working with Rijkswaterstaat to take follow-up steps so that our solution can become part of the generic Internet of Things intake chain.

At the bridge over the Biesbosch lock in Werkendam, we therefore worked on setting up a hazard detection system to support the bridge operators during their work. Two cameras will detect anomalous situations on the bridge when the barriers go down. For example, there may be a bicycle still attached to the gate or a person still on the bridge deck when operators want to open the bridge. In the event of an anomalous situation, the operator receives a notification with 'watch out, something is happening here' and an image. This provides the operator with additional information to decide whether or not a bridge can safely open for a passing vessel.

We set to work on recording images to train an artificial intelIigence network so that the system learns to recognise anomalous situations. For this purpose, we recorded images of the bridge under normal and anomalous situations.

We are working on this test setup as part of the Smart Patrol programme to create an AI model for the Generic RWS IOT intake chain with Rob Michels from Rijkswaterstaat. In the video, Karin de Jong from Rijkswaterstaat and Thomas de Wolf from Pegamento, among others, explain what we have been doing.

Read more about it here: rijkswaterstaat.nl.

Testing the AI network

After collecting the visual data, we trained the AI networks to recognise anomalous situations in addition to detecting people, cyclists, and vehicles. We eventually performed a live test of this solution on the bridge over the Biesbosch lock, to see how well the system is able to deal with these situations.

We saw that the system is good at detecting and alerting in response to visible people, cyclists, and vehicles. It also alerted correctly for most anomalous situations, but there is still some room for improvement here to ensure that this continues to happen consistently in all types of circumstances.

We are currently taking a follow-up step in this project via an SBIR innovation competition to start looking at how well the AI networks behind the solution can generalise to other locations and applications.

Read more about it here: rijkswaterstaat.nl.