SINTRA field lab develops next-generation security solutions in the Port of Moerdijk
Critical industrial and civil infrastructures, such as airports, ports, power plants, and road networks, frequently suffer from disruptions caused by a wide range of man-made physical safety and security threats, from well-organised criminal activities to low-level but costly actions like vandalism. The ITEA project SINTRA aims to improve the resilience and protection of these critical infrastructures by developing an open multi-modal sensing platform to bring real-time situational awareness to control room officers.
Field lab in Port of Moerdijk
As part of the SINTRA project, the consortium partners have built a field lab to monitor organised crime activities in one of the Netherlands’ main harbours - the Port of Moerdijk. Located at a very busy spot in the port, with substantial day- and night-time operations, the field lab can detect hidden and complex behaviour typical of organised crime, such as drug smuggling, people trafficking, large-scale burglary and logistics of unauthorised goods. It can also identify lower-level criminal actions, including theft, vandalism, violence and environmental pollution. The field lab collects both training and real-world data on these activities, enabling the development of an innovative multi-modal AI model for criminal activity detection. Coupled with a visualisation dashboard, the solution will provide enhanced situational awareness to harbour security officers, police, and customs.
AI-driven analysis reveals hidden and subtle activities
Egor Bondarev, head of the AI for Multi-modal Sensing (AIMS) research lab at the Eindhoven University of Technology and co-leader of the SINTRA consortium, explains: “The core innovation of the project is in combining data from different sensor modalities in a single AI-empowered analysis model, to enable detection of subtle, hidden and context-dependent actions, which could never be detected with just visual cameras.” He adds as an example: “Think of a group of immigrants hiding inside a container. While visual sensors are not able to see them through the container, our infrared sensor can see a clear trace of a body that is leaning against the container wall. Also, our acoustic sensor array is able to detect that someone is talking inside the container.”
The combination of multiple sensor modalities improves the detection rate for simple criminal behaviours, such as violence or vandalism, from 88 to 92 AUC (Area Under the Curve). For more complex activities, including hidden immigrants or drug transfers, the SINTRA solution enables detectability in general, which would be difficult or impossible with traditional single-modality systems.
A demonstration video of the SINTRA field lab can be viewed here:
Collaboration driving innovation
The SINTRA field lab combines cutting-edge technology from several consortium partners: a powerful 4-GPU computation server that runs AI-based detection models from TU/e and analyses sensor data in real time; acoustic sensor arrays installed at multiple locations by Sorama; a set of visual and infrared sensors from Bosch/Keenfinity; multispectral sensors from MantiSpectra; and UAV drone and ground rovers with visual/IR/multispectral sensors from Avular and ViNotion. The visualisation dashboard that renders the detected events for control room officers is developed by Omines.
Next steps
Halfway through the project, the field lab is delivering promising results towards the development of next-generation security solutions. In the remaining phases of the project, which runs until August 2027, the SINTRA field lab will be used for 24/7 testing and validation of the full-fledged SINTRA platform against actual criminal activities. It will also enable the consortium to gather feedback from end-users, such as control room officers, to prepare the platform for further exploitation and market entry.
More information: https://itea4.org/project/sintra.html and https://www.sintra-ai.nl/
